Oak Ridge develops powerful intrusion detection systems

Scientists at the Energy Department’s Oak Ridge National Laboratory have developed an attack analysis tool that uses machine learning to filter noise and increase the effectiveness of commercial intrusion detection systems.

Oak Ridge Cyber Analytics was developed as part of a Lockheed Martin-funded project to develop a self-healing network. “ORCA effectively sits on top of off-the-shelf intrusion detection systems, and its correlation engine processes information and learns as cyber events arrive,” said Justin Beaver, the research scientist leading the development team.

The correlation engine supplements or replaces the preset rules used by most intrusion detection systems to detect attacks or other malicious events. “The problem is there is a sweet spot with the rules,” Beaver said. “The existing IDSes are great for existing problems. But if you open them up, you can get swamped.”

GCN’s William Jackson writes that if the rules are all turned on, the system will produce more information than can be effectively used. If they are tuned too finely to limit reported information, they can miss important events. The ORCA correlation engine allows operators to open the intrusion detection system by using all of the rules and then sending the data to a correlation engine for analysis before the system issues alerts.

“We use machine learning,” Beaver said. “We are not using rules; we are using examples. But if you don’t have a way for an operator to say, ‘These are the kinds of things that are important,’ the system doesn’t work.” An analysis engine provides that capability. “That is the part that I am proudest of.”

The analysis engine allows an operator to look at intrusion detection system data in a variety of ways, including:

• Clusters that provide high-level views of activity that can be highlighted or excluded to reveal patterns.
• A timeline graph to show temporal relationships of different kinds of events.
• Swarm intelligence, an innovative technique that provides what Beaver called “a nature-inspired view of computational intelligence.”

Jackson writes that swarm intelligence uses a bird flock model based on the idea that “birds of a feather flock together.” Dots representing specific types of traffic behavior from specific IP addresses are placed on a grid, and a simulation tool allows them to move around the grid and congregate when two dots representing similar behavior come together. As some dots flock together, a pattern should emerge that shows similar behaviors from different IP addresses.

When the system identifies an attack pattern, the operator can identify that behavior in a swarm or flock. With that information, the analysis engine could identify zero-day attacks that rules have not previously identified.

ORCA is an operational prototype incorporated in Lockheed Martin’s experimental Defense and Self-Healing Network. In addition to a number of other metrics-collecting tools, ORCA uses Splunk, a search-and-reporting tool that gathers log and other data from applications, servers and network devices and indexes them for searches. That should make ORCA compatible with most commercial intrusion detection systems, Beaver said. “I think we could adapt to anything that puts out a log file thanks to Splunk,” he said.

ORCA’s advanced capabilities require operators to spend some time training the tool. “When you first fire the thing up, you are going to have to invest some time upfront in laying out the examples” for the analysis engine, Beaver said. That learning period probably will last several weeks before the engine knows enough to handle detection and alerts on its own.

For now, the examples are expected to be unique to each location where ORCA operates. But Oak Ridge researchers are hoping to develop a collaboration tool that would let users share examples so that each developer does not have to build them from scratch.