JULY 2, 2008

The Difficulty With Testing Blacklists

I was recently directed to Greg Hoglund's new blog, Fast Horizon. So far the few posts he's made have been excellent reading, and I generally agree with most of what he's been saying over there. Just recently he put up a post entitled "Whitelists are the new snake-oil" where he convincingly outlines both why blacklisting is a failing approach to stopping malware (if it ever really worked in the first place), and why the whitelisting approach that many vendors are moving toward is equally doomed to failure. Blacklisting is the current industry standard approach to stopping malware and is used in technologies such as anti-virus and anti-spyware software. Blacklisting is also the approach that most network security device vendors take to blocking network attacks and exploits. While firewalls are the obvious exception to this rule as they are essentially whitelist based, the vast majority of packets that get blocked, filtered, or identified by other network security devices such as IDS and IPS systems are done so via a match against a blacklist of traffic signatures. These signatures are analogous to fingerprints of the known malicious data that traverses the network. The following quote from Greg's post nicely sums up the difficulties with this approach:

"Blacklisting sounds ideal, but it doesn’t work. New malware emerges daily that has no corresponding blacklist signature. The malware must first be detected, and then processed. There is always a time window where Enterprises have no defense. Recent figures suggest that the AV vendors are falling so far behind this curve that they will never catch up with the deluge of new malware arriving daily. It can take weeks for a signature to become available.

This deluge of new malware is due to several factors. First, there is more money behind malware development than ever before. Second, we weren’t really that good at capturing malware in the past. Today, new malware can be automatically collected, without human intervention. The slow trickle of malware turned into a flood as honeypot technology emerged. Sensor grids can obtain new malware samples with efficiency - they automatically ‘drive by’ (aka spidering) malicious websites to get infections and leave open ports on the ‘Net so automated scanners will exploit them. In parallel to the automated collection efforts, cybercrime has risen to epic levels. Finally, the barrier to entry has dropped for the cyber criminal. Cyber weapon toolkits have become commonly available. Anti-detection technology is standard fare. New variants of a malware program can be auto-generated. A safe bet is to expect thousands of new malware to hit the Internet per day."

What he describes happening in the malware space is also happening within the attack and exploitation landscape. Network attacks and exploits are becoming increasingly more dynamic and harder to fingerprint for a number of reasons. As individual exploits increase the number of targets they can attack, which usually consist of various combinations of target software, the operating system running that software, patch-levels of both the software and operating system, the hardware architecture that both are running on, and any number of other target environmental factors, identifying a single exploit or vulnerability based on it's related network traffic becomes much harder. When you then factor in all of the evasion techniques such as data massaging, reordering, randomizing, encoding, encrypting, tunneling, and any number of others, many of which have become standard in publicly available tools such as the Metasploit Exploitation Framework, the problem gets exponentially harder and you begin to approach the number of potential variants of a single attack or exploit that the malware folks are seeing in their space.

When faced with an ever-changing hostile network environment of malicious traffic, blacklists of signatures and filters that are developed to detect this data must be equally robust, and must be tested and verified with test cases that approach the dynamic nature and variation that you would see in the wild.

For further reading on dynamic security tests, please see my previous post entitled "File Format Vulnerabilities and Dynamic Exploit Generators" from a few weeks ago. It helps make Greg's point in that it illustrates how many variants of a malicious file must be properly detected when even only a handful of fields of data within that file can change and it still be a working file format exploit.