Apr 13, 2011

NSS Labs Firewall Report Highlights the Need to Rethink Firewall Testing

by Alexander Karstens

By Tim Walker

The latest NSS Labs findings on firewall security won't be good news to CISOs. As reported yesterday in Network World, the independent testing organization just published its “Network Firewall 2011 Comparative Test Results” report, which covers six devices from top firewall vendors. The evaluations, which were conducted in part using BreakingPoint products, found that half of the firewalls crashed and failed closed when hit with traffic designed to test their stability. Worse, five out of the six succumbed to a well-known attack that we (among others) have been talking about for the past 18 months.

The devices evaluated are serious machines that companies count on to give them real network security. And all of them did function as advertised under a variety of performance, security, and stability scenarios. Yet most of them failed to handle challenges that should be considered routine. Consider this: the failed stability tests were conducted at a maximum of 350 Mbps, which is a far cry from the multi-Gigabit throughput rates claimed by their makers.

The TCP Split Handshake Strikes Again

As mentioned above, five out of the six firewalls failed to stop the “TCP Split Handshake” spoof that NSS Labs threw at them. The Network World article explains that this attack “lets a hacker remotely fool the firewall into thinking an IP connection is a trusted one behind the firewall.”

One of our engineers unearthed this vulnerability in a November 2009 post titled “TCP Portals: The Handshake’s a Lie!” and we informed device manufacturers about the vulnerability around that time. All BreakingPoint CTMs have been able to generate this attack since late 2009, and we know it’s been a topic of continuing interest for our audience, because the original post has remained a perennial favorite on this blog. Rick Moy, the president of NSS Labs, has noted that the TCP handshake vulnerability is basic and well documented.

Rethink Firewall Testing — and Validate for Yourself

In discussing the scenarios used for the NSS Labs evaluations, Moy and his colleague Vik Phatak, the CTO of NSS Labs, have called for ongoing validation of equipment against these conditions. As reported in a Security Week article covering the same report, NSS Labs also concluded, “Measuring performance based upon RFC-2544 (UDP) does not provide an accurate representation of how the firewall will perform in live real-world environments.”

We couldn’t agree more. If you’re buying a piece of equipment that you’re going to count on for the security of your network, you owe it to yourself to validate that the device will perform under the actual conditions that it will face in your production environment.

Take a look at the NSS Labs report (you’ll need to be an NSS Labs client to see full results), and then let us know how we can help.