Setting The Standard

Next week is my favorite week of the year. It's the Sales Operations meetings held at our headquarters in Austin. Each year we bring the sales people and sales engineers together to review the previous year and preview the year moving forward. More importantly I get to show off.

2009, from all facets, was an incredible year here at BreakingPoint. Sales had an amazing year, with huge growth. Our employee base grew by nearly 30%, much of that being our heavy investment in the security group. We put out 3 major releases and 3 minor releases of our firmware for the BreakingPoint Elite. And our application protocol list now tops more than 100 and our strikes are over 4,300.

This news is certainly exciting, but that was last year. And this is a completely new year and we are ramping up in engineering like you could not imagine. The next firmware release will once again improve the performance of everything from our application protocols, security engine and our SSL. And, of course this is all done without having to replace your blades and at no extra cost. Bet your other vendors don't say that every year.

Next month I'll be putting together a screencast showing you all the features in our next release. I'll save all the juicy bits for then, but here is a teaser of what to expect:

• Five new test labs, including huge strides for mobile networks.
• Changes in the way we are using the NetLogic network processor.
• Switch from using our network processor cores to do SSL, to leveraging the encryption engine on the chip itself (the impact this has on our number of handshakes is staggering).

Last year we changed the way people test their network equipment, this year we will set the standard.

Reminds me of when I worked at Cisco many years ago and Kevin Kennedy (Vice President) would show a slide in which Cisco was compared to other similar companies. There must have been 30 companies listed and at the time 3Com was below us, Lucent ahead of us and all the way at the top were companies like HP. At the time HP was 10x the size of Cisco. Today, Cisco is tens of billions of dollars ahead of HP, with a third of the employees. 

Every year that presentation showed Cisco passing yet another company. We have the same chart for our industry and the same goals, and some companies were ahead of us at the beginning of 2009. During 2009 we passed four of them and this year we will pass four more. And one day, like Cisco, we'll be at the top of everyone else's list.

NOTE: Sometimes Cisco didn't pass a company, the company fell. I'm seeing a lot of that lately, maybe I should send some flowers.

Proxies

Proxies

A Look at Some of the New Features in BreakingPoint Elite

Today I wanted to take a look at the latest version of BreakingPoint Elite, which is available immediately to BreakingPoint users, and includes more than 30 new features. I thought I would post a few of the features that we featured in the news release and embed a product screencast from CTO Dennis Cox demonstrating some of the features.

1) Dual Stack IPv4/IPv6 Testing Capabilities and Support for Current IPv6 Standards

BreakingPoint has the unique ability to generate blended stateful application traffic mixed with live security attacks at line-rate speeds and high session counts, delivered from the same address space. Using BreakingPoint Elite's IPv6 dual stack testing you get the industry’s most comprehensive and up to date IPv6 capable testing, allowing you to:

• Simulate IPv6 traffic through each testing component including Client Simulator for load testing IPv6 capable application servers.
• Ensure compliance and validate performance with the latest IPv6 standards and blended IPv4 and IPv6 traffic.

2) Capture and Recreate Functionality

BreakingPoint Elite has the largest capture history buffer, with 8 Gigabytes per blade, to reduce the time it takes to debug network equipment and application servers. This latest product upgrade adds in advanced filtering of packet captures to capture and report on only the data needed. This includes:

• Post-processing ability to compress the capture export, allowing you to specify start and end frames.
• Berkley Packet Filter (BPF) capability to isolate traffic by IP address, port, protocol and more.
• Ingress pre-processing filtering parameters based on VLAN ID, Source IP, Destination IP, Source Port and Destination Port.

Additionally you now have the ability to incorporate data originating from a real network into tests, replaying stateful traffic using the TCP stack, including three-way handshakes and any necessary retransmissions. This includes raw playback of traffic to test for common issues with ARP, RARP, TCP/UDP headers, TCP SYN floods, DDoS, invalid packets and more.

3) Pattern Matching (PCRE) for Full Data Validation of Network Security Devices

BreakingPoint Elite regular expression pattern matching validates traffic flows utilizing a user-defined data pattern by comparing that pattern against incoming network traffic. QA and R&D can quickly identify sequence errors and data errors in a network device or application server, including validation that the network equipment is generating the appropriate network traffic. BreakingPoint Elite also supports PERL Compatible Regular Expressions (PCRE), to allow users to match substrings in data packets, as well as provide the ability to store the matched data for future use.

4) Impairment Support for Realistic Wide-Area Network (WAN) Simulation

Support for simulating impairments within IP traffic for realistic WAN simulation by configuring impairments within BreakingPoint Elite Network Neighborhood and easily replicating and fine-tuning for any DUT.

5) Enhanced Test Automation, Reporting and Layer 2/3 Testing Capabilities

More extensive test automation including Attack Plan Iteration that enables security testers to loop security attacks. Also, reporting enhancements include the ability to customize content to be included in reports, and export in additional formats. Finally, numerous Layer 2/3 testing capabilities have been added including UDP and enhanced HTTP support in Session Sender, addressing expansion, stateless TCP in Routing Robot and expanded frame sizes.

4 Product Bakeoff Pitfalls and How to Avoid Getting Played

Most IT buyers take product data sheet claims with a grain of salt. Capabilities are overblown and performance numbers are guesstimated using only Layer 2/3 or HTTP traffic. We all know this isn't real. So, we don't trick ourselves into believing that the numbers will hold true in a real network.

Beyond the performance numbers, there's also the question of features and functionality. When it comes to network and security equipment, one size does not fit all. Each reacts differently to traffic on your network and offers a unique set of features and performance levels. The top network equipment and server vendors are also combining multi-purpose capabilities to introduce all-in-one products in an effort to differentiate their offerings. Evaluating the best firewall, IPS, or server load balancer to meet your needs just became even more bewildering. How do make head-to-head comparisons with products that operate differently? How do you generate a realistic mix of traffic and still evaluate all of the products with the same mix?

A product bakeoff is the most important step in the hardware evaluation(PDF), negotiation, and planning process because it offers the best way to understand the real capabilities and performance of the devices you are considering for purchase. Yet lack of planning, bad bakeoff behavior, and the use of synthetic traffic often leave buyers with poor decision-making data and a false sense of security. As well all know, a bit of preparation on the front end is always required to get accurate results. Combine that with insider knowledge and you can ensure an accurate and deterministic network device evaluation. Here are 4 product bakeoff secrets every IT buyer should understand before embarking upon any head-to-head product comparison:

1. Bits will bite you. Bit blasting, the use of IMIX or even PCAPs will not give you an accurate view of how a product will perform under load and under attack--in other words--in your network. Mike Hamilton effectively characterizes the problem with IMIX in this post: "Frequently I am asked about UDP "packet blasting", IMIX, RFC 2544 and other testing procedures. My answer is always the same; these testing "methodologies" are not realistic. Testing a firewall with 100% stateless UDP traffic is pointless when the actual traffic it will see consists of a wide variety of applications over both UDP and TCP."

It is absolutely vital that you adequately categorize the makeup of the traffic on your network and simulate these conditions under load during your product bakeoff. As the chart below illustrates, you will likely see several magnitudes of difference in throughput between competing products when exposed to real network traffic.

2. You can have the best of both worlds: realistic and random. Once you've moved up the stack to realistic application layer traffic, you also need to ensure the traffic generated is somewhat random so that you can make direct comparisons between products and generate the same traffic in the same way to produce deterministic results. A pseudo-random mix of traffic will provide more accurate results, help you identify, resolve and validate fixes, and prevent vendors from gaming the system with clever programming tricks. This may sound mutually exclusive, but let me assure you it is not. The answer lies in the use of pseudo random number generator (PRNG).

While most testing products do not provide PRNG capabilities, BreakingPoint leverages PRNG to generate a pseudo-dynamic blend of traffic that is both real and repeatable. Once a seed value is set, BreakingPoint will create data by generating all data variants in the same way for each test executed. You can pick a number and generate the attacks or application traffic the same way every time to run repeated tests on different products or to repeat tests on the same product or network to determine the source or resolution to a problem.

3. Don't be left with a false sense of security. Another important reason to evaluate security equipment with random mix of traffic is because it is the only way to truly evaluate vulnerability vs. exploit variant detection capabilities. It's generally agreed that vulnerability filtering provides superior security coverage. Yet many products can’t perform the complex matching needed to deliver a vulnerability filter. They offer products that are programmed to identify exploits. The problem with this approach is that slight variants of the exploits these systems are programmed to identify can easily bypass the system. Vendors then release new filters and hackers develop new variants to bypass the same vendors again. The Conficker worm is a prime example of new variants bypassing security products every few months.

A dirty little secret of synthetic testing vendors is that their exploits are branded with trademarks or other recognizable content. Vendors can easily exploit this code by programming their products to recognize the code and trigger filters to easily pass product validation. While it may appear that these products are working as promised, this is no indication that the equipment is capable of recognizing and filtering real security attacks in a production network. PRNG eliminates the possibility that devices under test can be programmed to recognize and react to codes embedded in test traffic.

4. Stick to a proven test methodology. One of the challenges for buyers who are structuring their product bakeoff, is the lack of good solid test methodologies for thoroughly validating equipment. They use RFC's that are outdated, specifications become obsolete quickly, and most of the methodologies you’ll find online are many years old. Since they were published, network equipment has evolved, so you need a current product bakeoff test methodology designed specifically for the devices you are evaluating. Here are a few the BreakingPoint Labs team has published: Firewall Testing methodology, Server Load Balancer Testing methodology, IPS Testing methodology and other test methodologies. However, more are needed.

Tip of the Product Bakeoff Iceberg?
These are probably just the start so I’d like to hear your product bakeoff advice and your methodology wish list. By sharing tips, collaborating on test methodologies, and exposing secrets, we can all get the most out of product bakeoffs.

Show Us Your Board

During my interview last week with BreakingPoint CTO Dennis Cox (When NP Doesn't Equal Network Processors) it became apparent how important it is to use real network processors within network testing tools. Also important was asking to see the board to determine if your testing tool is using a true network processor. I asked Dennis to show us the BreakingPoint Elite board and that footage is below.

Now it's your chance to get in the fun. We have created a YouTube room so you can show off your board. Whether you are in the testing tools space, manufacture network equipment, build custom PCs or anything else, head on over to the room and show us your boards!