May 05, 2011

Prerequisites for Network Device Evaluations: 15 Minutes to a Better Bakeoff

by Mike Hamilton

By Mike Hamilton

It’s almost Interop week, my second-favorite week of the year (No. 1 is the week after Christmas). A BreakingPoint crew will, of course, be on the show floor—in Booth 1667—to answer all of your questions about our products and services. We’ll even have several of our customers (Cisco, Fortinet, McAfee, etc.) giving presentations in our theater.

Now for my shameless plug: Each day I’ll be presenting a talk called “Contenders and Pretenders.” The talk will lay out what you need to know about setting up and executing a great network device bakeoff. In 15 minutes on the Interop floor we will discuss the prerequisites for ensuring the success and usefulness of a device evaluation. Here is a preview of what to expect:

Network Device Evaluation Prerequisite 1: Understand Your Requirements.

The pre-planning stage of a bakeoff is probably the most difficult, and the part that I dislike the most. As an engineer, I find myself wanting to jump right in and start building tests, tweaking things, and playing around to see how different input stimuli affect output behavior. But in my experience I’ve found that a little bit of time spent on definition of requirements makes everything else go more smoothly. During this phase you’ve got to ask yourself some tough but basic questions about what you’re trying to accomplish.

There could be any number of reasons for going through a device bakeoff. Many times, business needs are the primary reason: increased network demand, requirements for more security, or even a need for more stability. Those three drivers are familiar to any network engineer, but differences in how important each one is to your business could lead to vastly different technical requirements for your bakeoff. Make sure you completely outline these prior to beginning the bakeoff.

Network Device Evaluation Prerequisite 2: Understand Your Metrics.

Given what you discovered about your business drivers in the preceding step, what should you be measuring? How many times have you heard device vendors say, “My firewall is a 150-gig device”—as though throughput were the only thing that mattered? While throughput is certainly an important piece of the puzzle, it’s just that: one piece. On the other hand, how many times have you heard, “My device can handle 350,000 connections per second”? Probably not very many—yet the connection rate might be the most important metric for your particular bakeoff.

If your business driver is to support increased network demand for an application, a higher-throughput network device might be the answer, but the rate at which new sessions can be established may actually be even more important. You must understand the relevant metrics to be able to adequately judge among multiple competing devices in a bakeoff. Otherwise you have no benchmark against which to measure the devices.

Network Device Evaluation Prerequisite 3: Know Their Score.

device evaluation The final way to prepare for a custom bakeoff is to use a standards-based scoring method that takes into account performance, security, and stability. For example, the BreakingPoint Resiliency Score process performs a straightforward evaluation of the resiliency of your devices and scores it, with 100 being the top score.

The BreakingPoint Resiliency Score is easy to configure and provides a great deal of baseline insight into how your devices will perform under the pounding they will take in your network. It looks at many factors—throughput, connection rate, security, handling of corrupted traffic—to give you an easy way to get a clear picture of the whole puzzle. With that knowledge in hand, you can easily start eliminating devices that don’t satisfy your business needs so that you can focus on your short list of devices for custom testing, with a much better idea of what you’re looking for.

So here’s my personal invitation: If you’re going to be at Interop next week, I highly encourage you to stop by the BreakingPoint booth to see my presentation. Many of the concepts I’ve talked about will become even clearer with the added context of a live product demonstration. And you never know: We might even see a published BreakingPoint Resiliency Score from one of our most important customers.