7 Ways to Reduce Time-to-Test

UPDATE: In Spring of 2010 BreakingPoint unveiled the pioneering Cyber Tomography Machine to help you with problems such as the ones described in this post. Read more.

Test engineers have a tough job, testing more and more complex network devices that demand a more realistic testing scenario and at the same time needing to do this faster and most likely with limited resources. Because of these challenges I'm going to use my blog posts over the next few days to excerpt sections from our paper "7 Ways to Reduce Time-to-Test". This paper, which came out this week, provides seven practical tips that can help reduce time-to-test, but also and perhaps more importantly, helps speed up product development time. Today we'll start with the introduction and the first two tips. If you want to jump ahead you can download the paper here. Let us know what you think of the tips, either here in the blog or on Twitter.

Today’s network equipment and application servers are getting so sophisticated that it is not possible to test every configuration or deployment scenario. To make matters worse, test engineers are being asked to accomplish their testing faster and with fewer resources. This unique combination of increased complexity and economic pressure actually creates a significant opportunity for test engineers to shine. The following seven steps can help reduce time-to-test, lower overall testing costs, speed product development and provide your customers with superior product, not to mention make you look like a hero at schedule time.

1) Keep It Simple Stupid

Traditionally, test labs have had to cobble together several disparate tools to create the realistic testing environment needed to evaluate today’s network devices and application servers, resulting in an enormous drain on testing time, resources, and budget. A dissimilar mix of testing tools in your lab forces you to patch together different testing systems, resulting in wasted days of set-up time. It has gotten to the point where legacy testing tool vendors have started talking about forming industry alliances in order to get their solutions to play well with others. It remains to be seen whether these alliances will ever deliver the integration requirements. In the mean time, test engineers are forced to spend time learning a mix of tools, setting up tests across multiple products, adding more equipment to the lab and integrating reports to provide meaningful test results.

Look for testing tools that can conduct comprehensive Layer 2-7 testing using realistic application traffic at speeds of 10+ Gigabits per second and faster in a single system. By consolidating Layer 2-7 testing you can eliminate the need to cable, maintain and develop complex configuration scripts, simplify your test harness and sync your reports. Recent tools even have built-in automation that can help eliminate the need to do complex Tcl scripting, which we will hit on more during step number three.

You can imagine the immediate benefits of simplifying this once complex configuration of racks of servers and disparate testing tools and the resulting reduction in time-to-test, not to mention the cost savings from the elimination of all that hardware.

2) Know Thy Enemy

Testing properly means understanding the types of application traffic you will be supporting and making sure your test equipment can easily support this mix. Devices run on production networks that support, on average, 30 different applications, not to mention facing hundreds if not thousands of security threats. Determine the traffic and scenarios that will be running on the network device or application server, and emulate these complex scenarios. To do this efficiently your testing tools should have, out of the box, many of the stateful application protocols and security threats needed to emulate your network and provide a realistic measure of performance and security.

One question to ask your testing tool vendor is: How often and how quickly can you provide new protocols and security threats? No company can afford the delays and costs of employing their own team of researchers to provide these protocols and threats, nor can you wait weeks, let alone months, for a test vendor to support an application protocol.

Find a testing tool that has the means to easily provide you with the latest stateful protocols and security threats on a consistent basis and one that can create specific protocols for testing in days not months.

Finally, once you have the ability to test with stateful applications and security threats, you can further reduce your time-to-test by employing application traffic profiles. Profiles should be easily set-up within your testing tool and based on industry standard mixes, which combine application traffic, security attacks and protocol fuzzing. With a few clicks you are testing with a realistic scenario.

Testing today with realistic application and security traffic means eliminating problems tomorrow, helping to reduce your time-to-test and accelerating your product development.

Posted by Kyle Flaherty

Printer Friendly Create PDF

0 comments
Tags: Layer 2-7 Traffic // Network Traffic Generation //