JANUARY 26, 2009

7 Ways to Reduce Time-to-Test: Tips 5-7

Last week we started to excerpt from our paper, "7 Ways to Reduce Time-to-Test", talking about automation, resource sharing, access to stateful traffic protocols and more. The final three tips in the paper look at removing a layer of abstraction between your network configuration and IP addresses and routing information, easily reproducing bugs for QA and how to stop babysitting your test equipment.

Download the full paper here (no registration required).

5) The Devil is in the Details

Re-configuring and developing new scripts every time you need to change a testing requirement is a large waste of resources. By creating a layer of abstraction between your network configuration and IP addresses and routing information you can eliminate the need to re-configure and develop new scripts every time you need to change your testing requirements. This step can eliminate days of configuration, allowing you to change an existing test in a matter of minutes.

Whether changing a test between a switched, routed, NAT or VLAN environment or changing IP address ranges, by creating this layer of abstraction you can quickly and easily designate addressing information for each test interface. You will also be able to create network domains in moments that will be reflected instantly across all tests and eliminate the need to tear down and reconfigure equipment. Some test gear excel at this, while others fall short. Other ways to create layers of abstraction is by leveraging physical layer switching devices such as the APCON®. These will remove the need to rewire your lab for each test.

6) Get Your Developers What They Need

Reproducing bugs for developers can sometimes be a chore, even as we realize the importance to the overall production. During these times you will want to simply provide an instant snapshot of any bugs for the developer. Most test devices allow you to capture and replay traffic in order to see what is causing any problems, however these captures are often too shallow and do not represent a true sample. Furthermore you want the traffic sample to be usable outside of your own equipment in an open format, e.g. PCAP.

When reviewing testing tools look that you are getting enough capture history per interface and that you can export the traffic sample as an open format. This will accelerate the debugging of devices as you witness the effects of real traffic by importing live traffic packet captures and coverting it automatically for stateful replay.

Additionally, device under test (DUT) Automation, which we wrote about above, also helps automate the regression testing your developers will need. DUT automation helps to eliminate manual power-cycle reboots or the need for a separate switch and allows a test engineer to connect the DUT, run a pre-saved test configuration, view the report and send it to the developer.

7) Unlock Yourself from the Lab

Babysitting test equipment is never a good use of your time. Why not monitor tests remotely and give yourself a chance to leave the lab on time for once! Before you get the idea that this is just about convenience, consider this: how many times have you come in to the surprise that your test failed in the middle of the night? This results in lengthy delays and costs.

Look for devices that do not require a client and have remote management capabilities, including the ability to securely log into the unit from a remote location to start, monitor, and conclude a test. Furthermore you need the ability to manage the DUT or SUT remotely, for example to cycle power and verify that it has been reset between test runs. Combine this remote functionality with automated testing and the ability to send results as email attachments and you might never have to go to the lab on a Saturday again.