Oct 13, 2010

Load Testing Software Is Never Enough: Scaling LoadRunner to Deliver Resilient Application Services

by Tim Walker

By Tim Walker

In IT environments, just as in our personal lives, sometimes the only way to understand real behavior is to examine a subject under stress. I was reminded of that last week when I saw the results of a recent project we completed with the IT staff of a prospective customer looking to improve their private cloud computing performance and security.

Our new friends within the group responsible for the company’s virtualized infrastructure were running headfirst into the performance limitations of load testing software and also wanted to factor in security considerations. Historically, for this group and many other large IT groups, recreating the conditions needed to properly validate application infrastructures has been a pricey proposition. As we’ve covered in previous blog posts, this is essentially due to the performance limitations of software-based load testing solutions.

Our lead systems engineer, Kristi, showed the group how they could combine the BreakingPoint Storm CTM and HP’s LoadRunner product to put the company’s virtualized application infrastructure through its paces in ways that they had never seen before. She ran through several scenarios to establish baseline levels of performance, then did a detailed analysis of how the virtualized infrastructure responded to LoadRunner alone, and then to LoadRunner operating in tandem with the BreakingPoint Storm CTM. The analysis produced a huge volume of data (all of it captured, of course, by our product), but for this post I’ll highlight one particular comparison.

Using LoadRunner to simulate 300 users for one of the applications running on the virtualized infrastructure with no other activity, Kristi put the application through a series of routine tasks -- a user accessing the home page, entering text in a search box, clicking a link to another page, etc. Each of these tasks, performed over and over, is represented by one of the lines in the graph below. Average transaction response times looked like this:

Good to know . . . but far too clean -- with just a few exceptions -- to reflect the user load and complex interactions that these applications will face in the real world. Here, by contrast, is the same exercise run with the BreakingPoint Storm CTM adding real-world user load on top of LoadRunner’s functional test. Again, each line on the graph represents how long it took to execute a different function within the application repeatedly.

As you can see, this presents quite a different picture, one that’s far more cautionary -- and realistic -- for the IT pros who have the job of ensuring that this manufacturer’s network and application infrastructures can face stiff challenges. Three trends stand out here:

Note how many more transaction response times went above various thresholds -- 5 seconds, 20 seconds, 30 seconds, and so on. If you were responsible for this infrastructure, you’d want to know that you should expect these slower times.
The LoadRunner-only simulation produced unrealistically smooth lines for most transaction response times; the second simulation, by contrast, portrays the variability of response times that network engineers should expect in the real world.
The information on the second chart suggests another set of questions -- about how the interplay of different processes affects system performance -- that could be explored through further simulations. The first chart doesn’t suggest any need for that kind of exploration.

If you want to see how your applications will perform in the real world when they’re deployed across your infrastructure, you have to move beyond traditional load testing for private clouds and subject them to real-world levels and types of stress. Doing this gives you a much more accurate picture of your infrastructure’s performance. And, once you add in our security strikes and protocol-fuzzing abilities, you’re a whole lot closer to the conditions you’ll face in the real world.

Find out how to address your application performance challenges with the BreakingPoint Application Load Evaluation Service.

Tags: