Will Cloud Computing Be the Tipping Point for Change in the Test Industry?
by Pam O'NealThe cloud computing space is heating up once again with IBM, Sun, Cisco and others introducing the Open Cloud Manifesto this week. The document was designed, in the words of the group’s founders, to start a conversation around standards and help clients ask the right questions about cloud interoperability.
It strikes me that Cloud Computing is doomed to fail unless someone starts asking questions about performance, security, and load testing for cloud-based environments. How are the major cloud players actually testing their networks and applications within the cloud for availability, performance and security? Or are they even testing at all with the cloud in mind?
Most of us have experienced at least one negative experience with cloud apps such as Google Docs, Amazon’s S3, Microsoft Azure or another service. We’re not talking sluggish performance. We are seeing outages of 10 to 20 hours or more. When it comes to business applications, that’s not considered acceptable down time.
Clearly, better performance monitoring and testing is in order to avoid what Greg Ferro, of EtherealMind.com, calls the “Vista effect,” where constant coverage of technology failures makes the acceptance of the technology nearly impossible.
No one disputes that more comprehensive and realistic testing of cloud networks is critical to mainstream adoption of cloud computing. And, while availability is obviously the most critical factor, performance and security are also weighing heavy on the minds of IT organizations that must deliver to SLAs whether they manage the environment or not. What’s more, cloud networks make enticing targets for hackers as one breach exposes the content of many users from many different companies.
Cloud Testing Has Its Challenges
The real issues behind the lack of testing and subsequent performance problems are the cost and complexity of testing on this scale using legacy tools. Testing cloud applications and networks demands a wide mix of application traffic, current security coverage, and incredibly high-performance and throughput. Essentially, you need to create a realistic testing environment with an ever-changing mix of applications and increasingly sophisticated security attacks. That makes delivering high performance a moving target.
That’s part of the reason performance and load testing on an enterprise or Internet scale has always been so painful, inefficient and costly. Now, push this to the cloud and you can begin to see a hint of the challenges ahead. Consider the added dynamic of routine maintenance, patches and upgrades and the need to factor in the time to test how the patches and upgrades will impact performance.
Limited Options for Performance and Load Testing in the Cloud
Historically, vendors have been left with few options, most of which would erode the very value propositions that make cloud computing so attractive—cost savings and agility. You could purchase or rent hundreds of servers to emulate users and application traffic, buy a whole bunch of testing software, purchase a slew of applications, license security attacks, and tie it together with a bunch of custom scripting. Or you could contract with a testing service provider and wait for a test window. Unfortunately, it’s not just a one and done proposition. Networks and applications change.
No one is arguing that testing in a cloud-based environment isn’t complex. It’s about as challenging as it gets. I’m just saying that performance, security and load testing has been far too difficult and expensive for far too long. And, the time for change has been dramatically accelerated with the onset of cloud computing.
Traditional testing tools were simply not designed for this dynamic, complex and high performance computing environments, and haven’t evolved to keep pace. Even a heavy hitter like HP LoadRunner is considered past its prime for today’s market needs. Without radical change in the performance and realism of today’s testing products, what we continue to refer to "truth in testing", cloud vendors will continue to be stuck between a rock and a hard place leading to the ultimate FAIL of cloud computing. I mean, if Google can’t do it with their immense resources, can it be done?
In my next post on cloud testing I’ll take a look at the requirements for testing cloud networks and ways to introduce realistic high performance conditions without significant downtime or huge investments in legacy testing tools and server farms.
Download the BreakingPoint Server Load Resiliency Methodology, a step-by-step guide to measuring the performance, security and stability of your cloud infrastructure.
