Cloud Computing Testing

Ensuring Performance and Security of Cloud Infrastructures

The challenges of testing cloud-based environments go well beyond just the size and complexity of the environment. The dynamic nature of cloud infrastructures means QA must effectively test for the ever-changing and the unknown:

• Unlimited web services and applications
• Elastic demand
• Morphing usage patterns
• Dynamic resource allocation

The constantly evolving characteristics of this adaptive environment create an unlimited number of testing variables. It’s a bit like building a skyscraper on shifting sands. It can be done. You just need the right tools or a whole lot of time and money.

When it comes to innovation, security and load testing products have lagged behind the hardware and software they were designed to test. This has stifled the pace of delivering stable next-generation products and services. In  conversations with a number of cloud vendors, the same pattern appears to hold true. Julien Sobrier, QA engineer for Zscaler, a provider of multi-tenant SaaS security services stated recently: “Right now, we are using the same tools that appliance vendors are using: Protos for fuzzing, regular HTTP performance tools (Autobench), etc., and custom tools to create a bigger variety of traffic.”

In an attempt to simulate realistic conditions, cloud vendors like Zscaler and larger cloud vendors like Microsoft, Amazon, and others must use legacy tools, some originally designed for traditional LAN-based environments, placed on hundreds of servers to simulate load. The net result: an amalgamation of tools and workarounds that is costly, brittle, and not ideally suited for the task at hand.

Testing Cloud Infrastructure: Four Important Factors to Consider

Because of the dynamic and shared characteristics of cloud infrastructures, the scale of the issue of how to effectively test the environment is unique. In this real-time, adaptive environment, four factors are paramount: elasticity, scalability, realism and security.

Elasticity
Renata Budko, VP of Products and Marketing of HyTrust, sums up the dynamic nature of the cloud: “Cloud infrastructure is very different from the traditional set up, where applications make exclusive use of the server resources. In the cloud, resources are pooled, access infrastructure is shared and resource allocation changes dynamically. The underlying infrastructure is flexible and changes often, and every major revision of hardware or software can result in significant performance impact and, of course, needs to be tested. However, in the case of cloud, the process is so dynamic that discrete testing following major upgrades is simply not sufficient. You need to understand how services behave when deployed together.”

Testing in such a dynamic environment must closely reflect the elastic nature of usage patterns. Conditions change frequently, demand is elastic, resources are shared, and more frequent releases require continuous testing. There is little time for cumbersome test configurations and scripts. Extensive automation is a must in order to replicate a wide range of usage patterns. What’s more, dynamic resource allocation means that applications cannot be tested in isolation from one another. In addition to a high performance, highly scalable testing platform, vendors also need agile, automated and easier-to-use testing products designed for a fast-paced, dynamic environment.

Realism
In today’s frenetic Web services/dynamic application/mashup world, it is impossible to emulate all of the different types of traffic that traverse the cloud. However, vendors still need to emulate a broad mix of traffic. That means more realistic testing tools that support an ever-changing mix of applications, services and an incredibly high volume of sessions and memory usage all hit with sophisticated security attacks. Otherwise, you are left to run small tests with a limited mix of applications, then extrapolating the results. Ultimately, you are making assumptions about how things might work with very few real data points. This is not sufficient to ensure the SLAs that cloud vendors must deliver to business application users.

Gomez’ CTO Imad Mouline echoes the need for more realistic testing, underscoring the need to create more realistic transactions with real-user monitoring and reporting. According to Mouline, “It is important to simulate load to the infrastructure that is coming in from different IP locations, different networks and different places in the world.”

Scalability
Mouline also talked a lot about the fact that we assume too much when we sign up for cloud computing services. One of the largest and most dangerous assumptions is stability in the face of peak demand. Prior to deployment, vendors must offer assurances that services will perform reliably under a variety of load conditions.

Simulating that load is easier said than done, however. Again, Sorbier: “Most tools I've worked with simulate one client and one server. We need to simulate thousands of clients and servers, with different IP addresses, to be closer to reality.” Imagine the number of servers and the millions in LoadRunner fees that it would take to run the tests needed to emulate the typical load these infrastructures see on a daily basis, much less under peak conditions. With the state of legacy testing tools, it would take a dedicated hydroelectric plant and a government bailout. Many have suggested using the massive computational power of the cloud to simulate that load, but we have yet to see this live up to the performance needs of growing cloud vendors.

Security
Possibly the biggest challenge lies in the security arena, in part due to the historical practice of conducting security and performance testing in isolation. In traditional hardware/software testing scenarios, security and performance organizations are typically siloed. However, as security breaches become more frequent and severe, and as network equipment vendors embed more security functionality into their core network products, organizations are integrating their security and performance testing efforts. But change has not come fast enough for the cloud. In this more open and accessible environment, the stakes are higher.

Security attacks are not just dangerous; the protection against these strikes can have an immense effect on overall performance. Vendors must recognize the impact of security on application performance—specifically, web services—and test accordingly by emulating the real world, where hackers are exploiting the cloud to spread viruses and malware and to attack critical network infrastructure.

In a recent presentation on Cloud Computing Security, Eva Chen, CEO of Trend Micro, reported “a new virus is created every 2 seconds”. Clearly, you need massive computing horsepower and a wide range of current security attacks to test for this in order to remain secure. That’s going to require a new type of testing product designed to evolve along with the security landscape. Ensuring effective protection for cloud networks will require constant vigilance to keep testing tools current.

Advancing the State of Cloud Testing

New testing tools are emerging, but the unique obstacles presented by the dynamic, shared cloud infrastructure have set the bar almost impossibly high, leading the vendors we spoke with to rely on home grown in-house options. Clearly, these vendors are in need of new and more scalable, flexible, realistic and cost-effective options.

Request a Personalized Demonstration

To learn more about BreakingPoint Cloud Testing from a BreakingPoint expert, sign up for your demo today.