Mar 15, 2011

Know the Score: Measuring the Performance and Capacity of Cloud and Virtualized Infrastructures

by Kristi Thiele

UPDATE: BreakingPoint's Application Load Evaluation Service is now available to help you you properly test cloud and virtualized infrastructures.

By Kristi Thiele

Imagine this scene: You’re logged in to an online kitchen retailer. You find exactly the right saucepan and click “Add to Cart”. You go through the checkout process, click “Buy,” and . . . nothing happens. You click again. Your browser now shows you a blank screen and keeps cycling, but without actually doing anything. A bad scenario for any online business and a frustrating one for any consumer.

Multiply that scenario by hundreds or thousands and you’re describing the nightmare of companies that operate big data centers. These concerns aren’t just limited to retailers, either. Any organization that relies on servers for their business knows that the experience for the end user, whether a consumer or an employee, is critical.

Validating Cloud and Virtualized Data Center Environments

Preventing that nightmare from happening is why we introduced the BreakingPoint Data Center Resiliency Score, which is an empirical way to measure and optimize virtualized and cloud-based resources using a BreakingPoint CTM. Moving from physical to virtual data center architecture is an art at best: You can’t just read a set of specs from a virtualization vendor and know how to optimize the new architecture. The Data Center Resiliency Score takes out the guesswork and provides tangible results that give data center operators guidance for resource allocation.

The Data Center Resiliency Score comes with reference virtual machines (VMs) for file, Web, database, and mail servers built in. The goal is to allow you to have a quick and easy setup of common applications that you would deploy within a virtualized data center environment. The Data Center Resiliency Score that is calculated represents the number of end users who could concurrently use an application server or data center. For example, if you were going to deploy Web servers in a DMZ using VMs, you could use the provided Web server VM to find out the number of Web end users who could be supported. You can also mix and match the different types of servers to suit your particular data center implementation. We also provide an Application Server Resiliency Score, which focuses on one or more applications active within a single VM. We can cover the practicality of that on a different day.

Data Center Resiliency Score: Step-by-Step Instructions

Let’s walk through a sample scenario for Data Center Resiliency. Thinking about the online shopping experience from above, it’s clear you would need a Web server. To keep it simple, let’s say we are considering deploying two Web servers to support a new online application. Each VM has the opportunity to have a certain amount of virtual hardware resources allocated to it. For example, should I allocate two virtual CPUs (vCPUs) and 8G of RAM? Or can I get by with one vCPU and 4G of RAM? These are exactly the questions that can be answered by assigning the resources and then running the Data Center Resiliency Score to validate them.

First, let’s get the servers ready. The first step is to create the VMs that you will use to represent your application server that will eventually be deployed. We even provide step-by-step instructions to help you through the process of deploying a template generator that creates these VMs:

Using the generator provided, you simply answer a few networking questions, and the VMs get created automatically. Once you have deployed your two VMs and determined the resource allocation for the first scenario, you are ready to start the Data Center Resiliency Score process within the BreakingPoint CTM. For the first run, I have assigned one vCPU and 4G of RAM to each of the VMs.

Simply fill in the required information, so that the BreakingPoint CTM knows the proper network information for communicating with the Web VMs, as in this screenshot:

Reserve your ports for this scoring run by clicking Choose Your Ports.
Next, click Create New under “Network Configuration.” As shown below, a step-by-step wizard will guide you through setting the client IP addresses used for communicating with the Web VMs, as well as indicate the Web VM IP addresses that were assigned when the template generator created the VMs.

Getting back to the main setup screen, you’ll see that “Device Capacity” has two options:
- “At Least” — Use this option if you want the process to start at the designated number of users (1,000 in my example) and add more users over time until the server can no longer sustain at least a 90% successful transaction rate.
- “Exactly” — Use this option if you want the process to jump immediately to the designated number of users and hold that for 20 minutes, provided that the servers can sustain a successful transaction rate of 90% or more.
Give the report a name and click the Validate button. The validation process ensures that everything is configured properly. Once complete, select Start Test on the next screen and the process will begin.

During the Data Center Resiliency Score process, a screen will indicate real-time progress as well as the instantaneous quality of experience (QoE), user count, and latency. The other types of VMs are grayed out, as they are not active for this scenario. We certainly could run Web, File, Database, and Mail at the same time and get a simultaneous view of the status on each. As you look to optimize and rightsize your data center, you should select the VMs that closely reflect your desired production environment.

This test will run until the QoE drops below 90%. QoE is a measure of the ratio of successful transactions to total attempted transactions.

When the whole process is complete, a Data Center Resiliency Score will be displayed:

For the given resources allocated to my two Web VMs, I can expect to support 8,740 users concurrently as they actively request content. Looking at the column of icons on the right-hand side of the graphic, it’s easy to tell that this particular validation was run against two Web servers and no file, database, or mail servers, but keep in mind that you could easily include any or all of those back at the main setup screen if other server types are relevant to your needs.

Besides the score itself, the BreakingPoint CTM also generates a report that summarizes the test results. For example, I can review the following graph and see how successful transactions varied as more and more users were added, eventually falling below the acceptable range.

Optimizing Virtualized Environments: Where Do We Go from Here?

If the Data Center Resiliency Score achieved is acceptable for my new Web application, then I am done. If I know that my virtualized or cloud environment must support more users, I can simply add more resources to my two VMs and see how that affects the user count. Alternately, I could stay with the configuration of one vCPU and 4G of RAM per VM but add more VMs to see if that works better.

I have tried both of these options and discovered that, in this case, four VMs yield a user count of 15,000 — almost twice the user count for two VMs. If I stay with two VMs, I can allocate two vCPUs and 8G of RAM to support the same 15,000 users. So in my case, I could support about the same number of users using two VMs with more resources or four VMs with fewer resources allocated to each. If I were the data center operator, I would know that going with the two-VM setup would save me from having more VMs to manage than necessary. If I wanted to, I could refine this even more by using the performance monitoring inside VMware’s vCenter management tool to observe memory consumption, CPU utilization, and more.

The goal is to optimize the proper resource allocation for each VM based on the type of application server it will be supporting. For each hardware environment and each type of application server, the results will vary — which means that the only way to know for sure what resources you need is to give the Data Center Resiliency Score a try in your own environment.

Tags: