Testing With Twitter; The Importance of "Real" Applications

We have talked here about the overall importance of testing…particularly the need to test with REAL application traffic, live security strikes and doing it all at speeds of 10Gps and faster. These days that first criterion, testing with real apps, is getting more and more important. A great example comes from many of the applications popping up, seemingly every day, in the ‘social networking’ and Web 2.0 space.

Twitter is one of those social networking apps that emerged seemingly overnight and rapidly became a social phenomenon. It transcends description but has caught fire and become so mainstream, so fast, it is now regularly mentioned on CNN, USA Today and even CSI (Miami, of course). This exposure, and the applications usefulness, has created enormous growth. Depending on who you talk with there are an estimated 340,000 public Twitter accounts with 60,000 new accounts starting each month, or potentially 2.2M Twitter users (as of July 22^nd). Twitter has also seen great usage as a customer service tool, a social network and a lifesaver. 

With great success comes great disappointment and scrutiny of course, and Twitter has been getting plenty of poor coverage because of their inability to keep their network functioning at peak performance. It’s a worry of every company, from Fortune 500 to a start-up…if you provide a service that relies on a network, you best know that the components of that network can handle all the application traffic, security strikes and do it with blazing fast speeds.  We take this to heart and have actually added Twitter to the real applications folks can now perform protocol testing using BreakingPoint (check out our TweetLease).

Folks like Biz Stone at Twitter and other business heads (I’m talking to you Mr. Zuckerberg) must push their network equipment manufacturers to make sure they are properly testing network devices for today’s sophisticated application usage.  Twitter is simply one example of apps that can affect your network performance, but it is something that was not tested on your network devices before they were deployed. Tomorrow there will be more and you need to make sure these are being used to test your equipment.

Of course, this topic brings us into the ‘architecture discussion’ and how it is crucial that your testing tool is being updated with realistic blended apps and security strikes on a regular basis, and by “regular basis” I don’t mean every three months.  There is no telling where the next heavy use application traffic will come from but let’s just say my money is on Plurk…just because I love the name.

Latency and the Global Exchange

Earlier this week Cisco put out its study on global exchanges showing that there are at least 50 different cababilities an exchange needs to achieve peak performance.  The study focuses on the top ten, which can be seen in the cross-hairs of this graphic.  Coming in at number 5 was latency:

Ivy Schmerkmen at Wall Street & Technology has a terrific in-depth look at the study, including this nugget:

Before going through a major upgrade, the London Stock Exchange could handle up to 450 transactions at the peak second. After a four-year IT overhaul project, the LSE's new trading system TradeElect is now able to handle up to 2,500 transactions at the peak second and it cuts the time it took to process an order to 10 milliseconds from 140 milliseconds. Low-latency trading platforms are critical to exchanges that want to go after the algorithmic trading business, according to the study.

This hit home with me since this week I had also been reading a report put out by TABB Group in April, "The Value of a Millisecond" where they estimated, “Up to 10 milliseconds of latency could result in a 10% drop in revenues.”  I knew from previous work in the financial services industry that time was money...but wow. It is obvious that exchanges need to have the most up to date network equipment, but in the rush to get these devices deployed, measuring their performance and security is, d'uh, critical (you knew I'd bring it back around to test, right?). 

The reason I'm knee deep in all of this research was because this week we announced our support for the FIX/FIXT protocol.  FIX (Financial Information eXchange) is the global messaging standard for the electronic communication of "trade-related data" between financial firms such as banks, broker-dealers, exchanges, and industry utilities. If these exchanges need the latest and greatest network equipment, they need that equipment to have been tested using the FIX protocol (and many others, obviously). Just trying to do our part to grab those milliseconds back!

What Do You Think?

It has been a few months since we launched our new website and we wanted to get your reaction and thoughts. We put together a quick survey (9 questions) and we really want to know what you think. For your trouble you will be entered into a drawing to receive an iPod Touch or one of several $25 gift certificates to Amazon.com.

Traffic Analysis

As I may have previously mentioned, I love traffic analysis reports. I have several locations where I grab information and a script that combines them and collates all the data for me on a monthly basis. One of the sites I get my data from is Internet2.

The data shown on the Internet2 backbone is vastly different in terms of application percentages (less P2P for the most part). However, all the other numbers line up with almost all other sources. Check this out; 90% of all packets are between 1401 and 1500 bytes. Makes you wonder what your test rig is set to? It also reminds me of our conversation about IMIX, the description from CAIDA's site again is: "IMIX derives from analysis of NLANR traces and is tri-modal (e.g., 58% at 40 bytes, 18% at 576 bytes, and 23% at 1518 bytes)."

Netflow on Internet2 says 100 byte to 1400 byte packets are 9.15% and sub 100 byte is 0.46%, whereas IMIX is 18% at 576 bytes and 58% at 40 bytes. Again - why are people running IMIX? Crazy. After all, IMIX is based on real data, just like the data I'm provided, it's just I'm using current data. Unless your sending your device back in a time machine, I suggest you stop using IMIX.

My final point - how many applications are you testing - because Internet2 is a small backbone (in terms of users) yet according to the netflow data they have more than 40 applications running that they can measure.

P.S. 41.33%  of the traffic is unidentifiable :) That's a big piece of data that's missing.

Do the Math

I love math, although the funny thing is, I was really bad at it going through school and had to have a math tutor in high school. These days, I actually consider going back to school to study math from time to time. Talk about a change. What intrigues me the most is the ability to use math to come up with clever ways to solve problems. This became apparent to me since I actually honed my math skills by majoring in Music Composition.

During freshman year my professor played a recording of a piece of music, roughly 5 minutes long. He would play the piece once and it was our job, after the piece was played, to write the general score down. The challenge; we could only write for the first 30 seconds and the last 10 seconds of the recording! Five minutes of music with 40 seconds of writing to get the general score? Impossible we all said, it would take hours to transcribe 5 minutes of music with this many parts. We would need to listen to it over and over again to get it right...we all stared at him like he was crazy.

By my Junior year I could do it with only 10 seconds of writing at the beginning of the piece and a second at the end. It was dead simple, because it was math. What you need is the data that starts it and ends it. You can determine the path from those two points. [NOTE: This applies mostly to classical music]

The same thing applies to network equipment testing - I see too many engineers test their equipment without data. What does traffic look like in the real world? Sometimes a customer will quote an RFC, but the RFC is 10 years old and the standards, while not documented, have changed completely. If you don't know what the network looks like, even just 10 seconds of the beginning and 1 second at the end, you can't test your product correctly. After all, just because you know networks have HTTP in it, doesn't mean that's the only thing running. To complete the analogy, it would be like listening just for the drums to figure out the tune; you need to hear/see the whole thing for a good picture.

This resonated with me again when I saw a competitor just announced P2P support. Just announced? P2P is over 30 percent of network traffic and some test tools are just adding it? Heck some don't even have it yet. If you don't have the data, things will never add up.