AUGUST 25, 2010

The Costly Lessons Learned When Network Traffic Shaping Devices Fail

Often, the most informative blog posts come from our team in the field, which is why I’m writing today. We were engaged in a professional services project recently, when a customer relayed the details of a pricey lesson learned at a large mobile carrier. We are sharing the story in the hope that it might help other organizations avoid this costly mistake.

Network Bandwidth Leasing and Traffic Shaping

Service providers, in an effort to provide the widest possible coverage and best performance for their customers, lease network coverage from wholesale providers. They pay a lot for this coverage, especially if they breach certain limits. It’s kind of like a toll road, but instead of drivers footing the bill as they pass over the road, the city pays based on the volume of cars running on the roads. They pay based on the volume of traffic, agree to limit this traffic and are penalized heavily if they exceed.

Because this network coverage is so expensive for providers, and because traffic volume is growing unabated, they often deploy traffic shaping devices to regulate traffic running over the leased networks. If all goes as planned and devices work as advertised, this allows them to plan for and control costs and remain profitable. And, that’s where we come to our costly lesson learned.

Unfortunately, most network devices do not work in the real world as advertised. The reason is simple and something we’ve discussed before; they were not evaluated using the mix of high volume, application-rich traffic that one finds on typical networks. When neither the manufacturer of the device, nor the buyer of the product choose to evaluate the performance of the traffic shapers using a real blend of application traffic under extreme load, they can only guess how it will perform in the real world. In this case, when the devices were deployed they ran into major performance issues including a complete open failure. And, that failure gave “free access” for users to those toll roads.

Consequently, the company exceeded the bandwidth allocation from their wholesaler on several occasions, resulting in significant, unforeseen charges. Ouch. This has to be extremely frustrating for everyone from the CFO to the line-of-business manager tasked with managing costs—a cost containment measure ultimately resulting in unexpected cost overruns. You can imagine the political ramifications. And all it would take is some realistic validation. Lesson learned: Why guess when you can know.

Posted by Pam O'Neal