

Next week is my favorite week of the year. It's the Sales Operations meetings held at our headquarters in Austin. Each year we bring the sales people and sales engineers together to review the previous year and preview the year moving forward. More importantly I get to show off.
2009, from all facets, was an incredible year here at BreakingPoint. Sales had an amazing year, with huge growth. Our employee base grew by nearly 30%, much of that being our heavy investment in the security group. We put out 3 major releases and 3 minor releases of our firmware for the BreakingPoint Elite. And our application protocol list now tops more than 100 and our strikes are over 4,300.
This news is certainly exciting, but that was last year. And this is a completely new year and we are ramping up in engineering like you could not imagine. The next firmware release will once again improve the performance of everything from our application protocols, security engine and our SSL. And, of course this is all done without having to replace your blades and at no extra cost. Bet your other vendors don't say that every year.
Next month I'll be putting together a screencast showing you all the features in our next release. I'll save all the juicy bits for then, but here is a teaser of what to expect:
Last year we changed the way people test their network equipment, this year we will set the standard.
Reminds me of when I worked at Cisco many years ago and Kevin Kennedy (Vice President) would show a slide in which Cisco was compared to other similar companies. There must have been 30 companies listed and at the time 3Com was below us, Lucent ahead of us and all the way at the top were companies like HP. At the time HP was 10x the size of Cisco. Today, Cisco is tens of billions of dollars ahead of HP, with a third of the employees.
Every year that presentation showed Cisco passing yet another company. We have the same chart for our industry and the same goals, and some companies were ahead of us at the beginning of 2009. During 2009 we passed four of them and this year we will pass four more. And one day, like Cisco, we'll be at the top of everyone else's list.
NOTE: Sometimes Cisco didn't pass a company, the company fell. I'm seeing a lot of that lately, maybe I should send some flowers.
As the year runs down I thought I would look back at some of our blog posts that made the biggest impact in 2009. Over the next few days I'm going to reprint portions of posts here at BreakingPoint Labs, just in case you might have missed them during the year. The list was created by examining the most read, the most shared and the most commented on posts throughout 2009.
First up was a post written in February around the testing of the Juniper SRX:
The impact of testing using real blended application traffic and live security attacks was never more apparent than at last week’s event at Juniper Networks. The company brought in Dennis Cox and other BreakingPoint experts to deliver a full day of presentations and live demos to underscore the importance of this approach to 70+ Juniper engineers and technical marketing experts.
This morning, Ron Gula tweeted a link regarding the possible discontinuation of Cisco Security Agent (CSA). Gula, the CEO/CTO of Tenable Security, pondered whether this was the first of many Cisco security products to be discontinued. While I think he may be right in that regard, I was hoping CSA remained alive. Patrick Ogenstad wrote the actual blog post in Network Lore to which Gula referred in his tweet. It's a wonderful article and I agree with most of what Ogenstad writes, with the exception of a sentence in the last paragraph:
"Perhaps Cisco is the wrong vendor to have this specific product in its portfolio, and perhaps someone else will buy it."
While I do hope someone picks up the product, I actually think Cisco is the best company to own CSA. This was its trojan horse into the desktop to disseminate a whole host of other products and services. Perhaps even TelePresence?
Cisco has the manpower, a technical sales force and a strong technical support organization. Those are key factors, in my opinion, to make CSA successful. CSA reminds me a lot of Network Flight Recorder (NFR), acquired by Check Point in 2006. The products are (were?) both extremely powerful. You could do most anything you wanted and neither product required constant upgrading. The general feedback on both, however, was that they were complicated and required knowledgeable people to set them up and get the most out of them. I really dislike that as an argument for their demise: "I'm too lazy to read the manual and do a few Bing searches, Mr. Vendor. Just make it all auto-magically happen for me.".
Sorry, buddy, but networking doesn't work that way and network security definitely doesn't work that way. It's a detail-oriented profession and if you are not detailed enough to understand the difference between UDP and TCP, get out of networking. You are not doing anybody any favors by judging everything on presets and defaults. You sir, are the type of person being mocked in beer commercials.
We see this all the time in testing. Vendor A has built a new content-aware firewall, and its QA team tests the product using a bit blaster to see how many UDP packets can go through at any given packet size and any second. When does that happen on a real network? Never. The QA team is doing what they did in the past and is now simply being lazy. They are not helping the product succeed in the real world. Here is a suggestion to anyone with a content-aware firewall, test with some actual content, and you'll be surprised by the results.
As I noted in my last blog post, network administrators are once again failing to secure their networks properly, whether it's failing to update their routers and switches with the latest Cisco patches or not deploying solid security solutions such as CSA. It leads me to a couple of important questions for the peanut gallery: Why is CSA leaving the market (or is it)? And what could Cisco have done to save it?
Oh, one last thing, what are you doing to save your product? God knows I hope you're testing.
Last week Cisco released patches in their semi-annual security announcement. The publication includes 11 advisories that address 12 individual vulnerabilities. Ten of the advisories address vulnerabilities in Cisco IOS and one advisory addresses a vulnerability in Cisco Unified Communications Manager. Together these can affect routers and switches that not only use the Cisco Unified Communications Manager, but any device relying on the Cisco IOS operating system. To put it bluntly, this means a ton of devices critical to any network, and these vulnerabilities leave businesses and government agencies exposed to a barrage of attacks including denial-of-service (DDoS) or policy bypass.
Much has been written about the announcement of the vulnerabilities. However, details are lacking and there are more questions than answers. This lack of information leads me to believe Cisco does not take security seriously and continues to not know how to work with the security community. Considering the lack of details and opinions, I thought I would provide a few of my own.
The number of vulnerabilities patched by Cisco is not the issue. It is the potential danger these vulnerabilities pose. One of the IOS vulnerabilities allows unauthenticated attackers to bypass access control policies when the “Object Groups for Access Control Lists (ACLs)” feature is used. Your company is most likely protecting your critical components by leveraging ACLs, now imagine they are no longer in place. The human resources database with all that W-2 information? Hackers now have your salary, your direct deposit account, your medical history and of course your social security number. To make matters worse, replace that HR database with our government’s nuclear secrets; don’t you think Iran is aware of the Cisco vulnerabilities?
Scary stuff, for sure, but how long has the vulnerability been around and recognized. The answer is unknown. The only fact we have is that each of these eleven vulnerabilities may have been around for at least six months. That is an eternity in the security space and has given hackers too much time to walk in through an open door.
Microsoft is often a punching bag when it comes to vulnerabilities and it is sometimes warranted, but let’s be honest, the company does a good job of patching issues on a regular basis. With Microsoft, you know that you are going to get a patch each month and important details that help you make an informed security decision. Cisco should examine its patching schedule in light of the September 24th announcement; every six months is not acceptable.
You can never diminish the importance of a switch or router to your network infrastructure. They are the core to any network whether in a home, a large Enterprise or the Federal Government. If one fails you know it. However, if a vulnerability let’s people through due to a hack do you know it? While everyone remembers to patch their Mac or Windows laptop, how often do they patch the router, firewall or switch?
To see how up-to-date folks are with their Cisco firmware I ran a quick test. During a 1-hour scan of the Internet I found 420 responding systems and NONE were patched with any fixes from this cycle or the last. That means 420 systems, at a minimum, are susceptible to a years worth of vulnerabilities.
Microsoft had enough of people not patching and now it force feeds the patches. While I’m not a fan of that solution, it does work. Cisco needs to apply the same method to its products. It is irresponsible for Cisco to run its business in a way that could cause mass disruption to critical network infrastructures including government and military services.
Cisco is not the only one to blame in this mess, the people responsible for getting their routers, switches and other network equipment up-to-date also must be held accountable. How many of you updated with the patches on September 24th, the day of the announcement? The quick scan I did is telling me not many. Kelly Jackson Higgins of Dark Reading put it best, “The dirty little secret about patching routers is that many enterprises don't bother for fear of the fallout any changes to their Cisco router software could have on the rest of the infrastructure.”
In this case we have a great example of why every network device needs to be realistically tested under a variety of scenarios, both security and performance driven. Obviously, testing must occur at the NEMs level throughout the product lifecycle, but the enterprise must also test this equipment before it is deployed and after updates like these are made. Having the ability to quickly test equipment and the network after making updates is critical.
There is no room for excuses anymore. We have been able to become more adept at updating and testing equipment and software that are given more regular patches. Just look at how Microsoft Tuesday has become a habit. Other vendors have realized that this approach, ultimately, is better for everyone. I would encourage manufacturers of any network equipment to do the same.
The reason this is important is because the United States is currently fighting in two wars, heavily dependent on network technologies. The Department of Defense and other military agencies have concluded that the next major war will be waged, in great part, in cyberspace. If Cisco and other vendors guilty of the same security concerns do not get their act together it will be a war we cannot win.
Until March 24, 2010, when the next Cisco bulletin is due.
Tags: layer 2-7 // ddos and botnet simulation // custom applications and attacks // performance testing // application servers // server load testing // unified threat management // security updates // cyber warfare // tutorial // deep packet inspection // ids ips // vpn gateways // test methodology // network traffic generation // unified computing // 10-40-100 gige // iptv // wireless // virus and spam filters // load balancers // application protocol fuzzing // resiliency testing // proxies // voip // anti-malware // routers and switches // network management tools // blog post // wan optimization // ipv4-ipv6 // firewalls // data center planning and consolidation // cloud computing and virtualization //