Firewall Testing Methodology & Webinar

Wanted to give folks a quick preview of our upcoming high-performance firewall testing methodology which will be available on December 10th. That day we have put together a panel of network security experts for an interactive webinar on the new rules for testing firewalls. Panelists will include:

• William Graham, Check Point Architectural Engineering Manager
• Dennis Cox, BreakingPoint co-founder and CTO
• HD Moore, BreakingPoint Labs Director

The moderator for the panel will be Mike Hamilton, our Director of Product Marketing.  Below you can watch Mike introduce this latest testing methodology and be sure and sign up now for the firewall testing webinar!

 

StrikePack Email Alerts Keep You Informed and Help Me Wrap Up a Theme

A bit of a public service announcement today for BreakingPoint users.  We started to offer StrikePack alert emails that will notify you when we release a new StrikePack.  You can opt-in for these emails by heading to the "Member's Access" page and hitting on "Preferences" and of course you can opt-out at any time.  You will typically see about one email a week.

Never one to let an opportunity go, this simple announcement becomes a convenient way for me to wrap up my theme this week and I wish I could say I did it all on purpose. As you know, BreakingPoint is consistently keeping users up to date with the latest application protocols and security vulnerabilities to ensure realistic testing. In just the past month BreakingPoint has released four StrikePack’s that include:

• 57 new Security Strikes, including coverage for October 2008 Microsoft Security Bulletins.
• Strikes for exercising and fuzzing various packets in the Routing Information Protocol most commonly used by 'routed for UNIX'.
• Addition of the Gnutella Peer-to-Peer protocol, used most commonly by the iMesh, Morpheus and Limewire clients.
• Updates to the Stack Scrambler component to switch rates once per second when the Rate Type is set to a non-static value.
• Addition of the Apple Safari and Google Chrome client profiles to the HTTP application protocol.
• ...and more.
  

This is exactly what I'm talking about when I wrote "Eliminating Complexity in Testing Tools" and "Are the Apps you Need for Testing at your Fingertips?" this week. You need the ability to evolve your testing as your network requirements and traffic evolve...and you should be able to do it easily and cost-effectively. Starting today you can get alerted to these updates even faster.

 

Are the Apps you Need for Testing at your Fingertips?

The numbers are impressive when you look at peer-to-peer (P2P) network traffic. Recent reports estimate that more than 100 million files are exchanged each day using P2P networks and global P2P network traffic totals 61% of all upstream network traffic and 22% of downstream bandwidth consumption. Large numbers, but nothing compared to the growth estimates with P2P expected to surpass 100 exabytes per month by 2015.

I wrote the above last week when we announced support for testing network devices with encrypted BitTorrent. BreakingPoint obviously already had coverage for eDonkey, BitTorrent, Gnutella but writing this release and realizing these numbers really hit home for me the importance of using actual application traffic when you are testing network equipment.  Additionally I was doing research on our next supported application protocol and those numbers, in a relative sense, were even more staggering (hint: instant messaging protocol in a particular geography).

Encrypted BitTorrent adds an important element into the mix, since it has become a possible liability for service providers with many folks trying to avoid QoS policies, not to mention possible issues with file obfuscation, law enforcement avoidance and exploit prevention. Overall the immensity of P2P traffic has a severe affect on network devices, both from a performance and security standpoint, so as much testing you can do with this traffic the better.

However, beyond the particular application, this hits home on my gripe from Monday around the often artificial barriers put up that add complexity into the network equipment testing industry. When you are testing network devices you need to be using the traffic that will ultimately go through this device. Dennis wrote about this in "The World is Lacking Background Traffic". The question you have to ask yourself: do you have those application protocols at your fingertips during testing and if not are they easy to get?

Eliminating Complexity in Testing Tools

A statement I heard on Friday: "Testing is hard, testing equipment should be easy."

My response: "Testing is hard, some tools make it harder and yes, it should be easy."

We've talked here about how to make testing easier, through better network equipment testing tools, but typically we focus on product features, script automation or an intuitive user interface. It goes beyond features however. Often companies introduce new testing products that may have been better served, for users, to be an added feature to the product they have already purchased. I realize that legacy testing vendors are cornered because of their original architecture and I also acknowledge the need for specialized testing products for particular needs, but all too often you are adding testing tools to your process, not to mention additonal costs, simply to get a feature you need.

This is not unique to network equipment testing, in fact it is prevalent in the enterprise software space where adding product feature layers and subsequent liscense fees has become an art form.  It is not wonder that the SaaS movement gained so much traction and that people rush over to open source. Many times, particularly at the onset, a vendor can actually use this complexity to generate a great deal of revenue and FUD. The product complexity model works when there are limited players in the market to challenge the norm and before users have grown tired of trying to piece all of these different products together.

Making your testing life easier is not simply about product features, I get that, instead it should stem from the way the product is designed, sold, updated, packaged and how you provide training and support. This ability to minimize complexity and maximize functionality, particularly in this economic environment, will serve companies well in their success.  Kevin Costner may have heard "Build it and they will come", had he been a network test engineer his internal dialogue may have muttered "Build it, make it all-in-one and eliminate product redundancy and licensing complexity and they will come".

Network Equipment Testing in APAC

I recently sat down with Leonard Zhang out of our Beijing office, to chat about the network equipment testing market in the Asia Pacific (APAC) region.  Wanted to share some of our conversation and see if it resonates with some of the experiences many of you have had in global organizations.

What current trends are you seeing in APAC?

In the past months, we have been interacting with many potential customers throughout the region and observed an increasing trend of interest around application and security testing. It is something good to see. The APAC technical wave mostly follows the U.S. trend, maybe 1-2 years delay. But we see the gap closing, for sure. In my personal opinion, the big picture APAC needs to overcome some cultural issues first in order to take the lead at least in some area.

How is your market different in terms of mindset?

A typical Asian puzzle is the conservative mindset. For example, it happens that when we see some potential customer, the first response might be "we never heard of such a testing tool, so no interest" or "a new tool, we don't believe well in new tools...". Such an enclosed mindset, I guess, is not rare to see in some Asian markets. People usually doubt the new innovations but forgot that every great company/product came with a new hat.

With that in mind, how will BreakingPoint make an impact?

Application and security testing is very complicated, and as I stated above it is seeing very high levels of interest, so I am glad that I joined BreakingPoint who understands the complexity of this market. Application and security testing is constantly changing, so it is always new.

What is your goal in bringing innovative network equipment testing to APAC?

I hope we can speed up the process of the "appreciating new stuff" attitude in APAC through our efforts. Only open minds can lead to a position of pioneers, which Asia is really lacking today. Japan would be a good role model to its neighbors in this area.

Fighting JavaScript with JavaScript

In response to my previous blog post regarding a new clickjacking technique using link object href target replacement hooked to some of the link object's mouse events, I got a lot of questions asking how this target URL swap technique could be detected short of analyzing all the JavaScript on the page looking for this type of malicious code.  Automated detection through code analysis is a fairly difficult problem for a number of reasons, so I began thinking about this problem from the approach of detecting the result of the malicious code rather than the code itself.  Using that approach I wondered if you could fire potentially unknown event handlers, whatever they may be, without actually causing the event to happen.  Turns out you can:

<html>
   <head>
      <script id="SwapDetetctor" type="text/javascript">
         <!--
         function swapdetect() {
            var tp = window.frames[0];
            var pagelinks = tp.document.links;

            // Preserve the current links array
            var oldlinks = new Array();
            for( i=0; i<pagelinks.length; i++ ) {
            oldlinks[i] = pagelinks[i].href;
}

            // Fire all the links' mousedown and click events
            links = tp.document.getElementsByTagName('a');

            for( i=0; i<links.length; i++ ) {
               clickevent = tp.document.createEvent("MouseEvents");
               clickevent.initEvent("mousedown", true, true);
               links[i].dispatchEvent(clickevent);

               clickevent = tp.document.createEvent("MouseEvents");
               clickevent.initEvent("click", true, true);
               links[i].dispatchEvent(clickevent);
            }

            // Compare old values versus new values
            for( i=0; i<pagelinks.length; i++ ) {
               if( oldlinks[i] != pagelinks[i].href ) {
                    alert("Link target changed due to mouse event:\r\n" +
                    oldlinks[i] + "\r\nis now\r\n" +
                    pagelinks[i].href);
                 }
                }
         }

         setTimeout("swapdetect()", 10000);

         //-->
      </script>
   </head>

   <frameset rows="*" frameborder="no" border="0" framespacing="0">
      <frame name="targetpage" src="targetpage.html" height="100%" width="100%"/>
   </frameset>
</html>


What this bit of HTML and JavaScript does is load a target page into a frame, waits 10 seconds, and then calls the function swapdetect().  I used frames to approach interacting with the page as if it were in a pseudo-sandbox, however you could easily modify the above JavaScript to operate within the current window rather than target a frame or other window, just be sure to call swapdetect() after the page has finished loading and any other handlers for the page's 'onload' events have been called.

The function swapdetect() first creates a backup copy of the page's links array's href targets for later comparison.  It then creates event objects for the 'mousedown' and 'click' events, and uses those artificial events to dispatch the event handlers using the dispatchEvent() function (use fireEvent() on IE) for them (if any) on each link object found in the page's links array.  Due to this approach, only the event handlers are called; the actual event doesn't take effect.  This will cause the link swapping code in my previous blog post to do it's thing, which we can then detect.  The links array is then compared against the backup copy with the original targets to determine if any of the links' targets have changed, and create alert dialogs if so.  If you were so inclined, the same technique used in the link swapping code could be used in place of the alerts to restore the original link's href target value.

BreakingPoint LiveLook: QA for QA

We're back with another BreakingPoint LiveLook on this Friday morning. In today's video, Dennis talks to Chuck Summers about writing QA scripts for our network equipment testing products. The two of them also chat about work commutes and Friday beer, not to mention Chuck's plea at the end.

Clickjacking Technique Using the 'onmousedown' Event

A few weeks ago, Tod Beardsley wrote about (not) 'clickjacking' here on the BreakingPoint Systems blog.  He covered a number of techniques to accomplish generating a 'popup' window without triggering any of the traditional popup protections that some browsers feature.  The idea was essentially to cause the user to 'request' the popup, thus making it legitimate in the eyes of the browser.  Later, he covered his speculation on the 'real clickjacking' attack, which didn't use JavaScript at all but rather did some interesting CSS overlay trickery to hijack a link out from under the user as they clicked on it.

During some research that I was recently performing that I'll likely post about a little later, I discovered another technique that's a bit of a middle-ground between the two methods that Tod was discussing in his blog posts.  He came close to this one with his hooking of the 'onmouseup' event, however he was having it spawn a completely new window (the popup) in addition to following the link rather than 'jacking the click' and sending it somewhere entirely different.  This is essentially the same type of event hooking technique, but it is used to accomplish actual replacement of the link's target URL.

The following JavaScript function accepts as arguments a link object such as you would find in the document object's links array and a URL that you want to override the original link's URL with:

function AddJacker(link, url) {
if ( link.addEventListener ) {
      link.addEventListener("mousedown", function(e){link.href=url;}, false);
  } else if ( link.attachEvent ) {
     link.attachEvent("onmousedown", function(e){link.href=url;});
} else {
var oldhandler = link["onmousedown"];
     if ( oldhandler ) {
        link["onmousedown"] = function(e){oldhandler(e);link.href=url;};
     } else {
        link["onmousedown"] = function(e){link.href=url;};
     }
  }
}

What this essentially does is create an event handler for the 'onmousedown' event for the target link. When the user clicks on the link, the 'onmousedown', 'onclick', and 'onmouseup' events are fired.  Since the 'onmousedown' event happens first, the event handler is called which replaces the link object's href value with the new target URL, which happens before the user is sent on their way to that link's target URL.

The interesting bit about this technique in comparison to the 'onmouseup' technique that Tod was using is that it doesn't result in the user both going to the original target as well as the new target; they are only redirected to the new target, completely overriding the original target.  Like Tod's technique, because the new target URL is hiding in a function that is handling the 'onmousedown' event, a mouseover of the link in the browser indicates that it is still targeting the link's original URL.  The replacement of the URL doesn't happen until the user actually clicks on the link.

Clickjacking Aftermath

To get new readers up to speed: In the middle of September, researchers Rsnake and Jeremiah Grossman were planning on a talk at OWASP regarding a purportedly new web attack dubbed "Clickjacking." At the request of Adobe, they withheld proof of concept code and salient details until a patch was available to help mitigate Clickjacking's effects. Within two weeks or so, Adobe did, in fact, release APSA08-08 for Flash. With that coast clear, Rsnake published his and Grossman's paper here, which lays out the gory details. No problem, since Adobe's patched now, right?

Hmm, not so much. Clickjacking -- the technique of incorporating obscured, invisible, or otherwise counter-intuitive links from external sources via iframes in order to trick users into clicking them -- affects much, much more than merely resetting Flash permissions. It's essentially a means by which malicious web site operators and web content authors can subvert the Same Origin Policy of web browser to get victims to perform actions on the attacker's behalf.

Firefox users can take some solace in Giorgio Maone's addition of ClearClick to his most excellent NoScript extension, however everyone else (which should be taken as, "pretty much everyone") needs to rely on their favorite sites taking their own countermeasures via frame busting, secondary authentication, or other mechanisms to avoid being driven via malicious iframes.

With this in mind, I decided to take a quick peek at Alexa's top ten web sites to see how they handled being an iframe source. The (completely harmless) test page is below if you'd like to see my tests and notes for yourself.

http://www.planb-security.net/notclickjacking/iframe_madness.html

The short story: Of the most trafficked web sites in the world, sensitive personal information editing functions can be trivially incorporated in an attacker's iframe. These results didn't come as a complete surprise, of course. However, I was kind of hoping these hugely popular site operators might have implemented something in the month and a half since Clickjacking made the news. I guess we'll just have to wait for the (inevitable) series of wildly successful, high-profile clickjacking-enabled attacks before the Googles, Yahoos, and Facebooks of the world take on the threat on their own properties.

10G = $9.5B Says Research Firm

According to a new report from Infonetics Research, the market for 10 Gigabit per second (10Gbps) networking equipment will grow to $9.5B in 2008...this is up from $7.3B just last year.  That is quite the revenue jump, particularly in today's economy. Sean Michael Kerner at InternetNews.com has the full story which expands into the quest for 100 Gig Ethernet:

The researcher (Infonetics) is forecasting growth for 40G to have a compound annual growth rate of 59 percent from 2007 to 2011. Michael Howard, Infonetics' principal analyst and co-founder, said he expects service providers will take up 40G in the near term even though the faster 100G speed is also on the horizon.

100G is expected by Infonetics to start making inroads in 2009, though broader adoption and revenues are not expected until 2013, according to the study. Howard added that 100G is important as it will likely be in use until at least 2025.

In a conversation about these forecasts I thought our CTO Dennis Cox had an interesting comment:

"I think these predictions are a bit early; 10G in the back room, definitely, but you really need some 10G to the desktop (a minor portion at least) to make that much damage. Obviously, some applications are REALLY pushing bandwidth right now and bonding is getting much more common than even I thought it would become, so perhaps that is driving demand, which always skews analysts (in a good way). The other thing to think about; 10G chipsets are really expensive right now -1G is down to 40 bucks a port (which is really cheap) since 5 years ago it was 200 bucks a port."

Dennis is right, we are seeing 10G network equipment in the back room, and his look at the need up front and the chipset costs gel with many questions I have about these numbers. Since I do not have access to the report I'm not sure if Infonetics takes on the challenge that the delayed 40G/100G standard may pose, or the potential of an even steeper economic downturn. Would love to hear from Infonetics on this topic.

OK, here's your chance, tell me what you think of these predictions around 40G/100G, leave comments below or get involved with the conversation on Twitter.

Penetration Testing IPv6 & Infrastructure 2.0

Perusing the web today and found two interesting articles, both with a forward looking edge; some nice reading for you all on this Monday afternoon. The first comes from BreakingPoint's own HD Moore and it dives into Exploiting Tomorrow's Internet Today: Penetration Testing with IPv6. The paper takes a look at how IPv6-enabled systems "...with link-local and auto-configured addresses can be compromised using existing security tools". My first thought was to question the overall importance, since IPv6 has not been emphatically embraced, at least not compared to the promise. However, HD correctly points:

"Even though most networks are not “IPv6” ready, many of the machines on those networks are. The introduction of a new protocol stack introduces security challenges that are not well-known and often overlooked during security evaluations."

Check out the paper and let us know your thoughts; the end lesson for me: "Be Prepared".

The next article that caught my eye today was "The Next Tech Boom: Infrastucture 2.0" by Gregory Ness. Although I have a pet peeve concerning the moniker "2.0", Mr. Ness hits on some important points around static infrastructure, dynamic infrastructure and the players that sit within these camps. The piece dives into, among other topics, how network management costs will continue to rise and that the static players, particularly within a recession, must add more intelligence and automation to their devices and software. The author put it aptly in a reply to a comment I posted:

"They'll benefit from the spread of TCP/IP but not to the extent they could if they became more strategic to the emerging initiatives."

It would be interesting to add the testing factor into this equation for these industry players and how, with the obvious need to rapidly introduce content-aware network equipment, proper testing is not the only prerequisite.  The testing itself must be automated, intelligent, quick, easy and ultimately cost-effective. A lot of criteria for network equipment testing but interestingly all things mentioned in the video from last Friday.

BreakingPoint LiveLook: The Future of Network Equipment Testing

No, no, I didn't sit down with our latest network equipment testing product for an interview, I did a little better than that, chatting with Dennis Cox, the CTO and co-founder of BreakingPoint.  Dennis reviews the critical aspects of network equipment testing, why testing can be something to enjoy and bit of the company's history.  The interview is a good one for anyone involved in testing network equipment performance and security.

Protocols, Protocols, Protocols: Recap of Broadband World Forum

Jumping on a flight to attend CheckPoint's "Network Security and Performance Testing for Next Generation Networks" lunch and learn in Dallas (come join us if you are in the area). Thought it was a perfect time to give a recap of my recent trip to Broadband World Forum in Brussels.

BreakingPoint received a warm reception despite the cold and rainy weather last week at the show. We previewed the BreakingPoint Elite chassis to an audience of equipment vendors and service providers.  Three common points were brought up from folks on what they were looking for in performance and security testing:

1. High Performance: sessions, sessions, sessions
2. Fast: 20 Gigabits per second and faster
3. Realistic: Live security attacks, actual application traffic…oh, and the ability to easily blend them

The idea of realism in testing did, however, dominate my conversations. One of the visitors to our booth sought us out because he wanted to test using more than 300 protocols to reproduce realistic network conditions. The reason? He wanted to thoroughly test the DPI functionality of his network equipment. As the VP of Marketing, he wasn’t comfortable shipping a product that wasn’t fully tested using realistic traffic. He had been told that BreakingPoint had the most application protocols, the ability to integrate proprietary protocols, and do it all at 20 Gigabits per second and faster.  Testing true performance and security for today’s next-gen devices means you must be testing with actual application traffic.

Some other hot topics while talking with people at our booth and venturing out onto the show floor: DPI and ways to test DPI functionality, SIP testing, green initiatives, and—given current economic conditions—ways to save money and reduce testing TCO.

Overall the show was a great place to introduce the new product and to hear how folks are getting excited about new possibilities in testing equipment. And that certainly made the trip worthwhile.

BreakingPoint Elite Launches with 15M TCP Sessions, 20 Gbps of Layer 4-7 Traffic and more

This morning we launched our latest product, BreakingPoint Elite.  Wanted to share with you all some information around the launch and let you know that I'm going to try and get up some video from Dennis later today about the new product.  In the meantime here are some great links for you to check out BreakingPoint Elite:

News Release

BreakingPoint Elite Overview

BreakingPoint Elite Datasheet

This is all exciting stuff for us since BreakingPoint Elite is the first testing solution to generate 15M TCP sessions and 20 Gigabits per second of Layer 4-7 traffic in a 4 rack unit chassis.  Stay tuned for some video, check out the links above and here is a quick feature overview of the new BreakingPoint Elite:

• 15 million concurrent TCP sessions
• 1.5 million TCP sessions per second
• 20 Gigabits per second of Layer 4-7 traffic in a 4 rack unit chassis
• 80 Gigabits per second of Layer 2-3 traffic in a 4 rack unit chassis
• Scales easily with multiple BreakingPoint Elite's to generate 200 Gigabits per second of Layer 4-7 traffic and 800 Gigabits of Layer 2-3 traffic while retaining a single interface, test configuration and reporting
• An intuitive management interface, multi-user capabilities, integrated Layer 2-7 reporting and extensive automation 
• More than 60 application protocols that can be blended with more than 3,600 security attacks and the ability to integrate proprietary application traffic
• Industry-leading vulnerability coverage including complete Microsoft^® Tuesday coverage

 

Five for Friday: Matt Sarrel & Network Equipment Testing on Wall Street

This week for "Five for Friday" I talked with Matt Sarrel, the Founder and Executive Director of Sarrel Group.  Sarrel Group is a New York City based product testing, editorial services and consulting firm.  Matt and his staff have worked with an exclusive clientele of small hedge fund and financial management companies for more than 10 years building networks, implementing algorithm based trading systems, and building sophisticated data analysis clusters. As we discussed a month ago Matt's group is using BreakingPoint as part of their testing environment. It seemed to make sense, with all the economic news,  that we start off tapping Matt's brain about testing on Wall Street. Enjoy!

1) How sophisticated is network equipment testing for firms on Wall Street?

I can’t speak for the larger firms because most of the work I’ve done on Wall Street has been with smaller hedge funds.  I’m sure the larger firms must test network equipment before implementing it, but to tell you the truth, smaller firms take the network for granted.  They are very focused on their software and the algorithms that they run and will test those almost infinitely.  It surprises me that these firms assume that the network functions properly (the same way most people assume that the there will be a dial tone when they pick up the phone) because minimizing latency is crucial for many of these projects.  In my opinion, network equipment and network links should be tested more often.  I’ve seen companies spend hundreds of thousands of dollars optimizing code and then turn around and run it over a tediously slow WAN link.  Taking the network for granted increases the number of unknowns (not a good thing in finance) and puts many projects in potential jeopardy.

2) How will the latest economic alter testing for these types of companies?

There are two ways to look at this. The first is that companies will simply cut their test budgets.  The trouble is that most business people don’t understand that systems should be tested before implementation.  That’s just some issue that techies need to deal with and times are tough so let’s cut the budget.  What those people don’t realize is that the cost of not testing far exceeds the money saved by not testing.  Would you want to invest in a fund that is going to use your money to test their system when it goes live?  I didn’t think so. 

On the other hand, and this is what I hope happens, is that Wall Street realizes that they no longer have the luxury of testing on a live system.  In these troubled times, controlling and limiting risk is essential – both in your systems and in your investments.  It would make more sense to me for companies to increase the amount of testing in order to limit the risk of systems failing when they go live.  There is enough uncertainty in the market; you don’t need that uncertainty to extend into your systems.  Spending the money to test and verify that systems work as needed will reduce risk and allow companies on Wall Street to focus on their financial management, investments, trades, and algorithms rather than being sidetracked by something as simple as an overloaded network.

3) No matter the organization, when you are running tests on devices what do you look to do from the start? What is the most important first few steps? 

The first few steps occur before the actual testing starts.  The most important aspect of any test is understanding how the device will be used in that particular environment and how the device itself works.  Without that solid understanding, you’re testing without context, and testing without context is meaningless.  It’s like if your son comes home from school and says, “I got a 25 on my test.”  What does that mean?  A 25 out of 25, a 25 out of 100?  What was the average score?  What did he get on the last test?  Is this in an easy subject or a hard one?  Understanding how a device will fit into a particular environment and then testing it congruent to that implementation is what we at Sarrel Group refer to as “real-world testing”.  Testing in a void won’t teach you anything about how a device will perform in the real world.  Sometimes it takes more time and resources to create an appropriate test bed than it does to run the tests.  This is one thing in particular which BreakingPoint greatly simplifies because we can go to a client site and capture real network traffic, then return to the lab and replay it to test network equipment.

4) What are you looking for in your test results? Does this differ from what the vendors are looking to get out of the testing? 

As I mentioned in my previous answer, Sarrel Group focuses on real-world testing.  My technicians and I have years of consulting experience and have built hundreds if not thousands of networks.  We know the issues that customers face when they implement a new technology, and Wall Street firms frequently implement new technology in order to gain a competitive advantage.  Most test labs, and I’m being generous here because as far as I can tell it is almost all other test labs, don’t employ technicians with real-world experience.  The result is testing under unrealistic conditions which yield results that are inappropriate for most implementations.  Interestingly enough, this is almost exactly what most vendors want because almost every device will perform better in a sterile lab environment than in a complex real-world environment.  Vendors want test results that max out their equipment and show how fast it is and many times test labs will resort to unrealistic test scenarios to meet these goals. 

We see little value in that at Sarrel Group, and I would like to think that the industry has gotten smart enough over the years that corporate buyers see little value in that kind of testing also.  For example, I tested an IPSec firewall earlier this year.  The company said it could process traffic at wire speed.  The sales and marketing materials all said “wire-speed”.  When we got it into the lab it was a wholly different story.  It could process non-encrypted traffic at wire speed.  Turn on IPSec and performance was cut by a third.  Then, we advanced from simple Layer 2 bit blasting to run advanced application traffic through the device and performance was cut by another third.  So all of a sudden, the 100 Mbps VPN is now a 30 Mbps VPN.  You can only get meaningful results by testing under real world conditions.

5) How has testing evolved since you started in the industry? 

In the old days, meaning 1990, in order to test we needed to build complex networks with hundreds and sometimes thousands of PCs to generate load.  This meant that we need to write all kinds of scripts to coordinate and manage the PC’s and need a few technicians around whose sole purpose was to maintain the testbed.  Then automated test tools started to appear and it became possible to simulate an enterprise network environment more easily.  But then network equipment and applications got more sophisticated and testing took a lot more knowledge and resources to do properly.  I think that this is where most test labs stopped advancing and stayed focused on lower layer simple testing which many of us call bit blasting. 

Other test labs, like Sarrel Group, developed methodology to test more complex layer 4-7 devices, but using the old equipment it wasn’t all that easy.  One of the reasons why we’ve partnered with BreakingPoint is that these test devices were designed to conduct more complex testing at higher layers and this is the direction the industry is moving in.  The days of simple bit blasting are over and the test labs who realize this earlier will be able to test bleeding edge products more quickly and easily than those that don’t.

Denial of Service Attacks

Recently there was a Dark Reading article on a new type of TCP DOS attack, of course the authors have learned their PR lessons and it's all hush-hush until the next big conference or product launch. Of course that leaves us all guessing as to what the attack could be. So I thought I would cover some basics about Denial of Service Attacks for people interested.

Normal TCP Connection
Client             Server
SYN       ->
              <-       SYN-ACK
ACK       ->      
DATA      ->       
               <-       ACK
FIN         ->
               <-       FIN-ACK


SYN Flood
SYN Floods are one of the most basic Denial of Service attacks. The basic way they work is the attacker sends a large number of TCP requests to the victim (a server or network device). This ties up all his connection entries and he can no longer process incoming requests. To jump down to a even more basic level -

To make a connection to a server a number of things have to happen, one of them is generally a TCP connection. The connection involves a three way hand shake. The first is called the SYN, which the client sends, then the server says SYN-ACK - meaning I read you loud and clear. The next stage (third step in the three way handshake), is the client saying okay - we have a connection or ACK. Now we are in established state. So if my devices supports 5,000 connections then and you sent 5,000 SYNs I can't receive any more connections. So I have denied service. Make sense?

SYN Flood
Client             Server
SYN     ->
            <-       SYN-ACK

SYN     ->
            <-       SYN-ACK

repeat

This technique has had a solution since 1996 (I know, I have the first patent on it). There are so many ways to solve this problem. One is to make a list of connecting and established connections. When the connecting list is full you throw away the oldest of those in the connecting state or favor something existing in the established list. Hence, clients that you have already spoken to are more trusted then those talking to you the first time. Again, dozens of solutions to the SYN Flood problem.

Connection Attack
A connection attack is basically the same thing as Denial of Service, except in this case I complete the three way hand shake. Now, I've gone onto your established list and eaten up your entries. You can also do this (but it's painful) by focusing on a single IP address and eating up every possible port number (16 bit value), but that requires a lot of connections (~65k) and generally speaking big sites use load balancers and dozens of IP addresses, so it's not useful.

Connection Attack
Client             Server
SYN     ->
             <-       SYN-ACK
ACK     ->      

Other forms of Denial of Service
You can play with TCP stacks to create other types of Denial of Service issues. Sending resets can interrupt connections or playing with the Window Size. I could make the victim think that I'm on a really slow link and it will leave the connection open forever. This works great on network devices in particular because they tend to shortcut the TCP fields to save on memory and speeding up their data searching (hashing, lists, trees, etc).

I did a recent test the other day on making the victim think I am on a slow link. I found devices with specific field size for the TCP header were susceptible to having this type of attack more often. For example, a Windows 2003 Server took 10 packets per connection, while a popular network firewall took 4 packets per connection. This type of attack is intensive in the number of packets (not large one's mind you, but still enough) and it fools even the best load balancers and firewalls. It definitely can bypass SYN-Cookies without a problem.

A good example on TCP Window Size is:
"In theory the TCP window size should be set to the product of the available bandwidth of the network and the round trip time of data going over the network. For example if a network had a bandwidth of 100 Mbits/s and the round trip time was 5 msec, the TCP window should be 100x10^6 times 5x10^-3 or 500x10^3 bits (65 kilobytes)" - so If I want to go slower?

This type of attack is really interesting because an article in Dark Reading recently talks about some folks that think they found the next DOS attack, no details have surfaced so who knows, but my guess it's this type of attack.

Application Denial of Service
Another Denial of Service is to run a CPU intensive program/script/query on the device you are after. For example, if I run a select query that is highly complex on a database I may be denying service to users that want to query as well. Run enough of these and you can starve people out. You could also send malformed data to the program, if the program crashes or runs really slow you deny service. A recent favorite we saw was VoIP related, some compression algorithms are more CPU intensive then others. By selecting a horribly slow one and creating a dozen calls, all voice calls start to suffer in quality - or worse can't happen at all. So you can see Denial of Service isn't just TCP related and there are dozens of ways to make it happen.

Network Device Denial of Service
Network equipment also suffers from Denial of Service problems. How many ARP entries can a device handle? How big is the connection table? Does it have a control path for routing protocols? If I send routing protocol packets, like RIP updates does it slow down packet processing? What if I send a flood of them? If I create a worse case packet for an IDS that leverages Regular Expressions, does it miss traffic? If so, I can sneak my attack by unnoticed.

So it seems that Denial of Service that will end the Internet is the new craze. To be honest, Denial of Service attacks aren't rocket science, to find one you just need to look, it really is that easy. It all comes back to testing; if you test the worst case scenarios in your product, you will come up with simple solutions to protect from these and minimize the possibility of a Denial of Service attack.

BreakingPoint LiveLook: Tcl Interface & Corvette interiors

In the latest BreakingPoint LiveLook, Dennis has an interesting conversation with Kris Raney.  When you can go from Tcl interfaces to rebuilding a Corvette you might have a shot at being a member of our illustrious media.

 

Toorcon X Mini Wrap-Up

On the whole, I was very happy to have attended the 10th Toorcon in San Diego, CA. Toorcon is probably my favorite small con. The attendance isn't massive but the people are generally more interested and knowledgeable in hacking and security. Not to mention that downtown San Diego is a blast and the weather is absolutely perfect. These were my highlights:

The Future of Lockpicking, datagram
I was glad to see a talk on lock picking that went beyond the realm of a simple how to or a single type attack.  Datagram didn't spend too long explaining the lockpicking techniques even though he did have some good animated visual aids. Instead, he focused a lot on how a lock vendor would react to new attacks getting media publicity. Just like in the software security world, some vendors wouldn't ever go beyond a PR response. Some vendors would add a metal plate in a certain place, much like a software patch, and others still redesigned the locks entirely. Some very interesting industry parallels.

Owning Telephone Entry Systems, Joshua Brashers
So many apartment complexes, condos, and gated communities have computerized panels that visitors can use to ask permission to gain entry. The talk outlined many different types of attacks against these types of systems. Most of these appear to be serviced by 3rd parties and allow you to remotely dial-in. And of course, default passwords are rarely changed. He showed how he was able to “back door the front gate” by adding a new entry that played a “Rick Roll” instead of calling a resident and later opened the gate. Another and more scary attack he outlined was the ability to proxy normal entry calls to his apartment using a VoIP server to perform the MiTM. This looked like it was a lot of fun.

How To Impress Girls With Browser Memory Protection Bypasses, Alexander Sotirov
This was a great talk. Although Dowd and Sotirov gave this talk at Blackhat almost two months ago. It was still a fun and entertaining talk to sit though. Alex outlined the implementations of the newer Microsoft memory protection schemes like SafeSEH, DEP, and ASLR. Then showed how and why none of them were effective in defending Internet Explorer from attacks and how much that impressed the ladies. The paper is here.

Visualizing the Twitter Social Network

Back in July, BreakingPoint added support for the Twitter API to our Application Simulator. The method I used in developing this AppSim protocol was to reference the API documentation, writeup a small client, and capture network packets. Looking at how API clients and the Twitter servers behave makes developing a realistic simulation of Twitter pretty simple to do. One question we commonly hear, especially concerning our Security and Application Simulator components, is, "How do you verify correctness?" I want to give one specific example of this by talking about a side project I worked on recently.

Once a month, I attend a local meeting of people interested in computer security. The format for the meetings allows anyone to do a talk typically between five and fifteen minutes in length. I always wait until a few days before the meetings to start thinking about prospective topics. Since I'd been working on some Trac tickets related to Twitter, I thought I might do a short topic related to mapping out the Twitter network. My original goal was to find all the cool security celebrities as a way to find all the cool links everyone is talking about.

Twitter is like most social networks, in that it tracks 'friend' relationships between users. I thought it would be interesting to visualize some of the friend/follower relationships on the site. I started to write yet-another-twitter-client. After about 10 minutes, I saw that I was reimplementing the same code I'd already done for AppSim. I decided I might as well leverage the code I'd written for AppSim. This is one way to address the question of application correctness. If I take the code in the BreakingPoint product that implements the Twitter API, and can really communicate with Twitter servers, then I would call our implementation correct. It's also a good way to make sure I don't have any bugs in Twitter, too.

I started implementing a directed graph in ruby. About 15 seconds later I started looking for a library for doing directed graphs. I found RGL, the Ruby Graph Library. There's a ton of usefulness in that library, and I know I'm not scratching the surface of how I could use it. I hadn't even started thinking about visualization yet, but I found that RGL supports Dot files, the input supported by GraphViz. I was further convinced that RGL was going to be great to work with. I had about a day before the meeting, and didn't want to write any layout code. I just wanted something done quickly (and with the least amount of effort on my part).

I wrapped the AppSim code that implements the Friends/Followers Twitter API calls, and started putting the list of Twitter users into a queue. I then dequeue the first user from the front queue, and get her friends and followers, and repeat.

This graph shows the social network from the perspective of the @BreakingPoint Twitter account. This graph adds new nodes in whatever order they are returned in the API calls.

Ok, so there are some users that show prominently, but it's all a jumble and hard to really extract anything useful.

My next method was to order insertion into the queue by the number of follwers each friend had. This technique had an interesting effect; the user I start graphing at (again, @BreakingPoint) remains in the center of the graph. You'll notice that most of the edges are directed out of the @BreakingPoint user. This is due to the fact that I process all the friends first, then all the followers. One attribute of this view is that people we follow that have a large following are prominent. It's a nice side-effect of how GraphViz lays out the graph.

The biggest problem doing this is that you end up with many of the same users populating graphs, even when starting from different initial users. When you order by the number of followers, eventually someone in your network is following @CNN or @the_Onion, and graphs from different runs start looking very similar in terms of what users are prominent. Also, once you hit a user with a follower count in the tens-of-thousands, progress in mapping the network slows as you retrieve followers in batches of 100 at a time, which is a requirement of the API. If anyone in the network you're graphing follows, say, @BarackObama, you should go to your local Alamo Drafthouse and catch a movie.

It became obvious that filtering is key if you want to get any interesting results from the data.

Here is a graph generated by limiting the users included in the graph to those with between one hundred and one thousand followers. I have also modified the graph to show our biggest followers.

An image like this gives you an idea of the most potentially influential followers you have. If you were a marketer like @KyleFlaherty, you could use this information to start trying to influence the biggest influencers that follow you. In the marketing world, word-of-mouth is a goldmine. This seems especially true in social networks where your friends are hand-selected.

This is just a first step into the topic of data visualization for me. I have a feeling I'm going to have to come up with something better than GraphViz for visualization. I've also had a request for making a web application out of this and make it available to a wider audience. I'd like to use the comments to gauge demand for this tool. If it looks like people would find this useful, I might just try and get it cleaned up and make a simple version available.

The number one thing people have said they'd want is interactivity. I think I'll go get working on that. I have tons of features I'd like to see, and I still don't have a talk for the upcoming meeting.

Update: the first thumbnail and linked image were corrected.

The Inevitable Failure of Content-Aware Network Devices

Wanted to point out the new IDC White Paper we posted to the site last week.  The report is titled, "IDC Vendor Spotlight: The Inevitable Failure of Content-Aware/DPI Network Devices - and How to Mitigate the Risk." For those of you in the industry, you may recognize the author, Elisabeth Rainge, who also put together IDC's "Worldwide Network Test and Measurement 2008-2012 Forecast."

The report highlights the benefits of testing network equipment using a broad range of applications and live security strikes at 10 Gigabits per second speeds with heavy network traffic. Some good data in the report for anyone looking at network equipment testing. Check out the report and let us know what you think.

Real Clickjacking?

After Michael Zalewski's WHATWG post spilled enough beans to show definitely that yesterday's pop-up evaders weren't "clickjacking," I put together another demo this afternoon (link below), which uses a combination of opacity and z-index settings on an iframe. Again, it's just speculation.

 

http://www.planb-security.net/notclickjacking/iframetrick.html

 

This seems to fit the bill: No Javascript required, uses iframes, and gives the ability to seemingly overlay one UI on top of another. By the way, the demo is mostly harmless -- it just turns your Myspace profile from private to public. I started down the path of masking my brokerage's trading app, but masking out keystrokes for stock orders seemed to be overkill for a simple speculative demo.

 

Not Clickjacking (Almost Certainly)

[UPDATE: See updated post]

A couple months back, I got to thinking about how to weasel around the built-in pop up blockers for Mozilla Firefox and Microsoft Internet Explorer. I can't remember now why I started down this path, but it consumed my afternoon, and in the end I had about a dozen variations on three or so core strategies.

Since I have no fake watches or mortgages to sell, I thusly shelved it all once I was done. We couldn't think of a reasonable application for these techniques here in BreakingPoint.

Well, maybe something was in the air, because it looks like RSnake and Jeremiah Grossman were working on something similar at around the same time. Only, they were going to present their findings at OWASP this week.

Unfortunately, we can't know what the extent of their trickery is, since some vendors asked them to keep it quiet -- but it's almost certainly more devious than simply skipping around the pop up blockers.

So, in the interest of quelling any speculation on the nature of "ClickJacking," here's two or three mildly interesting examples of what it almost certainly isn't. Note, the below example links are all in an iframe from planb-security.net. So, if you have a tabbed browser, you may notice an illusion the iframe's parent window (this blog post) being replaced by the fake pop up link (in this case, example.com).

---

---

If you're in the business of speculating on what other people's findings are, then you can safely check these techniques off your list. Since they all require Javascript to work their anti-anti-pop up magic, NoScript is quite effective in preventing these attacks. Based simply on this observation, and discounting for the moment the inherent lameness of merely subverting mouse clicks, it's virtually certain Jeremiah and RSnake's stuff is much, much more interesting / dangerous.

Network Management Practices

Just finished reading the document Comcast supplied the FCC (PDF) on their network management practices and reading the recaps from CircleID and Wired's Threat Level. This is, as folks point out, more of a historical document since Comcast has already submitted their new practices, but nonetheless it is interesting on a few points. Since we talk constantly about traffic analysis and of course P2P, I've been keeping up with the stories throughout.

What I found the most interesting in light of today's news was not the info around P2P or their new "application-agnostic" practices, it was the fact that Comcast's Jason Livingood who is in their National Engineering & Technical Operations, is commenting on these blog posts and supplying info, links and in some cases taking suggestions. Certainly a thankless job, but he seems to be helping bring a technical voice to the discussion.  What have you thought as you've read these stories?

Obviously people will be watching all of this closely and I'm looking forward to hearing your opinions on the new practices they outlined.  I'm going to bet some folks have a few things to say about the 250GB cap?

BreakingPoint LiveLook: SSL, Emmy Awards and a Harley

In San Francisco right now attending the Frost & Sullivan Award ceremony tonite and hoping to get some video to you all tomorrow, but I really wanted to post this video that Dennis shot with Mohsin Lari in our QA group. Mohsin and Dennis talk about QA and SSL, but also the fact that Mohsin has an Emmy Award winner in the family and is on a quest for a Harley. Something we all can relate to...well except for winning an Emmy, the award tonite will have to do for now.

 

Let us know if you have anything that you would like us to video at BreakingPoint or folks you want to see and hear.

Testing Network Devices using Proprietary Applications

As you know, BreakingPoint has a Custom Application Toolkit feature, which allows you to generate stateful application traffic from proprietary apps and then integrate this traffic with BreakingPoint’s testing platform.  You can blend the proprietary apps with our standard apps and security strikes or just run them alone, and test their effects on high performance network devices. 

Mike Hamilton, our Director of Product Marketing and Tod Beardsley in BreakingPoint Labs got together to demonstrate how to put this feature to good use.  The full screencast can be found here (along with our other screencasts), but I thought that this would be of interest to many of you so I've also embedded it below.

Bit Twiddling with Ruby: PacketFu

Since I started here at BreakingPoint, I've become a huge fanboy of Ruby. The Application Simulators I write are about 90% Ruby and 10% XML, and over all, it's a pretty fun language to code in. However, outside of BreakingPoint, Ruby is almost universally perceived as a language specifically for driving web applications. As a result, it can be hard to find good libraries for non-web chores -- in my case, I couldn't come up with a good way to natively read and write raw network packets.

So, I wrote a packet library for Ruby. It's called PacketFu, and over the weekend, I talked about it at Lone Star Ruby Conf here in Austin. In my talk, I spent a fair amount of time discussing the design ideas behind PacketFu, the most excellent BinData Ruby gem, and demoing various tasks that PacketFu endeavors to assist in -- packet sniffing, parsing, and rewriting.

PacketFu, now at version 0.0.3, is triple-secret alpha pre-release level code, but the demos and example applications work, and the external interfaces are pretty much nailed down and documented here. Look for a more sane beta release in the next few weeks, and a major version release before the end of the year. If all goes well, at next year's LSRC, I'll be talking about all the fab apps written using PacketFu.

DPI Coverage Getting it Wrong

Over the past two days news has trickled out that NebuAd's CEO has resigned and the company is now suspending their web-monitoring plans. Blogs and mainstream media are not only calling this a win for online privacy, but a few have talked about how this will kill off deep packet inspection (DPI) as a technology.  ZDnet's "Between the Lines" more specifically wrote:

"...it doesn’t sound like there’s a long prosperous future ahead for deep packet inspection..."

As Dennis commented on the ZDnet post, this is a bit of 'throwing the baby out with the bathwater'.  DPI capabilities go way beyond marketing uses, including enhancing network security and data leak prevention.  To collectively dismiss DPI because of the NebuAd example is short-sighted and a bit irresponsible.

To elaborate on these points, here are the DPI slides we shared a few weeks ago from Dennis' portion of the DPI webinar. These should be helpful for folks who are only looking at one angle of this story.

BreakingPoint LiveLook: Hitting the Bullseye with UI Design