Protocol Reverse Engineering with the BreakingPoint Storm CTM Custom Application Toolkit
by Mike HamiltonPCAP replay has been used for many years as a technique for attempting to simulate realistic traffic in a testing environment. Unfortunately, this technique has important flaws. First, you are only able to get as many unique traffic flows as can be contained in your PCAP file. Second, without using a third-party tool, you are unable to modify any flow within the PCAP. Luckily, though, there are alternatives. Realistic application traffic and extremely high speeds are crucial when validating the resiliency of network devices and infrastructure. Part of realism is understanding that traffic patterns and traffic content are extremely dynamic and are always changing.
BreakingPoint supports hundreds of application protocols, including some you may never have heard of. For protocols we don’t already cover, our Application and Threat Intelligence (ATI) team is always ready to deliver new protocols at our customers’ request. Sometimes, though, customers may want to keep what they’re working on under wraps. In these situations, a great way to properly test resiliency is to use the BreakingPoint Custom Application Toolkit to reverse engineer the protocol in question. This guide will show you how to do this on the BreakingPoint Storm Cyber Tomography Machine (CTM)™.
To Reverse or Not to Reverse
For the purposes of this guide, we will be reverse engineering the TR-069 protocol. I do want to call attention to the fact that BreakingPoint already supports TR-069 as a built-in protocol in the ATI pack. TR-069 just represents an interesting protocol to demonstrate the functionality of the BreakingPoint Custom Application Toolkit. Certain packet capture tools in the industry, such as Wireshark, have already done extensive research on both public and proprietary protocols. Many times, these types of tools have in-depth dissectors available that break down various protocols into the specific headers and fields, which makes reverse engineering particularly easy -- someone else has already done the work. This guide will be focused, instead, on a protocol that does not have this luxury.
TR-069
TR-069 is the protocol over which Customer Premise Equipment (CPE) devices such as DSL and cable modems speak to Auto-configuration Servers (ACS). ACS devices maintain tables of authentication credentials and ensure that only properly authenticated devices are allowed to access network resources.
The more granular details of TR-069 are unnecessary for the purposes of this document. More details on the specifics of the TR-069 protocol are available from the publishers of the TR-069 protocol, the Broadband Forum.
XML
TR-069, like many other applications, is based on eXtensible Markup Language (XML) over hyper text transfer protocol (HTTP). Because the Custom Application Toolkit also uses XML as the input language, this introduces some complexities that this post will address below.
Procedure
Because TR-069 is based on XML over HTTP on an arbitrary port -- in this case 7013 -- Wireshark does not have a native dissector for it.
Thus, we will use the ‘Follow Stream’ option in Wireshark to view the ASCII characters. Once we’ve captured the ASCII output, we will then import that output into the Custom Application Toolkit language for further processing.
NOTE: You must be careful of non-printable characters. Carriage return/line feeds are generally easy to catch, but others such as TAB or NULL may not be.
For the purposes of this document, only the section of protocol displayed in the ‘Follow TCP Stream’ window will be reverse engineered. But the exact same procedure can be used to continue the reverse engineering process for other sections.
Now that the ASCII text of the transaction has been copied into a text editor, we can start to port this into the XML necessary for the Custom Application Toolkit. (Here is where we begin to run into some of the complexities mentioned above.)
The first step is to realize that the ‘\r\n’ characters in the ASCII output were doubled when copying to a text file. Both ‘\r’ and ‘\n’ are recognized as line feeds in my particular text editor. Once you’ve completed the task of removing the doubled characters, your ASCII output will look much more like a typical HTTP request.
As referenced earlier, the fact that TR-069 is an XML-based protocol introduces an interesting dilemma when using the Custom Application Toolkit. Because both are XML-based, the '<' and '>' symbols will cause our tags to get out of sync in the Custom Application XML. There is the option of using the "![CDATA[]]" tag, but that’s better covered in a different example. The way I chose to address this is by using escaped hex. Using your favorite text editor or Perl script, replace each '<' symbol with '\x3c' and each '>' symbol with '\x3e'.
You will also need to attach the HTTP/XML '\r\n' characters at the end of each line, like this:
Following the commands in Figure 5 provides the final ASCII output that you will need going forward.
Now that the ASCII output has been properly processed, it must be copied into the appropriate Custom Application XML format. The "application" tag must be applied using port 7013, and the ASCII itself must be put into a "client" tag since this data is originating from the client. Once these changes have been applied, the resulting XML is almost ready for final editing.
You will notice that I have done a few things that are a bit out of the ordinary. First, I put the POST content into a dblock store prior to the headers. The reason for this is that we must calculate the content length before we can actually output the headers. Once we’ve calculated the content length via the ‘ascii_length’ dblock, we can use that in place of the captured static content length. The reason for this will become clear below.
At this point, the XML is almost ready for importing on the BreakingPoint Storm CTM, but all it would really be doing at this point is standard PCAP replay. Here at BreakingPoint, that just is not good enough. The first step is to identify the fields that must be changed from flow to flow. At first glance, I can see that the “Host:” field must actually reflect the destination IP of the connection. I also notice that the “Authorization:” field, the manufacturer, OUI, product class, and serial numbers could be randomized. These field are input into the XML as dblock stores in the configuration block. The variables are then called in place throughout the rest of the XML. Once these steps have been completed, our XML is ready to be imported into the BreakingPoint Storm CTM.
Import and Run
Now that we have created a valid XML for a simplified TR-069 transaction, we can import that XML and run a test to verify that we’ve completed our mission -- reverse engineering TR-069.
First, we must create a Super Flow using the Custom Application flow.
We must then edit the “Process DBlock XML” action to use the tr-069.xml file we created in the previous section.
Once we’ve created the Super Flow, we’ll need to create an Application Profile containing the Super Flow we just created.
We can then create an Application Simulator component to generate this Application Profile.
Once we’ve run the test, we now have created potentially millions of unique TR-069 transactions as illustrated in the PCAP/ASCII output shown here. Take note that each of the parameterized fields we specified is new to this particular flow.
Conclusion
Often, simple PCAP replay will frankly not get the job done. Statically replaying traffic does not show the type of variation that would be seen in the real world. This document has shown you a simple and quick way to use the Custom Application Toolkit to create highly customized and programmatically random applications. The beauty of using this method is that it’s easily extensible and updatable. If TR-069 ever becomes TR-070, we can use the XML just created as the basis for creating our next version, too. While not shown in this particular example, we could also unlock the true power of the BreakingPoint Storm CTM architecture by using token substitution (which will be the subject of future posts). All of these are things you just cannot get from a simple application emulation engine.
If you’ve ever worked with TR-069, done any other reverse engineering, or just have questions about what I’ve written, we’d love to get your thoughts in the comments thread. Fire away!
