BreakingPoint Labs

Use Recreate in Testing; Eliminate the Need for External Tools

Introduction

Hello everyone, welcome to my first BreakingPoint blog post. Prior to joining the BreakingPoint team, I spent several years as an Intrusion Prevention/Detection Systems developer. In that position, I spent considerable time decoding various protocols and using tools like Wireshark, Tcpreplay, Tcprewrite, and Netdude to view and manipulate packet captures. I was excited to adopt the BreakingPoint Recreate component and implement new features that allow Recreate to assist software developers, quality assurance engineers, and testers in ways that previously would have required external tools.

Background

The Recreate component allows users to incorporate data originating from their real network into the tests conducted in their test environment. Recreate operates in self-described fashion: it recreates traffic based on the data contained in a capture file from any libpcap-based sniffer, such as wireshark or tcpdump, modifying the traffic attributes as necessary to conform to the test network specifications. Prior to the upcoming release (which will be coming soon), raw playback (discussed below) was not possible; instead, the Recreate component rewrote the data to match the traffic parameters specified for the domain, importing only the raw payload. This method had two modes of operation:

  • Use capture file settings
  • Use user-specified settings
Using the first setting, Use capture file settings, Recreate will use the data rate, maximum simultaneous sessions, sessions per second, test duration, inter-packet delays, application payloads, and destination ports from the supplied PCAP file. All other fields/parameters will be taken from the Parameters tab. The source port will be randomized.

Using the second setting, Use User-specified settings, Recreate will only use the application payload and destination ports from the PCAP file. The source ports will be randomized and all other fields/parameters will be taken from the Parameters tab. The interpacket delays will be set to ‘0’.
<
The purpose behind these two settings is to allow you to use the application payload from the PCAP file, but still have some control over how the file is replayed. Use capture file settings essentially lets you replay the PCAP as recorded, whereas Use User-specified settings enables you to control how fast or slow the traffic is replayed while still preserving the original payload.

Importing IPv6 Capture Files

 

Importing an IPv6 capture file is no different than importing an IPv4 file. All imported capture files must be libpcap-compatible. Once the capture file has been successfully uploaded, it will be listed under the Capture File Name list on the Recreate editor screen and selectable from the Filename drop down menu in the Recreate component’s parameters list.

To import an IPv4 or IPv6 capture file:

1.
Select Managers > Traffic Manager from the Menu bar.
2.
Click the Import Capture button.
 

A new window will display, which will allow you to upload a PCAP file:

3.

Enter a name in the Capture Name field.
Note:
This will be the name displayed for the PCAP in the Traffic Manager. Note that capture file names can only contain alphanumeric characters, spaces, and dashes.
4.
Click the Browse button.
5.
Navigate to the location of the PCAP file and select the file.
6.
Click the Open button.
7.
Select the Allow Overwrite option if you want to overwrite an existing file with the same name (as defined in the Capture Name field).
8.
Click the Upload button.


After the file has been properly imported, it will become available in the Capture File drop down box:

breakingpoint testing tools

 

Importing gzipped Capture Files
Often it is valuable to view the transmitted and/or received packet buffers exported from the ports used in one of our tests. This can be accomplished by exporting the packet buffer as shown in the next screenshot:

 

This will launch an options window (shown below) allowing the end user to choose which slot(s) to export, select transmitted and/or received packet buffers for export, and configure a BPF filter and snaplen for each selection, respectively. Once the Export button is pressed, a lotXPortX.xxxxxxxxxxxxx.pcap.gz file will be downloaded. For increased test efficiency, this release allows Recreate to re-import the generated SlotXPortX.xxxxxxxxxxxxx.pcap.gz file without first having to decompress it on your external file system:

 

 

 

 

 

Raw playback

While Recreate's layer 4 payload modes of operation discussed above are still the default, the upcoming release introduces additional raw playback functionality similar to tcpreplay for layers 2-7. This new mode's performance cannot match the payload-only mode due to disk I/O constraints, and the reporting is limited to interface statistics, but it can replay traffic exactly as captured, which can be very useful when testing layer 2-, layer 3-, and layer 4-related issues such as:

  • Customer or vendor supplied packet capture files
  • Layer 2 ARP and RARP
  • Layer 3 ICMP,  IPv4, and IPv6
  • TCP/UDP headers (traditional mode replayed payloads only)
  • TCP SYN floods, denial of service attacks, and invalid packets

 Raw playback can be enabled by toggling the Replay capture file without modification option from false to true:

 

BPF Filtering

The raw playback mode also supports BPF filter expressions after importing, but prior to sending, the traffic. This can be useful if you want to isolate traffic by IP address, port, protocol, etc. without permanently modifying the capture file. The tcpdump manpage is an excellent source for BPF filter syntax.

The filter string depicted in the following screenshot will display all TCP traffic to and from TCP port 80 involving host 10.10.10.41. This filter will generate traffic exactly as if the PCAP had originated from executing: tcpdump -r ipv-http.pcap ip host 10.10.10.41 and tcp and port 80:

 

Capture File Looping
Recreate's new raw playback mode also supports looping the capture file from 1 to 10,000 times.

 If we look at the Wireshark summary of a PCAP that we are about to loop:

 

 

Set the Number of times to loop capture file to 3 and then Save and Run test:

 

The Wireshark summary of the capture file exported off of our Slot2Port0 interface shows the proper loop count below:

 

TCP and UDP Port Rewriting

 

 

 

 

 

TCP/UDP port rewriting occurs during packet transmission and does not alter the capture file imported onto the disk. Prior to this release, changing the ports in a capture file required using a tool like tcprewrite or netdude to modify the file prior to importing.

To rewrite a port, navigate to the Recreate Parameters tab and choose Modification Options.Rewrite source and destination ports. The format of this parameter is "originalport:newport":

 

The original imported capture file containing IPv6 TCP port 80 (HTTP) traffic:

The exported packet buffer from the Slot2Port0 interface after the test was completed showing the IPv6 addresses rewritten to IPv4 addresses and the TCP ports rewritten from 80 to 8080:

 

breakingpoint testing toolsPort Independent Protocol Classification

In previous releases, Recreate classified protocols based upon TCP and UDP port only. If SSH, normally bound to port 22, was bound to port 80, Recreate would classify the protocol incorrectly as HTTP. If SSH was bound to port 2222, it would be classified as other. Utilizing regular expressions taken from the l7-filter project  and libpcre, the internal protocol classifier can now properly detect many standard protocols (SSH, IRC, HTTP, FTP, etc.) bound to non-standard ports as well as protocols that often do not have a standard port associated with them, such as many popular P2P (Bittorrent, Gnutella, etc.) clients. Recreate currently identifies 38 protocols.

Here is a screenshot of SSH running on TCP port 2222. Wireshark does not contain any protocol classification code and as you can see, Wireshark labels the protocol TCP. With Recreate's new classification feature, the generated reports will correctly identify this as SSH:

 

Prior to this release, the ssh traffic on TCP 2222 would have been reported as other. As you can see below, the protocol is now properly classified. Expect the protocols that we support to be expanded in future releases:

Conclusion

The inclusion of these new features will allow the tester to spend less time obtaining, compiling, learning, and using third party tools to manipulate and replay packet captures files. We have several new features planned, which will make Recreate even more powerful, that I will write about in a future blog post.

2 comments
Tags: ipv4-ipv6 // layer 2-7 // blog post // tech talk //

recreate this

Posted by Andre Gironda at 2009-06-04 11:18
TL;DR !!!

j/k... thanks, kirby. it would have been nice to have this sort of thing 10 years ago. very cool stuff!

Port Independent Protocol Identification

Posted by Erik H at 2009-06-05 11:50
I’m happy to see you’ve implemented Port Independent Protocol Classification!

L7-filter is a great (and freely available) method for identifying the application layer protocol, but it has some limitations… The regular expressions for several protocols are computationally expensive, while others have high rates of false positives or false negatives.

I realized this when I was looking at the possibility to implement L7-filter in my network forensic tool NetworkMiner. I’ve therefore done quite extensive research in the field of traffic classification in order to complete a better method for identifying the application layer protocol. The result is an algorithm called the Statistical Protocol IDentification (SPID) algorithm.

You can find a proof-of-concept implementation of SPID at SourceForge:
http://sourceforge.net/projects/spid

There is also a research paper available about the algorithm here:
http://spid.sourceforge.net/sncnw09-hjelmvik_john-CR.pdf

I hope you will find the SPID algorithm useful for your purposes!

/erik

Videos

More >


Interact





LinkedIn

YouTube

Newsletter


Subscribe to BreakingPoint Labs blog by email:

Type in your email, hit submit and quickly verify your address.


Subscribe to our RSS feed