JANUARY 30, 2009

Block: Fast Application Protocols in Ruby

Combined with our hardware acceleration, our Ruby-backed AppSim engine can easily pump out traffic at 10 or more gigabits per second.

In this library, protocol information is represented by Ruby objects called Blocks. These blocks can contain protocol-specific configuration options (like URIs, usernames or passwords), protocol state information, and can even contain other block objects. Block objects inherit still more information from their parent classes.

By creating custom protocol libraries with nested blocks, the end result is a simple to use way to create complex network messages. Also, using custom nested blocks leaves the resulting static strings with a multitude of valuable type information that we can use. I will cover more on how we can use that with my next blog post.

The Block Object

The Block object is the core of the Block library. It is the base class of every single block type that we create to define our protocols. Using this class as a base, a developer can create a specific block type like "HTTP::Request::Get", for example.

By combining lists of custom block types, a developer can typically reduce a protocol action (like in the AppManager UI) into a single command using Block. This is aided by convention that any options that don't get specified by a user will be randomly generated to look as realistic as possible.

# Simple FTP USER Command
b(FTP::Commands::USER).gen # => "USER akQA12KN\r\n"

# Simple HTTP GET Request
b(HTTP::Request::Get, :uri => "/index.html").gen # => "GET /index.html HTTP/1.1\r\nHost: a21naasd\r\n\r\n"

Block Syntax

This is the basic way to call into the Block library. The 'b' function is a globally accessible object factory that builds any type of your choosing. You can nest blocks within your block, and give them attributes or parameters.

The syntax used draws heavily on use of Ruby code block syntax. While it looks hairy at first, it simplifys things a bit. It reads a lot like XML, in that you have a element type and then that type can contain any number of other types.

Eventually you will take the object you just created and call the "gen" method on it. This lets Block decide how to best create a string matching the parameters that have been specified.

# Basic form
# b(TypeClass,:attribute=>'value') { |d| [BLOCK VALUE] }

# Static string generation
b(ByteString){|d| "hello"}.gen # => "hello"

# Static integer generation
b(Int::UINT16LE){|d| 0x01020304}.gen # => "\x04\x03\x02\x01"

# Random generation
b(String::AlphaNumeric, :min => 4).gen # => "mUw9"

# Compound Lists
b(ByteString){|d| ["one","two"]}.gen # => "onetwo"

# Deeply nested structure
b(Block){|d|[ # anon function returns array, note array commas
b(Block){|d| attr(:stored_object)},
b(Repeat,:times=>2){|b|[ # can use static attributes
b(Char){|d| "H"},
b(Char){|d| "E"},
b(Char){|d| "L"},
b(Char){|d| "O"},
]},
b(EOL) # uses default or random value for EOL ("\r\n")
]}.gen # => "(StoredObject)HELOHELO\r\n"

Custom Block Types

The real power of the Block library is in creating your own custom Block types. When a full suite of custom Block types are implemented for a protocol, generating realistic traffic becomes straight-forward and simple.

class FTP < Block
module Commands
class USER < Command
@operation = b(Operation){|d|"USER"}
@username = b(AlphaNum,:min=>1,:max=>24)
def encode(data)
b(FTP::Command::Line,:operation=>attr(:operation),:arguments=>[attr(:username)])
end
end
end
end

Block + AppSim

The BreakingPoint AppSim architecture exists to provide high-performance realistic application traffic with the easiest possible user interface. The user is able to supply a multitude of options that fine tune all of this network traffic.

Block was orginally created as an aid for the BreakingPoint Labs to help deal with the often very complex sets of options coming in from the users. Since Block can handle very complex options and auto-generate content, just passing on the AppSim options to a simple command like "b(HTTP::Request::Gen,@user_options)" can transform a simple HTTP message into a very realistic looking one. Easy:

GET /~sean/authd/index.html HTTP/1.1
Host: localhost
Connection: close
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727)
Accept: */*

Accept-Language: en-us
Accept-Encoding: gzip,deflate
UA-CPU: x86
Authorization: Digest username="httpuser",realm="private",nonce="408b6c576464477d675f04eb86609d90",uri="/~sean/authd/index.html",qop=auth,nc=00000001,cnonce="c9ea9ac0",response="da59792735e59ab97ddf3db1c7a5acde",opaque="5b168ba031c8cfa782b94ae587b79f67"