TCP Portals: The Handshake's a Lie!

Whenever I interview someone for an Application Engineer or Security Research position, my favorite introductory question is, "Can you describe for me the TCP three-way handshake?". It is a fine baseline question to understand a candidate's knowledge of modern networking. Answers range from "SYN, SYN/ACK, ACK,", to a full description of ARP, to initial sequence number generation. It's a good springboard question, because then you can start talking about spoofing addresses, port scanning, the significance of IPIDs, and more.

We are hiring a lot here at BreakingPoint, which means I'm asking this question a lot. After the fourth or fifth interview, I decided one morning to look over RFC 793 to make sure that I really did know everything there is to know about the handshake. That is when I found out that we've all been living a lie.

If you've spent any reasonable amount of time around network protocols, you're probably familiar with some version of this diagram:

Here, we see the client on the left starting up a conversation with the server on the right. All pretty normal and familar, right? Well, when I was reviewing the RFC again, I noticed something very, very, odd. Disturbing, even. Allow me to quote at some length:

  The synchronization requires each side to send its own initial
  sequence number and to receive a confirmation of it in acknowledgment
  from the other side.  Each side must also receive the other side's
  initial sequence number and send a confirming acknowledgment.

    1) A --> B  SYN my sequence number is X
    2) A <-- B  ACK your sequence number is X
    3) A <-- B  SYN my sequence number is Y
    4) A --> B  ACK your sequence number is Y

  Because steps 2 and 3 can be combined in a single message this is
  called the three-way (or three message) handshake.

Do you see what I see? Because I'm thinking, "this is not a three-way handshake. This is a four-way handshake." The handshake is a lie, born of coalescing steps 2 and 3.

Now, surely, if I just decided to ACK a SYN, then send my own SYN, that couldn't possibly work, right? Enter PacketFu, my little Ruby library for crafting packets. Turns out, 28 years or so after this RFC was written, clients behave rather strangely when you decide to actually honor ol' RFC 793. After some experimentation, I have a pretty decent proof-of-concept stack that behaves like so:

This is the point where things get a little weird. What's happening here is:

    1) A --> B  SYN my sequence number is X
    2) That's nice. I'm not going to bother to ack that, because...
    3) A <-- B  SYN my sequence number is Y.
    4) A --> B  ACK your sequence number is Y, and my sequence number is X.
    5) A <-- B ACK your sequence number is X

Does this work? You betcha! Take a look at the packet captures, collected from Linux (stock Ubuntu), Apple (stock OSX), and Microsoft (stock Windows XP). These three desktop operating systems are all totally cool with this crazy backwards TCP portal.

But what does it mean? Is this simply a parlor trick, where you can reverse the roles of client and server? How does this affect stateful firewalls? How about inspection devices like IPSes, which often need to have an idea of who the "real" client and server are? How about NAT devices, where the idea of "relatedness" is absolutely tied up with where SYN packets come from.

Clearly, there is a ton of testing work to be done here. Lucky for me, I happen to work at a really advanced testing equipment manufacturer, so I've dropped this nugget in the next StrikePack. Now, strikes can employ the "SneakAckHandshake" TCP override option, and all servers simulated will behave in accordance with this crazy backwards handshake. We'll see how well network inspection gear detects clientside attacks when the client is tricked into behaving like a server.

At the very least, now I have better interview questions and I should at least be able to detect if the next candidate is reading this blog. :)

Protocol Realism: HTTP POST Content-Type

As any semi-competent web application developer could tell you, when you fill out a form and submit it via HTTP, there are a few options your browser has in formatting the response. However, since I am not a semi-competent web application developer, I had to go look this up because we have a BreakingPoint Elite user who recently asked for exactly this functionality in our HTTP Application Simulator.

So, as of the latest Application Protocol and Strike Pack, the HTTP Application Simulator now asks for an optional "Content-boundary" value, to go along with the usual "Content-Type" header, like so:

To illustrate, if we had a test that specified the options as shown above, along with a text file with the specified boundary token of "BPS123" to demarcate each of the HTTP form's values, the result would look something like this:

POST /submit.html HTTP/1.1
Host: default
Connection: Keep-Alive
User-Agent: BreakingPoint/1.x (http://bpointsys.com/)
Accept: */*
Content-Type: multipart/form-data; boundary=BPS123
Content-Length: 395

--BPS123
Content-Disposition: form-data; name="value_one"

This is my first value. I just want to say
--BPS123
Content-Disposition: form-data; name="value_two"

Yo value_one, I'm really happy for you and I'mma let you finish,
but application/x-www-form-urlencoded is the one of the best form
encoding types of all time.

One of the best form encoding types of all time!
--BPS123--

If we had, instead, set the Content-Type to the more typical POST encoding of "application/x-www-form-urlencoded," we could specify another appropriately formatted text file and generate a result that HTTP-aware content inspection devices would understand:

POST /submit.html HTTP/1.1
Host: default
Connection: Keep-Alive
User-Agent: BreakingPoint/1.x (http://bpointsys.com/)
Accept: */*
Content-Type: application/x-www-form-urlencoded
Content-Length: 188

value_one=First+value&value_two=%09This+is+the+first+line%0d%
0aSecond+line,+and+honestly,+this+kanye+thing+isn%27t+all+that+funny+anymore.
%0d%0aNo,+really,+let%27s+just+end+this+now.

Of course, there's also the plain text encoding, which virtually nobody uses. But, just because it's unpopular doesn't mean that your inspection device shouldn't be tested on it. To do so, it's as simple as setting "text/plain" as the Content-Type, and providing the text to look for:

POST /submit.html HTTP/1.1
Host: default
Connection: Keep-Alive
User-Agent: BreakingPoint/1.x (http://bpointsys.com/)
Accept: */*
Content-Type: text/plain
Content-Length: 298

value_one=This is the first value.
value_two=Yo value_one, I'm really hap--
value_three=Yo value_two, I'm really happy for you, and I'mma let you 
finish, but the fake form value interruption in example one was the best 
"I'mma let you finish" joke of this entire blog post.

This entire blog post!

It's a little thing, of course -- just a few UI settings to make sure that the form data you're blasting out at dangerous speeds is formatted according to your fancy. But it's the little things like this that make the case for realistic testing. If a vendor claims a device inspects outgoing HTTP data, then, by gum, it ought to be handling all the vagaries of form encoding. And if the vendor claims it does that, then by double-gum, now you can test it to be sure.

Port Forwarding Fun

Now that I'm spending a fair amount of time writing and testing our new Client Simulator Super Flows, I often find myself with a need for rapid deployment of services to test against. Normally, this is no problem; we have a VMWare server dedicated to Client Simulator development testing hooked up to our BreakingPoint Elite, and the image runs a friendly stock Ubuntu Server.

This week, I started in on ironing out our new Microsoft SQL Cient Simulator, and as great a product as Microsoft SQL Server is, it sadly doesn't install on Linux very well, and since I'm uncannily lazy, I didn't want to go through the hour or two of building out another VMWare image.

Lucky for me, Kirby Kuehl had already installed MSSQL on his own test server (running QEMU) in case we needed any Network Processor magic to make the MSSQL Client Simulator work correctly.

So, I have this VMWare server sitting over here, and a QEMU guest OS sitting over there, in a different network. "No problem," says I, "I'll just set up some port forwarding love!"

Up until now, whenever I have a need to forward network ports around, I default to using ssh tunnels. But in this case, I really wanted the traffic to be cleartext all along the way since I hadn't decided yet where exactly I'd be sniffing the traffic to verify my implementation. Netcat is a fine cleartext alternative for ad-hoc forwarding, but I got it in my head that I wanted a more "real" service running on my VMWare instance.

Since I had already tested out the diagnostic services using this same host -- Chargen, Echo, Daytime, etc -- I had the venerable xinetd installed to run these basic protocols. Somehow, in my many networking adventures, I had forgotten how powerful xinetd can be. Turns out, with an edit to /etc/services (adding 'externsql' as the name for 1433/tcp), adding "kirbys-target" to /etc/hosts, and an entry under /etc/xinetd.d like so...

# default:on
# description: forward various sql services. In this case, 
various is only one.

service externsql
{
	disable = no
	flags = REUSE
	socket_type = stream
	wait = no
	user = root
	redirect = kirbys-target 1433
}

...I'm able to forward my VMWare guest's 1433/tcp traffic across to Kirby's QEMU machine. Xinet.d also features some pretty easy access control configuration and logging (see Jose Nazario's ancient LinuxJournal article on the subject), so if you're in a more hostile network, that might work out for you.

Once that step was out of the way, I noticed that the bridged interface of Kirby's machine was out of whack (it wasn't set to come up on boot), so I couldn't talk to it directly from my VMWare image. Again, I'd like to reference my supreme sloth: instead of proper troubleshooting with brctl, another xinetd redirect (from the QEMU host to the QEMU guest) had me up and running, shuttling MSSQL packets between the BreakingPoint Elite and the far-off SQL Server.

Now that I'm wrapping up my Client Simulator development (look for it in a StrikePack in the near future), it struck me funny that I had so many layers of redirection in order to test traffic. So funny, in fact, I put together this network cartoon so I could share the hilarity with you.

If you're going to be performing any serious level of performance testing, I probably wouldn't recommend such a labyrinthine setup; QEMU and VMWare both can get a little pokey under load, and while xinetd is pretty lightweight, all those stops along the way are adding friction to your ridiculously fast testing gear. But, in the case of functional R&D, this set up is ideal, if a little silly; I'm never going to recable our test Elite again.

Constantly Hacking Ruby Constants

Here at BreakingPoint, we write all of our application simulation code in Ruby. Lately, I've been working on adding a slew of new behaviors to our IMAPv4 implementation so users can fine-tune their IMAP Application Simulator and Client Simulator flows. At first, this seemed like it would mean a whole lot of typing to wire up twelve new actions.

Instead of copying and pasting all over the place (and dreading the possibility of fixing the same bug in fifteen zillion places), I needed to come up with a code reuse technique that takes advantage of the existing codebase written using standardized naming conventions. Since I'm swimming in these standard names, I figured there must be a way to use Ruby's dynamic typing and extensible classes to make this easier on myself, both now and in the future.

The first trick is to programmatically figure out which application profile class to use when I'm in a particular protocol. For example, if we're in a function in the "Imap" object, I need to get protocol configuration from the "ImapProfile" singleton object. This is pretty easy with Ruby's introspection and the nifty Kernel.const_get() function.

So, let's say we have a (simplified) ImapProfile class:

class ImapProfile
 def self.config
	{:username => "todb",
	 :password => "Shadowfax" # Unguessable! 
	}
 end
end

In the Imap subclass of Application Manager, I'll want to get a hold of those configuration parameters. I can do so with something like this:

module AppManager
 class Imap

  # Get my class name, strip off the superclass
  def my_protocol
  self.class.name.to_s.split("::").last
  end

  # get_profile_params() takes the string from 
    my_profile_object(),
  # gets the associated constant, and invokes 
    the config() method. def get_profile_params
  Kernel.const_get(my_protocol + "Profile").send :config
   end

   end
end

Now we can call the profile object's "config" method by deriving the class name from our own class's name:

irb(main):001:0> @app = AppManager::Imap.new
=> #
irb(main):002:0> @app.get_profile_params
=> {:username="todb", :password=>"Shadowfax"}

That's pretty neat and all, but the real trick is to figure out how to do the same thing with a method name, since (as you'll see) they bear a resembelence to individual action classes. After a little bit of research, it turns out we can perform something similar with the Kernel.caller() function, and again use some string manipulation to get what we want:

def caller_action_to_constant
	caller[1] =~ /`([^']+?)'/
	$1 =~ /^do_(client|server)_(.*)/
	$2.split("_").map {|s| s.capitalize}.join
end

This function takes the second element of the execution stack, extracts the calling method's name (the first regex), extracts the part we care about (the second regex), then splits on the underscores in order to CamelCase the result. In the end, the string:

"do_client_send_user_name"

becomes 

SendUserName

Why not the first element of the call stack array? Well, I'm wrapping this up in an intermediary function, called the action_executor, which takes this string and performs another const_get to actually use it for something:

def action_executor(args={})
 Kernel.const_get
(my_protocol + caller_action_to_constant + "Cmd").send :data
end

So, from now on, the do_ actions can call the action_executor in order to track down the right classes to get the data from:

def do_client_send_user_name(args={})
	action_executor(args)
end

Pretty neat, if you ask me. A complete code listing should be available here, at Pastie.

In the end, this strikes me as an implementation of the OO Delegation design pattern. However, it includes some extra smarts about where the delegatee is, all based on a common naming convention for classes and methods. While the example code is sparse, in reality, the application actions I'm replacing in IMAP were each around 15 lines, and this technique compresses them down to one. I also get the added bonus of centralizing a common function to one spot, to ease future tweaks to the way application protocols work, or, the laughably remote possibility that there's ever a bug discovered there.

Protocol Realism: OSCAR File Transfer

The last Protocol Realism article we published here on the BreakingPoint Labs Blog was about AIM, and how it's possible to fingerprint AIMv6 over TLS by observing packet lengths. The latest Application Protocol & Strike Pack now supports simulating AIMv6 file transfers with the AIM6-Rendezvous Application Simulator component.

AIM file transfers are interesting for a couple reasons. First off, for a 'proprietary protocol,' the OFT protocol (OSCAR File Transfer) is surprisingly well-documented. This is due, in no small part, to the efforts of Jonathan Clark and his paper, On Sending Files via OSCAR. Although the paper is now a couple years old, OFT has remained essentially unchanged. If you're interested in the nitty-gritty of OFT, I can't recommend Clark's paper enough.

One aspect that struck me, though, is that if you're using the AOL-provided client (AIM version 6.8.14.6 for Windows), your file transfers are proxied through AOL's servers on port 443, just like your usual chat channels. However, unlike regular chat, these file transfers are not encrypted. This is great for IPS folks who like to watch file transfers for worm and trojan content, but maybe not so much for people who have been assuming that their port 443 file transfers have been enjoying the benefits of TLS encryption.

Note that third-party clients, such as Pidgin, have a configuration setting to prefer direct IP connections over AIM's proxies, which causes file transfers to be negotiated directly between peers over random ports. However, this also exposes that peer's IP address to other peers. This is not usually that big of a deal, but sometimes people get touchy about that sort of thing. The moral of the story here is, there can be no assumption of privacy when using OSCAR file transfers.

While Clark's paper made implementing our Application Simulator component relatively easy, his paper defers detailing the checksum algorithm used to other, existing implementations -- specifically, Joscar's mJava-based and Gaim's (now Pidgin's) C-based implementations. Existing, open source implementations are great references for writing Application Simulator components, so all I had to do was reimplement it in Ruby.

The checksum algorithm itself has been described variously as "nonintuitive" (Clark) and "the weirdest I've ever seen" (Gaim). Since Ruby code is generally pretty readable, I've included an implementation below, so you can decide if you've seen any weirder.

# OFT Checksum calculator. 
def calc_checksum(buffer='',prev_checksum=0xffff,odd_start=false)
  checksum = (prev_checksum >> 16) & 0xffff
  buffer.split(//).each_with_index do |x,i|
    old_checksum = checksum
    # Even bytes are high, odd are low, unless the last chunk ended up starting
    # this chunk on an odd byte, in which case the reverse is true. Note that
    # the first chunk (and often the only chunk) is always starting on 0, thus,
    # even. This is the typical use case. Implementations should set the 
    # odd_start flag and the prev_checksum value if this is the case.
    if odd_start
      value = (!(i % 2).zero? ? x[0] << 8 : x[0])
    else
      value = ((i % 2).zero? ? x[0] << 8 : x[0]) # usually this one.
    end	
    checksum -= value
    checksum -= 1 if checksum > old_checksum 
  end
  2.times {checksum = (checksum & 0x0000ffff) + (checksum >> 16) }
  checksum << 16
end

Granted, this (sometimes backwards) high byte/low byte business is not as straight forward as the standard Internet checksum, nor nearly as useful as a proper cryptographic hash. I can't know why AOL felt the need to reinvent the wheel with this, but my suspicion is that it was introduced purely to irritate third-party OSCAR implementations. Since that didn't work, AOL has no doubt learned that there is no stopping the efforts of dedicated protocol reverse engineers.