APRIL 10, 2009

Protocol Realism: OSCAR File Transfer

The last Protocol Realism article we published here on the BreakingPoint Labs Blog was about AIM, and how it's possible to fingerprint AIMv6 over TLS by observing packet lengths. The latest Application Protocol & Strike Pack now supports simulating AIMv6 file transfers with the AIM6-Rendezvous Application Simulator component.

AIM file transfers are interesting for a couple reasons. First off, for a 'proprietary protocol,' the OFT protocol (OSCAR File Transfer) is surprisingly well-documented. This is due, in no small part, to the efforts of Jonathan Clark and his paper, On Sending Files via OSCAR. Although the paper is now a couple years old, OFT has remained essentially unchanged. If you're interested in the nitty-gritty of OFT, I can't recommend Clark's paper enough.

OFT Privacy

One aspect that struck me, though, is that if you're using the AOL-provided client (AIM version 6.8.14.6 for Windows), your file transfers are proxied through AOL's servers on port 443, just like your usual chat channels. However, unlike regular chat, these file transfers are not encrypted. This is great for IPS folks who like to watch file transfers for worm and trojan content, but maybe not so much for people who have been assuming that their port 443 file transfers have been enjoying the benefits of TLS encryption.

Note that third-party clients, such as Pidgin, have a configuration setting to prefer direct IP connections over AIM's proxies, which causes file transfers to be negotiated directly between peers over random ports. However, this also exposes that peer's IP address to other peers. This is not usually that big of a deal, but sometimes people get touchy about that sort of thing. The moral of the story here is, there can be no assumption of privacy when using OSCAR file transfers.

The OFT Checksum

While Clark's paper made implementing our Application Simulator component relatively easy, his paper defers detailing the checksum algorithm used to other, existing implementations -- specifically, Joscar's mJava-based and Gaim's (now Pidgin's) C-based implementations. Existing, open source implementations are great references for writing Application Simulator components, so all I had to do was reimplement it in Ruby.

The checksum algorithm itself has been described variously as "nonintuitive" (Clark) and "the weirdest I've ever seen" (Gaim). Since Ruby code is generally pretty readable, I've included an implementation below, so you can decide if you've seen any weirder.

# OFT Checksum calculator. 
def calc_checksum(buffer='',prev_checksum=0xffff,odd_start=false)
  checksum = (prev_checksum >> 16) & 0xffff
  buffer.split(//).each_with_index do |x,i|
    old_checksum = checksum
    # Even bytes are high, odd are low, unless the last chunk ended up starting
    # this chunk on an odd byte, in which case the reverse is true. Note that
    # the first chunk (and often the only chunk) is always starting on 0, thus,
    # even. This is the typical use case. Implementations should set the 
    # odd_start flag and the prev_checksum value if this is the case.
    if odd_start
      value = (!(i % 2).zero? ? x[0] << 8 : x[0])
    else
      value = ((i % 2).zero? ? x[0] << 8 : x[0]) # usually this one.
    end	
    checksum -= value
    checksum -= 1 if checksum > old_checksum 
  end
  2.times {checksum = (checksum & 0x0000ffff) + (checksum >> 16) }
  checksum << 16
end

Granted, this (sometimes backwards) high byte/low byte business is not as straight forward as the standard Internet checksum, nor nearly as useful as a proper cryptographic hash. I can't know why AOL felt the need to reinvent the wheel with this, but my suspicion is that it was introduced purely to irritate third-party OSCAR implementations. Since that didn't work, AOL has no doubt learned that there is no stopping the efforts of dedicated protocol reverse engineers.