TCP Portals: The Handshake's a Lie!

Thats freaking odd man... I wouldnt think that that should work that way... Kinda makes for some interesting thoughts, I will keep looking for any updates you put here ;)

Page 29 last paragraph of rfc 793 states

"The three-way handshake
reduces the possibility of false connections. It is the implementation of a trade-off between memory and messages to provide
information for this checking.

The simplest three-way handshake is shown in figure 7 below.

TCP A TCP B

1. CLOSED LISTEN

2. SYN-SENT --> <SEQ=100><CTL=SYN> --> SYN-RECEIVED

3. ESTABLISHED <-- <SEQ=300><ACK=101><CTL=SYN,ACK> <-- SYN-RECEIVED

4. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK> --> ESTABLISHED

5. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK><DATA> --> ESTABLISHED

Basic 3-Way Handshake for Connection Synchronization

Thanks for the quotation, Robert.

The surprising thing, at least to me, is that I presumed that the "three-way" in "three-way handshake" requires exactly three packets. This is clearly not the case.

The fact that TCP B can optionally choose to ACK TCP A's SYN introduces some ambiguity that third-parties might trip up on when they're watching for TCP session establishment.

There's already an ambiguity as far as firewalls are concerned; Microsoft Windows XP's stock firewall is fine with this setup, but Ubuntu's stock iptables is very much not cool with this (incoming non-synacks are dropped). So, which one is behaving correctly?

I too have wondered that. There definitely is a need for some clarifications.

Of course we all know TCP is suppose to be a 3-way handshake.

I am not sure why the end of page 26 was included in the original rfc.

Quote:
" The synchronization requires each side to send it's own initial
sequence number and to receive a confirmation of it in acknowledgment
from the other side. Each side must also receive the other side's
initial sequence number and send a confirming acknowledgment.

1) A --> B SYN my sequence number is X
2) A <-- B ACK your sequence number is X
3) A <-- B SYN my sequence number is Y
4) A --> B ACK your sequence number is Y

[Page 27]

September 1981
Transmission Control Protocol
Functional Specification

Because steps 2 and 3 can be combined in a single message this is
called the three way (or three message) handshake."

The only thing I can assume is that some vendors have adopted this even though it clearly states in the philosophical and technical areas of rfc 793 that lines that steps 2 and 3 can be combined. Which as you say would makes it look like a 4-way handshake.

This would lead me to think either; 1)Vendors did not fully or still do not read the rfc hence some of these things were/are included.
2)The rfc should not have included that example.
3)maybe tcp also supported a 4-way handshake. Who know I did not have discussions with DOD and researchers when the original rfc was created.

The most logical answer 28 years deserves a rewrite.

Actually, this makes sense if you follow the RFC. Its not that you can't use more than three packets, its just that three is the minimum required.

For example once you reach established state we ACK and send data in the same packet. Nothing in the spec says you can't use two packets instead, its just not as efficient.

Re on firewalls:
Depends on the product. Some monitor flag state and some do not. Based on my testing Cisco would block this but CP and Netscreen would let it though.

As for which is correct, XP or Netfilter, the RFC does not say you "must" use four packets, so the XP operation is probably more correct. Its for the wrong reason however because the XP firewall is not watching flag state. You could just as easily generate a state entry with an ACK.

RE on IDS/IPS:
I have two different vendor products here in the lab and results are not good.

Another interesting point is that Wireshark tags Packet #4 as a dup ACK even though it is not.

Great work!
Chris

This contradicting post reminds me taking off on board on an airplane. I always pray that the laws of gravitation still work! When I read the post, I thought: oh, no, that cannot be possible! ;( I am dreaming!
Do the laws of TCP handshake still hold?
This is a very interesting topic ;))) --- TRUE or FALSE – “Is TCP 3 handshake a lie?” This sounds a philosophical topic.

No, no we have not been deceived all these years! All these points are interesting but the TCP 3-handshake is true!

RFC states that “The TCP is intended to provide a reliable process-to-process communication service in a multi-network environment.” The key term is “process-to-process” (or – well-known - socket-to-socket given that processors own sockets) but your paper focuses ONLY on client-to-server environment. The TCP handshake takes place in client-to-client or host-to-host environment as well.
In your article and your simulation you are assuming that all TCP handshakes take place in a client-to-server environment. (Note the title of the figure X below)

Below are the snippets from RFC 793
-------------------------
Figure A
Basic 3-Way Handshake for Connection Synchronization

TCP A TCP B
1. CLOSED LISTEN
2. SYN-SENT --> <SEQ=100><CTL=SYN> --> SYN-RECEIVED
3. ESTABLISHED <-- <SEQ=300><ACK=101><CTL=SYN,ACK> <-- SYN-RECEIVED
4. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK> --> ESTABLISHED
5. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK><DATA> --> ESTABLISHED


In line 2 of figure 7, TCP A begins by sending a SYN segment indicating that it will use sequence numbers starting with sequence number 100. In line 3, TCP B sends a SYN and acknowledges the SYN it received from TCP A. Note that the acknowledgment field indicates TCP B is now expecting to hear sequence 101, acknowledging the SYN which occupied sequence 100. At line 4, TCP A responds with an empty segment containing an ACK for TCP B's SYN; and in line 5, TCP A sends some data. Note that the sequence number of the segment in line 5 is the same as in line 4 because the ACK does not occupy sequence number space (if it did, we would wind up ACKing ACK's!).
--------------------------

TCP achieves reliability by assigning sequence numbers to each packet transmitted and an ACK (which is different than a reset) is required from the receiving host. Flow control and transmission between two processors are made possible through the utilization of sequence numbers space acknowledgement with a window size acknowledgement (buffer).
In the 3rd step (which is really the second step of the handshake), the SYN-RECEIVED is a just a stand-by waiting stage. The receiving-node is sending a SYN and only when both nodes have received their respective SYNs sequence, the connection is confirmed with an ACK.
Each session packet is represented by an initial sequence number (ISN) of 32 bits. And each ISN is respectively acknowledged “ACKed” from each side the sender and the receiver. The main reason why RFC calls is the 3 handshake is because SYN / ACK (2 steps) from the receiving socket can be combined into 1 step.

Also in response to Chris, I can see one possible reason why Wireshark interprets the last sequence “ACK’ed” from receiving client as “DUP’ed” (in Tod’s example (Packet # 4) A --> B ACK your sequence number is Y, and my sequence number is X)
In reference to figure A from RFC 793 example above); since the ESTABLISHED session has the same sequence number (301) between steps 4 and 5; it is presumably possible that Wireshark interprets it as a “DUP” (like “too late to ACK dude, since session has already been established, I ignore you!”). It is interesting to note that ACK does not consume sequence space number session but only SYN does. In line 4, the ACK sequence from the receiving node in the SYN-ACK, is an empty segment. The last ACK from the receiving node can just be interpreted as redundant since it is a “duplicate acknowledgement” (like “DUP to late”) and Wireshark ignores it since TCP ignores duplicate acknowledgements.

So, to come back to the 1st motivation about this post, it would be interesting to challenge the conclusions by simulating various experiments such as client-to-client or host-to-host, asynchronous vs. synchronous environment and etc. We cannot assume that all TCP handshake take place in client-to-server environment. You may also like to craft packets with RST and asynchronous segment space numbers and challenge the “client-to-server” environment. You may also put Wireshark and capture traffic on both sides before the firewall and behind / between the firewall and an internal host. It will be interesting to see the results. ;))

Cool topic !!!! since i had to review and dig into the philosophical realms of TCP (RFC 793)

./fabie

"There's already an ambiguity as far as firewalls are concerned; Microsoft Windows XP's stock firewall is fine with this setup, but Ubuntu's stock iptables is very much not cool with this (incoming non-synacks are dropped). So, which one is behaving correctly?"

is this a problem? isn't resend/'un-ack-ed things don't exist' way of things going to fix it? establishing the connection might take considerably longer then normal.

In your method you can actually and totally ignore your step 1. If you follow the rest of your steps, you are following a "standard" 3-way handshake. The only difference is that YOU are attempting to initiate the connection via your first SYN packet that contains your seq #. If side B does stateful packet inspection (meaning it knows and tracks when a full handshake has occured), you will get blocked being that you are sending a wild SYN that was not expected (it just so happens that in your case it's immediately after host A tried initiating a connection with you)... what are the odds? :)

Right.

But the punchline is, client OS'es are fine with responding to "wild" syns on their ephemeral ports, provided they're in the SYN_SENT state. In other words, you can set up a session with a port that technically was never in LISTENING.

With that said, the only odd thing I see occuring by doing this is the connection is established between BOTH ephemeral ports. Side A would have established the connection
on its ephemeral port by nature.

So, if I made a client that reacted the same way your server does to a SYN (ignore and SYN), then tried to connect to your server, would we just keep SYN'ing each other back and forth indefinitely?

If your client cared naught for state, probably.

The way to make it work is to ensure that your listening side moves along the state path like:

LISTEN -> SYN_RECV -> SYN_SENT -> SYN_RECV -> ESTABLISHED

..while the sending side follows the path of:

CLOSED -> SYN_SENT -> SYN_RECV -> SYN_SENT -> ESTABLISHED

It would all be predicated on where you start the state process.

Cool work

This is simply the result of allowing simultaneous open to work (see fig 8 of RFC793). Personally, I am not convinced that allowing simultaneous open was a good decision, given the scope for confusion.

For bonus points you can work out why simultaneous open doesn't work for TLS connections, despite being ok for the underlying TCP stream.

TCP Simultaneous Open gets a lot of treatment in RFC 5382 because there are a lot of stupid middleboxes implemented by people who didn't read RFC 793 very carefully. It remains to be seen whether anyone will read RFC 5382.

It seems like there are a few things going on here. First, a flawed simultaneous open implementation whereby the client goes to SYN_RCVD instead of sending RST. Second, client TCP stacks aren't properly implementing non-piggybacked SYN/ACKs. Third, stateful firewalls don't recognize non-piggybacked SYN/ACKs are related.

In your second TCP exchange, the 5th line has A and B reversed, which tripped me up when I was trying to read it.

Thanks Tim -- Fixed (5) so it reads more sensibly.

Very amusing. I got this one "wrong" on a El Goog phone screen :-).

-----N

I thought everyone knew that!

Just kidding, that's definitely an eye-opener. And the impact on security device identification of client and server definitely seems like a valid concern. But are those roles decided by the last step in the handshake process, or by the first?

It's somewhat surprising that equipment will work with this, but I guess it's understandable since the stack is meant to be robust.

But this is very disturbing. Even if high-end traffic monitoring hardware and software is updated to maintain the correct view of who initiated a connection, there will ALWAYS be cheap, poorly implemented technology out there in use, and unless this issue becomes more well known I think it'll be taken advantage of.

An excellent find.

Using ipfw (stock Mac OS X) we could drop established connections that don't have a dynamic rule:

ipfw -q add check-state

ipfw -q add deny log tcp from any to any established

See: http://codesnippets.joyent.com/posts/show/1267

This contradicting post reminds me taking off on board on an airplane. I always pray that the laws of gravitation still work! When I read the post, I thought: oh, no, that cannot be possible! ;( I am dreaming!
Do the laws of TCP handshake still hold?
This is a very interesting topic ;))) --- TRUE or FALSE – “Is TCP 3 handshake a lie?” This sounds a philosophical topic.

No, no we have not been deceived all these years! All these points are interesting but the TCP 3-handshake is true!

RFC states that “The TCP is intended to provide a reliable process-to-process communication service in a multi-network environment.” The key term is “process-to-process” (or – well-known - socket-to-socket given that processors own sockets) but your paper focuses ONLY on client-to-server environment. The TCP handshake takes place in client-to-client or host-to-host environment as well.
In your article and your simulation you are assuming that all TCP handshakes take place in a client-to-server environment. (Note the title of the figure X below)

Below are the snippets from RFC 793
-------------------------
Figure A
Basic 3-Way Handshake for Connection Synchronization

TCP A TCP B
1. CLOSED LISTEN
2. SYN-SENT --> <SEQ=100><CTL=SYN> --> SYN-RECEIVED
3. ESTABLISHED <-- <SEQ=300><ACK=101><CTL=SYN,ACK> <-- SYN-RECEIVED
4. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK> --> ESTABLISHED
5. ESTABLISHED --> <SEQ=101><ACK=301><CTL=ACK><DATA> --> ESTABLISHED


In line 2 of figure 7, TCP A begins by sending a SYN segment indicating that it will use sequence numbers starting with sequence number 100. In line 3, TCP B sends a SYN and acknowledges the SYN it received from TCP A. Note that the acknowledgment field indicates TCP B is now expecting to hear sequence 101, acknowledging the SYN which occupied sequence 100. At line 4, TCP A responds with an empty segment containing an ACK for TCP B's SYN; and in line 5, TCP A sends some data. Note that the sequence number of the segment in line 5 is the same as in line 4 because the ACK does not occupy sequence number space (if it did, we would wind up ACKing ACK's!).
--------------------------

TCP achieves reliability by assigning sequence numbers to each packet transmitted and an ACK (which is different than a reset) is required from the receiving host. Flow control and transmission between two processors are made possible through the utilization of sequence numbers space acknowledgement with a window size acknowledgement (buffer).
In the 3rd step (which is really the second step of the handshake), the SYN-RECEIVED is a just a stand-by waiting stage. The receiving-node is sending a SYN and only when both nodes have received their respective SYNs sequence, the connection is confirmed with an ACK.
Each session packet is represented by an initial sequence number (ISN) of 32 bits. And each ISN is respectively acknowledged “ACKed” from each side the sender and the receiver. The main reason why RFC calls is the 3 handshake is because SYN / ACK (2 steps) from the receiving socket can be combined into 1 step.

Also in response to Chris, I can see one possible reason why Wireshark interprets the last sequence “ACK’ed” from receiving client as “DUP’ed” (in Tod’s example (Packet # 4) A --> B ACK your sequence number is Y, and my sequence number is X)
In reference to figure A from RFC 793 example above); since the ESTABLISHED session has the same sequence number (301) between steps 4 and 5; it is presumably possible that Wireshark interprets it as a “DUP” (like “too late to ACK dude, since session has already been established, I ignore you!”). It is interesting to note that ACK does not consume sequence space number session but only SYN does. In line 4, the ACK sequence from the receiving node in the SYN-ACK, is an empty segment. The last ACK from the receiving node can just be interpreted as redundant since it is a “duplicate acknowledgement” (like “DUP to late”) and Wireshark ignores it since TCP ignores duplicate acknowledgements.

So, to come back to the 1st motivation about this post, it would be interesting to challenge the conclusions by simulating various experiments such as client-to-client or host-to-host, asynchronous vs. synchronous environment and etc. We cannot assume that all TCP handshake take place in client-to-server environment. You may also like to craft packets with RST and asynchronous segment space numbers and challenge the “client-to-server” environment. You may also put Wireshark and capture traffic on both sides before the firewall and behind / between the firewall and an internal host. It will be interesting to see the results. ;))

Cool topic since i had to review and dig into the philosophical real of TCP !! ;)

./fabie

Let me start by saying that ultimately, what you are pointing out is certainly interesting and potentially troublesome to NAT devices. And my issues with the test DO NOT suggest that your results are invalid.

I believe your client machine sends a second SYN because the first one is not being ACKnowledged.

I don't think this means 3 way handshake is a lie. It's not like the systems you tested with avoided using three way handshake for TCP, your hack was splitting out the functions that would normally be put into one packet, so you were breaking the RFC rules, not the other systems.

The nature of TCP is to be resilient and so I'm not surprised that the operating systems handled the trickery.

Before TCP sends the initial SYN is sets up a socket, which, as you know, is a unique combination of src/dst IP address and src/dst port numbers. The combination of all four make it a unique socket. (I know you know this already)

After a TCP sends a SYN, it is expecting two things back: (1) an ACK and (2) a reciprocal SYN. TCP is getting both, just not at the same time.

In fact the SYN you are sending in your manipulated packet matches the socket that the TCP on the initiating system has already setup - it is expecting that SYN so the client and server have not changed roles at all.

Changing the browser on a specific OS doesn't change the behavior and rightly so - the TCP stack is implemented in the OS.

I did notice that XP & Ubuntu have the same behavior, but OSX is slightly different.

XP & Ubuntu:
[SYN]->
<-[SYN]
[SYN/ACK]->
<-[ACK]

OSX:
[SYN]->
<-[SYN]
[ACK]->
[SYN/ACK]->
<-[ACK]

Here's where I think the test is flawed:
Your program on the server side is not ACKing until it receives a second SYN.

If you look at the OSX traces, the OSX TCP is waiting almost a full second for the [ACK] and then is doing a retransmit of the [SYN] - this is classic dropped packet behavior.

XP and Ubuntu are similar, except they do a "fast retransmit" - as soon as your [SYN] comes to them without an [ACK] they retransmit the [SYN]

It looks like your program is intercepting the SYN/ACK being sent by the server and simply flipping the ACK field. If you intercepted that packet and made two copies of it, flipping off ACK in one packet and flipping off SYN in the other and transmitting both, the test would be correct.

If you do this and send the ACK before the SYN, you should have a clean four way handshake, where all OS will only send one SYN.

If you zero out the ACK field in the SYN only packet it would ensure that Wireshark doesn't flag a duplicate ACK.

All in all - very interesting - and fun in a geeky way!