TCP Exploits by Prabhaker Mateti

CEG 499/699:
Internet Security

College of Engineering & CS
Wright State University
Dayton, Ohio 45435-0001

TCP Exploits


Prabhaker Mateti

Abstract:  In these lectures, we go over the exploits that are possible on the Internet using design weaknesses of TCP rather than software errors.
This work is supported in part by NSF DUE-9951380.

Table of Contents

  1. Educational Objectives
  2. TCP Exploits
    1. Connection Killing by RST
    2. Closing a Connection by FIN
    3. The SYN Flood
    4. Connection Hijacking
  3. Lab Experiment
  4. Acknowledgements
  5. References

Educational Objectives

  1. Understand the role of sequence numbers in TCP.
  2. Understand several TCP exploits.
  3. Recognize design weaknesses of TCP.

TCP Exploits

Faking a UDP packet is relatively simple. The header of a UDP packet is very simple due to its connectionless nature. So, you can fill in any IP address you like and pretend to be somebody else. Faking a TCP connection is much harder.

Sequence Number Prediction

TCP exploits are typically based on IP spoofing, and sequence number prediction.  In establishing a TCP connection, both the server and the client generate a sequence number from which they will start counting the packets transmitted. This sequence number is (should be) generated at random, and should be hard to predict. However, some implementations of the TCP/IP protocol make it rather easy to predict this sequence number. The attacker either sniffs the current SEQ/ACK of the connection, or can algorithmically predict them.  For more on this, read [Steve Bellovin 1989] and [Brecht Claerhout].

It is worth recalling the TCP details regarding opening and closing of a connection between two hosts XX and YY.

The TCP protocol uses a 3-way handshake between the two parties to start a connection.  The client starts the process by sending a SYN packet to synchronize communication. The host then sends one of two answers back to the visitor; a RST packet resets the connection, thus refusing the request for service, while an ACK packet acknowledges the request and awaits the client's first packet.

The TCP flag FIN indicates "no more data from sender".  This flag is used when closing a connection down the normal way.  The receiving host enters the CLOSE­WAIT state and starts the process of gracefully closing the connection. Each end of the connection sends a packet with the FIN flag.  The receiver is expected to acknowledge a received FIN packet by sending a FIN packet.  So four packets are used to close a TCP connection.  

Closing a connection can also be done by using the RST flag. The RST flag of a TCP packet of a sender indicates to the receiver that a reset should occur.  On receiving this packet, the host enters the CLOSED state and frees any resource associated with this instance of the connection. The RST packet is not acknowledged. Any new incoming packet for that connection will be dropped. The receiver accepts the RST packet provided the sequence number is correct.

Below, we show the relevant contents of packets of an attack in progress.   Assume that the "sniper" S (on host ZZ) is on the same subnet as XX. YY need not be on the same subnet.  The data portion of the packets is often irrelevant, so it is dropped.  The SEQ and ACK numbers are in hex.  These were obtained by running a sniffer while the attack was in progress.

Connection Killing by RST

The attacker (on host ZZ) is going to wait for packets in a connection between XX and YY. He will calculate, from YY's ACK packets to XX, the sequence number for XX's packets, and fire off a bogus RST packet from ZZ (faking to be XX) to YY.

Here is an actual attack.   XX on port 1810 and YY on port 23 are connected for a while.

  1. Attacker S waits for a packet to get current SEQ/ACK.  Several packets, as seen by S,  are shown below.  S is ready to send his RST after the very first.

    XX.1810-YY.23 SEQ: 57E1F2A6 ACK: B8BD7679 FLAGS: -AP--- Window: 3400
    YY.23-XX.1810 SEQ: B8BD7679 ACK: 57E1F2A8 FLAGS: -AP--- Window: 2238
    XX.1810-YY.23 SEQ: 57E1F2A8 ACK: B8BD767B FLAGS: -A---- Window: 3400
    YY.23-XX.1810 SEQ: B8BD767B ACK: 57E1F2A8 FLAGS: -AP--- Window: 2238
    XX.1810-YY.23 SEQ: 57E1F2A8 ACK: B8BD7691 FLAGS: -A---- Window: 3400

  2. The attacker sends off to YY.23 his RST packets, with source forged as XX.1810.  His program sent two RST packets. One after the ACK of B8BD7679.   The reply from YY was:

    YY.23-XX.1810 SEQ: B8BD7679 FLAGS: ---R--

    But this packet arrived much later, in this particular attack, at that particular time because of LAN load or whatever.  Connection is still on.  So off goes another RST from the sniper after  B8BD7691.   The reply from YY was:

    YY.23-XX.1810 SEQ: B8BD7691 FLAGS: ---R--

    The second RST from the sniper was the packet that killed the connection.  The first reset packet has been buffered somewhere on our system, because the Ethernet segment was busy when we wanted to send it. This is the 'unexpected thing' that happens some times.  Here the data stream cooled down fast. When it doesn't cool down so fast, the attacker could miss his RST (or the connection will be killed a little later than when we wanted).

Closing a Connection by FIN

The attacker constructs a spoofed FIN packet.  It will have the correct SEQ numbers so that it is accepted by one of the two hosts.  This host would believe the (spoofed) sender did not have any data left. Any packets that may follow would be ignored as bogus. Here is an actual attack

  1. Connection between XX:23 and YY:1072 is running.  Sniper (the attacking programs name) is started on host ZZ  and waits for a packet to take action.

    XX.23-YY.1072 SEQ: 19C6B98B ACK : 69C5473E FLAGS: -AP--- Window: 3400

    plus 2 data bytes
  2. Sniper detected it, and will send a bogus packet (ZZ as YY to XX). Sniper sets the SEQ as ACK of above  packet. Sniper calculates his ACK as: SEQ of above packet + data length of that packet = 19C6B98B + 2 = 19C6B98D.

    YY.1072-XX.23 SEQ: 69C5473E ACK: 19C6B98D FLAGS: -A---F Window: 7C00
  3. Host XX to YY: "Okay, you ended the session, so here is my last data."

    XX.23-YY.1072 SEQ: 19C6B98D ACK: 69C5473E FLAGS: -AP--- Window: 3400 XX.23-YY.1072 SEQ: 19C6B998 ACK: 69C5473F FLAGS: -A---- Window: 3400
    XX.23-YY.1072 SEQ: 19C6B998 ACK: 69C5473F FLAGS: -A---F Window: 3400

  4. Host XX now has flushed its buffer and sent his turn FIN's the connection. Sniper intercepts this packet and now knows the host XX fell for the spoof and the killing was a success!
  5. We impersonated YY, making XX believe we had no further data. But YY doesn't know that and continues to send packets.

    YY.1072-XX.23 SEQ: 69C5473E ACK: 19C6B98D FLAGS: -A---- Window: 3750

    Host XX has that connection closed, and thus thinks the real packets of YY are spoofed (or at least bogus)! So host XX sends some RST packets.

    XX.23-YY.1072 SEQ: 19C6B98D FLAGS: ---R--

  6. This goes on for a couple of packets.

The SYN Flood

When the TCP protocol was designed, there was no limit set on the wait after receiving the SYN.  Once a TCP session begins (i.e., after the next packet comes in) each TCP packet has a timeout. If a packet takes too long, it times out, and eventually, the connection itself will time out, but because of the way the state transition diagrams were specified so many years ago, in this particular state of opening the connection, there are no timeouts.

Suppose ZZ initiates a connection to YY but impersonates as though the packet is from XX. This is the first packet (SYN connection init) of the three-way TCP handshake. YY replies with the second packet of the handshake, and will now wait for response from host XX.  If  the spoofer sends a packet to the server, the server will send an acknowledgement back to the "real" client. If host XX is reachable, there is no easy way for ZZ to prevent the packet from reaching XX. This real client does not know what to do with this ACK.  XX will tell host YY (with a RST) that it did not initiate a connection. YY concludes it received a bogus packet, and will ignore the SYN, and normally nothing more will happen with XX. Remember that YY is wholly unaware of ZZ.  So, the spoofer may try to flood the real client XX with dummy packets at such a rate that its buffers will overflow, and the ACK from the server will be lost.  

But suppose host XX is turned off, ZZ some how observed this, and chose to impersonate XX. Or that the spoofed source IP address that ZZ used looks legitimate but is not assigned to any real host. So if XX is unreachable, YY has a rather long wait for the third packet of the handshake.

Also there is no explicit specification in TCP for the number of simultaneous channels permitted on a given port. The vast majority of systems in the Internet allow upto1024 simultaneous sessions. Suppose an attacker sends 1024 packets to such a computer with the SYN bit set to port 80, say.  spoofing the source address so that each packet has a different source host number.  After receving these 1024 packets, the target system will stop listening to that port. Note that such a packet is about 40 bytes long, so the total amount of traffic is only 40,960 bytes, taking a few seconds on a modem,  or about 1/20 of a second on  T1.  That is, one second's worth of packets results in a system hang.  Many systems probably run out of internal space to store the uncompleted connections before the second passes and crash. 

This TCP design bug cannot be fixed at the router or using some add-on product. The only real fix is to modify operating systems so that it times out on uncompleted TCP sessions.  As of June 2000, we do know of such OS fixes for all major OS including Linux and Windows NT.

Connection Hijacking

YY trusts the packets from XX because of its correct SEQ/ACK numbers. So if there was a way to mess up XX's SEQ/ACK, YY would stop believing XX's real packets. We could then impersonate to be XX, but using correct SEQ/ACK numbers from the perspective of YY. This results in ZZ hijacking the connection: host XX is confused, YY thinks nothing is wrong as ZZ sends 'correct' data to YY. We can confuse XX's SEQ/ACK numbers as seen by YY by simply inserting a data packet into the stream at the right moment (ZZ as XX->YY). YY would accept this data, and update ACK numbers. XX would continue to send it's old SEQ numbers, as it's unaware of our spoofed data.  Here is an actual attack.
  1. Attacker examines with a sniffer, and sees the port number 2035, and also that the user is in an interactive shell via telnet to YY. He starts hijack (the programs name) on host ZZ as 'hijack XX 2035 YY'. The following packet from XX->YY is detected.

    XX.1040-YY.23 SEQ: 5C8223EA ACK: C34A67F6 FLAGS: -AP--- Window: 7C00 XX.1040-YY.23 45 E 00 . 00 . 29 ) CA . F3 . 40 @ 00 . 40 @ 06 . C5 . 0E . 9D . C1 . 45 E 3F ? 9D . C1 . 2A * 0B . 04 . 10 . 00 . 17 . 5C \ 82 . 23 # EA . C3 . 4A J 67 g F6 . 50 P 18 . 7C | 00 . 6D m 29 ) 00 . 00 . 6C l ~~~~

  2. Server YY echoes that data byte (happens to be a  'l')

    YY.23-XX.1040 SEQ: C34A67F6 ACK: 5C8223EB FLAGS: -AP--- Window: 2238 YY.23-XX.1040; 45 E 00 . 00 . 29 ) B5 . BD . 40 @ 00 . FC . 06 . 1E . 44 D 9D . C1 . 2A * 0B .; 9D . C1 . 45 E 3F ? 00 . 17 . 04 . 10 . C3 . 4A J 67 g F6 . 5C \ 82 . 23 # EB .; 50 P 18 . 22 " 38 8 C6 . F0 . 00 . 00 . 6C l

  3. XX ACKnowledges that:

    XX.1040-YY.23 SEQ: 5C8223EB ACK: C34A67F7 FLAGS: -A---- Window: 7C00

    (data is irrelevant)
  4. Now the hijack program on ZZ impersonates XX with further data to YY.  The hijacker sends some Backspace (0x08) characters and some Enter (0x0A) characters to clean up the command line of the user. This may generate some error messages back from the shell. But hijack program should  handle that too.

    XX.1040-YY.23 SEQ: 5C8223EB ACK: C34A67F6 FLAGS: -AP--- Window: 7C00 XX.1040-YY.23 45 E 00 . 00 . 32 2 31 1 01 . 00 . 00 . 45 E 06 . 99 . F8 . 9D . C1 . 45 E 3F ? 9D . C1 . 2A * 0B . 04 . 10 . 00 . 17 . 5C \ 82 . 23 # EB . C3 . 4A J 67 g F6 . 50 P 18 . 7C | 00 . AE . F5 . 00 . 00 . 08 . 08 . 08 . 08 . 08 . 08 . 08 . 08 . 0A . 0A .

  5. The following is the echo of the spoofed data from the server YY.

    YY.23-XX.1040 SEQ: C34A67F7 ACK: 5C8223F5 FLAGS: -AP--- Window: 2238 YY.23-XX.1040 45 E 00 . 00 . 3C < B5 . BE . 40 @ 00 . FC . 06 . 1E . 30 0 9D . C1 . 2A * 0B . 9D . C1 . 45 E 3F ? 00 . 17 . 04 . 10 . C3 . 4A J 67 g F7 . 5C \ 82 . 23 # F5 . 50 P 18 . 22 " 38 8 26 & 7C | 00 . 00 . 5E ^ 48 H 5E ^ 48 H 5E ^ 48 H 5E ^ 48 H 5E ^ 48 H 5E ^ 48 H 5E ^ 48 H 5E ^ 48 H 0D . 0A . 0D . 0A .

    Note that ACK = 5C8223F5 = 5C8223EB + 0A. This is how the hijacker detects success.  At this point the connection is controlled by the hijacker, and XX's SEQ/ACK numbers are completely messed up with respect to YY. At this point the real telnet client's session hangs. Most people ignore this as a network problem and re-login after a while, accepting the disconnection as Murphy's law. Indeed, connection hangs do happen without any spoofing involved because of other reasons.

  6. Hijack will now send the strings of text it wants to be executed. Each time a packet arrives at a host out of sequence, the host should answer it with correct SEQ/ACK. This provides the hijacker with the certainty that a lot of packets are going to be send with correct (and not changing) SEQ/ACK numbers.

    XX.1040-YY.23 SEQ: 5C8223EB ACK: C34A67F7 FLAGS: -AP--- Window: 7C00 YY.23-XX.1040 SEQ: C34A680B ACK: 5C8223F5 FLAGS: -A---- Window: 2238

  7. We are back on track (or at least hijack is, because this is going very fast). The hijacker boasts with a shell command. 'echo "echo Hijacked" >> ~/.profile'

    XX.1040-YY.23 SEQ: 5C8223F5 ACK: C34A680B FLAGS: -AP--- Window: 7C00 XX-YY.23 45 E 00 . 00 . 4D M 31 1 01 . 00 . 00 . 45 E 06 . 99 . DD . 9D . C1 . 45 E 3F ? 9D . C1 . 2A * 0B . 04 . 10 . 00 . 17 . 5C \ 82 . 23 # F5 . C3 . 4A J 68 h 0B . 50 P 18 . 7C | 00 . 5A Z B6 . 00 . 00 . 65 e 63 c 68 h 6F o 20 22 " 65 e 63 c 68 h 6F o 20 48 H 41 A 43 C 4B K 45 E 44 D 22 " 20 3E > 3E > 24 $ 48 H 4F O 4D M 45 E 2F / 2E . 70 p 72 r 6F o 66 f 69 i 6C l 65 e 0A . 00 .

  8. Now we wait for this data to be confirmed. ACK = 5C8223F5 + 0x25 = 5C82241A.

    YY.23-XX.1040 SEQ : C34A680B ACK: 5C82241A FLAGS: -AP--- Window: 2238 YY.23-XX.1040 ... data ...
  9. The connection between ZZ and XX contnues. Now the hijacker can execute more commands.  The hijack program must stay on track of SEQ/ACK, and eventually FINish the connection.

Lab Experiment

All work should be carried out in Operating Systems and Internet Security (OSIS) Lab, 429 Russ.   Use any of the PCs numbered 19 to 30.  No other WSU facilities are allowed. 

Objective: Run the TCP exploit programs (sniper-fin, sniper-rst, and hijack) based on the above with a goal of learning how the attack actually works in detail.  Observe the details of the packets being transmitted using a sniffer.  Capture sequences of packets to and fro and explain how the specific attack worked. Examine the source code of sniper-fin, sniper-rst, and hijack programs both because you need to learn to invoke them, and because a later assignment will use the source code.

  1. Download file ipspoof.tgz. Untar it into your own Zip disk, and build the three programs sniper-fin, sniper-rst, and hijack using the Makefile.
  2. Setup a network of at least three PCs, P0, P1, P2 with P1 as a router as in the first lab experiment.
  3. You will be running  a sniffer, and sniper-fin, sniper-rst, and hijack programs on P1.
  4. Verify that the sniffer is running properly.
  5. Telnet from P0 to P2.  Invoke sniper-fin on P1 to kill this connection.  Observe and copy-and-paste into a text file the sniffer output as the connection gets killed.  Add your annotations and observations to this text file.
  6. Do step 6 but with sniper-rst.
  7. Do step 6 but with hijack.  Append the line of "evil data" to the text file journal you are keeping.
  8. Edit the journal file so that it is a lab report, and turn it in.


The material in this lecture is based on articles by Brecht Claerhout and others appearing in the Phrack magazine over the years.


  1. Steve Bellovin, Security Problems in the TCP/IP Protocol Suite, Computer Communication Review, Vol. 19, no. 2 (April 1989) pages 32-48; [Local copy.]  Required Reading.
  2. Brecht Claerhout, A short overview of IP spoofing: Parts I and II,  1996. The papers are found in many web archives, but do not seem to be published in any formal way.   ipspoof.tgz is a collection of files from these papers. Reference.
  3. Daemon9 / route / infinity, "Project Neptune", Phrack Magazine, Volume Seven, Issue Forty-Eight, File 13 of 18, July 1996.  Explains, in detail, SYN flooding. Reference only.
  4. Laurent Joncheray, "Simple Active Attack Against TCP,"   5th USENIX UNIX Security Symposium, pp 7-19, June 5-7, 1995,  [Local copy] Required Reading.
  5. Biswaroop Guha, Biswanath Mukherjee, "Network Security via Reverse Engineering of TCP Code: Vulnerability Analysis and Proposed Solutions". Proc. of the IEEE Infocom'96 , San Francisco, CA, March 1996, pp. 603-610. [Abstract][PDF][PostScript].  Reference only.
  6. Prabhaker Mateti, IP and ICMP Exploits, June 2000.  A lecture from a course on Internet Security. . Explains IP spoofing.  Required Reading.
  7. Prabhaker Mateti, TCP/IP Refresher, June 2000.  A lecture from a course on Internet Security. .  Required Reading.
  8. Robert T. Morris, A Weakness in the 4.2BSD Unix TCP/IP Software.
  9. Mike D. Schiffman,The Libnet Reference Manual,  v.02 / 10.27.99, manual/lrm.html  TCP/IP library routines.  Become familiar with it.
04/30/01 02:24:01 PM
Open Content Copyright © 2000