Are you tired of digging through lists of thousands of packets in Wireshark to troubleshoot a connection problem?
Why don't you use graphs instead? They are a powerful tool in a packet analyst's craft. Having a basic understanding of how Wireshark visualizes packet information in a TCP graph, you can take shortcuts in the analysis workflow and avoid spending many hours looking for patterns that are barely visible in the limited perspective of the packet list.
Wireshark offers a couple of graphs for TCP analysis: RTT, throughput, window scaling, and the time sequence graphs.
This article will teach you how to interpret TCP connections using the TCP time-sequence graphs. There are two versions of the time sequence graph, the more straightforward Stevens graph** and the more advanced TCP trace graph. In this PacketSafari article, we will start from the basics, work our way to more advanced concepts, and provide clear patterns to look for when troubleshooting a TCP connection.
TCP Graphs Basics
All TCP graphs, whether Stevens or the trace flavor, represent one TCP connection in one direction. To open a graph, you must select a packet belonging** to the relevant connection in the packet list. Selecting any packet in the packet list is sufficient.
The TCP Graph (Stevens)
Next, open one of the TCP diagrams and plot the connection. Click Statistics, TCP Stream Graphs, and TCP Graph (Stevens).
A new window opens with the Stevens TCP graph. The graph's direction is in the upper part of the window. In this case, from 131.188.12.211:37674 to 192.168.1.178:61127. The x-axis is the relative time since the beginning of the connection in seconds. The unit of the y-axis is sequence numbers, hence the name time-sequence diagram. Since numbers count bytes sent or transmitted, sequence numbers plotted over time show a connection's throughput (data per second) in one direction. In the example, we see a reasonably stable data transfer from 131.188.12.211 to 192.168.1.178.
By switching the direction of the TCP graph using the Switch Direction button at the bottom right of the TCP graph window, you can examine the transmitted data in the reverse direction (192.168.1.178 to 131.188.12.211).
Not much to see here, just a flat line. The IP address 192.168.1.178 barely transmits any data. The sequence number does not increase at all. This is a typical case of a download without bidirectional data transmission.
Getting to know more advanced graphs
Using the Stevens graph, you can troubleshoot the performance of a TCP connection. However, taking the time to learn about the more advanced TCP trace graph is worthwhile to ee the symptoms and the causes of a bad connection.
Side-by-side, the Stevens and trace graphs look similar at first glance.
Zooming in, some differences become apparent. The trace graph does not only plot the blue sequence numbers of packets. We can see a red, yellow and, green line.
By clicking on "switch direction", the difference becomes more apparent. The scale of the y-axis is entirely different.
But why? How does the TCP trace graph work? How can it help us to find the cause of TCP problems?
Let's find out...
TCP Graph (Trace) and sequence numbers
As in the simpler TCP Stevens graph, the TCP sequence numbers are represented over time. In the TCP graph, you see this as a blue line. Zooming in, it turns out that the blue line is composed of many small vertical lines. These are the individual TCP segments.
The line length tells you how many bytes are carried within one packet.
Sequence numbers enable many features of TCP, such as the reliable transmission of data, detection of packet loss, and flow control. Other protocols such as IPv4, TLS, RTP, and QUIC also use sequence numbers when fragmenting data. However, Wireshark only has ready-made graphs for TCP.
Sequence numbers are direction-dependent
Sequence numbers are a counter of the bytes transmitted. Since two communication partners are involved in each TCP connection and they each send data, there are two sequence number counters for each TCP connection: one for the client side and one for the server side.
The TCP graph plots a direction in a window. However, you can change direction and analyze the other side. Here you see how a server sends data (relatively) continuously to a web browser.
After clicking on Switch Direction at the bottom right, you will see the receiving side. Since it does not send much data, its sequence number remains constant.
Sequence numbers over time
With each byte sent, the sequence number is incremented. The TCP graph represents these sequence numbers on the y-axis and records them over time on the y-axis. Thus, the graph gives you insight into the throughput of a connection. If the graph increases steadily, the connection runs well.
If the slope of the curve increases, the throughput increases. At lower gradients, the throughput decreases.
Here you see a data transfer within a LAN. The connection does not have high throughput. We transfer 5 megabytes within 30 seconds. But the transfer is steady. This suggests that the transfer is bandwidth-bound. The connection performs well within the environment, albeit slowly, for today's standards.
Comparing this curve with the browser download via the Internet, the influence of the WAN in the graph is apparent. While the connection is fast because more bandwidth is available, it is not as stable as the local connection.
Both connections are unproblematic insofar as the user does not complain about the performance of the downloads. Both the Stevens and the Trace Graph visualize this behavior.
For comparison, here is a problematic connection. No data is transferred for a short time at the first marked point. At the second point, the throughput of the transmission decreases. The slope of the curve flattens out.
The trace graphs allow us to examine TCP stack properties. For example, during the TCP Slow Start, the transmitter constantly doubles its transmission rate. This algorithm results in an exponential curve in the graph.
Remember: If the blue line does not rise steadily, this indicates a problem with the connection (packet loss, out of order, retransmission, almost retransmission).
Dark yellow ACK line
One line that is not found in the Stevens graph is the dark yellow ACK line. Here the ACK number of the other side, the receiver side, is displayed**. This allows you to see when a sequence number has been confirmed**.
If the green line is too far apart from the blue line, packets will not be acknowledged in time. If the green line does not rise, the transmitter waits for acknowledgments. This is symptomatic of disconnections or performance issues.
Out of Order or Retransmission
Here you see how the blue packets skip a sequence number and send it a moment later. This can be either an out-of-order or retransmission.
DUP ACKs can be recognized in the TCP trace graph because the ACK line does not increase, and small strokes below the line are visible. DUP acks are used in packet loss to trigger fast retransmissions. Too many DUP acks can be an indication of too many packet losses.
Selective ACKs
Selective ACKs are shown in red. Here you see how SACKs are sent for a short time, but within 100ms, the problem is solved by retransmitting the requested bytes.
Green Receive Window line
The distance between the current sequence number (blue packets) and the Receive Windows (top green line) of the opposite side is shown in the graph. The distance between the two indicates how many bytes may still be sent by the sender before confirmation of the transmitted bytes must arrive. You see how the receiver enlarges the Receive Window, possibly because he could pass received packets on to the application and now has more memory in the kernel for this connection.
You see a download that evolves in bursts. The receiver immediately confirms the bursts of data packets. When zooming in closer, such a bursty behavior is often seen during downloads. The closer the bursts are together, the smoother the download flows.
The top line shows the recipient's Receive Window. As we do not have many bytes in flight (unacknowledged bytes), the distance to the blue packets should always be quite large. This allows the transmitter to continue sending. If the packet bursts in blue approach the Receive Window line, we may soon receive a window full message and the sender may not transmit any more packets. The user would complain about a stalled or even aborted download. At the highlighted spot, we have such a case.
In the positive case, the Receive Window always increases when packets are confirmed. Thus, we get a stair-like graph for the Receive Window line. With a "bursty" transmission behavior, this becomes visible at low resolution.
Analyzing bytes in flight
In addition to the information already discussed, the bytes in flight are shown in the screenshot below. The bytes in flight are the distance of the blue packets to the dark yellow line (the ACK line). As soon as the ACK line rises, packets are confirmed. You can see nicely how the ACKing of packets provides more space in the upper Receive Window line.
Packets outside the sequence number order - Out of Orders
You recognize out-of-orders packets by the fact that sequence numbers are skipped at first, but then a short time later, a blue packet with the previous sequence numbers arrives. So the blue lines do not rise steadily, but there is a gap that is then filled later. Too many out-of-orders can cause problems, especially if it takes advantage of the Receive Window.
TCP Analysis Flags - Cheatsheet
| Explaination | Analysis | Display filter |
|---|---|---|
| A segment that is not in the trace has been acknowledged | Indicates that not all packets have been recorded or a route has been flapped | tcp.analysis.ack_lost_segment |
| The receiver tries to trigger Fast Retransmissions | Case dependent (few usually no problem) | tcp.analysis.duplicate_ack |
| The sender sends a Keep Alive | Good, the connection continues | tcp.analysis.keep_alive |
| The receiver sends a Window Update | Good, the sender is allowed to send again | tcp.analysis.window_update |
| The receive window is completely used by bytes in flight. | Can be bad for performance | tcp.analysis.window_full |
| The receive window is 0, the sender is not allowed to send anymore. | Indication of performance problem or transition fast to slow link. | tcp.analysis.zero_window |
| Retransmission of a packet for which there is already a Seq # in the trace | If the sender sends too many, it might congest the network needlessly | tcp.analysis.spurious_retransmission |
| Segment lost, either really or just not recorded in the trace | Case dependent (few usually no problem) | tcp.analysis.lost_segment |
| Not seen Seq # was retransmitted | Case dependent (few usually no problem) | tcp.analysis.retransmission or tcp.analysis.fast_retransmission |
| Packets are in the wrong Seq # order | Few out of orders are no problem, if the out of orders become larger than the receive window, there will be problems | tcp.analysis.out_of_order |
| A port pair is reused | Case dependent, if ports are used too often it indicates port exhaustion. | tcp.analysis.reused_ports |
TCP Field Hints
| Explanation | Display filter |
|---|---|
| Long RTT time for ACKs | tcp.analysis.ack_rtt > 1.0 |
| No window scaling is used | tcp.window_size_scalefactor==-2 |
| Immediate reset | tcp.flags.reset == 1 and tcp.seq == 1 |
| Zero Window (without resets) | tcp.window_size == 0 && tcp.flags.reset == 0 |
| TCP segment error | tcp.segment.error || tcp.segment.overlap || tcp.segment.toolongfragment || tcp.segment.overlap.conflict |
| TCP handshake without SACK and MSS | tcp.flags.syn == 1 and tcp.hdr_len < 28 |
| D-SACKs | tcp.ack > tcp.options.sack_le and tcp.options.sack.count == 1 |
| Many NOPs | tcp.options contains 01:01:01:01 |
