The new syntax addresses filtering limitations that have been present for many years.
If you analyze network protocols like IPv4, ICMP, IPv6, ICMPv6, TLS, and GRE, this article is for you.
In packets that contain the same protocol more than once, it was previously impossible to distinguish between these protocols using a display filter. The filter expression limitation has been an issue on the Wireshark bug tracker for a long time - 13 years: Filter expression syntax needs to handle tunneling better.
Why does this matter? Well, maybe you deal with protocols that:
- Tunnel the same protocols multiple times (IP-in-IP)
- Quote other protocols in a reply (ICMP)
- Appear more than once in a single packet
That's where these enhancements make your filtering job easier.
Matching a specific layer in the protocol stack
Not every packet in a PCAP is just a simple Ethernet / IPv4 / TCP packet. Packets can get a lot more complex, including repeating the same protocol twice (tunneling) or repeating the same protocol field twice within the same packet layer.
This can be confusing for people who are trying to read a PCAP, because they might not expect to see a header twice in the same packet.
When troubleshooting network issues, it's important to be able to read a PCAP and understand what's going on. If you see a packet that has two IP headers, it's likely that the packet has been tunneled or quoted. This means that the original packet was encapsulated in a new packet, with a new IP header added on top.
Let's consider ICMP as an example.
ICMP as example
ICMP is an excellent protocol for network analysts because, correctly interpreted, we can diagnose a problem right away. So we might use a display filter like icmp.
In practice, you will notice that we get way too many packets, and you will quickly find yourself narrowing down the filter a bit more. The ICMP Echo Request / Reply messages, better known as Ping, don't indicate a problem and are common background noise in networks; network engineers use them for troubleshooting or monitoring.
So let's enhance the filter. Echo Request has the type field in the ICMP header set to 8, whereas an Echo Reply sets it to 0. So the filter should be
icmp and not (icmp.type == 0 or icmp.type == 8) 
You will notice that this naive filter does not quite work. It removes the ping, yes. But unfortunately, some other ICMP messages that indicate errors are filtered, too. The highlighted packets are destination unreachable packets (icmp.type ==35) that should not be removed from the packet list in this case.
The problem
Suppose a network engineer sends an ICMP Echo Request to a network that can not be routed. In that case, the router with the missing path sends an ICMP Destination Unreachable, Network Unreachable message quoting the original packet.
The router doesn't quote the whole packet, usually just the IPv4 and TCP/UDP/ICMP header. So, in this case, the packet sent by the router contains two ICMP headers. The one of the original packet the network engineer sent and one of the ICMP error message.
Using the naive filter above, we would remove the packet from the packet list, as it contains not that applies to all ICMP layers.
The solution
To get around this limitation, we can rewrite the filter not to use any negation like this.
icmp and ((icmp.type > 0 and icmp.type < 8) or (icmp.type>8)) This filter will include all packets with an error code instead of removing the pings.
While the filter above works fine, it is not intuitive. In Wireshark 4.0, this filter becomes straightforward:
icmp.type#1 != 8 and icmp.type#1 != 0The #1 means that we want to match the first occurrence of the protocol in the packet, not on the quoted messages**.
The Wireshark developers give other examples in the release notes, like IP-over-IP tunneling.
A syntax to match a specific layer in the protocol stack has been added. For example in an IP-over-IP packet
ip.addr#1 == 1.1.1.1matches the outer layer addresses andip.addr#2 == 1.1.1.2matches the inner layer addresses.
Side note
When configuring a column even before version 4.0 of Wireshark, it is possible to target a specific occurrence by using the occurrence option in the column preferences.
The default column configuration displays all occurrences in the Type column.
You can specify the occurrence by right-clicking on the column and selecting Edit column on the top
Now only the first occurrence of the value is displayed in the column as we changed the occurrence value from 0 (show all) to 1.
This behaves slightly differently because the column's occurrence will allow us to select fields within the same layer. Whereas the new 4.0 syntax only will enable us to distinguish between fields in different layers.
IPv4 example
The following screenshots illustrate tunneling and quoting of the IPv4 and IPv6 protocols. We can filter tunneled packets using ip#2 and ipv6#2.
IPv4 in IPv4 GRE Tunnel.
IPv6 quoted in ICMPv6.
The screenshots of these two examples are from the Ultimate PCAP.
TLS may be another case
The TLS dissector decodes multiple TLS layers within the same packet even though they are not quoted or tunneled. In this case, we can specify which layer we want to filter: tls#2. I'm not sure of the practical application of this yet. Maybe TLS fingerprinting; if you have an idea for a good display filter, contact me via @packetsafari.
Trying to filter the same display field within the same layer
The following screenshot has a column with all occurrences of the bit-string field, even within the same layer.
We can have multiple instances of the same field in a column. However, filtering the first two fields with the same field name within the same layer using the new syntax doesn't work, as the #2 syntax refers to the second goose layer and not the second bit_string within the first goose layer. As a result, we don't have a single match.
Universal quantifiers
Additionally, we can now use universal quantifiers to check if all layers or fields fulfill a condition. Before 4.0, any would be the default.
Universal quantifiers "any" and "all" have been added to any relational operator. For example the expression "all tcp.port > 1024" is true if and only if all tcp.port fields match the condition. Previously only the default behaviour to return true if any one field matches was supported.
Arithmetic support for numeric fields
Arithmetic support allows us to do more complex expressions
Arithmetic is supported for numeric fields with the usual operators “+”, “-”, “*”, “/”, and “%”. Arithmetic expressions must be grouped using curly brackets (not parenthesis).
As an example we can use the following expression to find anomalies in packet length.
{tcp.len + tcp.hdr_len + ip.hdr_len} + 14 < frame.len and frame.len gt 64The packet in question does have 1260 bytes TCP payload but in total is 1689 bytes. This doesn't quite add up, as the normal header lengths are: TCP: 20 bytes, IP: 20 bytes, and Ethernet: 14 bytes. To avoid accounting for runts (ethernet frames < 64 bytes ) we exclude those in the filter.
The matching frame is an interesting case, as the sum of the headers and the TCP payload does not match the frame length. What about the other 375 remaining bytes? Well, in this case, we have a corrupted Ethernet frame. Good to know, especially nowadays, because the default profiles in Wireshark don't enable checksum validation anymore.
Summary
To summarize, we see a lot of enhancements in the display filter syntax. But remember that you might need to adapt your display filters (especially coloring rules) to be compatible with version 4.0. For further info, check the official announcement.
ICMP Filters
| Name | Filter |
|---|---|
| All ICMP | icmp |
| Bad ICMP (not version, inprecise) | icmp and not (icmp.type == 0 or icmp.type == 8) |
| Bad ICMP (precise version, pre 4.0) | icmp and ((icmp.type > 0 and icmp.type < 8) or (icmp.type>8)) |
| Bad ICMP (4.0 | icmp.type#1 != 8 and icmp.type#1 != 0 |
| ICMP Errors only | icmp && icmp.type in {3,4,11,12} |
More changes
There are more significant changes that we quote here.
New display filter functions
New display filter functions max(), min() and abs() have been added. Functions can accept expressions as arguments, including other functions. Previously only protocol fields and slices were syntactically valid function arguments.
Negative indexes
It is now possible to index protocol fields from the end using negative indexes. For example the following expression tests the last two bytes of the TCP protocol field: tcp[-2:] == AA:BB. This was a longstanding bug that has been fixed in this release.
Elements must be separated using a comma
Set elements must be separated using a comma, e.g: {1, 2, "foo"}. Using only whitespace as a separator was deprecated in 3.6 and is now a syntax error.
New strict equality operator
A new strict equality operator "===" or "all_eq" has been added. The expression "a === b" is true if and only if all a’s are equal to b. The negation of "===" can now be written as "!==" (any_ne). The aliases "any_eq" for "==" and "all_ne" for "!=" have been added. The operator "~=" is deprecated and will be removed in a future version. Use "!==", which has the same meaning instead.
