In a SYN flood attack, server resources are targeted to consume TCP memory by sending SYN, and after the server responds with a SYN+ACK, force the server to hold state information while waiting for the client’s ACK message. As the server maintains state for half-open connections, server resources are constrained. The server might no longer be able to accept TCP connections, resulting in a DoS condition.
Trellix IPS uses a specific choice of initial TCP number as a defense against SYN flood attacks. The SYN cookie feature is a mechanism to counter SYN flood attacks. This feature is an adjunct to the existing statistical anomaly-based DoS detection. In cases where a DoS attack is already underway and there is no time for learning a long-term profile, Trellix IPS provides the ability for the Sensor to proxy all inbound three-way handshakes.
You can enable SYN cookies for both inbound and outbound traffic.
The Sensor supports a configurable threshold for SYN arrival rate, above which the Sensor begins using SYN cookies to avoid reserving connections during the three-way handshake. The SYN cookie feature has a threshold configured for inbound and outbound. Threshold value indicates the number of half-open connections on the Sensor, that is the connections where the three-way handshakes are not yet complete. If it goes beyond the configured threshold, the Sensor changes to SYN cookie mode. When this happens, the Sensor triggers an alert indicating IP: syncookie proxy <direction> activated. Similarly, when the number of half-open connections comes down below the threshold value, the Sensor triggers another alert indicating IP: syncookie proxy <direction> deactivated. You can view the alert details in the Attack Log.
In SYN cookie mode, the Sensor acts as a TCP proxy, that is it responds to SYN requests received from the client with a SYN+ACK that includes the cookie. The SYN+ACK also contains an Initial Sequence Number (ISN), uniquely generated for every packet, using the information present in the incoming SYN packet, and a secret key. A valid ACK is received only if the connection request is from a legitimate host, after which the Sensor initiates a three-way handshake with the server. If the connection request is not from a legitimate host, a TCP session is not created, and a potential DoS condition is averted. After the three-way handshake is established, the Sensor proxies all further data packets.
The Sensor does not maintain a counter for the number of half-open connections. Hence, there are no capacity limitations for the number of half-open connections.
The Manager provides an interface for the user to enable and disable the SYN cookie feature. It also provides a user the ability to configure threshold values for such SYN cookies. The recommended threshold value is 10% of the supported flows. For example, you can check the number of half-open connections on the Sensor over a normal period of time, and based on that set a threshold that is 2 to 3 times the number of connections. The threshold value assigned is applicable to all interfaces and not to specific interfaces.
Following are few points to be noted when you enable the SYN cookie feature:
Sensors using SYN cookie settings must be in inline mode. If you do not have any ports in inline mode, configure at least one port pair to be inline.
When the Sensor is in SYN cookie mode, it acts as a proxy for the TCP connection between the host and the server. When using VLAN-tagged traffic, based on the placement of the VLAN router, there is a possibility that the packets from the host passes through the same interface pair of the Sensor twice to reach the server. In such cases, a separate sub-interface must be configured for each VLAN to ensure that a packet is not seen more than once.
Packets seen twice by the Sensor.png "GUID-4C551F2D-285C-4248-A19C-6B8C65D5D695-low.png")
Do not enable SYN cookies when passing MPLS traffic through a Sensor. SYN cookie cannot be supported on MPLS traffic because there is no specific MPLS tag to use in the return direction when the Sensor proxy code responds to the SYN packets. When there is a combination of a SPAN port with MPLS traffic and an inline port without MPLS traffic, the SYN cookie still prevents SYN flood as it parses the normal traffic through the inline port pair.
You can view the number of invalid SYN connections received by the Sensor under the show flowsCLI command. For more information on the CLI command, refer to the CLI Commands section.