Question

If my server implements SYN Cookies to avoid DoS attacks, but an attacker knows the server utilizes SYN Cookies, is it possible that they could create half/fully open connection simply by sending an ACK?

I know that SYN Cookies use an algorithm to create the unique initial connection, and if the attackers handshake is incomplete the SYN is dropped and can only be recreated by receiving a valid SYN-ACK.

But could an attacker still somehow manage?

Answer 1

No, it should not be possible for an attacker to know what the SYN initial sequence value is in order to complete the TCP 3 way handshake. Further more it is not possible for any tcp port to be in a half-open state when they are using SYN Cookies. The answer is rooted in cryptography.

An implementation of SYN Cookies could use a Symmetric Cipher to generate sequence id’s. For instance, when the machine boots it will generate a random secret key to be used for all TCP sequence id’s. When the machine receives and incoming SYN packet to an open port it will generate a sequence id by encrypting the the Server’s IP address, the Client’s IP address and the port numbers being used. The server doesn’t need to keep track of the SYN initial sequence id it sent out, so it doesn’t have a per-client state and this idea of a “half-open” tcp socket doesn’t really apply (at-least in terms of DoS). Now when the client sends back its SYN-ACK packet it needs to contain the SYN initial sequence ID. When the server gets this initial sequence id back from the client in a SYN-ACK packet it can work backwards, by encrypting the the Server’s IP address, the Client’s IP address and the port numbers being used.