Below is a C# program demonstrating the problem.
The server starts listening on a socket. The client connects to the server, sends a message, uses Shutdown(SocketShutdown.Send) to close its send half of the connection to let the server know where the end of the message is, and waits for a response from the server. The server reads the message, does some lengthy computation (simulated here with a sleep call), sends a message to the client, and closes the connection.
On Windows, the client’s Receive call always fails after exactly 2 minutes with “A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond” even though the timeout is set to infinite.
If I run the program in Linux with Mono, the timeout does not occur even if I set the “lengthy operation” to 10 minutes, but it happens in Windows whether I run it with Mono or .NET. If I set the timeout to 1 second, it times out after 1 second. In other words, it times out in the timeout I set or 2 minutes, whichever is less.
A similar sample program in which the server sends a message to the client, with no message from client to server and no half-close, works as expected with no timeout.
I can get around this by modifying my protocol to use some other method of indicating to the server when a message is complete (perhaps prefixing the message with the length of the message). But I want to know what’s going on here. Why does Socket.Receive time out on a half-closed connection when the timeout is set to infinite?
From what I understand, a connection with only its send half closed should be able to continue receiving data indefinitely. It seems unlikely that there would be a bug in such a fundamental part of Windows. Am I doing something wrong?
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Net.Sockets;
using System.Net;
using System.Threading.Tasks;
using System.Diagnostics;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
// Start server thread
Thread serverThread = new Thread(ServerStart);
serverThread.IsBackground = true;
serverThread.Start();
// Give the server some time to start listening
Thread.Sleep(2000);
ClientStart();
}
static int PortNumber = 8181;
static void ServerStart()
{
TcpListener listener = new TcpListener(new IPEndPoint(IPAddress.Any, PortNumber));
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
Task connectionHandlerTask = new Task(ConnectionEntryPoint, client);
connectionHandlerTask.Start();
}
listener.Stop();
}
static void ConnectionEntryPoint(object clientObj)
{
using (TcpClient client = (TcpClient)clientObj)
using (NetworkStream stream = client.GetStream())
{
// Read from client until client closes its send half.
byte[] requestBytes = new byte[65536];
int bufferPos = 0;
int lastReadSize = -1;
while (lastReadSize != 0)
{
lastReadSize = stream.Read(requestBytes, bufferPos, 65536 - bufferPos);
bufferPos += lastReadSize;
}
client.Client.Shutdown(SocketShutdown.Receive);
string message = Encoding.UTF8.GetString(requestBytes, 0, bufferPos);
// Sleep for 2 minutes, 30 seconds to simulate a long-running calculation, then echo the client's message back
byte[] responseBytes = Encoding.UTF8.GetBytes(message);
Console.WriteLine("Waiting 2 minutes 30 seconds.");
Thread.Sleep(150000);
try
{
stream.Write(responseBytes, 0, responseBytes.Length);
}
catch (SocketException ex)
{
Console.WriteLine("Socket exception in server: {0}", ex.Message);
}
}
}
static void ClientStart()
{
using (Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
// Set receive timeout to infinite.
socket.ReceiveTimeout = -1;
// Connect to server
socket.Connect(IPAddress.Loopback, PortNumber);
// Send a message to the server, then close the send half of the client's connection
// to let the server know it has the entire message.
string requestMessage = "Hello";
byte[] requestBytes = Encoding.UTF8.GetBytes(requestMessage);
socket.Send(requestBytes);
socket.Shutdown(SocketShutdown.Send);
// Read the server's response. The response is done when the server closes the connection.
byte[] responseBytes = new byte[65536];
int bufferPos = 0;
int lastReadSize = -1;
Stopwatch timer = Stopwatch.StartNew();
try
{
while (lastReadSize != 0)
{
lastReadSize = socket.Receive(responseBytes, bufferPos, 65536 - bufferPos, SocketFlags.None);
bufferPos += lastReadSize;
}
string responseMessage = Encoding.UTF8.GetString(responseBytes, 0, bufferPos);
Console.WriteLine(responseMessage);
}
catch (SocketException ex)
{
// Timeout always occurs after 2 minutes. Why?
timer.Stop();
Console.WriteLine("Socket exception in client after {0}: {1}", timer.Elapsed, ex.Message);
}
}
}
}
}
The following program prefixes messages with a 4-byte message length rather than using socket.Shutdown(SocketShutdown.Send) to signal end of message. The timeout does not occur in this program.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net.Sockets;
using System.Net;
using System.Threading.Tasks;
using System.Diagnostics;
using System.Threading;
namespace WithoutShutdown
{
class Program
{
static void Main(string[] args)
{
// Start server thread
Thread serverThread = new Thread(ServerStart);
serverThread.IsBackground = true;
serverThread.Start();
// Give the server some time to start listening
Thread.Sleep(2000);
ClientStart();
}
static int PortNumber = 8181;
static void ServerStart()
{
TcpListener listener = new TcpListener(new IPEndPoint(IPAddress.Any, PortNumber));
listener.Start();
while (true)
{
TcpClient client = listener.AcceptTcpClient();
Task connectionHandlerTask = new Task(ConnectionEntryPoint, client);
connectionHandlerTask.Start();
}
listener.Stop();
}
static void SendMessage(Socket socket, byte[] message)
{
// Send a 4-byte message length followed by the message itself
int messageLength = message.Length;
byte[] messageLengthBytes = BitConverter.GetBytes(messageLength);
socket.Send(messageLengthBytes);
socket.Send(message);
}
static byte[] ReceiveMessage(Socket socket)
{
// Read 4-byte message length from the client
byte[] messageLengthBytes = new byte[4];
int bufferPos = 0;
int lastReadSize = -1;
while (bufferPos < 4)
{
lastReadSize = socket.Receive(messageLengthBytes, bufferPos, 4 - bufferPos, SocketFlags.None);
bufferPos += lastReadSize;
}
int messageLength = BitConverter.ToInt32(messageLengthBytes, 0);
// Read the message
byte[] messageBytes = new byte[messageLength];
bufferPos = 0;
lastReadSize = -1;
while (bufferPos < messageLength)
{
lastReadSize = socket.Receive(messageBytes, bufferPos, messageLength - bufferPos, SocketFlags.None);
bufferPos += lastReadSize;
}
return messageBytes;
}
static void ConnectionEntryPoint(object clientObj)
{
using (TcpClient client = (TcpClient)clientObj)
{
byte[] requestBytes = ReceiveMessage(client.Client);
string message = Encoding.UTF8.GetString(requestBytes);
// Sleep for 2 minutes, 30 seconds to simulate a long-running calculation, then echo the client's message back
byte[] responseBytes = Encoding.UTF8.GetBytes(message);
Console.WriteLine("Waiting 2 minutes 30 seconds.");
Thread.Sleep(150000);
try
{
SendMessage(client.Client, responseBytes);
}
catch (SocketException ex)
{
Console.WriteLine("Socket exception in server: {0}", ex.Message);
}
}
}
static void ClientStart()
{
using (Socket socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp))
{
// Set receive timeout to infinite.
socket.ReceiveTimeout = -1;
// Connect to server
socket.Connect(IPAddress.Loopback, PortNumber);
// Send a message to the server
string requestMessage = "Hello";
byte[] requestBytes = Encoding.UTF8.GetBytes(requestMessage);
SendMessage(socket, requestBytes);
// Read the server's response.
Stopwatch timer = Stopwatch.StartNew();
try
{
byte[] responseBytes = ReceiveMessage(socket);
string responseMessage = Encoding.UTF8.GetString(responseBytes);
Console.WriteLine(responseMessage);
}
catch (SocketException ex)
{
// Timeout does not occur in this program because it does not call socket.Shutdown(SocketShutdown.Send)
timer.Stop();
Console.WriteLine("Socket exception in client after {0}: {1}", timer.Elapsed, ex.Message);
}
}
}
}
}
This behavior is by design. When the client has closed its half on the connection and the server acknowledges the close, the client is in the FIN_WAIT_2 state, waiting for the server to close the connection. http://support.microsoft.com/kb/923200 states that there is a FIN_WAIT_2 timeout of 2 minutes. If no data is received in a 2 minute period when a connection is in the FIN_WAIT_2 state, the client forcibly closes the connection (with a RST).
This old Apache article suggests the reason for the timeout: malicious or misbehaving applications could keep the other end of the connection in FIN_WAIT_2 indefinitely by never closing their end of the connection and thus tie up operating system resources.
Linux apparently has a timeout as well You can check the value with
$ cat /proc/sys/net/ipv4/tcp_fin_timeout
I’m not sure why the timeout wasn’t occurring for me on Linux. Perhaps because it was a loopback connection and therefore DoS attacks are not a concern or loopback connections use different code that does not use the tcp_fin_timeout setting?
Bottom line: The operating system has a good reason for making the connection time out. Avoid using Shutdown as an application-layer signalling mechanism and use an actual application-layer method instead.