Question

I have client-server architecture build in python, unfortunately the original design was made that each request to server is represented by one TCP connection and I have to send requests in large groups (20 000+) and sometimes there occurs socket error #10055.

I’ve already found out how to handle it in python:

>>> errno.errorcode[10055]
'WSAENOBUFS'
>>> errno.WSAENOBUFS
10055

And build a code that is able to handle that error and reconnect (of course with little time delay to give server time to do whatever it has to do):

class MyConnect:
    # __init__ and send are not important here

    def __enter__(self):
        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

        # Try several reconnects
        for i in range(0,100):
            try:
                self.sock.connect((self.address, self.port))
                break

            except socket.error as e:
                if e.errno == errno.WSAENOBUFS:
                    time.sleep(1)
                else:
                    raise

        return self

    def __exit__(self, type, value, traceback):
        self.sock.close()

# Pseudocode
for i in range(0,20000):
    with MyConnect(ip,port) as c:
        c.send(i)

My questions are:

• is there any “good practice” way to do this?
• is e.errno == errno.WSAENOBUFS multi-platform? If not so, how to make it multi-platform?

_{Note: I’ve tested in just on Windows yet, I need it to work on Linux too.}

Answer 1

You are clogging your TCP stack with outgoing data and all the connection establishment and termination packets.

If you have to stick to this design, then force each connection to linger until its data has been successfully sent. That is to say, that by default, close() on the socket returns immediately and further delivery attempts and connection tear-down happen “in the background”. You can see that doing so over 20000+ times in a tight loop can easily overwhelm the OS network stack.

The following will force your socket close() to hang on for up to 10 seconds trying to deliver the data:

import struct
s.setsockopt(socket.SOL_SOCKET, socket.SO_LINGER, struct.pack('ii', 1, 10))

Note that this is not the same as Python socket.sendall() – that one just passes all the bytes to the kernel .

Hope this helps.