Question

I’m writing a script that opens a text file and loops through each line (pausing a few seconds between each one). For each line, it opens a transient client socket connection and sends the text to a host server. A host response may or may not come; it doesn’t matter either way.

I’ve already bumped into the Python socket limitation where you can’t reconnect with an existing socket object (because doing so triggers the exception EBADF, 'Bad file descriptor'). So I’m creating a new socket instance for each transient connection. The trick then of course becomes how to avoid a memory leak.

The way I’ve approached this is to push the whole portion of creating, using, and closing the socket to a function — relying on Python’s garbage collection to remove each instance after I’m done with it:

    import socket,select,time

    def transientConnect(host,port,sendData):
        response = ''
        sendSocket = socket.socket()
        sendSocket.connect((serverHost,serverPort))
        sendSocket.send(line)
        gotData = select.select([sendSocket],[],[],2)
        if (gotData[0]):response = sendSocket.recv(65535)
        sendSocket.close()
        return response

    scriptLines = open('testScript.txt','r').readlines()
    serverHost  = '127.0.0.1'
    serverPort  = 15004
    for line in scriptLines:
        response = transientConnect(serverHost,serverPort,line)
        print(response)
        time.sleep(3.0)

My questions: (1) Will this approach work to avert any memory leaks? (2) Is there a more direct way to make sure each instance is eliminated after I’m finished with it?

Answer 1

First off, it is normal to only use a socket for a single exchange. See the socket HOWTO.

One of the nice things about python is that in general you don’t have to worry about garbage collection. And you shouldn’t unless you have real memory use problems.

From this webpage, keep in mind that:

“Python won’t clean up an object when it goes out of scope. It will clean it up when the last reference to it has gone out of scope.”

So if the socket created inside the function isn’t referenced elsewhere, it should go out of scope and be deallocated (but not gc-ed). What follows is probably specific to cpython. Read the documentation of gc.set_threshold() for an idea how garbage collection works in cpython. Especially:

“When the number of allocations minus the number of deallocations exceeds threshold0, collection starts.”

The standard values for the thresholds (in cpython) are:

In [2]: gc.get_threshold()
Out[2]: (700, 10, 10)

So there would heva to be a fair number of allocations before you get a gc run. You can force garbage collection by running gc.collect().