Question

I am developing a client-server application where whenever a new client connects to the server, the server spawns a new process using the multiprocessing module. Its target function is a function where it takes the socket and does I/O. The problem I have is once the TCP connection is closed between the client and the process on the server how/where do I put the .join() function call to end the child process? Also do I need to do any waitpid in the parent process like in C?

Server code:

def new_client(conn_socket):
    while True:
        message = conn_socket.recv(BUFFER_SIZE)
        conn_socket.send(message)  
        #just echo the message
        #how to check to see if the TCP connection is still alive?
        #put the .join() here??



def main():
    #create the socket
    server_socket = socket(AF_INET,SOCK_STREAM)

    #bind the socket to the local ip address on a specific port and listen
    server_port = 12000                               
    server_socket.bind(('',server_port))
    server_socket.listen(1)

    #enter in a loop to accept client connections
    while True:
        connection_socket, client_address = server_socket.accept()       
        #create a new process with the new connection_socket
        new_process = Process(target = new_client, args = (connection_socket,))
        new_process.start()
        #put the .join() here or what??

if __name__ == '__main__':
    main()

Also for this setup would it be more beneficial to use threads in the thread module or stay with processes? The server code is being developed for heavy usage on a server with “average” specs(how to optimize this setup).

Answer 1

You need to check the return value of recv. If it returns zero then the connection is closed nicely, if negative then there was an error.

And the join call should be in the process that creates the sub-process. However, be carefull because join without argument will block the calling process until the sub-process is done. Put the processes in a list, and on regular intervals call join with a small timeout.

Edit: Simplest is to add, at the end of the infinite accept loop, to iterate over the list of processes, and check if it’s is_alive. If not then call join and remove it from the list.

Something like:

all_processes = []
while True:
    connection_socket, client_address = server_socket.accept()       
    #create a new process with the new connection_socket
    new_process = Process(target = new_client, args = (connection_socket,))
    new_process.start()

    # Add process to our list
    all_processes.append(new_process)

    # Join all dead processes
    for proc in all_processes:
        if not proc.is_alive():
            proc.join()
    # And remove them from the list
    all_processes = [proc for proc in all_processes if proc.is_alive()]

Note that purging of old processes will only happen if we get a new connection. This can take some time, depending on if you get new connections often or not. You could make the listening socket non-blocking and use e.g. select with a timeout to know if there are new connections or not, and the purging will happen at more regular intervals even if there are no new connections.