Question

I am building a web server in python using the select() function – I/O multiplexing. I am able to connect to multiple clients which in my case are web browsers (safari, chrome, firefox) and accept each clients HTTP 1.1 GET requests. Once i receive the request I return the html page content to the the browser where the html page is displayed.

The problem i am getting is when i try to keep the connection open for a while. I realized that i am not able to display anything in the browser until i close the connection using fd.close().

Here is the function i am using to accept and respond to the browser request. The problem is after i use fd.sendall(), i dont want to close the connection but the page wont display until i do. Please help! Any help or suggestion is appreciated..

def handleConnectedSocket():
    try:
        recvIsComplete = False
        rcvdStr = ''

        line1 = "HTTP/1.1 200 OK\r\n"
        line2 = "Server: Apache/1.3.12 (Unix)\r\n"
        line3 = "Content-Type: text/html\r\n" # Alternately, "Content-Type: image/jpg\r\n"
        line4 = "\r\n"

        line1PageNotFound = "HTTP/1.1 404 Not Found\r\n"
        ConnectionClose = "Connection: close\r\n"

        while not recvIsComplete:
            rcvdStr = fd.recv( 1024 )

            if rcvdStr!= "" :

# look for the string that contains the html page
                recvIsComplete = True
                RequestedFile = ""
                start = rcvdStr.find('/') + 1 
                end = rcvdStr.find(' ', start)
                RequestedFile = rcvdStr[start:end] #requested page in the form of xyz.html

                try:
                    FiletoRead = file(RequestedFile , 'r')
                except:
                    FiletoRead = file('PageNotFound.html' , 'r')
                    response = FiletoRead.read()
                    request_dict[fd].append(line1PageNotFound + line2 + ConnectionClose + line4) 
                    fd.sendall( line1PageNotFound + line2 + line3 + ConnectionClose + line4 + response )
#                    fd.close()   <--- DONT WANT TO USE THIS
                else:    
                    response = FiletoRead.read()
                    request_dict[fd].append(line1 + line2 + line3 + ConnectionClose + line4 + response)
                    fd.sendall(line1 + line2 + line3 + line4 + response)
#                    fd.close()   <--- DONT WANT TO USE THIS
            else:
                recvIsComplete = True
#Remove messages from dictionary
                del request_dict[fd]    
                fd.close()

The client (browser) request is in HTTP 1.1 form as shown:

GET /Test.html HTTP/1.1
Host: 127.0.0.1:22222
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8) AppleWebKit/536.25 (KHTML, like    Gecko) Version/6.0 Safari/536.25
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive

Answer 1

Connection: close indicates to the browser that you’ll tell it when you’re done sending data by closing the connection. Since you don’t want to do that, you’ll probably want to use a different value for Connection, like Keep-Alive. If you use that, though, then you’ll also need to send Content-Length or do something else so the browser knows when you’re done sending it data.

Even if you’re not using Keep-Alive, Content-Length is a good thing to send, because it allows the browser to know the current progress in downloading the page. If you have a big file you’re sending and don’t send Content-Length, the browser can’t, say, show a progress bar. Content-Length enables that.

So how do you send a Content-Length header? Count up the number of bytes of data you’ll send. Turn that into a string and use that as the value. It’s that simple. For example:

# Assuming data is a byte string.
# (If you're dealing with a Unicode string, encode it first.)
content_length_header = "Content-Length: {0}\r\n".format(len(data))

---

Here’s some code that’s working for me:

#!/usr/bin/env python3
import time
import socket

data = b'''\
HTTP/1.1 200 OK\r\n\
Connection: keep-alive\r\n\
Content-Type: text/html\r\n\
Content-Length: 6\r\n\
\r\n\
Hello!\
'''


def main(server_address=('0.0.0.0', 8000)):
    server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, True)
    server.bind(server_address)
    server.listen(5)
    while True:
        try:
            client, client_address = server.accept()
            handle_request(client, client_address)
        except KeyboardInterrupt:
            break


def handle_request(client, address):
    with client:
        client.sendall(data)
        time.sleep(5)  # Keep the socket open for a bit longer.
        client.shutdown(socket.SHUT_RDWR)


if __name__ == '__main__':
    main()