Home/ Questions/Q 8385117
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-09T17:35:15+00:00

Background: I am a complete beginner when it comes to servers, but I know

  • 0

Background: I am a complete beginner when it comes to servers, but I know my way around programming in Python.

I am trying to setup a simple server using the basic Python 2.7 modules (SimpleHTTPServer, CGIHTTPServer, etc). This server needs to load a global, read-only variable with several GB of data from a file when it starts; then, when each user accesses the page, the server uses the big data to generate some output which is then given to the user.

For the sake of example, let’s suppose I have a 4 GB file names.txt which contains all possible proper nouns of English:

Jack
John
Allison
Richard
...

Let’s suppose that my goal is to read the whole list of names into memory, and then choose 1 name at random from this big list of proper nouns. I am currently able to use Python’s native CGIHTTPServer module to accomplish this. To start, I just run the CGIHTTPServer module directly, by executing from a terminal:

python -m CGIHTTPServer

Then, someone accesses www.example-server.net:8000/foo.py and they are given one of these names at random. I have the following code in foo.py:

#!/usr/bin/env python

import random

name_list = list()
FILE = open('names.txt','r')
for line in FILE:
    name = line[:-1]
    name_list.append(name)

FILE.close()
name_to_return = random.choice(name_list)

print "Content-type: text/html"
print
print "<title>Here is your name</title>"
print "<p>" + name_to_return + "</p>"

This does what I want; however, it is extremely inefficient, because every access forces the server to re-read a 4 GB file.

How can I make this into an efficient process, where the variable name_list is created as global immediately when the server starts, and each access only reads from that variable?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    Just for future reference, if anyone ever faces the same problem: I ended up sub-classing CGIHTTPServer‘s request handler and implementing a new do_POST() function. If you had a working CGI script without global variables, something like this should get you started:

    import CGIHTTPServer
import random
import sys
import cgi

class MyRequestHandler(CGIHTTPServer.CGIHTTPRequestHandler):
    global super_important_list
    super_important_list = range(10)
    random.shuffle(super_important_list)

    def do_POST(s):    
        """Respond to a POST request."""
        form = cgi.FieldStorage(fp=s.rfile,headers=s.headers,environ={'REQUEST_METHOD':'POST','CONTENT_TYPE':s.headers['Content-Type'],})
        s.wfile.write("<html><head><title>Title goes here.</title></head>")
        s.wfile.write("<body><p>This is a test.</p>")
        s.wfile.write("<p>You accessed path: %s</p>" % s.path)
        s.wfile.write("<p>Also, super_important_list is:</p>")
        s.wfile.write(str(super_important_list))
        s.wfile.write("<p>Furthermore, you POSTed the following info: ")
        for item in form.keys():
            s.wfile.write("<p>Item: " + item)
            s.wfile.write("<p>Value: " + form[item].value)
        s.wfile.write("</body></html>")

if __name__ == '__main__':
    server_address = ('', 8000)
    httpd = CGIHTTPServer.BaseHTTPServer.HTTPServer(server_address, MyRequestHandler)
    try:
        httpd.serve_forever()
    except KeyboardInterrupt:
        sys.exit()

    Whenever someone fills out your form and performs a POST, the variable form will be a dictionary-like object with key-value pairs which may differ for each user of your site, but the global variable super_important_list will be the same for every user.

    Thanks to everyone who answered my question, especially Mike Steder, who pointed me in the right direction!

    • 0