Question

I’m trying to make a small testing server for Bugzilla so I can test out changes I make before they are deployed to the main Apache based server. I’m most familiar with Python, and I was wanting to make Python’s built-in HTTP server run Bugzilla’s CGI programs.

Unfortunately, Bugzilla has lots more than CGI apps. It has a bunch of css and other data that is served up directly. This means the handler needs to deal with those as well. I would like to set up a WSGI handler that looks at the request URL and appropriately routes the request to either one of the Bugzilla CGI scripts or pulls the data directly from the filesystem.

Is there a better way to accomplish what I want to do? If there isn’t, is there a WSGI app out there already that will set up a CGI environment and call out to a CGI app via Python’s subprocess module?

Answer 1

Here is a solution that works, though it’s rather ugly and kind of slow. It demands a separate CGIApplication object for each CGI script you want to run. So if you have a directory full of them, you will need to instantiate a different CGIApplication object for each one. When you instantiate it, of course, is up to you. You can choose to instantiate new ones for every request, but you will likely save a bit of time if you somehow avoid this.

import os
import os.path as _osp
import re
import subprocess
import io
import email.parser

env_forward = re.compile('^[A-Z][A-Z0-9_]*$')
header_match = re.compile(b'^(.*?\\n[ \\t\\r]*\\n)(.*)$', re.M | re.S)
env_whitelist = frozenset(('AUTH_TYPE', 'CONTENT_LENGTH', 'CONTENT_TYPE',
                           'DOCUMENT_ROOT', 'QUERY_STRING', 'PATH_INFO',
                           'PATH_TRANSLATED', 'REMOTE_ADDR', 'REMOTE_PORT',
                           'REMOTE_IDENT', 'REMOTE_USER', 'REQUEST_METHOD',
                           'REQUEST_URI', 'SCRIPT_NAME',
                           'SERVER_ADDR', 'SERVER_ADMIN', 'SERVER_NAME',
                           'SERVER_PORT', 'SERVER_PROTOCOL',
                           'SERVER_SIGNATURE', 'SERVER_SOFTWARE'))

class CGIApplication(object):
    def __init__(self, appfname):
        self._appfname = _osp.abspath(appfname)

    def __call__(self, environ, start_respose):
        appenv = {item[0]: item[1] \
                      for item in environ.items() \
                      if ((item[0] in env_whitelist) or
                          item[0].startswith('HTTP_'))}
        appenv['GATEWAY_INTERFACE'] = 'CGI/1.1'
        appenv['PATH'] = '/usr/local/bin:/usr/bin:/bin'
        appenv['SCRIPT_FILENAME'] = self._appfname
        nbytes_for_cgi = appenv.get('CONTENT_LENGTH', '')
        nbytes_for_cgi = (int(nbytes_for_cgi) if nbytes_for_cgi != '' else 0)

        args = [self._appfname]
        query = environ.get('QUERY_STRING', None)
        query = query.replace('+', ' ')
        if '=' not in query:
            args.append(query)
        proc = subprocess.Popen(args,
                                stdin=subprocess.PIPE,
                                stdout=subprocess.PIPE,
                                stderr=subprocess.PIPE,
                                env = appenv,
                                cwd = _osp.dirname(self._appfname))
        bytes_read = 0
        data_for_cgi = io.BytesIO()
        while bytes_read < nbytes_for_cgi:
            data = environ['wsgi.input'].read(nbytes_for_cgi - bytes_read)
            bytes_read += len(data)
            data_for_cgi.write(data)
            data = None
        data_for_cgi = data_for_cgi.getvalue()
        output, errdata = proc.communicate(data_for_cgi)
        data_for_cgi = None
        proc.stdin.close()
        proc.stdout.close()
        proc.stderr.close()
        try:
            errdata = errdata.decode('utf-8')
        except UnicodeDecodeError:
            errdata = errdata.decode('iso8859-1')
        environ['wsgi.errors'].write(errdata)
        errdata = None
        if proc.returncode != 0:
            start_respose('500 Internal Server Error',
                          [('Content-Type', 'text/plain')])
            return (b"CGI application died with non-zero return code.\n",)
        else:
            output_hdr = header_match.match(output)
            output_hdr, output = output_hdr.groups()
            parser = email.parser.HeaderParser()
            headers = parser.parsestr(output_hdr.decode('iso8859-1'))
            status = headers.get_all('Status', ['200 OK'])[-1]
            del headers['Status']
            start_respose(status, list(headers.items()))
            return (output,)