Question

I have a small web.py Python application that I would like to serve under Apache using mod_python. The web.py framework, as well as other third-party Python modules the application in question relies upon, are installed in a virtual environment. The virtual environment is created inside: /home/ayaz/Sandbox/Scrapper/

The relevant snippet from the virtual host configuration for Apache that I am using in order to set up this application is this:

<Location "/api">
    PythonPath "['/home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/', '/home/ayaz/project/'] + sys.path"
    #PythonHandler wsgiref.modpython_gateway::handler
    PythonHandler modpython_gateway::handler
    SetHandler python-program 
    PythonOption wsgi.application device_api::main
    PythonOption device_api /api/
</Location>

On the browser when I try to access the /api URL however, I get a 500 from the server with the error in the logs saying that the web module imported from within the device_api.py file (which is my application) could not be found; in other words, I see an ImportError. I am not sure why it is unable to find the web module.

I know that the PythonPath directive is working, at least partially, judging from the fact that the Python interpreter is able to find the device_api.py file from the path defined in that directive. But, it is unable to find the rest of the modules for which the path is also defined in the same directive.

Any help with this will be deeply appreciated.

Thanks!

UPDATE #1

Ned’s reply had me looking
through the site-packages directory
for the virtual environment I have.
While the permissions looked fine
inside the directory to me, I realized
that mod_python/Apache was not able
to read the eggs. Since I installed
all the packages inside the virtual
environment using easy_install, they are all in the forms
of eggs. So, for example, if I moved
the directory
/home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/web.py-0.34-py2.5.egg/web/
into
/home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/
(essentially taking it out of the egg
file/directory), mod_python stopped
complaining about the missing web
module (of course, it then started
complaining about the missing rest of
the third-party modules).

I have Apache configured to run as my
user and group, that is ayaz, and I
checked that the permissions on the
/home/ayaz/.python-eggs directory
were fine.

I then used this Using eggs with
mod_python tricked explained on
the Django deployment documentation
page. But it didn’t help.
Eventually, I bit the bullet, and for
each third-party module that
mod_python complained it couldn’t
find, I moved the actual directory
from inside the egg for that module
outside of the egg and into the
site-packages directory for the
virtual environment. This made
mod_python find the modules.

Now, I am really not sure why this
is happening, and why mod_python isn’t
able to read through the eggs.

Answer 1

So as to answer this question, I am going to copy the update paragraphs from my question and post them as the answer (as they seem to explain to some extent how I manage to get rid of the problem).

Ned’s reply had me looking through the
site-packages directory for the
virtual environment I have. While the
permissions looked fine inside the
directory to me, I realized that
mod_python/Apache was not able to read
the eggs. Since I installed all the
packages inside the virtual
environment using easy_install, they
are all in the forms of eggs. So, for
example, if I moved the directory
/home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/web.py-0.34-py2.5.egg/web/
into
/home/ayaz/Sandbox/Scrapper/lib/python2.5/site-packages/
(essentially taking it out of the egg
file/directory), mod_python stopped
complaining about the missing web
module (of course, it then started
complaining about the missing rest of
the third-party modules).

I have Apache configured to run as my
user and group, that is ayaz, and I
checked that the permissions on the
/home/ayaz/.python-eggs directory were
fine.

I then used this Using eggs with
mod_python tricked explained on the
Django deployment documentation page.
But it didn’t help. Eventually, I bit
the bullet, and for each third-party
module that mod_python complained it
couldn’t find, I moved the actual
directory from inside the egg for that
module outside of the egg and into the
site-packages directory for the
virtual environment. This made
mod_python find the modules.

Now, I am really not sure why this is
happening, and why mod_python isn’t
able to read through the eggs.