Question

I imported a numpy library in two different ways. The first time with from numpy.random import mtrand and the second time after messing with the sys.path.

However, the output of these two module imports was totally different:

>>> from numpy.random import mtrand
>>> dir(mtrand)
['RandomState', '__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__', '_rand', 'beta', 'binomial', 'bytes', 'chisquare', 'dirichlet', 'exponential', 'f', 'gamma', 'geometric', 'get_state', 'gumbel', 'hypergeometric', 'laplace', 'logistic', 'lognormal', 'logseries', 'multinomial', 'multivariate_normal', 'negative_binomial', 'noncentral_chisquare', 'noncentral_f', 'normal', 'np', 'pareto', 'permutation', 'poisson', 'power', 'rand', 'randint', 'randn', 'random_integers', 'random_sample', 'rayleigh', 'seed', 'set_state', 'shuffle', 'standard_cauchy', 'standard_exponential', 'standard_gamma', 'standard_normal', 'standard_t', 'triangular', 'uniform', 'vonmises', 'wald', 'weibull', 'zipf']

And the second one:

>>> sys.path.insert(0, '/usr/lib/pymodules/python2.7/numpy/random')
>>> import mtrand
>>> dir(mtrand)
['__builtins__', '__doc__', '__file__', '__name__', '__package__']

How is this behaviour possible?

Edit:

• These two tests were executed in different python processes.
• Messing with the sys path is stupid, I know that. But this is not for a normal program, it’s for an autocompletion. I certainly don’t want to import the whole numpy package. I just want to be able to make a dir(mtrand)

Answer 1

As Endophage pointed out, it is indeed stupid to play with the sys.path. But I guess I started it, because I did not want to execute any Python code, which would be quite cool for an auto-completion. I didn’t think that this is a big issue for c_builtin modules. However, there are modules, which really need the correct package path (see the comment about segfaults further down).

I even did a workaround for PyQt4, but noticed that this would not be the only workaround:

sys.path.append('/usr/lib/python2.7/dist-packages/PyQt4')
try:
    import QtCore
except SystemError:
    QtCore = sys.modules['PyQt4.QtCore']
    debug.warning('Loaded a module with SystemError.') 

This included catching a SystemError and then using it from sys.modules. Indeed this is very stupid, because I don’t know about any side effects of such operations. As I tested PySide (which is another Qt wrapper), segfaults happend under certain circumstances.

So I came up with the more pythonic solution to load modules like this again: from PyQt4 import QtCore and from numpy.random import random.

Interestingly, the memory footprint for loading those modules with their full path, was about as much as messing with the sys.path.