Question

tl;dr: How do I create a new namespace in some __init__.py, and copy select bits of another namespace into it, without requiring a sub-module in a new directory?

Full question:

Let’s say I have a pair of modules in a package, with the directory structure as follows:

foo/mod_1.py
foo/mod_2.py
foo/__init__.py

mod_1.py is as follows:

import numpy

__all__ = ['test1']

def test1():
    return numpy.zeros(10)

mod_2.py is as follows:

import numpy

def test2():
    return numpy.zeros(20)

and __init__.py as follows:

from mod_1 import *
import mod_2

del mod_1

This yields more or less the desired namespace:

In [7]: dir(foo)
Out[7]: 
['__builtins__',
 '__doc__',
 '__file__',
 '__name__',
 '__package__',
 '__path__',
 'mod_2',
 'test1']

Except that numpy is now in foo.mod_2 namespace. If I try to delete it in __init__.py, with del mod_2.numpy, it no longer exists for that module, so mod_2.test2 is broken.

I can achieve what I want by creating an extra directory mod_2 containing mod_2.py, but I want to keep the directory structure as-is if possible.

Hence my initial question.

Edit: I don’t mind doing a bit of name mangling to stop one name treading on another. Let’s say putting the relevant bits of mod_2.py into foo._mod_2.

Answer 1

You don’t want to do that, as you discovered you break your code.

Python is a dynamic language, introspection gives us unfettered access to almost everything the interpreter can reach. This applies to your module too.

If it makes you feel any better, the same thing applies to the standard library:

>>> import unittest
>>> unittest.sys
<module 'sys' (built-in)>
>>> unittest.types
<module 'types' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/types.pyc'>
>>> dir(unittest)
['FunctionTestCase', 'TestCase', 'TestLoader', 'TestProgram', 'TestResult', 'TestSuite', 'TextTestRunner', '_CmpToKey', '_TextTestResult', '_WritelnDecorator', '__all__', '__author__', '__builtins__', '__doc__', '__email__', '__file__', '__metaclass__', '__name__', '__package__', '__unittest', '__version__', '_makeLoader', '_strclass', 'defaultTestLoader', 'findTestCases', 'getTestCaseNames', 'main', 'makeSuite', 'os', 'sys', 'time', 'traceback', 'types']

Everything a module needs in it’s global namespace can be seen from the outside. This is normal.

As you already discovered, use __all__ to control from modulename import * wildcard importing, and leave your module namespace alone otherwise.

Of course, if you want to provide users of your package a clean API, there is nothing stopping you from importing the names that are part of the API into the top-level package __init__.py, or by using a api.py module.

Importing just means adding the same name as a variable in the current module; mod_2.numpy is the exact same thing as sys.modules['numpy'], or numpy in your current module after you ran import numpy.