Imports in _Init_.Py and 'Import As' Statement

Imports in init.py and 'import as' statement

You incorrectly assume that one cannot have an alias with from ... import, as from ... import ... as has been there since Python 2.0. The import ... as is the obscure syntax that not many know about, but which you use by accident in your code.

PEP 0221 claims that the following 2 are "effectively" the same:

import foo.bar.bazaar as baz
from foo.bar import bazaar as baz

The statement is not quite true in Python versions up to and including 3.6.x as evidenced by the corner case you met, namely if the required modules already exist in sys.modules but are yet uninitialized. The import ... as requires that the module foo.bar is injected in foo namespace as the attribute bar, in addition to being in sys.modules, whereas the from ... import ... as looks for foo.bar in sys.modules.

(Do note also that import foo.bar only ensures that the module foo.bar is in sys.modules and accessible as foo.bar, but might not be fully initialized yet.)

Changing the code as follows did the trick for me:

# import pkg.subpkg.two_longname as two
from pkg.subpkg import two_longname as two

And code runs perfectly on both Python 2 and Python 3.

Also, in one.py you cannot do from pkg import subpkg, for the same reason.

To demonstrate this bug further, fix your one.py as above, and add the following code in tst.py:

import pkg
import pkg.subpkg.two_longname as two

del pkg.subpkg

from pkg.subpkg import two_longname as two
import pkg.subpkg.two_longname as two

Only the last line crashes, because from ... import consults the sys.modules for pkg.subpkg and finds it there, whereas import ... as consults sys.modules for pkg and tries to find subpkg as an attribute in the pkg module. As we just had deleted that attribute, the last line fails with AttributeError: 'module' object has no attribute 'subpkg'.

As the import foo.bar as baz syntax is a bit obscure and adds more corner cases, and I have rarely if ever seen it being used, I would recommend avoiding it completely and favouring from .. import ... as.

Module imports and init.py

A couple things you could do to improve your organization, if only to adhere to some popular Python conventions and standards.

If you search this topic, you will inevitably run across people recommending the PEP8 guidelines. These are the de facto canonical standards for organizing python code.

Modules should have short, all-lowercase names. Underscores can be
used in the module name if it improves readability. Python packages
should also have short, all-lowercase names, although the use of
underscores is discouraged.

Based on these guidelines, your project modules should be named like this:

foo/
    __init__.py
    foo.py
    module1.py
    module2.py
    module3.py

I find it's generally best to avoid importing modules unnecessarily in __init__.py unless you're doing it for namespace reasons. For example, if you want the namespace for your package to look like this

from foo import Foo

instead of

from foo.foo import Foo

Then it makes sense to put

from .foo import Foo

in your __init__.py. As your package gets larger, some users may not want to use all of the sub-packages and modules, so it doesn't make sense to force the user to wait for all those modules to load by implicitly importing them in your __init__.py. Also, you have to consider whether you even want module1, module2, and module3 as part of your external API. Are they only used by Foo and not intended to be for end users? If they're only used internally, then don't include them in the __init__.py

I'd also recommend using absolute or explicit relative imports for importing sub-modules. For example, in foo.py

Absolute

from foo import module1
from foo import module2
from foo import module3

Explicit Relative

from . import module1
from . import module2
from . import module3

This will prevent any possible naming issues with other packages and modules. It will also make it easier if you decide to support Python3, since the implicit relative import syntax you're currently using is not supported in Python3.

Also, files inside your package generally shouldn't contain a

if __name__ == '__main__'

This is because running a file as a script means it won't be considered part of the package that it belongs to, so it won't be able to make relative imports.

The best way to provide executable scripts to users is by using the scripts or console_scripts feature of setuptools. The way you organize your scripts can be different depending on which method you use, but I generally organize mine like this:

foo/
    __init__.py
    foo.py
    ...
scripts/
     foo_script.py
setup.py

What is init.py for?

It used to be a required part of a package (old, pre-3.3 "regular package", not newer 3.3+ "namespace package").

Here's the documentation.

Python defines two types of packages, regular packages and namespace packages. Regular packages are traditional packages as they existed in Python 3.2 and earlier. A regular package is typically implemented as a directory containing an __init__.py file. When a regular package is imported, this __init__.py file is implicitly executed, and the objects it defines are bound to names in the package’s namespace. The __init__.py file can contain the same Python code that any other module can contain, and Python will add some additional attributes to the module when it is imported.

But just click the link, it contains an example, more information, and an explanation of namespace packages, the kind of packages without __init__.py.

Importing modules in Python and init.py

__init__.py turns a folder into a package. This is useful to create a sort of hierarchy of modules, where you can have import-statements like this:

import mymodule.cool.stuff

This is not possible without packages.

Python import in init()

Imported names are bound to the current scope, so an import inside a function binds to a local name only.

If you absolutely have to import something in __init__ that then needs to be globally avaliable, mark the imported name as global first:

>>> def foo():
...     global sys
...     import sys
... 
>>> sys
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'sys' is not defined
>>> foo()
>>> sys
<module 'sys' (built-in)>

but this usually leads to strange and wonderfully difficult to locate bugs. Don't do that, just make your imports at module scope instead.

If you need the imported name within other class methods, you could also assign the imported name to a instance variable:

class Foo(object):
    def __init__(self):
        import os
        self.join = os.path.join

but again, that's not the best practice to use.

How to import from the init.py in the same directory?

I see 'import from parent module' as an anti-pattern in Python. Imports should be the other way around. Importing from modules's __init__.py is especially problematic. As you noticed, importing module foo.bar from foo/bar.py involves importing foo/__init__.py first, and you may end up with a circular dependency. Adding a print("Importing", __name__) to your init files helps see the sequence and understand the problem.

I'd suggest that you moved the code you want to import in conditions.py from __init__.py to a separate lower-level module, and just import some names from that module in __init__.py to expose it at higher level.

Let's suppose that you had some class Bar in your __init__.py. I'd reorganize it the following way.

__init__.py:

from bar import Bar  # exposed at the higher level, as it used to be.

bar.py:

class Bar(object): ...

conditions.py:

from . import Bar  # Now it works.

Ideally an __init__.py should contain nothing but imports from lower-level modules, or nothing at all.

Python: How to import from an init.py file?

Try relative imports

from . import db

Python submodule imports using init.py

You probably already understand that when you import a module, the interpreter creates a new namespace and executes the code of that module with the new namespace as both the local and global namespace. When the code completes execution, the module name (or the name given in any as clause) is bound to the module object just created within the importing namespace and recorded against its __name__ in sys.modules.

When a qualified name such as package.subpackage.module is imported the first name (package) is imported into the local namespace, then subpackage is imported into package's namespace and finally module is imported into package.subpackage's namespace. Imports using from ... import ... as ... perform the same sequence of operations, but the imported objects are bound directly to names in the importing module's namespace. The fact that the package name isn't bound in your local namespace does not mean it hasn't been imported (as inspection of sys.modules will show).

The __init__.py in a package serves much the same function as a module's .py file. A package, having structure, is written as a directory which can also contain modules (regular .py files) and subdirectories (also containing an __init__.py file) for any sub_packages. When the package is imported a new namespace is created and the package's __init__.py is executed with that namespace as the local and global namespaces. So to answer your problem we can strip your filestore down by omitting the top-level package, which will never be considered by the interpreter when test.py is run as a program. It would then look like this:

test.py
subpackage/
    __init__.py
    hello_world.py

Now, subpackage is no longer a sub-package, as we have removed the containing package as irrelevant. Focusing on why the do_something name is undefined might help. test.py does not contain any import, and so it's unclear how you are expecting do_something to acquire meaning. You could make it work by using an empty subpackage/__init__.py and then you would write test.py as

from subpackage.hello_world import do_something
do_something()

Alternatively you could use a subpackage/__init__.py that reads

from hello_world import do_something

which establishes the do_something function inside the subpackage namespace when the package is imported. Then use a test.py that imports the function from the package, like this:

from subpackage import do_something
do_something()

A final alternative with the same __init__.py is to use a test.py that simply imports the (sub)package and then use relative naming to access the required function:

import subpackage
subpackage.do_something()

to gain access to it in your local namespace.

With the empty __init__.py this could also be achieved with a test.py reading

import subpackage.hello_world
subpackage.hello_world.do_something()

or even

from subpackage.hello_world import do_something
do_something()

An empty __init__.py will mean that the top-level package namespace will contain only the names of any subpackages the program imports, which allows you to import only the subpackages you require. This gives you control over the namespace of the top-level package.

While it's perfectly possible to define classes and functions in the
__init__.py , a more normal approach is to import things into that namespace from submodules so that importers can just import the top-level package to gain access to its contents with a single-level attribute
reference, or even use from to import only the names you specifically want.

Ultimately the best tool to keep you straight is a clear understanding of how import works and what effect its various forms have on the importing namespace.

How to import classes defined in init.py

'lib/'s parent directory must be in sys.path.

Your 'lib/__init__.py' might look like this:

from . import settings # or just 'import settings' on old Python versions
class Helper(object):
      pass

Then the following example should work:

from lib.settings import Values
from lib import Helper

Answer to the edited version of the question:

__init__.py defines how your package looks from outside. If you need to use Helper in settings.py then define Helper in a different file e.g., 'lib/helper.py'.


.
|   `-- import_submodule.py
    `-- lib
    |-- __init__.py
    |-- foo
    |   |-- __init__.py
    |   `-- someobject.py
    |-- helper.py
    `-- settings.py

2 directories, 6 files

The command:

$ python import_submodule.py

Output:

settings
helper
Helper in lib.settings
someobject
Helper in lib.foo.someobject

# ./import_submodule.py
import fnmatch, os
from lib.settings import Values
from lib import Helper

print
for root, dirs, files in os.walk('.'):
    for f in fnmatch.filter(files, '*.py'):
        print "# %s/%s" % (os.path.basename(root), f)
        print open(os.path.join(root, f)).read()
        print

# lib/helper.py
print 'helper'
class Helper(object):
    def __init__(self, module_name):
        print "Helper in", module_name

# lib/settings.py
print "settings"
import helper

class Values(object):
    pass

helper.Helper(__name__)

# lib/__init__.py
#from __future__ import absolute_import
import settings, foo.someobject, helper

Helper = helper.Helper

# foo/someobject.py
print "someobject"
from .. import helper

helper.Helper(__name__)

# foo/__init__.py
import someobject

Imports in _Init_.Py and 'Import As' Statement