How to Check If a Python Module Exists Without Importing It

How to check if a Python module exists without importing it

TL;DR) Use importlib.util.find_spec(module_name) (Python 3.4+).

Python2: `imp.find_module`

To check if import can find something in Python 2, using imp:

import imp
try:
    imp.find_module('eggs')
    found = True
except ImportError:
    found = False

To find dotted imports, you need to do more:

import imp
try:
    spam_info = imp.find_module('spam')
    spam = imp.load_module('spam', *spam_info)
    imp.find_module('eggs', spam.__path__) # __path__ is already a list
    found = True
except ImportError:
    found = False

You can also use pkgutil.find_loader (more or less the same as the Python 3 part:

import pkgutil
eggs_loader = pkgutil.find_loader('eggs')
found = eggs_loader is not None

Python 3

Python 3 ≤ 3.3: `importlib.find_loader`

You should use importlib. I went about doing this like:

import importlib
spam_loader = importlib.find_loader('spam')
found = spam_loader is not None

My expectation being, if you can find a loader for it, then it exists. You can also be a bit more smart about it, like filtering out what loaders you will accept. For example:

import importlib
spam_loader = importlib.find_loader('spam')
# only accept it as valid if there is a source file for the module - no bytecode only.
found = issubclass(type(spam_loader), importlib.machinery.SourceFileLoader)

Python 3 ≥ 3.4: `importlib.util.find_spec`

In Python 3.4 importlib.find_loader Python documentation was deprecated in favour of importlib.util.find_spec. The recommended method is the importlib.util.find_spec. There are others like importlib.machinery.FileFinder, which is useful if you're after a specific file to load. Figuring out how to use them is beyond the scope of this.

import importlib
spam_spec = importlib.util.find_spec("spam")
found = spam_spec is not None

This also works with relative imports, but you must supply the starting package, so you could also do:

import importlib
spam_spec = importlib.util.find_spec("..spam", package="eggs.bar")
found = spam_spec is not None
spam_spec.name == "eggs.spam"

While I'm sure there exists a reason for doing this - I'm not sure what it would be.

Warning

When trying to find a submodule, it will import the parent module (for ALL of the above methods)!

food/
  |- __init__.py
  |- eggs.py

## __init__.py
print("module food loaded")

## eggs.py
print("module eggs")

were you then to run
>>> import importlib
>>> spam_spec = importlib.util.find_spec("food.eggs")
module food loaded
ModuleSpec(name='food.eggs', loader=<_frozen_importlib.SourceFileLoader object at 0x10221df28>, origin='/home/user/food/eggs.py')

Comments are welcome on getting around this

Acknowledgements

@rvighne for importlib
@lucas-guido for Python 3.3+ deprecating find_loader
@enpenax for pkgutils.find_loader behaviour in Python 2.7

How to check if python module exists and can be imported

You can use the same logic inside your function:

def module_exists(module_name):
    try:
        __import__(module_name)
    except ImportError:
        return False
    else:
        return True

There is no performance penalty to this solution because modules are imported only once.

How can I check if a module has been imported?

Test for the module name in the sys.modules dictionary:

import sys

modulename = 'datetime'
if modulename not in sys.modules:
    print 'You have not imported the {} module'.format(modulename)

From the documenation:

This is a dictionary that maps module names to modules which have already been loaded.

Note that an import statement does two things:

if the module has never been imported before (== not present in sys.modules), then it is loaded and added to sys.modules.
Bind 1 or more names in the current namespace that reference the module object or to objects that are members of the module namespace.

The expression modulename not in sys.modules tests if step 1 has taken place. Testing for the result of step 2 requires knowing what exact import statement was used as they set different names to reference different objects:

import modulename sets modulename = sys.modules['modulename']
import packagename.nestedmodule sets packagename = sys.modules['packagename'] (no matter how many addional levels you add)
import modulename as altname sets altname = sys.module['modulename']
import packagename.nestedmodule as altname sets altname = sys.modules['packagename.nestedmodule']
from somemodule import objectname sets objectname = sys.modules['somemodule'].objectname
from packagename import nestedmodulename sets nestedmodulename = sys.modules['packagename.nestedmodulename'] (only when there was no object named nestedmodulename in the packagename namespace before this import, an additional name for the nested module is added to the parent package namespace at this point)
from somemodule import objectname as altname sets altname = sys.modules['somemodule'].objectname
from packagename import nestedmodulename as altname sets altname = sys.modules['packagename.nestedmodulename'] (only when there was no object named nestedmodulename in the packagename namespace before this import, an additional name for the nested module is added to the parent package namespace at this point)

You can test if the name to which the imported object was bound exists in a given namespace:

# is this name visible in the current scope:
'importedname' in dir()

# or, is this a name in the globals of the current module:
'importedname' in globals()

# or, does the name exist in the namespace of another module:
'importedname' in globals(sys.modules['somemodule'])

This only tells you of the name exists (has been bound), not if it refers to a specific module or object from that module. You could further introspect that object or test if it’s the same object as what’s available in sys.modules, if you need to rule out that the name has been set to something else entirely since then.

How to check if a module is installed in Python and, if not, install it within the code?

EDIT - 2020/02/03

The pip module has updated quite a lot since the time I posted this answer. I've updated the snippet with the proper way to install a missing dependency, which is to use subprocess and pkg_resources, and not pip.

To hide the output, you can redirect the subprocess output to devnull:

import sys
import subprocess
import pkg_resources

required = {'mutagen', 'gTTS'}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing = required - installed

if missing:
    python = sys.executable
    subprocess.check_call([python, '-m', 'pip', 'install', *missing], stdout=subprocess.DEVNULL)

Like @zwer mentioned, the above works, although it is not seen as a proper way of packaging your project. To look at this in better depth, read the the page How to package a Python App.

How can I check on runtime that a python module is valid without importing it?

We already had a custom importer (disclaimer: I did not write that code I 'm just the current maintainer) whose load_module:

def load_module(self,fullname):
    if fullname in sys.modules:
        return sys.modules[fullname]
    else: # set to avoid reimporting recursively
        sys.modules[fullname] = imp.new_module(fullname)
    if isinstance(fullname,unicode):
        filename = fullname.replace(u'.',u'\\')
        ext = u'.py'
        initfile = u'__init__'
    else:
        filename = fullname.replace('.','\\')
        ext = '.py'
        initfile = '__init__'
    try:
        if os.path.exists(filename+ext):
            with open(filename+ext,'U') as fp:
                mod = imp.load_source(fullname,filename+ext,fp)
                sys.modules[fullname] = mod
                mod.__loader__ = self
        else:
            mod = sys.modules[fullname]
            mod.__loader__ = self
            mod.__file__ = os.path.join(os.getcwd(),filename)
            mod.__path__ = [filename]
            #init file
            initfile = os.path.join(filename,initfile+ext)
            if os.path.exists(initfile):
                with open(initfile,'U') as fp:
                    code = fp.read()
                exec compile(code, initfile, 'exec') in mod.__dict__
        return mod
    except Exception as e: # wrap in ImportError a la python2 - will keep
        # the original traceback even if import errors nest
        print 'fail', filename+ext
        raise ImportError, u'caused by ' + repr(e), sys.exc_info()[2]

So I thought I could replace the parts that access the sys.modules cache with overriddable methods that would in my override leave that cache alone:

So:

@@ -48,2 +55,2 @@ class UnicodeImporter(object):
-        if fullname in sys.modules:
-            return sys.modules[fullname]
+        if self._check_imported(fullname):
+            return self._get_imported(fullname)
@@ -51 +58 @@ class UnicodeImporter(object):
-            sys.modules[fullname] = imp.new_module(fullname)
+            self._add_to_imported(fullname, imp.new_module(fullname))
@@ -64 +71 @@ class UnicodeImporter(object):
-                    sys.modules[fullname] = mod
+                    self._add_to_imported(fullname, mod)
@@ -67 +74 @@ class UnicodeImporter(object):
-                mod = sys.modules[fullname]
+                mod = self._get_imported(fullname)

and define:

class FakeUnicodeImporter(UnicodeImporter):

    _modules_to_discard = {}

    def _check_imported(self, fullname):
        return fullname in sys.modules or fullname in self._modules_to_discard

    def _get_imported(self, fullname):
        try:
            return sys.modules[fullname]
        except KeyError:
            return self._modules_to_discard[fullname]

    def _add_to_imported(self, fullname, mod):
        self._modules_to_discard[fullname] = mod

    @classmethod
    def cleanup(cls):
        cls._modules_to_discard.clear()

Then I added the importer in the sys.meta_path and was good to go:

importer = sys.meta_path[0]
try:
    if not hasattr(sys,'frozen'):
        sys.meta_path = [fake_importer()]
    perform_the_imports() # see question
finally:
    fake_importer.cleanup()
    sys.meta_path = [importer]

Right ? Wrong!

Traceback (most recent call last):
  File "bash\bush.py", line 74, in __supportedGames
    module = __import__('game',globals(),locals(),[modname],-1)
  File "Wrye Bash Launcher.pyw", line 83, in load_module
    exec compile(code, initfile, 'exec') in mod.__dict__
  File "bash\game\game1\__init__.py", line 29, in <module>
    from .constants import *
ImportError: caused by SystemError("Parent module 'bash.game.game1' not loaded, cannot perform relative import",)

Huh ? I am currently importing that very same module. Well the answer is probably in import's docs

If the module is not found in the cache, then sys.meta_path is searched (the specification for sys.meta_path can be found in PEP 302).

That's not completely to the point but what I guess is that the statement from .constants import * looks up the sys.modules to check if the parent module is there, and I see no way of bypassing that (note that our custom loader is using the builtin import mechanism for modules, mod.__loader__ = self is set after the fact).

So I updated my FakeImporter to use the sys.modules cache and then clean that up.

class FakeUnicodeImporter(UnicodeImporter):

    _modules_to_discard = set()

    def _check_imported(self, fullname):
        return fullname in sys.modules or fullname in self._modules_to_discard

    def _add_to_imported(self, fullname, mod):
        super(FakeUnicodeImporter, self)._add_to_imported(fullname, mod)
        self._modules_to_discard.add(fullname)

    @classmethod
    def cleanup(cls):
        for m in cls._modules_to_discard: del sys.modules[m]

This however blew in a new way - or rather two ways:

a reference to the game/ package was held in bash top package instance in sys.modules:
```
bash\
  __init__.py
  the_code_in_question_is_here.py
  game\
    ...
```
because game is imported as bash.game. That reference held references to all game1, game2,..., subpackages so those were never garbage collected
a reference to another module (brec) was held as bash.brec by the same bash module instance. This reference was imported as from .. import brec in game\game1 without triggering an import, to update SomeClass. However, in yet another module, an import of the form from ...brec import SomeClass did trigger an import and another instance of the brec module ended up in the sys.modules. That instance had a non updated SomeClass and blew with an AttributeError.

Both were fixed by manually deleting those references - so gc collected all modules (for 5 mbytes of ram out of 75) and the from .. import brec did trigger an import (this from ... import foo vs from ...foo import bar warrants a question).

The moral of the story is that it is possible but:

the package and subpackages should only reference each other
all references to external modules/packages should be deleted from top level package attributes
the package reference itself should be deleted from top level package attribute

If this sounds complicated and error prone it is - at least now I have a much cleaner view of interdependencies and their perils - time to address that.

This post was sponsored by Pydev's debugger - I found the gc module very useful in grokking what was going on - tips from here. Of course there were a lot of variables that were the debugger's and that complicated stuff

Sample Image

Check if module exists, if not install it

Here is how it should be done, and if I am wrong, please correct me. However, Noufal seems to confirm it in another answer to this question, so I guess it's right.

When writing the setup.py script for some scripts I wrote, I was dependent on the package manager of my distribution to install the required library for me.

So, in my setup.py file, I did this:

package = 'package_name'
try:
    return __import__(package)
except ImportError:
    return None

So if package_name was installed, fine, continue. Else, install it via the package manager which I called using subprocess.

How to: Check if a module exists, import it in a loop and set a bit

This is the cleanest way I know how:

Create a separate python file in the same folder where you do all you imports:

try:
    from mymodule_1 import class_a as x
except ImportError:
    pass

try:
    from mymodule_2 import  class_b as y
except ImportError:
    pass

then in your main script, from the module moduleFinder import ModuleFinder and your script with the imports.

from modulefinder import ModuleFinder
import my_imports

Create a ModuleFinder object and make it run your script that has the imports.

finder = ModuleFinder()
finder.run_script('my_imports.py')

Finally, you can create a dictionary that replaces all those bit variables you had before

modules = {mod_name: True for mod_name in finder.modules}
modules.update({mod_name: False for mod_name in finder.badmodules})

And in your code, you can just do a lookup in the dictionary

if modules['mymodule_1']:
    pass

How to Check If a Python Module Exists Without Importing It