Using Pytest with a Src Layer

Using pytest with a src layer

Recommended approach for pytest>=7: use the pythonpath setting

Recently, pytest has added a new core plugin that supports sys.path modifications via the pythonpath configuration value. The solution is thus much simpler now and doesn't require any workarounds anymore:

pyproject.toml example:

[tool.pytest.ini_options]
pythonpath = [
"src"
]

pytest.ini example:

[pytest]
pythonpath = src

The path entries are calculated relative to the rootdir, thus the src entry adds path/to/project/src directory to sys.path in this case.

Multiple path entries are also allowed: for a layout

repo/
├── src/
| └── lib.py
├── src2/
| └── lib2.py
└── tests
└── test_lib.py

the configuration

[tool.pytest.ini_options]
pythonpath = [
"src", "src2",
]

or

[pytest]
pythonpath = src src2

will add both lib and lib2 modules to sys.path, so

import lib
import lib2

will both work.

Original answer

Adjusting the PYTHONPATH (as suggested in the comments) is one possibility to solve the import issue. Another is adding an empty conftest.py file in the src directory:

$ touch src/conftest.py

and pytest will add src to sys.path. This is a simple way to trick pytest into adding codebase to sys.path.

However, the src layout is usually selected when you intend to build a distribution, e.g. providing a setup.py with (in this case) explicitly specifying the root package dir:

from setuptools import find_packages, setup

setup(
...
package_dir={'': 'src'},
packages=find_packages(where='src'),
...
)

and installing the package in the development mode (via python setup.py develop or pip install --editable .) while you're still developing it. This way, your package my_package is correctly integrated in the Python's site packages structure and there's no need to fiddle with PYTHONPATH.

Ensuring py.test includes the application directory in sys.path

As you say yourself py.test basically assumes you have the PYTHONPATH setup up correctly. There are several ways of achieving this:

  • Give your project a setup.py and use pip install -e . in a virtualenv for this project. This is probably the standard method.

  • As a variation on this if you have a virtualenv but no setup.py use your venv's facility to add the projects directory on sys.path, e.g. pew add . if you use pew, or add2virtualenv . if you use virtualenv and the extensions of virtualenvwrapper.

  • If you always like the current working directory on sys.path you can simply always export PYTHONPATH='' in your shell. That is ensure the empty string on on sys.path which python will interpret as the current working direcotry. This is potentially a security hazard though.

  • My own favourite hack, abuse how py.test loads conftest files: put an empty conftest.py in the project's top-level directory.

The reason for py.test to behave this way is to make it easy to run the tests in a tests/ directory of a checkout against an installed package. If it would unconditionally add the project directory to the PYTHONPATH then this would not be possible anymore.

PATH issue with pytest 'ImportError: No module named YadaYadaYada'

Yes, the source folder is not in Python's path if you cd to the tests directory.

You have two choices:

  1. Add the path manually to the test files. Something like this:

     import sys, os
    myPath = os.path.dirname(os.path.abspath(__file__))
    sys.path.insert(0, myPath + '/../')
  2. Run the tests with the env var PYTHONPATH=../.

pytest cannot find module

Update for pytest 7 and newer: use the pythonpath setting

Recently, pytest has added a new core plugin that supports sys.path modifications via the pythonpath configuration value. The solution is thus much simpler now and doesn't require any workarounds anymore:

pyproject.toml example:

[tool.pytest.ini_options]
pythonpath = [
"."
]

pytest.ini example:

[pytest]
pythonpath = .

The path entries are calculated relative to the rootdir, thus . adds junk directory to sys.path in this case.

Multiple path entries are also allowed: for a layout

junk/
├── src/
| └── lib.py
├── junk/
│ ├── __init__.py
│ └── ook.py
└── tests
├── test_app.py
└── test_lib.py

the configuration

[tool.pytest.ini_options]
pythonpath = [
".", "src",
]

or

[pytest]
pythonpath = . src

will add both lib module and junk package to sys.path, so

import junk
import lib

will both work.

Original answer

Just put an empty conftest.py file in the project root directory:

$ pwd
/home/usr/tmp/junk
$ touch conftest.py

Your project structure should become:

junk
├── conftest.py
├── junk
│ ├── __init__.py
│ └── ook.py
└── tests
└── test_ook.py

What happens here: when pytest discovers a conftest.py, it modifies sys.path so it can import stuff from the conftest module. So, since now an empty conftest.py is found in rootdir, pytest will be forced to append it to sys.path. The side effect of this is that your junk module becomes importable.



Related Topics



Leave a reply



Submit