Pytest - Specify Log Level from the Cli Command Running the Tests

pytest - specify log level from the CLI command running the tests

Pytest 3.5 introduced a parameter

pytest --log-cli-level=INFO

For versions prior to 3.5 you can combine pytest commandline options and the python loglevel from commandline example to do the following:

Add the following to conftest.py:

import pytest
import logging

def pytest_addoption(parser):
    parser.addoption(
        "--log", action="store", default="WARNING", help="set logging level"
    )

@pytest.fixture
def logger():
    loglevel = pytest.config.getoption("--log")
    logger = logging.getLogger(__name__)

    numeric_level = getattr(
        logging,
        loglevel.upper(),
        None
    )
    if not isinstance(numeric_level, int):
        raise ValueError('Invalid log level: %s' % loglevel)

    logger.setLevel(numeric_level)
    return logger

and then request the logger fixture in your tests

def test_bla(logger):
    assert True
    logger.info("True is True")

Then run pytest like

py.test --log INFO

to set the log level to INFO.

Add custom log level for pytest tests to python the logging module

In this code there is a misunderstanding between a logger instance and the logger module.

To be able to call the test_log method in the original code you need to obtain a logger instance eg:

# After the existing code

logger = logging.getLogger()
logger.test_info("Some log data.")

As you have added the test_info method to the loggerClass it will be available to the logger instance.

To have the method available on the logging module the function needs to be created for that purpose and added to the module:

# After the existing code

def module_test_info(message, *args, **kwargs):
    logging.log(level_number, message, *args, **kwargs)

setattr(logging, level_name.lower(), module_test_info)

This will add the module_test_info function to the logging module with the name test_info you will be able to call it as such logging.test_level("Some log data.")

Logging within pytest tests

Works for me, here's the output I get: [snip -> example was incorrect]

Edit: It seems that you have to pass the -s option to py.test so it won't capture stdout. Here (py.test not installed), it was enough to use python pytest.py -s pyt.py.

For your code, all you need is to pass -s in args to main:

 pytest.main(args=['-s', os.path.abspath(__file__)])

See the py.test documentation on capturing output.

Change pytest's live log level during runtime

I'm afraid I have not found a way of accessing that manager.

Use the request fixture to access the plugin manager in tests:

def test_spam(request):
    mgr = request.session.config.pluginmanager
    logging_plugin = mgr.get_plugin("logging-plugin")

However, this won't be necessary for changing the live log level (and IIRC changing LoggingPlugin.log_cli_level will only have an effect in runtest hooks, not in the test cases or fixtures). The log capturing and live logging share the same log level, so simply use the caplog fixture. Here's a simple example:

import logging
import pytest

logger = logging.getLogger(__name__)

def emit_logs():
    logger.info('sample info')
    logger.warning('sample warning')
    logger.error('sample error')
    logger.critical('sample critical')

def test_spam():
    emit_logs()

will yield

-------------------------------- live log call --------------------------------
2020-12-11 16:27:41 [    INFO] sample info (test_spam.py:9)
2020-12-11 16:27:41 [ WARNING] sample warning (test_spam.py:10)
2020-12-11 16:27:41 [   ERROR] sample error (test_spam.py:11)
2020-12-11 16:27:41 [CRITICAL] sample critical (test_spam.py:12)

Now e.g. raising the level to CRITICAL:

def test_spam(caplog):
    caplog.set_level(logging.CRITICAL)
    emit_logs()

yields only

-------------------------------- live log call --------------------------------
2020-12-11 16:27:41 [CRITICAL] sample critical (test_spam.py:12)

Using caplog as context manager:

def test_spam(caplog):
    with caplog.at_level(logging.ERROR):
        emit_logs()

will yield

-------------------------------- live log call --------------------------------
2020-12-11 16:27:41 [   ERROR] sample error (test_spam.py:11)
2020-12-11 16:27:41 [CRITICAL] sample critical (test_spam.py:12)

Dynamic changing live log level also works,

def test_eggs(caplog):
    with caplog.at_level(logging.WARNING):
        logger.info("i am not logged")
    logger.info("but i am")

as well as moving live log level switch to fixtures:

@pytest.fixture
def errors_only(caplog):
    with caplog.at_level(logging.ERROR):
        yield

@pytest.mark.usefixtures("errors_only")
def test_bacon():
    logger.error("i am printed")
    logger.info("but i am not")

pytest: How to show messages from logging.debug in the function under test

To do it by default with every run, add to pytest.ini:

[pytest]
log_cli = true
log_cli_level = DEBUG

Note: if using a very old pytest, pre-3.5, you will need to set log_cli = true in pytest.ini to use the following commands.

Otherwise, for 3.5 and later, you can use just the command line for a case-by-case basis:

pytest --log-cli-level=10 func.py

Or more clearly:

pytest --log-cli-level=DEBUG func.py

The command line will always override pytest.ini settings.

How to pass multiple pytest command line options in the run configuration

pytest uses a mixture of single - and double -- hyphens preceding the command line options, some are used without an hyphen. In your example only --log-level uses a double hyphen.

The remaining two options log_cli and log_cli_level are written without any leading hyphen and have an underscore in the middle of the name. Those two are also "Configuration Options" and may need to be prefixed with -o.

A complete reference can be found in Command-line Flags.

So instead of:

--log-level=DEBUG --log-cli=True --log-cli-level=DEBUG

try:

--log-level=DEBUG -o log_cli=True -o log_cli_level=DEBUG

C:\path_to your_project\venv\Scripts\python.exe "C:\Program Files\JetBrains\PyCharm 2019.3.2\plugins\python\helpers\pycharm\_jb_pytest_runner.py" --path C:/path_to your_project/tests/basic_test.py -- --log-level=DEBUG -v -o log_cli=True -o log_cli_level=DEBUG
Testing started at 3:05 AM ...
Launching pytest with arguments --log-level=DEBUG -v -o log_cli=True -o log_cli_level=DEBUG C:/path_to your_project/tests/basic_test.py in C:\path_to your_project

============================= test session starts =============================
platform win32 -- Python 3.9.0, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- C:\path_to your_project\venv\Scripts\python.exe
cachedir: .pytest_cache
rootdir: C:\path_to your_project, configfile: pytest.ini
collecting ... collected 1 item

tests/basic_test.py::test_validate_args PASSED                      [100%]

============================== 1 passed in 0.02s ==============================

Process finished with exit code 0

Disabling specific logger in pytest

Pytest does not support this by default, but you can add a custom option to your conftest.py to turn off specific loggers.

import pytest
import logging

def pytest_addoption(parser):
    """Add a command line option to disable logger."""
    parser.addoption(
        "--log-disable", action="append", default=[], help="disable specific loggers"
    )

def pytest_configure(config):
    """Disable the loggers."""
    for name in config.getoption("--log-disable", default=[]):
        logger = logging.getLogger(name)
        logger.propagate = False

---

Related Topics