How to Generate Dynamic (Parameterized) Unit Tests in Python

How do you generate dynamic (parameterized) unit tests in Python?

This is called "parametrization".

There are several tools that support this approach. E.g.:

• pytest's decorator
• parameterized

The resulting code looks like this:

from parameterized import parameterized

class TestSequence(unittest.TestCase):
    @parameterized.expand([
        ["foo", "a", "a",],
        ["bar", "a", "b"],
        ["lee", "b", "b"],
    ])
    def test_sequence(self, name, a, b):
        self.assertEqual(a,b)

Which will generate the tests:

test_sequence_0_foo (__main__.TestSequence) ... ok
test_sequence_1_bar (__main__.TestSequence) ... FAIL
test_sequence_2_lee (__main__.TestSequence) ... ok

======================================================================
FAIL: test_sequence_1_bar (__main__.TestSequence)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/parameterized/parameterized.py", line 233, in <lambda>
    standalone_func = lambda *a: func(*(a + p.args), **p.kwargs)
  File "x.py", line 12, in test_sequence
    self.assertEqual(a,b)
AssertionError: 'a' != 'b'

For historical reasons I'll leave the original answer circa 2008):

I use something like this:

import unittest

l = [["foo", "a", "a",], ["bar", "a", "b"], ["lee", "b", "b"]]

class TestSequense(unittest.TestCase):
    pass

def test_generator(a, b):
    def test(self):
        self.assertEqual(a,b)
    return test

if __name__ == '__main__':
    for t in l:
        test_name = 'test_%s' % t[0]
        test = test_generator(t[1], t[2])
        setattr(TestSequense, test_name, test)
    unittest.main()

how to parameterize unittest test methods

Your numbered tests are really just parts of a single test. What you should define as separate tests are functions that use your parameter sets.

class TestProject(unittest.TestCase):
    def _create_project(self):
        ...
    def _page1(self, projectName, projectDescription):
       ...
    def _page2(self):
       ...
    def _finish(self):
       ...

    def _run(self, name, descr):
        self._create_project()
        self._page1(name, descr)
        self._page2()
        self._finish()

    def test001(self):
        self._run("foo", "do foo")

    def test002(self):
        self._run("bar", "do bar")

    # etc

An interesting module that can reduce some of the boilerplate is the ddt project.

import ddt

@ddt.ddt
class TestProject(unittest.TestCase):
    def _create_project(self):
        ...
    def _page1(self, projectName, projectDescription):
       ...
    def _page2(self):
       ...
    def _finish(self):
       ...

    @ddt.data(
      ("foo", "do foo"),
      ("bar", "do bar"),
      # etc
    )
    @ddt.unpack
    def test_run(self, name, descr):
        self._create_project()
        self._page1(name, descr)
        self._page2()
        self._finish()

Python's unittest and dynamic creation of test cases

For this you should use test generators in nose. All you need to do is yield a tuple, with the first being a function and the rest being the args. From the docs here is the example.

def test_evens():
    for i in range(0, 5):
        yield check_even, i, i*3

def check_even(n, nn):
    assert n % 2 == 0 or nn % 2 == 0

How to dynamically create parameterized test from the file (json or yaml) in python

In case you just want to pass params from file to single function:

Yes, you can use decorator, pass filename to it, load json from file and use as decorated func's params:

import json
from functools import wraps

def params_from_file(file):
    """Decorator to load params from json file."""
    def decorator(func_to_decorate):
        @wraps(func_to_decorate)
        def wrapper(self, *args, **kwargs):
            with open(file, 'r') as fh:
                kwargs = json.loads(fh.read())
                return func_to_decorate(self, **kwargs)
        return wrapper
    return decorator

Usage:

import unittest

class Test(unittest.TestCase):
    @params_from_file('file.txt')  # {"name":"url_1", "url":"http://check.com", "expected": "23"}
    def test_url(self, name, url, expected):        
        self.assertEqual(name, 'url_1')
        self.assertEqual(url, 'http://check.com')
        self.assertEqual(expected, '23')
        # print(name, url, expected)

if __name__ == "__main__":
    unittest.main()

In case you want to create multiple new tests from params set:

import json

def create_tests(func_name, file):
    def decorator(cls):
        func = getattr(cls, func_name)
        # Set new funcs to class:
        with open(file, 'r') as fh:
            data = json.loads(fh.read())['data']
            for i, params in enumerate(data):
                def tmp(params=params):  # need for http://stackoverflow.com/q/7546285/1113207
                    def wrapper(self, *args, **kwargs):
                        return func(self, **params)      
                    return wrapper          
                setattr(cls, func_name + '_' + str(i), tmp())
        # Remove func from class:
        setattr(cls, func_name, None)
        return cls
    return decorator

Usage:

import unittest

@create_tests('test_url', 'file.txt')
class Test(unittest.TestCase):
    def test_url(self, name, url, expected):
        print(name, url, expected)

if __name__ == "__main__":
    unittest.main(verbosity=2)

Output:

test_url_0 (__main__.Test) ... name1 https://example.com 175
ok
test_url_1 (__main__.Test) ... name2 https://example2.com 15
ok

----------------------------------------------------------------------
Ran 2 tests in 0.000s

How to automatically generate unit testing routines from special syntax in method docstring / comment?

It's called doctest and it's part of the standard library. But you'll have to change your syntax a bit:

def is_even(a, b):
    """Returns True if both numbers are even.

    >>> is_even(0, 2)
    True
    >>> is_even(2, 1)
    False
    >>> is_even(3, 5)
    False
    """
    return (a % 2 == 0 and b % 2 == 0)

You can run it with:

python -m doctest is_even.py

The syntax has been designed so that you can mostly copy and paste your tests from an interactive (C)Python interpreter session, so there is good reason not to try and change it. Moreover, other Python developers will already be familiar with this syntax, and not with anything custom you might come up with.

Python nose processes parameter and dynamically generated tests

as far as I can tell you just need to make sure that create_all is run in every test process. just moving it out of the __main__ test works for me, so the end of the file would look like:

# as above

# Create all the test cases dynamically
create_all()

if __name__ == '__main__':
    # Execute the tests
    logging.info('Start running tests.')
    nose.runmodule(name='__main__')
    logging.info('Finished running tests.')

Python parameterized unittest by subclassing TestCase

How about using pytest with to parametrize fixture:

import pytest

MyCollection = set
AnotherCollection = set

def maximise(collection, array):
    return 1

@pytest.fixture(scope='module', params=[MyCollection, AnotherCollection])
def maximise_fn(request):
    return lambda array: maximise(request.param, array)

def test_single(maximise_fn):
    assert maximise_fn([1]) == 1

def test_overflow(maximise_fn):
    assert maximise_fn([3]) == 1

If that's not an option, you can make a mixin to contain test function, and subclasses to provide maximise_fns:

import unittest

MyCollection = set
AnotherCollection = set

def maximise(collection, array):
    return 1

class TestCollectionMixin:
    def test_single(self):
        self.assertEqual(self.maximise_fn([1]), 1)

    def test_overflow(self):
        self.assertEqual(self.maximise_fn([3]), 1)

class TestMyCollection(TestCollectionMixin, unittest.TestCase):
    maximise_fn = lambda self, array: maximise(MyCollection, array)

class TestAnotherCollection(TestCollectionMixin, unittest.TestCase):
    maximise_fn = lambda self, array: maximise(AnotherCollection, array)

if __name__ == '__main__':
    unittest.main()

---

Related Topics