How to Repeat Each Test Multiple Times in a Py.Test Run

How can I repeat each test multiple times in a py.test run?

In order to run each test a number of times, we will programmatically parameterize each test as the tests are being generated.

First, let's add the parser option (include the following in one of your conftest.py's):

def pytest_addoption(parser):
    parser.addoption('--repeat', action='store',
        help='Number of times to repeat each test')

Now we add a "pytest_generate_tests" hook. Here is where the magic happens.

def pytest_generate_tests(metafunc):
    if metafunc.config.option.repeat is not None:
        count = int(metafunc.config.option.repeat)

        # We're going to duplicate these tests by parametrizing them,
        # which requires that each test has a fixture to accept the parameter.
        # We can add a new fixture like so:
        metafunc.fixturenames.append('tmp_ct')

        # Now we parametrize. This is what happens when we do e.g.,
        # @pytest.mark.parametrize('tmp_ct', range(count))
        # def test_foo(): pass
        metafunc.parametrize('tmp_ct', range(count))

Running without the repeat flag:

(env) $ py.test test.py -vv
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
collected 2 items 

test.py:4: test_1 PASSED
test.py:8: test_2 PASSED

=========================== 2 passed in 0.01 seconds ===========================

Running with the repeat flag:

(env) $ py.test test.py -vv --repeat 3
============================= test session starts ==============================
platform darwin -- Python 2.7.5 -- py-1.4.20 -- pytest-2.5.2 -- env/bin/python
collected 6 items 

test.py:4: test_1[0] PASSED
test.py:4: test_1[1] PASSED
test.py:4: test_1[2] PASSED
test.py:8: test_2[0] PASSED
test.py:8: test_2[1] PASSED
test.py:8: test_2[2] PASSED

=========================== 6 passed in 0.01 seconds ===========================

Further reading:

• https://pytest.org/latest/plugins.html#well-specified-hooks
• https://pytest.org/latest/example/parametrize.html

How do I repeat a python PyTest multiple times, utilizing values carried over from the previous test

One thing you could do is run a for-loop in the test.

def test_hardware():
    previous_id = 1
    set_hardware_id(1)
    for i in range(2, 100):
        current_id = get_hardware_id()
        # this fill fail at the first mismatch
        assert current_id == previous_id
        previous_id = current_id
        set_hardware_id(i)

If you want to run the whole test itself repeatedly, use the @pytest.mark.parametrize decorator.

@pytest.mark.parametrize("test_input", range(100))
def test_hardware(test_input):
    set_hardware_id(test_input)
    result = get_hardware_id()
    assert result == test_input

How to run a pytest method multiple times?

 @pytest.mark.run(order=1)
 def test_registerLink(self):
    for i in range(100):
        self.rg.register()
        self.rg.select_state_name()
        self.rg.select_city_name()
        self.rg.select_ready_wait()
        self.rg.select_ready_pay()
        self.rg.select_submit()
        self.rg.driver.back()
        self.rg.driver.refresh()

I was able to solve it by doing this

Running parameterized pytests in a loop

You could put the repeat on a class and then set the --repeat-scope argument to class. See the docs:

If you want to override default tests executions order, you can use --repeat-scope command line option with one of the next values: session, module, class or function (default). It behaves like a scope of the pytest fixture.

function (default) scope repeats each test count or repeat times before executing next test. session scope repeats whole tests session, i.e. all collected tests executed once, then all such tests executed again and etc. class and module behaves similar session , but repeating set of tests is a tests from class or module, not all collected tests.

@pytest.mark.repeat(2)
class Test:
    @pytest.mark.parametrize(
        "username, password",
        [("abc@abc.com", "aaa"), ("def@def.com", "ddd"), ("efg@efg.com", "eee")],
    )
    def test(self, username, password):
        print(username, password)

Then run pytest -s --repeat-scope class test.py:

============================ test session starts =============================
platform darwin -- Python 3.9.4, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /Users/flakes/workspace
plugins: requests-mock-1.9.2, cov-2.12.0, mock-3.6.1, repeat-0.9.1
collected 6 items                                                                

test.py abc@abc.com aaa
.def@def.com ddd
.efg@efg.com eee
.abc@abc.com aaa
.def@def.com ddd
.efg@efg.com eee
.

============================= 6 passed in 0.01s ==============================

---

Related Topics