What Is %Timeit in Python

What is %timeit in Python?

%timeit is an IPython magic function, which can be used to time a particular piece of code (a single execution statement, or a single method).

From the documentation:

%timeit

   Time execution of a Python statement or expression

   Usage, in line mode:
       %timeit [-n<N> -r<R> [-t|-c] -q -p<P> -o] statement

To use it, for example if we want to find out whether using xrange is any faster than using range, you can simply do:

In [1]: %timeit for _ in range(1000): True
10000 loops, best of 3: 37.8 µs per loop

In [2]: %timeit for _ in xrange(1000): True
10000 loops, best of 3: 29.6 µs per loop

And you will get the timings for them.

The major advantage of %timeit are:

• that you don't have to import timeit.timeit from the standard library, and run the code multiple times to figure out which is the better approach.

• %timeit will automatically calculate number of runs required for your code based on a total of 2 seconds execution window.

• You can also make use of current console variables without passing the whole code snippet as in case of timeit.timeit to built the variable that is built in another environment that timeit works.

How to use timeit module

The way timeit works is to run setup code once and then make repeated calls to a series of statements. So, if you want to test sorting, some care is required so that one pass at an in-place sort doesn't affect the next pass with already sorted data (that, of course, would make the Timsort really shine because it performs best when the data already partially ordered).

Here is an example of how to set up a test for sorting:

>>> import timeit

>>> setup = '''
import random

random.seed('slartibartfast')
s = [random.random() for i in range(1000)]
timsort = list.sort
'''

>>> print min(timeit.Timer('a=s[:]; timsort(a)', setup=setup).repeat(7, 1000))
0.334147930145

Note that the series of statements makes a fresh copy of the unsorted data on every pass.

Also, note the timing technique of running the measurement suite seven times and keeping only the best time -- this can really help reduce measurement distortions due to other processes running on your system.

Those are my tips for using timeit correctly.

Don't understand how Timeit works. Need an explanation

If you want to pass arguments to your function you might want to use timeit.Timer, but make your list global like this:

prod_nums = ['V475', 'F987', 'Q143', 'R688']

And then run this:

from timeit import Timer
t = Timer(lambda: search_fast(prod_nums))
print t.timeit() # In my case will print 0.336354970932
t = Timer(lambda: search_slow(prod_nums))
print t.timeit() # 0.374251127243

timeit is useful when you want to inspect small piece of code in your dev environment.

If your function looks like that:

def search_slow():
    prod_nums = ['V475', 'F987', 'Q143', 'R688']
    return_value = False
    for item in prod_nums:
        if item == 'R688':
            return_value = True
    return return_value

You can use timeit.timeit

import timeit
timeit.timeit(search_slow)
>>> 0.3833189010620117

This will not return any result thou, only the time it took. This is another scenario in which you can use decorator.
Basically you can use timeit to tell you how much time does it take for a function to execute, much like time sample_file.py in your terminal.

Based on python docs (https://docs.python.org/2/library/timeit.html):
This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.

What unit of time does timeit return?

The return value is seconds as a float.

It is the total time taken to run the test (not counting the setup), so the average time per test is that number divided by the number argument, which defaults to 1 million.

See the Time.timeit() documentation:

Time number executions of the main statement. This executes the setup statement once, and then returns the time it takes to execute the main statement a number of times, measured in seconds as a float. The argument is the number of times through the loop, defaulting to one million.

Measure : ipython timeit vs timeit method

I am not aware of anything standalone, but this can be easily extracted out of IPython, most of what it does is in the TimeitResult class which formats the output https://github.com/ipython/ipython/blob/8520f3063ca36655b5febbbd18bf55e59cb2cbb5/IPython/core/magics/execution.py#L55-L104

Then there's the nicer reporting of the compile time, worst run being much worse than the fastest, and getting the run number automatically like timeit's cli.

Reusing the code from IPython and python's timeit, and removing some checks that are probably unnecessary for the common use, we can get a simple function to do the timing in the same way IPython does:

import math
import timeit

def _format_time(timespan, precision=3):
    """Formats the timespan in a human readable form"""
    units = ["s", "ms", "\xb5s", "ns"]
    scaling = [1, 1e3, 1e6, 1e9]
    if timespan > 0.0:
        order = min(-int(math.floor(math.log10(timespan)) // 3), 3)
    else:
        order = 3
    scaled_time = timespan * scaling[order]
    unit = units[order]
    return f"{scaled_time:.{precision}g} {unit}"

class TimeitResult(object):
    """
    Object returned by the timeit magic with info about the run.

    Contains the following attributes :

    loops: (int) number of loops done per measurement
    repeat: (int) number of times the measurement has been repeated
    best: (float) best execution time / number
    all_runs: (list of float) execution time of each run (in s)
    compile_time: (float) time of statement compilation (s)
    """

    def __init__(self, loops, repeat, best, worst, all_runs, compile_time, precision):
        self.loops = loops
        self.repeat = repeat
        self.best = best
        self.worst = worst
        self.all_runs = all_runs
        self.compile_time = compile_time
        self._precision = precision
        self.timings = [dt / self.loops for dt in all_runs]

    @property
    def average(self):
        return math.fsum(self.timings) / len(self.timings)

    @property
    def stdev(self):
        mean = self.average
        return (
            math.fsum([(x - mean) ** 2 for x in self.timings]) / len(self.timings)
        ) ** 0.5

    def __str__(self):
        return "{mean} {pm} {std} per loop (mean {pm} std. dev. of {runs} run{run_plural}, {loops} loop{loop_plural} each)".format(
            pm="+-",
            runs=self.repeat,
            loops=self.loops,
            loop_plural="" if self.loops == 1 else "s",
            run_plural="" if self.repeat == 1 else "s",
            mean=_format_time(self.average, self._precision),
            std=_format_time(self.stdev, self._precision),
        )

def nice_timeit(
    stmt="pass",
    setup="pass",
    number=0,
    repeat=None,
    precision=3,
    timer_func=timeit.default_timer,
    globals=None,
):
    """Time execution of a Python statement or expression."""

    if repeat is None:
        repeat = 7 if timeit.default_repeat < 7 else timeit.default_repeat

    timer = timeit.Timer(stmt, setup, timer=timer_func, globals=globals)

    # Get compile time
    compile_time_start = timer_func()
    compile(timer.src, "<timeit>", "exec")
    total_compile_time = timer_func() - compile_time_start

    # This is used to check if there is a huge difference between the
    # best and worst timings.
    # Issue: https://github.com/ipython/ipython/issues/6471
    if number == 0:
        # determine number so that 0.2 <= total time < 2.0
        for index in range(0, 10):
            number = 10 ** index
            time_number = timer.timeit(number)
            if time_number >= 0.2:
                break

    all_runs = timer.repeat(repeat, number)
    best = min(all_runs) / number
    worst = max(all_runs) / number
    timeit_result = TimeitResult(
        number, repeat, best, worst, all_runs, total_compile_time, precision
    )

    # Check best timing is greater than zero to avoid a
    # ZeroDivisionError.
    # In cases where the slowest timing is lesser than a microsecond
    # we assume that it does not really matter if the fastest
    # timing is 4 times faster than the slowest timing or not.
    if worst > 4 * best and best > 0 and worst > 1e-6:
        print(
            f"The slowest run took {worst / best:.2f} times longer than the "
            f"fastest. This could mean that an intermediate result "
            f"is being cached."
        )

    print(timeit_result)

    if total_compile_time > 0.1:
        print(f"Compiler time: {total_compile_time:.2f} s")
    return timeit_result

nice_timeit("time.sleep(0.3)", "import time")

# IPython license
# BSD 3-Clause License
#
# - Copyright (c) 2008-Present, IPython Development Team
# - Copyright (c) 2001-2007, Fernando Perez <fernando.perez@colorado.edu>
# - Copyright (c) 2001, Janko Hauser <jhauser@zscout.de>
# - Copyright (c) 2001, Nathaniel Gray <n8gray@caltech.edu>
#
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright notice, this
#   list of conditions and the following disclaimer.
#
# * Redistributions in binary form must reproduce the above copyright notice,
#   this list of conditions and the following disclaimer in the documentation
#   and/or other materials provided with the distribution.
#
# * Neither the name of the copyright holder nor the names of its
#   contributors may be used to endorse or promote products derived from
#   this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
# SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
# CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
# OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Using timeit to compare the execution time in milliseconds between different statements

default_timer is not the best way to measure tiny pieces of code, since it can be easily affected by other system processes, and the time taken is so short it needs run multiple times to be significant.

Instead, use timeit.timeit as follows, specifying test-specific variables inside a setup block.

from timeit import timeit

setup = """
name = "Kimberley"
"""

method1 = """f'Hello, {name}, how is your day?'"""
method2 = """'Hello, {0}, how is your day?'.format(name)"""
method3 = """'Hello, %(who)s, how is your lamb?' % {'who': name}"""
method4 = """Hello, %s, how is your lamb?' % (name,)"""

for method in (method1, method2, method3, method4):
    print(timeit(setup=setup, stmt=method))
# 0.0563660740153864
# 0.17292902698274702
# 0.18335944903083146
# 0.09785140998428687

Note of course that these times (ms) are the time for timeit to execute each statement 1000000 times.

I've also added in method4 which is faster than 3 since it doesn't have to do the 'mapping' from who to name.

Show timeit progress

You can create your subclass of timeit.Timer that uses tqdm to track the total iterations performed.

from timeit import Timer, default_number
from tqdm import tqdm
import itertools
import gc

class ProgressTimer(Timer):
    def timeit(self, number=default_number):
        """Time 'number' executions of the main statement.
        To be precise, this executes the setup statement once, and
        then returns the time it takes to execute the main statement
        a number of times, as a float measured in seconds.  The
        argument is the number of times through the loop, defaulting
        to one million.  The main statement, the setup statement and
        the timer function to be used are passed to the constructor.
        """
        # wrap the iterator in tqdm
        it = tqdm(itertools.repeat(None, number), total=number)
        gcold = gc.isenabled()
        gc.disable()
        try:
            timing = self.inner(it, self.timer)
        finally:
            if gcold:
                gc.enable()
        # the tqdm bar sometimes doesn't flush on short timers, so print an empty line
        print()
        return timing

To use this object, we just need to pass in the script we want to run. You can either define it as a string (like below) or you can simply open the file for reading and read to a variable.

py_setup = 'import numpy as np'

py_script = """
x = np.random.rand(1000)
x.sum()
"""

pt = ProgressTimer(py_script, setup=py_setup)
pt.timeit()

# prints / returns:
100%|███████████████████████████████████████████████| 1000000/1000000 [00:13<00:00, 76749.68it/s]
13.02982600001269

---

Related Topics