How to Use Timeit Module

How to use timeit module

The way timeit works is to run setup code once and then make repeated calls to a series of statements. So, if you want to test sorting, some care is required so that one pass at an in-place sort doesn't affect the next pass with already sorted data (that, of course, would make the Timsort really shine because it performs best when the data already partially ordered).

Here is an example of how to set up a test for sorting:

>>> import timeit

>>> setup = '''
import random

random.seed('slartibartfast')
s = [random.random() for i in range(1000)]
timsort = list.sort
'''

>>> print min(timeit.Timer('a=s[:]; timsort(a)', setup=setup).repeat(7, 1000))
0.334147930145

Note that the series of statements makes a fresh copy of the unsorted data on every pass.

Also, note the timing technique of running the measurement suite seven times and keeping only the best time -- this can really help reduce measurement distortions due to other processes running on your system.

Those are my tips for using timeit correctly. Hope this helps :-)

How do I use timeit module inside a decorator

the problem as I mention is in the setup, doing that import get you the already decorated function, that when executed once again do the import and get the decorated function and goes and execute the import and soo on until it blow up the stack... and we don't want that.

The timeit module can also take function as its stmt, so with just building a lambda function it solve the problem and we can use the undercoated function that way

The stmt and setup parameters can also take objects that are callable without arguments. This will embed calls to them in a timer function that will then be executed by timeit(). Note that the timing overhead is a little larger in this case because of the extra function calls.

import timeit
from functools import wraps
    
def tool(func):
    """Include debug statement and timeit setup"""
    @wraps(func)
    def wrapper(*args):
        print(func.__doc__, args)
        res = func(*args)
        print(res)
        times = timeit.repeat(
            stmt=lambda: func(*args),
            repeat=3, number=10000)
        print(times)
    return wrapper

and a quick test

>>> @tool
def fun():
    return 42
>>> fun()
None ()
42
[0.0013318000000026586, 0.0013294000000314554, 0.0013452000000597764]
>>> 

What is %timeit in Python?

%timeit is an IPython magic function, which can be used to time a particular piece of code (a single execution statement, or a single method).

From the documentation:

%timeit

   Time execution of a Python statement or expression

   Usage, in line mode:
       %timeit [-n<N> -r<R> [-t|-c] -q -p<P> -o] statement

To use it, for example if we want to find out whether using xrange is any faster than using range, you can simply do:

In [1]: %timeit for _ in range(1000): True
10000 loops, best of 3: 37.8 µs per loop

In [2]: %timeit for _ in xrange(1000): True
10000 loops, best of 3: 29.6 µs per loop

And you will get the timings for them.

The major advantage of %timeit are:

• that you don't have to import timeit.timeit from the standard library, and run the code multiple times to figure out which is the better approach.

• %timeit will automatically calculate number of runs required for your code based on a total of 2 seconds execution window.

• You can also make use of current console variables without passing the whole code snippet as in case of timeit.timeit to built the variable that is built in another environment that timeit works.

timeit on a class function

The timeit.timeit() function runs the code you supply in the stmt argument, but it doesn't run in the same scope^{not sure this is the right word} as your other code. You can include an argument setup that contains code to run before the timed code in stmt, so that it can import modules, etc. In this statement, you can also get timeit to import stuff from your __main__ code like so:

print(timeit.timeit("bubble_small.sorts()", "from __main__ import bubble_small"))
print(timeit.timeit("bubble_large.sorts()", "from __main__ import bubble_large"))

Since your code takes quite some time to execute, it might be worth specifying the number of runs you want to time with the number=100 (or some suitably sized integer) argument to timeit()

print(timeit.timeit("bubble_small.sorts()", "from __main__ import bubble_small", number=100))
print(timeit.timeit("bubble_large.sorts()", "from __main__ import bubble_large", number=100))

Is this the proper way to use the timeit module?

Documentation link.

• Processes running on the system may interfere with the timing as
  timeit measures clock time, not CPU time.

• By default, timeit() temporarily turns off garbage collection during the timing. The advantage of this approach is that it makes independent timings more comparable. This disadvantage is that GC may be an important component of the performance of the function being measured.

To call timeit() repeatedly,

def test():
L = []
for i in range(10):
L.append(i)

if __name__ == '__main__':
import timeit
print(timeit.repeat("test()", setup="from __main__ import test"))


How to use python timeit when passing variables to functions?

Make it a callable:

if __name__=='__main__':
    from timeit import Timer
    t = Timer(lambda: superMegaIntenseFunction(10))
    print(t.timeit(number=1))

Should work

how to pass parameters of a function when using timeit.Timer()

The code snippets must be self-contained - they cannot make external references. You must define your values in the statement-string or setup-string:

import timeit

setup = """
A = 1
B = 2

def foo(num1, num2):
    pass

def mainprog():
    global A,B
    for i in range(20):
        # do something to A and B
        foo(A, B)
"""

t = timeit.Timer(stmt="mainprog()" setup=setup)
print(t.timeit(5))

Better yet, rewrite your code to not use global values.

How to use timeit for multiple values

Just a note: you need to use timeit.Timer instead of timeit.timer or you will get AttributeError: module 'timeit' has no attribute 'timer' although you might as well use timeit.timeit which is useful for single use timing.

---

I'd recommend looking at How to use timeit module for efficient ways to use the timeit module, specifically using the from __main__ import syntax:

def function(stuff):
    NotImplemented

def setup():
    NotImplemented

setup_code = """
from __main__ import function, setup
data = setup() 
"""
func_code = "function(data)"

result = timeit.timeit(func_code,setup_code)

This way the setup function is called to generate the necessary data for the benchmark. Because it is a function it also has access to the global namespace in your module, so it can use information created in the main program in part of its code:

import timeit

def function(stuff):
    NotImplemented

def setup():
    print("the setup_data is:",setup_data)

setup_code = """
from __main__ import function, setup
data = setup() 
"""
func_code = "function(data)"

for i in range(4):
    setup_data = i
    result = timeit.timeit(func_code,setup_code)
    print("result is",result)

So just define setup_data as the data necessary for the setup (dimensions of matrix for example) before running timeit.timeit and you are good to go!

---

It makes sense to use repeat instead of timeit if you want to run the setup multiple times for the benchmarks but not every time.

For example lets say you have a sorting algorithm that varies in speed depending on how close to sorted the list is already. if you just use timeit the results could vary by a lot since after setup it will use the same list every time. You also wouldn't want to generate a list every pass since it would interfere with the actual benchmark.

So in that case it would make sense to use repeat to sort different lists without including the list creation in the benchmark, plus you can then use the entries to look at the max or min times and average etc.

Don't understand how Timeit works. Need an explanation

If you want to pass arguments to your function you might want to use timeit.Timer, but make your list global like this:

prod_nums = ['V475', 'F987', 'Q143', 'R688']

And then run this:

from timeit import Timer
t = Timer(lambda: search_fast(prod_nums))
print t.timeit() # In my case will print 0.336354970932
t = Timer(lambda: search_slow(prod_nums))
print t.timeit() # 0.374251127243

timeit is useful when you want to inspect small piece of code in your dev environment.

If your function looks like that:

def search_slow():
    prod_nums = ['V475', 'F987', 'Q143', 'R688']
    return_value = False
    for item in prod_nums:
        if item == 'R688':
            return_value = True
    return return_value

You can use timeit.timeit

import timeit
timeit.timeit(search_slow)
>>> 0.3833189010620117

This will not return any result thou, only the time it took. This is another scenario in which you can use decorator.
Basically you can use timeit to tell you how much time does it take for a function to execute, much like time sample_file.py in your terminal.

Based on python docs (https://docs.python.org/2/library/timeit.html):
This module provides a simple way to time small bits of Python code. It has both a Command-Line Interface as well as a callable one. It avoids a number of common traps for measuring execution times.

---

Related Topics