How to Time a Code Segment for Testing Performance with Pythons Timeit

How can I time a code segment for testing performance with Pythons timeit?

You can use time.time() or time.clock() before and after the block you want to time.

import time

t0 = time.time()
code_block
t1 = time.time()

total = t1-t0

This method is not as exact as timeit (it does not average several runs) but it is straightforward.

time.time() (in Windows and Linux) and time.clock() (in Linux) are not precise enough for fast functions (you get total = 0). In this case or if you want to average the time elapsed by several runs, you have to manually call the function multiple times (As I think you already do in you example code and timeit does automatically when you set its number argument)

import time

def myfast():
   code

n = 10000
t0 = time.time()
for i in range(n): myfast()
t1 = time.time()

total_n = t1-t0

In Windows, as Corey stated in the comment, time.clock() has much higher precision (microsecond instead of second) and is preferred over time.time().

How to measure time taken between lines of code in python?

If you want to measure CPU time, can use time.process_time() for Python 3.3 and above:

import time
start = time.process_time()
# your code here    
print(time.process_time() - start)

First call turns the timer on, and second call tells you how many seconds have elapsed.

There is also a function time.clock(), but it is deprecated since Python 3.3 and will be removed in Python 3.8.

There are better profiling tools like timeit and profile, however time.process_time() will measure the CPU time and this is what you're are asking about.

If you want to measure wall clock time instead, use time.time().

How to time functions with timeit and save results

Function timeit does not have direct access to the global variables. You must provide the dictionary of the global variabes through the globals keyword. Since the variable ps is first read in the timed statement, it is by default considered local. Mark it as global:

# This part is only for testing
ps = []
def f(x,y): return x+y
A, B = 10, 11

timeit('global ps; ps += [f(A,B)]', number=1, globals=globals())

Run setup code exactly once for different test statements using timeit

The easiest way to achieve this is to put the setup code into a module setup.py, and then use

"from setup import name_1, name_2, name_3"

as setup parameter to timeit.timeit(). (Note that wildcard imports won't work correctly in that situation.)

If you don't want to use a separate module, put the code in the main module and use

"from __main__ import name_1, name_2, name_3"

Show timeit progress

You can create your subclass of timeit.Timer that uses tqdm to track the total iterations performed.

from timeit import Timer, default_number
from tqdm import tqdm
import itertools
import gc

class ProgressTimer(Timer):
    def timeit(self, number=default_number):
        """Time 'number' executions of the main statement.
        To be precise, this executes the setup statement once, and
        then returns the time it takes to execute the main statement
        a number of times, as a float measured in seconds.  The
        argument is the number of times through the loop, defaulting
        to one million.  The main statement, the setup statement and
        the timer function to be used are passed to the constructor.
        """
        # wrap the iterator in tqdm
        it = tqdm(itertools.repeat(None, number), total=number)
        gcold = gc.isenabled()
        gc.disable()
        try:
            timing = self.inner(it, self.timer)
        finally:
            if gcold:
                gc.enable()
        # the tqdm bar sometimes doesn't flush on short timers, so print an empty line
        print()
        return timing

To use this object, we just need to pass in the script we want to run. You can either define it as a string (like below) or you can simply open the file for reading and read to a variable.

py_setup = 'import numpy as np'

py_script = """
x = np.random.rand(1000)
x.sum()
"""

pt = ProgressTimer(py_script, setup=py_setup)
pt.timeit()

# prints / returns:
100%|███████████████████████████████████████████████| 1000000/1000000 [00:13<00:00, 76749.68it/s]
13.02982600001269

Using timeit to time algorithms without timing already sorted or setup

Your understanding is correct. However, there are some issues that could be improved.

Firstly, you usually want to measure only one algorithm at a time. If you measure each algorithm separately you can compare the two and get more accurate data. Also, I would stick to measuring it on the same graph multiple times and then average the time to get one more accurate (you can use copy.deepcopy() to make duplicates of the graph). Then do this on each of the graphs you have. Since each algorithm may be more efficient on different types of graphs (you would lose this information completely if measured as you propose).

Secondly, since timeit can repeat the measured operation multiple times but the setup is done only once before any measurement starts (see the docs) you would measure the algorithm on already traversed (or sorted as you put it) graphs. Which you correctly point out.

As a solution, I suggest manually timing the runs using the time.perf_counter_ns(). The code for measuring the bellmanFord algorithm could look as such:

import time
import copy
"""
Measures bellman ford algorithm ran on a *graph* *repetitions* number of times.
Returns time measured
"""
def measure_bellman_ford(repetitions, graph):
    graphs = [copy.deepcopy(graph) for x in range(repetitions)]
    
    # make measurements
    start = time.perf_counter_ns()
    for idx in range(repetitions):
        graphs[idx].bellmanFord(0)
    end = time.perf_counter_ns()
    
    return (end - start) / repetitions

---

Related Topics