How to Profile Python Code Line-By-Line

How can I profile Python code line-by-line?

I believe that's what Robert Kern's line_profiler is intended for. From the link:

File: pystone.py
Function: Proc2 at line 149
Total time: 0.606656 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   149                                           @profile
   150                                           def Proc2(IntParIO):
   151     50000        82003      1.6     13.5      IntLoc = IntParIO + 10
   152     50000        63162      1.3     10.4      while 1:
   153     50000        69065      1.4     11.4          if Char1Glob == 'A':
   154     50000        66354      1.3     10.9              IntLoc = IntLoc - 1
   155     50000        67263      1.3     11.1              IntParIO = IntLoc - IntGlob
   156     50000        65494      1.3     10.8              EnumLoc = Ident1
   157     50000        68001      1.4     11.2          if EnumLoc == Ident1:
   158     50000        63739      1.3     10.5              break
   159     50000        61575      1.2     10.1      return IntParIO

How to profile python 3.5 code line by line in jupyter notebook 5

You can use line_profiler in jupyter notebook.

1. Install it: pip install line_profiler
2. Within your jupyter notebook, call: %load_ext line_profiler
3. Define your function prof_function as in your example.
4. Finally, profile as follows: %lprun -f prof_function prof_function()

Which will provide the output:

Timer unit: 1e-06 s

Total time: 3e-06 s
File: <ipython-input-22-41854af628da>
Function: prof_function at line 1

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           def prof_function():
     2         1          1.0      1.0     33.3      x=10*20
     3         1          1.0      1.0     33.3      y=10+x
     4         1          1.0      1.0     33.3      return (y)

How do I use line_profiler (from Robert Kern)?

Just follow Dan Riti's example from the first link, but use your code. All you have to do after installing the line_profiler module is add a @profile decorator right before each function you wish to profile line-by-line and make sure each one is called at least once somewhere else in the code—so for your trivial example code that would be something like this:

example.py file:

@profile
def do_stuff(numbers):
    print numbers

numbers = 2
do_stuff(numbers)

Having done that, run your script via the kernprof.py^✶ that was installed in your C:\Python27\Scripts directory. Here's the (not very interesting) actual output from doing this in a Windows 7 command-line session:

> python "C:\Python27\Scripts\kernprof.py" -l -v example.py
2
Wrote profile results to example.py.lprof
Timer unit: 3.2079e-07 s

File: example.py
Function: do_stuff at line 2
Total time: 0.00185256 s

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
     1                                           @profile
     2                                           def do_stuff(numbers):
     3         1         5775   5775.0    100.0      print numbers

You likely need to adapt this last step—the running of your test script with kernprof.py instead of directly by the Python interpreter—in order to do the equivalent from within IDLE or PyScripter.

^✶Update

It appears that in line_profiler v1.0, the kernprof utility is distributed as an executable, not a .py script file as it was when I wrote the above. This means the following now needs to used to invoke it from the command-line:

> "C:\Python27\Scripts\kernprof.exe" -l -v example.py

How do I profile a Python script?

Python includes a profiler called cProfile. It not only gives the total running time, but also times each function separately, and tells you how many times each function was called, making it easy to determine where you should make optimizations.

You can call it from within your code, or from the interpreter, like this:

import cProfile
cProfile.run('foo()')

Even more usefully, you can invoke the cProfile when running a script:

python -m cProfile myscript.py

To make it even easier, I made a little batch file called 'profile.bat':

python -m cProfile %1

So all I have to do is run:

profile euler048.py

And I get this:

1007 function calls in 0.061 CPU seconds

Ordered by: standard name
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    1    0.000    0.000    0.061    0.061 <string>:1(<module>)
 1000    0.051    0.000    0.051    0.000 euler048.py:2(<lambda>)
    1    0.005    0.005    0.061    0.061 euler048.py:2(<module>)
    1    0.000    0.000    0.061    0.061 {execfile}
    1    0.002    0.002    0.053    0.053 {map}
    1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler objects}
    1    0.000    0.000    0.000    0.000 {range}
    1    0.003    0.003    0.003    0.003 {sum}

EDIT: Updated link to a good video resource from PyCon 2013 titled
Python Profiling
Also via YouTube.

commenting out @profile decorators when line-profiling

You could try to add something like this at the top of your script:

try:
    profile  # throws an exception when profile isn't defined
except NameError:
    profile = lambda x: x   # if it's not defined simply ignore the decorator.

That way you define the profile function as no-op decorator if it's not defined.

---

Related Topics