How to Step Through Python Code to Help Debug Issues

How to step through Python code to help debug issues?

Yes! There's a Python debugger called pdb just for doing that!

You can launch a Python program through pdb by using pdb myscript.py or python -m pdb myscript.py.

There are a few commands you can then issue, which are documented on the pdb page.

Some useful ones to remember are:

• b: set a breakpoint
• c: continue debugging until you hit a breakpoint
• s: step through the code
• n: to go to next line of code
• l: list source code for the current file (default: 11 lines including the line being executed)
• u: navigate up a stack frame
• d: navigate down a stack frame
• p: to print the value of an expression in the current context

If you don't want to use a command line debugger, some IDEs like Pydev, Wing IDE or PyCharm have a GUI debugger. Wing and PyCharm are commercial products, but Wing has a free "Personal" edition, and PyCharm has a free community edition.

How to debug a Python module run with python -m from the command line?

You can't do it now, because -m terminates option list

python -h
...
-m mod : run library module as a script (terminates option list)
...

That means it's mod's job to interpret the rest of the arguments list and this behavior fully depends on how mod is designed internally and whether it support another -m

Lets check out what's happening inside pdb of python 2.x. Actually, nothing intereseting, it only expects a script name to be supplied:

   if not sys.argv[1:] or sys.argv[1] in ("--help", "-h"):
        print "usage: pdb.py scriptfile [arg] ..."
        sys.exit(2)

    mainpyfile =  sys.argv[1]     # Get script filename
    if not os.path.exists(mainpyfile):
        print 'Error:', mainpyfile, 'does not exist'
        sys.exit(1)

    del sys.argv[0]         # Hide "pdb.py" from argument list

    # Replace pdb's dir with script's dir in front of module search path.
    sys.path[0] = os.path.dirname(mainpyfile)

    # Note on saving/restoring sys.argv: it's a good idea when sys.argv was
    # modified by the script being debugged. It's a bad idea when it was
    # changed by the user from the command line. There is a "restart" command
    # which allows explicit specification of command line arguments.
    pdb = Pdb()
    while True:
        try:
            pdb._runscript(mainpyfile)

Same for the currently released versions of python 3.x

Good news

The pull request that allows to do what you're asking has been merged 5 days ago. What a mysterious coincidence! Here's the code

So just wait a bit for the upcoming python 3.x versions to have this issue resolved )

How do you break into the debugger from Python source code?

import pdb; pdb.set_trace()

See Python: Coding in the Debugger for Beginners for this and more helpful hints.

Debug Python code that requires console input

If you're using Pycharm 2018.3 or above you can redirect your input to a file.
PS. I haven't tried this but it should work fine.

How does attrs fool the debugger to step into auto generated code?

Here's the definition of RawSerialPort from the documentation:

@target_factory.reg_resource
@attr.s(eq=False)
class RawSerialPort(SerialPort, Resource):
    """RawSerialPort describes a serialport which is available on the local computer."""
    def __attrs_post_init__(self):
        super().__attrs_post_init__()
        if self.port is None:
            ValueError("RawSerialPort must be configured with a port")

The class created by the class statement is not what is ultimately bound to the name RawSerialPort; it's whatever object is returned by target_factory.reg_resource. The decorator syntax desugars to

class RawSerialPort(SerialPort, Resource):
    """RawSerialPort describes a serialport which is available on the local computer."""
[docs]    def __attrs_post_init__(self):
        super().__attrs_post_init__()
        if self.port is None:
            ValueError("RawSerialPort must be configured with a port")

RawSerialPort = target_factory.reg_resource(attr.s(eq=False)(RawSerialPort))

The original class is passed first to
attr.s(eq=False), which returns a new, augmented class (one that contains the __init__ method referenced in the debugger session), and that class is passed to target_factory.reg_resource which (without actually looking up the definition), returns yet another class (possibly the same class) based on whatever the attr module created.

So the debugger isn't being fooled; quite the contrary, it's showing exactly what is happening, which isn't obvious from looking only at test.py.

Debugging: stepping through Python script using gdb?

Very interesting question. Here's my approach. Create signal_test.py:

import os
import signal

PID = os.getpid()

def do_nothing(*args):
    pass

def foo():
    print "Initializing..."
    a=10
    os.kill(PID, signal.SIGUSR1)
    print "Variable value is %d" % (a)
    print "All done!"

signal.signal(signal.SIGUSR1, do_nothing)

foo()

Then you can run it under gdb:

$ gdb --args python signal_test.py
GNU gdb (GDB) Red Hat Enterprise Linux (7.0.1-37.el5_7.1)
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /net/gs/vol3/software/modules-sw/python/2.7/Linux/RHEL5/x86_64/bin/python...done.

And when you run it, it will go until you reach the call to kill():

(gdb) run
Starting program: /net/gs/vol3/software/modules-sw/python/2.7/Linux/RHEL5/x86_64/bin/python signal_test.py
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000
[Thread debugging using libthread_db enabled]
Initializing...

Program received signal SIGUSR1, User defined signal 1.
0x0000003d340306f7 in kill () from /lib64/libc.so.6

You can then look at a backtrace:

(gdb) backtrace
#0  0x0000003d340306f7 in kill () from /lib64/libc.so.6
#1  0x00000000004d82dd in posix_kill (self=<value optimized out>, args=<value optimized out>)
    at ./Modules/posixmodule.c:4047
#2  0x000000000049b574 in call_function (f=0x8aca30, throwflag=<value optimized out>)
    at Python/ceval.c:4012
#3  PyEval_EvalFrameEx (f=0x8aca30, throwflag=<value optimized out>) at Python/ceval.c:2665
#4  0x000000000049c5cd in call_function (f=0x8ac560, throwflag=<value optimized out>)
    at Python/ceval.c:4098
#5  PyEval_EvalFrameEx (f=0x8ac560, throwflag=<value optimized out>) at Python/ceval.c:2665
#6  0x000000000049d3bb in PyEval_EvalCodeEx (co=0x2aaaae224f30, globals=<value optimized out>, 
    locals=<value optimized out>, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, 
    closure=0x0) at Python/ceval.c:3252
#7  0x000000000049d432 in PyEval_EvalCode (co=0x1a48, globals=0xa, locals=0x0) at Python/ceval.c:666
#8  0x00000000004bf321 in run_mod (fp=0x89ad60, filename=0x7fffffffb5b4 "signal_test.py", 
    start=<value optimized out>, globals=0x7e4680, locals=0x7e4680, closeit=1, flags=0x7fffffffaee0)
    at Python/pythonrun.c:1346
#9  PyRun_FileExFlags (fp=0x89ad60, filename=0x7fffffffb5b4 "signal_test.py", 
    start=<value optimized out>, globals=0x7e4680, locals=0x7e4680, closeit=1, flags=0x7fffffffaee0)
    at Python/pythonrun.c:1332
#10 0x00000000004bf5d8 in PyRun_SimpleFileExFlags (fp=<value optimized out>, 
    filename=0x7fffffffb5b4 "signal_test.py", closeit=1, flags=0x7fffffffaee0)
    at Python/pythonrun.c:936
#11 0x00000000004148cc in Py_Main (argc=<value optimized out>, argv=<value optimized out>)
    at Modules/main.c:599
#12 0x0000003d3401d994 in __libc_start_main () from /lib64/libc.so.6
#13 0x0000000000413b19 in _start ()

If you continue on, the rest of the program will run normally.

(gdb) continue
Continuing.
Variable value is 10
All done!

Program exited normally.

You can, instead, step through in the appropriate frame until you reach the statement you're interested in. You're probably going to want to run a debugging Python for this to make much sense.

---

Related Topics