Logging Uncaught Exceptions in Python

Logging uncaught exceptions in Python

As Ned pointed out, sys.excepthook is invoked every time an exception is raised and uncaught. The practical implication of this is that in your code you can override the default behavior of sys.excepthook to do whatever you want (including using logging.exception).

As a straw man example:

import sys
def foo(exctype, value, tb):
    print('My Error Information')
    print('Type:', exctype)
    print('Value:', value)
    print('Traceback:', tb)

Override sys.excepthook:

>>> sys.excepthook = foo

Commit obvious syntax error (leave out the colon) and get back custom error information:

>>> def bar(a, b)
My Error Information
Type: <type 'exceptions.SyntaxError'>
Value: invalid syntax (<stdin>, line 1)
Traceback: None

For more information about sys.excepthook, read the docs.

Store unhandled exceptions in log file

I would like to avoid encapsulating the whole code into a try: except statement.

I'd recommend doing the whole code in def main() then - it would be already useful if you decide to reuse parts of your code so that no code gets executed when importing the file. :)

Then you can do a catch-all thing in if __name__== "__main__"::

if __name__ == "__main__":
    try:
        main()
    except Exception as e: 
        logger.exception("Program stopped because of a critical error.")
        raise

[That's literally how I do it in my own code. It doesn't need any tricks, I literally added it for unexpected exceptions happened so that I could debug later.]

raise re-raises the same error that was caught - so it's uncaught to the outside world. And .exception does the same thing as .error but includes the exception info on its own.

Using python's logging module to log all exceptions and errors

It's probably a bad idea to log any exception thrown within the program, since Python uses exceptions also for normal control flow.

Therefore you should only log uncaught exceptions. You can easily do this using a logger's exception() method, once you have an exception object.

To handle all uncaught exceptions, you can either wrap your script's entry point in a try...except block, or by installing a custom exception handler by re-assigning sys.excepthook():

import logging
import sys

logger = logging.getLogger('mylogger')
# Configure logger to write to a file...

def my_handler(type, value, tb):
    logger.exception("Uncaught exception: {0}".format(str(value)))

# Install exception handler
sys.excepthook = my_handler

# Run your main script here:
if __name__ == '__main__':
    main()

Python, tkinter and imported classes: logging uncaught exceptions

Aha, I have found the issue.
The excepthook that is installed in the aforementioned way is only working on a higher level. I figured that out when I noticed that the exceptions that my class was throwing were still logging in the console while a keyboard interrupt did log in the file as I wanted.

Next thing I found was that you can have similar issues when using threads in Python. The main thread would still log as wanted but the exceptions in the other thread would not. Then I figured that classes can override my overriding of what to do with exceptions. Since all of my code is running in the tkinter mainloop I might need to tell that bit to do what I want it to do. This insight led me to this SO answer and with one line of code everything was fixed, all the way at the end of the main class:

# Run the app
root = tk.Tk()
root.report_callback_exception = exception_handler # this here.
app = ExceptMain(root)
root.mainloop()

Python: sys.excepthook and logging uncaught exceptions across multiple modules

sys.excepthook is global for your Python process. The last value that you set to it wins. Using unique names in different files for module loggers won't have any effect on it.

Logging every exception within custom Python package

Your exception hook can test to see if the exception is being raised in your package by seeing if it's one of your exceptions (use issubclass() to see whether the passed-in exception type is derived from your package's exception base class) or comes from one of your modules (check the passed-in traceback object; each stack frame's code object contains the filename in which the function was defined).

However, this only logs unhandled exceptions. If you want to log all exceptions, even if they are handled, the best place to do that is in the __init__ method of your exception base class. You'll need to use the inspect module to identify who is raising the exception since it hasn't actually been raised at that point, but it's doable.

Log exception with traceback in Python

Use logging.exception from within the except: handler/block to log the current exception along with the trace information, prepended with a message.

import logging
LOG_FILENAME = '/tmp/logging_example.out'
logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG)

logging.debug('This message should go to the log file')

try:
    run_my_stuff()
except:
    logging.exception('Got exception on main handler')
    raise

Now looking at the log file, /tmp/logging_example.out:

DEBUG:root:This message should go to the log file
ERROR:root:Got exception on main handler
Traceback (most recent call last):
  File "/tmp/teste.py", line 9, in <module>
    run_my_stuff()
NameError: name 'run_my_stuff' is not defined

How to write a unit test for uncaught exception handler

Instead of testing that your function is called for uncaught exceptions, it's probably best to instead test that the excepthook is installed, and that the function does the right thing when you call it manually. That gives you pretty good evidence that the excepthook will behave properly in real usage. You'll want to move your uncaught_exception_handler outside of init_uncaught_exception_logger so your tests can access it more easily.

assert sys.excepthook is uncaught_exception_handler
with your_preferred_output_capture_mechanism:
    try:
        1/0
    except ZeroDivisionError:
        uncaught_exception_handler(*sys.exc_info())
assert_something_about_captured_output()

If you want to actually invoke excepthook through an uncaught exception, then you'll need to launch a subprocess and examine its output. The subprocess module is the way to go for that.

How to log all exceptions in python 3

You could use decorators to wrap individual functions and methods with code that logs any uncaught exception.

This means one additional line of code in front of any function definition that should do this logging. Advantage is: Can be quickly added and removed.

The decorator also could get arguments. For example you could specify the base class of those exceptions that you want to log.

Here's a basic question & concise answer on how to write decorators.

---

Related Topics