Bad Idea to Catch All Exceptions in Python

Bad idea to catch all exceptions in Python

Because it's terribly nonspecific and it doesn't enable you to do anything interesting with the exception. Moreover, if you're catching every exception there could be loads of exceptions that are happening that you don't even know are happening (which could cause your application to fail without you really knowing why). You should be able to predict (either through reading documentation or experimentation) specifically which exceptions you need to handle and how to handle them, but if you're blindly suppressing all of them from the beginning you'll never know.

So, by popular request, here's an example. A programmer is writing Python code and she gets an IOError. Instead of investigating further, she decides to catch all exceptions:

def foo():
    try:
        f = open("file.txt")
        lines = f.readlines()
        return lines[0]
    except:
        return None

She doesn't realize the issue in his ways: what if the file exists and is accessible, but it is empty? Then this code will raise an IndexError (since the list lines is empty). So she'll spend hours wondering why she's getting None back from this function when the file exists and isn't locked without realizing something that would be obvious if she had been more specific in catching errors, which is that she's accessing data that might not exist.

How can I write a `try`/`except` block that catches all exceptions?

You can but you probably shouldn't:

try:
    do_something()
except:
    print("Caught it!")

However, this will also catch exceptions like KeyboardInterrupt and you usually don't want that, do you? Unless you re-raise the exception right away - see the following example from the docs:

try:
    f = open('myfile.txt')
    s = f.readline()
    i = int(s.strip())
except IOError as (errno, strerror):
    print("I/O error({0}): {1}".format(errno, strerror))
except ValueError:
    print("Could not convert data to an integer.")
except:
    print("Unexpected error:", sys.exc_info()[0])
    raise

Why is except: pass a bad programming practice?

As you correctly guessed, there are two sides to it: Catching any error by specifying no exception type after except, and simply passing it without taking any action.

My explanation is “a bit” longer—so tl;dr it breaks down to this:

1. Don’t catch any error. Always specify which exceptions you are prepared to recover from and only catch those.
2. Try to avoid passing in except blocks. Unless explicitly desired, this is usually not a good sign.

But let’s go into detail:

Don’t catch any error

When using a try block, you usually do this because you know that there is a chance of an exception being thrown. As such, you also already have an approximate idea of what can break and what exception can be thrown. In such cases, you catch an exception because you can positively recover from it. That means that you are prepared for the exception and have some alternative plan which you will follow in case of that exception.

For example, when you ask for the user to input a number, you can convert the input using int() which might raise a ValueError. You can easily recover that by simply asking the user to try it again, so catching the ValueError and prompting the user again would be an appropriate plan. A different example would be if you want to read some configuration from a file, and that file happens to not exist. Because it is a configuration file, you might have some default configuration as a fallback, so the file is not exactly necessary. So catching a FileNotFoundError and simply applying the default configuration would be a good plan here. Now in both these cases, we have a very specific exception we expect and have an equally specific plan to recover from it. As such, in each case, we explicitly only except that certain exception.

However, if we were to catch everything, then—in addition to those exceptions we are prepared to recover from—there is also a chance that we get exceptions that we didn’t expect, and which we indeed cannot recover from; or shouldn’t recover from.

Let’s take the configuration file example from above. In case of a missing file, we just applied our default configuration and might decide at a later point to automatically save the configuration (so next time, the file exists). Now imagine we get a IsADirectoryError, or a PermissionError instead. In such cases, we probably do not want to continue; we could still apply our default configuration, but we later won’t be able to save the file. And it’s likely that the user meant to have a custom configuration too, so using the default values is likely not desired. So we would want to tell the user about it immediately, and probably abort the program execution too. But that’s not something we want to do somewhere deep within some small code part; this is something of application-level importance, so it should be handled at the top—so let the exception bubble up.

Another simple example is also mentioned in the Python 2 idioms document. Here, a simple typo exists in the code which causes it to break. Because we are catching every exception, we also catch NameErrors and SyntaxErrors. Both are mistakes that happen to us all while programming and both are mistakes we absolutely don’t want to include when shipping the code. But because we also caught those, we won’t even know that they occurred there and lose any help to debug it correctly.

But there are also more dangerous exceptions which we are unlikely prepared for. For example, SystemError is usually something that happens rarely and which we cannot really plan for; it means there is something more complicated going on, something that likely prevents us from continuing the current task.

In any case, it’s very unlikely that you are prepared for everything in a small-scale part of the code, so that’s really where you should only catch those exceptions you are prepared for. Some people suggest to at least catch Exception as it won’t include things like SystemExit and KeyboardInterrupt which by design are to terminate your application, but I would argue that this is still far too unspecific. There is only one place where I personally accept catching Exception or just any exception, and that is in a single global application-level exception handler which has the single purpose to log any exception we were not prepared for. That way, we can still retain as much information about unexpected exceptions, which we then can use to extend our code to handle those explicitly (if we can recover from them) or—in case of a bug—to create test cases to make sure it won’t happen again. But of course, that only works if we only ever caught those exceptions we were already expecting, so the ones we didn’t expect will naturally bubble up.

Try to avoid passing in except blocks

When explicitly catching a small selection of specific exceptions, there are many situations in which we will be fine by simply doing nothing. In such cases, just having except SomeSpecificException: pass is just fine. Most of the time though, this is not the case as we likely need some code related to the recovery process (as mentioned above). This can be for example something that retries the action again, or to set up a default value instead.

If that’s not the case though, for example, because our code is already structured to repeat until it succeeds, then just passing is good enough. Taking our example from above, we might want to ask the user to enter a number. Because we know that users like to not do what we ask them for, we might just put it into a loop in the first place, so it could look like this:

def askForNumber ():
    while True:
        try:
            return int(input('Please enter a number: '))
        except ValueError:
            pass

Because we keep trying until no exception is thrown, we don’t need to do anything special in the except block, so this is fine. But of course, one might argue that we at least want to show the user some error message to tell him why he has to repeat the input.

In many other cases though, just passing in an except is a sign that we weren’t really prepared for the exception we are catching. Unless those exceptions are simple (like ValueError or TypeError), and the reason why we can pass is obvious, try to avoid just passing. If there’s really nothing to do (and you are absolutely sure about it), then consider adding a comment why that’s the case; otherwise, expand the except block to actually include some recovery code.

except: pass

The worst offender though is the combination of both. This means that we are willingly catching any error although we are absolutely not prepared for it and we also don’t do anything about it. You at least want to log the error and also likely reraise it to still terminate the application (it’s unlikely you can continue like normal after a MemoryError). Just passing though will not only keep the application somewhat alive (depending on where you catch of course), but also throw away all the information, making it impossible to discover the error—which is especially true if you are not the one discovering it.

---

So the bottom line is: Catch only exceptions you really expect and are prepared to recover from; all others are likely either mistakes you should fix or something you are not prepared for anyway. Passing specific exceptions are fine if you really don’t need to do something about them. In all other cases, it’s just a sign of presumption and being lazy. And you definitely want to fix that.

Handling all but one exception

The answer is to simply do a bare raise:

try:
    ...
except NoChildException:
    # optionally, do some stuff here and then ...
    raise
except Exception:
    # handling

This will re-raise the last thrown exception, with original stack trace intact (even if it's been handled!).

Catching all exceptions in Python

If you need to catch all exceptions and do the same stuff for all, I'll suggest you this :

try:
   #stuff
except:
   # do some stuff

If you don't want to mask "special" python exceptions, use the Exception base class

try:
   #stuff
except Exception:
   # do some stuff

for some exceptions related management, catch them explicitly :

try:
   #stuff
except FirstExceptionBaseClassYouWantToCatch as exc:
   # do some stuff
except SecondExceptionBaseClassYouWantToCatch as exc:
   # do some other stuff based
except (ThirdExceptionBaseClassYouWantToCatch, FourthExceptionBaseClassYouWantToCatch) as exc:
   # do some other stuff based

The exception hierarchy from the python docs should be a usefull reading.

Why is the Catch(Exception) almost always a bad Idea?

Because when you catch exception you're supposed to handle it properly. And you cannot expect to handle all kind of exceptions in your code. Also when you catch all exceptions, you may get an exception that cannot deal with and prevent code that is upper in the stack to handle it properly.

The general principal is to catch the most specific type you can.

Is it a good practice to use try-except-else in Python?

"I do not know if it is out of ignorance, but I do not like that
kind of programming, as it is using exceptions to perform flow control."

In the Python world, using exceptions for flow control is common and normal.

Even the Python core developers use exceptions for flow-control and that style is heavily baked into the language (i.e. the iterator protocol uses StopIteration to signal loop termination).

In addition, the try-except-style is used to prevent the race-conditions inherent in some of the "look-before-you-leap" constructs. For example, testing os.path.exists results in information that may be out-of-date by the time you use it. Likewise, Queue.full returns information that may be stale. The try-except-else style will produce more reliable code in these cases.

"It my understanding that exceptions are not errors, they should only
be used for exceptional conditions"

In some other languages, that rule reflects their cultural norms as reflected in their libraries. The "rule" is also based in-part on performance considerations for those languages.

The Python cultural norm is somewhat different. In many cases, you must use exceptions for control-flow. Also, the use of exceptions in Python does not slow the surrounding code and calling code as it does in some compiled languages (i.e. CPython already implements code for exception checking at every step, regardless of whether you actually use exceptions or not).

In other words, your understanding that "exceptions are for the exceptional" is a rule that makes sense in some other languages, but not for Python.

"However, if it is included in the language itself, there must be a
good reason for it, isn't it?"

Besides helping to avoid race-conditions, exceptions are also very useful for pulling error-handling outside loops. This is a necessary optimization in interpreted languages which do not tend to have automatic loop invariant code motion.

Also, exceptions can simplify code quite a bit in common situations where the ability to handle an issue is far removed from where the issue arose. For example, it is common to have top level user-interface code calling code for business logic which in turn calls low-level routines. Situations arising in the low-level routines (such as duplicate records for unique keys in database accesses) can only be handled in top-level code (such as asking the user for a new key that doesn't conflict with existing keys). The use of exceptions for this kind of control-flow allows the mid-level routines to completely ignore the issue and be nicely decoupled from that aspect of flow-control.

There is a nice blog post on the indispensibility of exceptions here.

Also, see this Stack Overflow answer: Are exceptions really for exceptional errors?

"What is the reason for the try-except-else to exist?"

The else-clause itself is interesting. It runs when there is no exception but before the finally-clause. That is its primary purpose.

Without the else-clause, the only option to run additional code before finalization would be the clumsy practice of adding the code to the try-clause. That is clumsy because it risks
raising exceptions in code that wasn't intended to be protected by the try-block.

The use-case of running additional unprotected code prior to finalization doesn't arise very often. So, don't expect to see many examples in published code. It is somewhat rare.

Another use-case for the else-clause is to perform actions that must occur when no exception occurs and that do not occur when exceptions are handled. For example:

recip = float('Inf')
try:
    recip = 1 / f(x)
except ZeroDivisionError:
    logging.info('Infinite result')
else:
    logging.info('Finite result')

Another example occurs in unittest runners:

try:
    tests_run += 1
    run_testcase(case)
except Exception:
    tests_failed += 1
    logging.exception('Failing test case: %r', case)
    print('F', end='')
else:
    logging.info('Successful test case: %r', case)
    print('.', end='')

Lastly, the most common use of an else-clause in a try-block is for a bit of beautification (aligning the exceptional outcomes and non-exceptional outcomes at the same level of indentation). This use is always optional and isn't strictly necessary.

Is it really that bad to catch a general exception?

Obviously this is one of those questions where the only real answer is "it depends."

The main thing it depends on is where your are catching the exception. In general libraries should be more conservative with catching exceptions whereas at the top level of your program (e.g. in your main method or in the top of the action method in a controller, etc) you can be more liberal with what you catch.

The reason for this is that e.g. you don't want to catch all exceptions in a library because you may mask problems that have nothing to do with your library, like "OutOfMemoryException" which you really would prefer bubbles up so that the user can be notified, etc. On the other hand, if you are talking about catching exceptions inside your main() method which catches the exception, displays it and then exits... well, it's probably safe to catch just about any exception here.

The most important rule about catching all exceptions is that you should never just swallow all exceptions silently... e.g. something like this in Java:

try { 
    something(); 
} catch (Exception ex) {}

or this in Python:

try:
    something()
except:
    pass

Because these can be some of the hardest issues to track down.

A good rule of thumb is that you should only catch exceptions that you can properly deal with yourself. If you cannot handle the exception completely then you should let it bubble up to someone who can.

---

Related Topics