Cost of Exception Handlers in Python

Cost of exception handlers in Python

Why don't you measure it using the timeit module? That way you can see whether it's relevant to your application.

OK, so I've just tried the following:

import timeit

statements=["""\
try:
    b = 10/a
except ZeroDivisionError:
    pass""",
"""\
if a:
    b = 10/a""",
"b = 10/a"]

for a in (1,0):
    for s in statements:
        t = timeit.Timer(stmt=s, setup='a={}'.format(a))
        print("a = {}\n{}".format(a,s))
        print("%.2f usec/pass\n" % (1000000 * t.timeit(number=100000)/100000))

Result:

a = 1
try:
    b = 10/a
except ZeroDivisionError:
    pass
0.25 usec/pass

a = 1
if a:
    b = 10/a
0.29 usec/pass

a = 1
b = 10/a
0.22 usec/pass

a = 0
try:
    b = 10/a
except ZeroDivisionError:
    pass
0.57 usec/pass

a = 0
if a:
    b = 10/a
0.04 usec/pass

a = 0
b = 10/a
ZeroDivisionError: int division or modulo by zero

So, as expected, not having any exception handler is slightly faster (but blows up in your face when the exception happens), and try/except is faster than an explicit if as long as the condition is not met.

But it's all within the same order of magnitude and unlikely to matter either way. Only if the condition is actually met, then the if version is significantly faster.

Using try except in python - what is the cost?

In the particular example you give, you can easily test for and avoid the IndexError case, so I would do that test. The ValueError is harder to check for so you should catch the exception.

for w in text:
    words = w.split()
    if len(words) >= 4:
        try:
            new = float(words[3].replace(',', '.'))
            if new < temp and words[1][3:5] == '12':
                temp = new
                date = w
        except ValueError:
            pass

Don't use readlines() on a file, you should just iterate over the file instead.

A good rule of thumb when it is equally easy to check before or to handle an exception, is to consider how often the exception would be thrown. If in normal cases the exception triggers very rarely then just handle it, on the other hand if it will be a common case then it may be better to avoid throwing it. Since I know nothing about your data, I don't know what proportion of lines will have fewer than 4 fields, so in this case the choice is pretty arbitrary.

Cheap exception handling in Python?

You might find this post helpful: Try / Except Performance in Python: A Simple Test where Patrick Altman did some simple testing to see what the performance is in various scenarios pre-conditional checking (specific to dictionary keys in this case) and using only exceptions. Code is provided as well if you want to adapt it to test other conditionals.

The conclusions he came to:

From these results, I think it is fair
to quickly determine a number of
conclusions:

1. If there is a high likelihood that the element doesn't exist, then
   you are better off checking for it
   with has_key.
2. If you are not going to do anything with the Exception if it is
   raised, then you are better off not
   putting one have the except
3. If it is likely that the element does exist, then there is a very
   slight advantage to using a try/except
   block instead of using has_key,
   however, the advantage is very slight.

Python -- efficiency of caught exceptions

Whenever you code there is a balancing of concerns: performance, readability, correctness, extendability, maintainability, etc.
Unfortunately, it is often not possible to improve code in each of these directions at the same time. What is fast may not be as readable for instance.

One of the reasons why try..except is encouraged in Python is because you often can not anticipate all the ways your code may be used, so rather than checking if a specific condition exists, it is more general to just catch any of a certain class of error that might arise. Thus try..except may make your code more reusable.

However, it is also true that try..except is slow if the except clause is often being reached.

Is there a way to code that block so that an exception is not being raised and use try..except to catch the less frequent condition?

Or if not, for the sake of efficiency, you may choose not to use try..except. There are few hard and fast rules in programming. You have to choose your way based on your balance of concerns.

Python FAQ: “How fast are exceptions?”

Catching exceptions is expensive, but exceptions should be exceptional (read, not happen very often). If exceptions are rare, try/catch is faster than LBYL.

The following example times a dictionary key lookup using exceptions and LBYL when the key exists and when it doesn't exist:

import timeit

s = []

s.append('''\
try:
    x = D['key']
except KeyError:
    x = None
''')

s.append('''\
x = D['key'] if 'key' in D else None
''')

s.append('''\
try:
    x = D['xxx']
except KeyError:
    x = None
''')

s.append('''\
x = D['xxx'] if 'xxx' in D else None
''')

for i,c in enumerate(s,1):
    t = timeit.Timer(c,"D={'key':'value'}")
    print('Run',i,'=',min(t.repeat()))

Output

Run 1 = 0.05600167960596991       # try/catch, key exists
Run 2 = 0.08530091918578364       # LBYL, key exists (slower)
Run 3 = 0.3486251291120652        # try/catch, key doesn't exist (MUCH slower)
Run 4 = 0.050621117060586585      # LBYL, key doesn't exist

When the usual case is no exception, try/catch is "extremely efficient" when compared to LBYL.

What's the cost of reraising an exception?

I've had a quick look at the assembler the compiler spews out for above code snippets. Turns out that the bytes right after jmp @HandleOnExeption contain data such as the exception class pointers you use in the on clauses (if any).

I'm not that well versed in assembler to know exactly what's going on, but enough to understand what is roughly going on and come to this conclusion:

I suspect System.pas' HandleOnException does a call @IsClass already, and passes the exception on if no suitable handler is found, so if you use on e:Exception and re-raise, this will add a little code and make two calls extra:

• one back to your exception handling section (in all cases)
• one call @RaiseAgain (in cases the exception gets re-raised)

So, there's a difference. A minor one, but still, it's there.

How much more expensive is an Exception than a return value?

Throwing an exception is definitely more expensive than returning a value. But in terms of raw cost it's hard to say how much more expensive an exception is.

When deciding on a return value vs. an exception you should always consider the following rule.

Only use exceptions for exceptional circumstances

They shouldn't ever be used for general control flow.

Python if vs try-except

You're setting alist only once. The first call to "tryway" clears it, then every successive call does nothing.

def tryway():
    alist = range(1000)
    try:
        while True:
            alist.pop()
    except IndexError:
        pass

def ifway():
    alist = range(1000)
    while True:
        if alist == []:
            break
        else:
            alist.pop()
if __name__=='__main__':
    from timeit import Timer
    print "Testing Try"
    tr = Timer("tryway()","from __main__ import tryway")
    print tr.timeit(10000)
    print "Testing If"
    ir = Timer("ifway()","from __main__ import ifway")
    print ir.timeit(10000)

>>> Testing Try
>>> 2.09539294243
>>> Testing If
>>> 2.84440898895

---

Related Topics