Catching an Exception While Using a Python 'With' Statement

Catching an exception while using a Python 'with' statement

from __future__ import with_statement

try:
    with open( "a.txt" ) as f :
        print f.readlines()
except EnvironmentError: # parent of IOError, OSError *and* WindowsError where available
    print 'oops'

If you want different handling for errors from the open call vs the working code you could do:

try:
    f = open('foo.txt')
except IOError:
    print('error')
else:
    with f:
        print f.readlines()

python exception handling inside with block

The exception from within the block governed by the with statement is propagated to your generator context manager through generator.throw() as shown in PEP 343: "Generator Decorator", which raises the exception at the point where the generator was paused. In other words you should wrap the yield in a try/except or try/finally:

@contextmanager
def test():
    print("Hello")
    try:
        # The block of the with statement executes when the generator yields
        yield

    finally:
        print("goodbye")

To quote the official documentation on the subject:

...If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred. Thus, you can use a try…except…finally statement to trap the error (if any), or ensure that some cleanup takes place. If an exception is trapped merely in order to log it or to perform some action (rather than to suppress it entirely), the generator must reraise that exception. Otherwise the generator context manager will indicate to the with statement that the exception has been handled, and execution will resume with the statement immediately following the with statement.

Catch exception throw from inside of the with statement

Yes, this is how you would want to handle exceptions from inside a with statement. ContextManagers (that implement the behavior behind the with statement) can handle exceptions but should only do so to properly clean up resources used by the object.

Here's a snippet from the relevant documentation:

If BLOCK raises an exception, the context manager’s exit() method
is called with three arguments, the exception details (type, value,
traceback, the same values returned by sys.exc_info(), which can also
be None if no exception occurred). The method’s return value controls
whether an exception is re-raised: any false value re-raises the
exception, and True will result in suppressing it. You’ll only rarely
want to suppress the exception, because if you do the author of the
code containing the ‘with‘ statement will never realize anything went
wrong.

How with is better than try/catch to open a file in Python?

For a start, it helps prevent the problem you've introduced in your try ... finally ... example.

The way you've structured it, if an exception is thrown while trying to open the file then you will never bind an open file to the name f, leading to either a NameError in the finally clause (if f has never been bound within scope) or something entirely unexpected (if it has).

The correct structure (equivalent to the with) is:

f = open(my_file)

try:
    do_stuff_that_fails()
finally:
    f.close()

(note - no need for an except clause if you've got nothing to do there).

Your second example similarly is wrong, and should be structured like:

try:
    f = open(my_file)

    try:
        do_stuff_that_fails()
    except EXPECTED_EXCEPTION_TYPES as e:
        do_stuff_when_it_doesnt_work()
    finally:
        f.close()

except (IOError, OSError) as e:
    do_other_stuff_when_it_we_have_file_IO_problems()

The second is (as stated in another answer) that you can't forget to call f.close().

BTW, the term is "context management", not "resource management" - the with statement manages contexts, some of which may be resources, but others not. For example, it's also used with decimal to establish a decimal context for a particular block of code.

Finally (responding to your comment to the previous answer) you should never rely on refcount semantics for handling resources in Python. Jython, IronPython and PyPy all have non-refcount semantics, and there's nothing preventing CPython from going the other way (though it's highly unlikely for the immediate future). In a tight loop (e.g. os.walk) it is very very easy to run out of file handles if code relying on refcount semantics is run on a VM with different behaviour.

Handling exceptions within if statements in Python

Well, there are 2 ways you could easily handle this. You could slice a single character which will be empty if it doesn't exist. E.g.,

if a[0:1] == "a":
    return("foo")
elif a[1:2] == "b":
    return("bar")
elif a[5:6] == "d":
    return("bar2")
elif a[2:3] == "c":
    return("bar3")
else:
    return("baz") 

Or, you could write a wrapper function to ignore the IndexErrors. E.g.,

def get(obj, index):
    try:
        return obj[index]
    except IndexError:
        return

if get(a, 0) == "a":
    return("foo")
elif get(a, 1) == "b":
    return("bar")
elif get(a, 5) == "d":
    return("bar2")
elif get(a, 2) == "c":
    return("bar3")
else:
    return("baz") 

Catch exception and continue try block in Python

No, you cannot do that. That's just the way Python has its syntax. Once you exit a try-block because of an exception, there is no way back in.

What about a for-loop though?

funcs = do_smth1, do_smth2

for func in funcs:
    try:
        func()
    except Exception:
        pass  # or you could use 'continue'

Note however that it is considered a bad practice to have a bare except. You should catch for a specific exception instead. I captured for Exception because that's as good as I can do without knowing what exceptions the methods might throw.

When does exception handling unexpectedly influence object lifetimes?

An exception stores a traceback, which stores all child frames ("function calls") between raising and excepting. Frames reference all local names and their values, preventing the garbage collection of local names and values.

This means that an exception handler should promptly finish handling exceptions to allow child locals to be cleaned up. Still, a function cannot rely on its locals being collectable immediately after the function ends.

As a result, patterns such as RAII are not reliable to be prompt even on reference counted implementations. When prompt cleanup is required, objects should provide a means for explicit cleanup (for use in finally blocks) or preferably automatic cleanup (for use in with blocks).

Objects, values and types

[…]
Programs are strongly recommended to explicitly close such objects. The ‘try…finally’ statement and the ‘with’ statement provide convenient ways to do this.

---

One can observe this with a class that marks when it is garbage collected.

class Collectible:
    def __init__(self, name):
        self.name = name
    def __del__(self, print=print):
        print("Collecting", self.name)

def inner():
    local_name = Collectible("inner local value")
    raise RuntimeError("This is a drill")

def outer():
    local_name = Collectible("outer local value")
    inner()

try:
    outer()
except RuntimeError as e:
    print(f"handling a {type(e).__name__}: {e}")

On CPython, the output shows that the handler runs before the locals are collected:

handling a RuntimeError: This is a drill
Collecting inner local value
Collecting outer local value

Note that CPython uses reference counting, which already leads to quick cleanup as soon as possible. Other implementations may further and arbitrarily delay cleanup.

Using python with statement with try-except block

1. The two code blocks you gave are
   not equivalent
2. The code you described as old way
   of doing things has a serious bug:
   in case opening the file fails you
   will get a second exception in the
   finally clause because f is not
   bound.

The equivalent old style code would be:

try:
    f = open("file", "r")
    try:
        line = f.readline()
    finally:
        f.close()
except IOError:
    <whatever>

As you can see, the with statement can make things less error prone. In newer versions of Python (2.7, 3.1), you can also combine multiple expressions in one with statement. For example:

with open("input", "r") as inp, open("output", "w") as out:
    out.write(inp.read())

Besides that, I personally regard it as bad habit to catch any exception as early as possible. This is not the purpose of exceptions. If the IO function that can fail is part of a more complicated operation, in most cases the IOError should abort the whole operation and so be handled at an outer level. Using with statements, you can get rid of all these try...finally statements at inner levels.

---

Related Topics