Explaining Python's '_Enter_' and '_Exit_'

Explaining Python's '__enter__' and '__exit__'

Using these magic methods (__enter__, __exit__) allows you to implement objects which can be used easily with the with statement.

The idea is that it makes it easy to build code which needs some 'cleandown' code executed (think of it as a try-finally block). Some more explanation here.

A useful example could be a database connection object (which then automagically closes the connection once the corresponding 'with'-statement goes out of scope):

class DatabaseConnection(object):

    def __enter__(self):
        # make a database connection and return it
        ...
        return self.dbconn

    def __exit__(self, exc_type, exc_val, exc_tb):
        # make sure the dbconnection gets closed
        self.dbconn.close()
        ...

As explained above, use this object with the with statement (you may need to do from __future__ import with_statement at the top of the file if you're on Python 2.5).

with DatabaseConnection() as mydbconn:
    # do stuff

PEP343 -- The 'with' statement' has a nice writeup as well.

How do __enter__ and __exit__ work in Python decorator classes?

the __exit__() method should accept information about exceptions that come up in the with: block. See here.

The following modification of your code works:

def __exit__(self, exc_type, exc_value, tb):
    if exc_type is not None:
        traceback.print_exception(exc_type, exc_value, tb)
        # return False # uncomment to pass exception through

    return True

Then you can try raising an exception in one of your with: blocks and it'll be caught in __exit__().

__enter__() takes exactly 3 arguments (1 given)

The __enter__ method is never given any arguments, so beyond self your signature should not have any other.

You should move those arguments to the __init__ method instead:

class FooBar(object):
    def __init__(self, param1, param2):
        # do something here ...

    def __enter__(self):
        # something else, perhaps return self

Creating an instance of FooBar() is a separate step. with calls __enter__ on the result of your mymod.FooBar("hello", 123) expression, the expression itself is not translated to an __enter__ call.

If it was, you couldn't also use it like this, but you can:

cm = mymod.FooBar("hello", 123)
with cm as x:
    # do something here with x, which is the return value of cm.__enter__()

Note that x is assigned whatever cm.__enter__() returned; you can return self from __enter__ or you can return something entirely different.

The expected methods __enter__ and __exit__ are documented in the With Statement Context Managers section of the Python Data model documentation:

object.__enter__(self)

Enter the runtime context related to this object. The with statement will bind this method’s return value to the target(s) specified in the as clause of the statement, if any.

as well as in the Content Manager Types section of the Built-In Types documentation:

contextmanager.__enter__()

Enter the runtime context and return either this object or another object related to the runtime context. The value returned by this method is bound to the identifier in the as clause of with statements using this context manager.

An example of a context manager that returns itself is a file object. File objects return themselves from __enter__() to allow open() to be used as the context expression in a with statement.

If you are interested in the exact interactions, see the original proposal: PEP 343 -- The "with" Statement; from the specification section you can see what with EXPR as VAR: BLOCK statement does under the hood:

mgr = (EXPR)
exit = type(mgr).__exit__  # Not calling it yet
value = type(mgr).__enter__(mgr)
exc = True
try:
    try:
        VAR = value  # Only if "as VAR" is present
        BLOCK
    except:
        # The exceptional case is handled here
        exc = False
        if not exit(mgr, *sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        exit(mgr, None, None, None)

Note the mgr = (EXPR) part; in your case, mymod.FooBar("hello", 123) is that part. Also note that (EXPR), __enter__ and __exit__ are not 'protected' by the try..except here, exceptions raised in the expression or when entering or exiting are not handled by the context manager!

Receiving AttributeError: __exit__ even when WITH object has EXIT defined

The value of expression after with keyword must be a valid context manager. That means the value of the expresion must have attributes __enter__ and __exit__ and these must accept parameters described in docs With Statement Context Managers. You can easily verify that the part aa.feed would be acceptable, but the value of whole expression is None and it has no necessary attribute. A difference between Python 3.5 and 3.6 is that one fails on missing __exit__ and the latter on missing __enter__. Nothing unexpected.

You also forgot self in the line def __call__(self, *args, **kargs): that would be a problem if you will use args and it is a pythonic pattern also with unused args.

__enter__ and __exit__ on class level in python 3

__enter__ and __exit__ are special methods, and as such only work correctly when defined on a object's type, not in it's instance dictionary.

Now Spam is a instance of type, and type(Spam).__enter__ and type(Spam).__exit__ do not exist. Therefore you get an attribute error.

To make this work, the methods would need to be declared on the metaclass of the class you want to use. Example:

class Spam(type):

    def __enter__(cls):
        print('enter')
        return cls

    def __exit__(cls, typ, value, tb):
        print('exit')

class Eggs(metaclass=Spam):
    pass

with Eggs:
    pass

Now Eggs is an instance of Spam (type(Eggs) == Spam, and therefore type(Eggs).__enter__ and type(Eggs).__exit__ do exist).

However defining a metaclass just to use an instance of it as a context manager seems a little over the top. The more straight forward solution starting from your example would be to just use

with Spam():
    pass

Or if you want to reuse the same instance later:

spam = Spam()
with spam:
    pass

with conn: AttributeError: __exit__ when using socket to receive message

In Python 2.7 the socket object does not support the context manager interface. You therefore cannot use the with statement with the socket object and have to explicitly call its close method after you're done using it.

Change:

        conn, addr = s.accept()
        with conn:
            print('Connected by', addr)
            data = conn.recv(1024)
            ...

to:

        conn, addr = s.accept()
        print('Connected by', addr)
        data = conn.recv(1024)
        ...
        conn.close()

Python missing __exit__ method

The error means that BacktestingDatabaseHelper is not designed to be used in a with statement. Sounds like the classes testingResource and BacktestingDatabaseHelper are not compatible with each other (perhaps your version of common.databaseHelper is out of date).

Conditional with statement in Python

If you want to avoid duplicating code and are using a version of Python prior to 3.7 (when contextlib.nullcontext was introduced) or even 3.3 (when contextlib.ExitStack was introduced), you could do something like:

class dummy_context_mgr():
    def __enter__(self):
        return None
    def __exit__(self, exc_type, exc_value, traceback):
        return False

or:

import contextlib

@contextlib.contextmanager
def dummy_context_mgr():
    yield None

and then use it as:

with get_stuff() if needs_with() else dummy_context_mgr() as gs:
   # do stuff involving gs or not

You alternatively could make get_stuff() return different things based on needs_with().

(See Mike's answer or Daniel's answer for what you can do in later versions.)

python swallow exception in context manager and go on

There's no way to continue running the body of the with statement after an exception makes it back to the context manager. The context manager can stop the exception from bubbling up further, but it can't do more than that.

What you might want is to use your context manager in several separate with statements:

suppress = swallow_exceptions([ZeroDivisionError])

with suppress:
    1 / 0              # this exception is suppressed

with suppress:
    float("String")  # this one is not

---

Related Topics