What's the Difference Between Eval, Exec, and Compile

What's the difference between eval, exec, and compile?

The short answer, or TL;DR

Basically, eval is used to evaluate a single dynamically generated Python expression, and exec is used to execute dynamically generated Python code only for its side effects.

eval and exec have these two differences:

eval accepts only a single expression, exec can take a code block that has Python statements: loops, try: except:, class and function/method definitions and so on.
An expression in Python is whatever you can have as the value in a variable assignment:
```
a_variable = (anything you can put within these parentheses is an expression)
```
eval returns the value of the given expression, whereas exec ignores the return value from its code, and always returns None (in Python 2 it is a statement and cannot be used as an expression, so it really does not return anything).

In versions 1.0 - 2.7, exec was a statement, because CPython needed to produce a different kind of code object for functions that used exec for its side effects inside the function.

In Python 3, exec is a function; its use has no effect on the compiled bytecode of the function where it is used.

Thus basically:

>>> a = 5
>>> eval('37 + a')   # it is an expression
42
>>> exec('37 + a')   # it is an expression statement; value is ignored (None is returned)
>>> exec('a = 47')   # modify a global variable as a side effect
>>> a
47
>>> eval('a = 47')  # you cannot evaluate a statement
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = 47
      ^
SyntaxError: invalid syntax

The compile in 'exec' mode compiles any number of statements into a bytecode that implicitly always returns None, whereas in 'eval' mode it compiles a single expression into bytecode that returns the value of that expression.

>>> eval(compile('42', '<string>', 'exec'))  # code returns None
>>> eval(compile('42', '<string>', 'eval'))  # code returns 42
42
>>> exec(compile('42', '<string>', 'eval'))  # code returns 42,
>>>                                          # but ignored by exec

In the 'eval' mode (and thus with the eval function if a string is passed in), the compile raises an exception if the source code contains statements or anything else beyond a single expression:

>>> compile('for i in range(3): print(i)', '<string>', 'eval')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    for i in range(3): print(i)
      ^
SyntaxError: invalid syntax

Actually the statement "eval accepts only a single expression" applies only when a string (which contains Python source code) is passed to eval. Then it is internally compiled to bytecode using compile(source, '<string>', 'eval') This is where the difference really comes from.

If a code object (which contains Python bytecode) is passed to exec or eval, they behave identically, excepting for the fact that exec ignores the return value, still returning None always. So it is possible use eval to execute something that has statements, if you just compiled it into bytecode before instead of passing it as a string:

>>> eval(compile('if 1: print("Hello")', '<string>', 'exec'))
Hello
>>>

works without problems, even though the compiled code contains statements. It still returns None, because that is the return value of the code object returned from compile.

>>> compile('for i in range(3): print(i)', '<string>'. 'eval')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    for i in range(3): print(i)
      ^
SyntaxError: invalid syntax

The longer answer, a.k.a the gory details

`exec` and `eval`

The exec function (which was a statement in Python 2) is used for executing a dynamically created statement or program:

>>> program = '''
for i in range(3):
    print("Python is cool")
'''
>>> exec(program)
Python is cool
Python is cool
Python is cool
>>>

The eval function does the same for a single expression, and returns the value of the expression:

>>> a = 2
>>> my_calculation = '42 * a'
>>> result = eval(my_calculation)
>>> result
84

exec and eval both accept the program/expression to be run either as a str, unicode or bytes object containing source code, or as a code object which contains Python bytecode.

If a str/unicode/bytes containing source code was passed to exec, it behaves equivalently to:

exec(compile(source, '<string>', 'exec'))

and eval similarly behaves equivalent to:

eval(compile(source, '<string>', 'eval'))

Since all expressions can be used as statements in Python (these are called the Expr nodes in the Python abstract grammar; the opposite is not true), you can always use exec if you do not need the return value. That is to say, you can use either eval('my_func(42)') or exec('my_func(42)'), the difference being that eval returns the value returned by my_func, and exec discards it:

>>> def my_func(arg):
...     print("Called with %d" % arg)
...     return arg * 2
... 
>>> exec('my_func(42)')
Called with 42
>>> eval('my_func(42)')
Called with 42
84
>>>

Of the 2, only exec accepts source code that contains statements, like def, for, while, import, or class, the assignment statement (a.k.a a = 42), or entire programs:

>>> exec('for i in range(3): print(i)')
0
1
2
>>> eval('for i in range(3): print(i)')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    for i in range(3): print(i)
      ^
SyntaxError: invalid syntax

Both exec and eval accept 2 additional positional arguments - globals and locals - which are the global and local variable scopes that the code sees. These default to the globals() and locals() within the scope that called exec or eval, but any dictionary can be used for globals and any mapping for locals (including dict of course). These can be used not only to restrict/modify the variables that the code sees, but are often also used for capturing the variables that the executed code creates:

>>> g = dict()
>>> l = dict()
>>> exec('global a; a, b = 123, 42', g, l)
>>> g['a']
123
>>> l
{'b': 42}

(If you display the value of the entire g, it would be much longer, because exec and eval add the built-ins module as __builtins__ to the globals automatically if it is missing).

In Python 2, the official syntax for the exec statement is actually exec code in globals, locals, as in

>>> exec 'global a; a, b = 123, 42' in g, l

However the alternate syntax exec(code, globals, locals) has always been accepted too (see below).

`compile`

The compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1) built-in can be used to speed up repeated invocations of the same code with exec or eval by compiling the source into a code object beforehand. The mode parameter controls the kind of code fragment the compile function accepts and the kind of bytecode it produces. The choices are 'eval', 'exec' and 'single':

'eval' mode expects a single expression, and will produce bytecode that when run will return the value of that expression:

>>> dis.dis(compile('a + b', '<string>', 'eval'))
  1           0 LOAD_NAME                0 (a)
              3 LOAD_NAME                1 (b)
              6 BINARY_ADD
              7 RETURN_VALUE

'exec' accepts any kinds of python constructs from single expressions to whole modules of code, and executes them as if they were module top-level statements. The code object returns None:

>>> dis.dis(compile('a + b', '<string>', 'exec'))
  1           0 LOAD_NAME                0 (a)
              3 LOAD_NAME                1 (b)
              6 BINARY_ADD
              7 POP_TOP                             <- discard result
              8 LOAD_CONST               0 (None)   <- load None on stack
             11 RETURN_VALUE                        <- return top of stack

'single' is a limited form of 'exec' which accepts a source code containing a single statement (or multiple statements separated by ;) if the last statement is an expression statement, the resulting bytecode also prints the repr of the value of that expression to the standard output(!).
An if-elif-else chain, a loop with else, and try with its except, else and finally blocks is considered a single statement.
A source fragment containing 2 top-level statements is an error for the 'single', except in Python 2 there is a bug that sometimes allows multiple toplevel statements in the code; only the first is compiled; the rest are ignored:
In Python 2.7.8:
```
>>> exec(compile('a = 5\na = 6', '<string>', 'single'))
>>> a
5
```
And in Python 3.4.2:
```
>>> exec(compile('a = 5\na = 6', '<string>', 'single'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    a = 5
        ^
SyntaxError: multiple statements found while compiling a single statement
```
This is very useful for making interactive Python shells. However, the value of the expression is not returned, even if you eval the resulting code.

Thus greatest distinction of exec and eval actually comes from the compile function and its modes.

In addition to compiling source code to bytecode, compile supports compiling abstract syntax trees (parse trees of Python code) into code objects; and source code into abstract syntax trees (the ast.parse is written in Python and just calls compile(source, filename, mode, PyCF_ONLY_AST)); these are used for example for modifying source code on the fly, and also for dynamic code creation, as it is often easier to handle the code as a tree of nodes instead of lines of text in complex cases.

While eval only allows you to evaluate a string that contains a single expression, you can eval a whole statement, or even a whole module that has been compiled into bytecode; that is, with Python 2, print is a statement, and cannot be evalled directly:

>>> eval('for i in range(3): print("Python is cool")')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    for i in range(3): print("Python is cool")
      ^
SyntaxError: invalid syntax

compile it with 'exec' mode into a code object and you can eval it; the eval function will return None.

>>> code = compile('for i in range(3): print("Python is cool")',
                   'foo.py', 'exec')
>>> eval(code)
Python is cool
Python is cool
Python is cool

If one looks into eval and exec source code in CPython 3, this is very evident; they both call PyEval_EvalCode with same arguments, the only difference being that exec explicitly returns None.

Syntax differences of `exec` between Python 2 and Python 3

One of the major differences in Python 2 is that exec is a statement and eval is a built-in function (both are built-in functions in Python 3).
It is a well-known fact that the official syntax of exec in Python 2 is exec code [in globals[, locals]].

Unlike majority of the Python 2-to-3 porting guides seem to suggest, the exec statement in CPython 2 can be also used with syntax that looks exactly like the exec function invocation in Python 3. The reason is that Python 0.9.9 had the exec(code, globals, locals) built-in function! And that built-in function was replaced with exec statement somewhere before Python 1.0 release.

Since it was desirable to not break backwards compatibility with Python 0.9.9, Guido van Rossum added a compatibility hack in 1993: if the code was a tuple of length 2 or 3, and globals and locals were not passed into the exec statement otherwise, the code would be interpreted as if the 2nd and 3rd element of the tuple were the globals and locals respectively. The compatibility hack was not mentioned even in Python 1.4 documentation (the earliest available version online); and thus was not known to many writers of the porting guides and tools, until it was documented again in November 2012:

The first expression may also be a tuple of length 2 or 3. In this case, the optional parts must be omitted. The form exec(expr, globals) is equivalent to exec expr in globals, while the form exec(expr, globals, locals) is equivalent to exec expr in globals, locals. The tuple form of exec provides compatibility with Python 3, where exec is a function rather than a statement.

Yes, in CPython 2.7 that it is handily referred to as being a forward-compatibility option (why confuse people over that there is a backward compatibility option at all),
when it actually had been there for backward-compatibility for two decades.

Thus while exec is a statement in Python 1 and Python 2, and a built-in function in Python 3 and Python 0.9.9,

>>> exec("print(a)", globals(), {'a': 42})
42

has had identical behaviour in possibly every widely released Python version ever; and works in Jython 2.5.2, PyPy 2.3.1 (Python 2.7.6) and IronPython 2.6.1 too (kudos to them following the undocumented behaviour of CPython closely).

What you cannot do in Pythons 1.0 - 2.7 with its compatibility hack, is to store the return value of exec into a variable:

Python 2.7.11+ (default, Apr 17 2016, 14:00:29) 
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = exec('print(42)')
  File "<stdin>", line 1
    a = exec('print(42)')
           ^
SyntaxError: invalid syntax

(which wouldn't be useful in Python 3 either, as exec always returns None), or pass a reference to exec:

>>> call_later(exec, 'print(42)', delay=1000)
  File "<stdin>", line 1
    call_later(exec, 'print(42)', delay=1000)
                  ^
SyntaxError: invalid syntax

Which a pattern that someone might actually have used, though unlikely;

Or use it in a list comprehension:

>>> [exec(i) for i in ['print(42)', 'print(foo)']
  File "<stdin>", line 1
    [exec(i) for i in ['print(42)', 'print(foo)']
        ^
SyntaxError: invalid syntax

which is abuse of list comprehensions (use a for loop instead!).

When to use eval mode rather than exec with Python compile()

Just like the eval builtin, the compile eval mode creates a code object that evaluates an expression and returns its result. In contrast, exec mode does not return a result.

>>> exec_code = eval(compile("1 + 2", "<stack overflow>", "exec"))
>>> eval(exec_code)  # no result for "exec" mode code
>>> eval_code = eval(compile("1 + 2", "<stack overflow>", "eval"))
>>> eval(eval_code)  # some result for "eval" mode code
3

But an expression is a statement.

This is not true. An expression evaluates to a value, a statement does not.

While expression statements mean an expression can be used "as a" statement, that does not make the two equivalent. An expression statement contains an expression; when run, the statement evaluates the expression but discards the result.

what is the difference between eval and exec in python?

The part of your question about storing the function name can be explained by the fact that this would be equivalent:

def test():
    print 'hello world'
func = test
func() # this will call test()

The call to eval() in your example is no different than a call like:

y = eval('x + 1')

I believe your second question is the same as this one, and the answers might be helpful.

Python 3 - exec() Vs eval() - Expression evaluation

How eval() works different from exec() ?

In your two cases, both eval() and exec() do, do the same things. They print the result of the expression. However, they are still both different.

The eval() function can only execute Python expressions, while the exec() function can execute any valid Python code. This can be seen with a few examples:

>>> eval('1 + 2')
3
>>> exec('1 + 2')
>>> 
>>> eval('for i in range(1, 11): print(i)')
Traceback (most recent call last):
  File "<pyshell#45>", line 1, in <module>
    eval('for i in range(1, 11): print(i)')
  File "<string>", line 1
    for i in range(1, 11): print(i)
      ^
SyntaxError: invalid syntax
>>> exec('for i in range(1, 11): print(i)')
1
2
3
4
5
6
7
8
9
10
>>>

exec statement with/without prior compile

There are a few differences that I see. Firstly, compile has slightly better semantics in the face of syntax errors than exec. I suspect that the real reason is that the definition of compile is very explicit with respect to the handling of new line characters where exec is a little less precise.

I was curious as to why compile and exec where being used in lieu of inner functions. I didn't know that compile/exec lets you control what globals are available. Very interesting.

Why should exec() and eval() be avoided?

There are often clearer, more direct ways to get the same effect. If you build a complex string and pass it to exec, the code is difficult to follow, and difficult to test.

Example: I wrote code that read in string keys and values and set corresponding fields in an object. It looked like this:

for key, val in values:
    fieldName = valueToFieldName[key]
    fieldType = fieldNameToType[fieldName]
    if fieldType is int:
        s = 'object.%s = int(%s)' % (fieldName, fieldType) 
    #Many clauses like this...

exec(s)

That code isn't too terrible for simple cases, but as new types cropped up it got more and more complex. When there were bugs they always triggered on the call to exec, so stack traces didn't help me find them. Eventually I switched to a slightly longer, less clever version that set each field explicitly.

The first rule of code clarity is that each line of your code should be easy to understand by looking only at the lines near it. This is why goto and global variables are discouraged. exec and eval make it easy to break this rule badly.

Usage of eval, exec, and ast.literal_eval

eval and exec are for dynamically executing simple Python expressions or more complex statements (respectively).

So, one practical example from the standard library, collections.namedtuple uses exec to dynamically define a __new__ method for the tuple subclass being returned:

# Create all the named tuple methods to be added to the class namespace

s = f'def __new__(_cls, {arg_list}): return _tuple_new(_cls, ({arg_list}))'
namespace = {'_tuple_new': tuple_new, '__name__': f'namedtuple_{typename}'}
# Note: exec() has the side-effect of interning the field names
exec(s, namespace)
__new__ = namespace['__new__']
__new__.__doc__ = f'Create new instance of {typename}({arg_list})'
if defaults is not None:
    __new__.__defaults__ = defaults

This sometimes makes sense, however, this is often abused by inexperience programmers to write unnecessarily dynamic code, e.g. dynamically creating a bunch of variables, when they should have used a list or a dict (or some other container) instead.

Access things outside of module with eval and exec

Yes!

exec has a few optional parameters, globals and locals. These basically tell it what global and local variables its allowed to use, in the form of a dictionary. Calling globals() or locals() functions returns the dictionary with all the global and local variables where you are calling from, so you can use:

myExec.py:

def myExec(code, globals_=None, locals_=None):  # the trailing underscore is so that there are no name conflicts
    exec(code, globals_, locals_)

main.py:

import myExec
def test():
    print("ok")
myExec.myExec("test()", globals())

Use of exec and eval in Python

I'll give an example in which I have used eval and where I think it was the best choice.

I was writing a simple software testing utility ... something to test whether student exercises were conforming to the assignment requirements. The goal was to provide a way for a simple configuration file to serve as a test specification (to get around a "chicken-and-egg" issue of using a programming language to describe/document/implement the test cases for elementary programming assignments).

I based my harness on the ConfigParser in the standard libraries. However, I did want the ability to represent arbitrary Python strings (including interpolations of \n, \t, and especially any interpolated hex encoded ASCII characters in the values read therefrom.

My solution was a try around an parsed_string=eval('''%s''' % cfg_read_item) followed by a try of the triple double-quoted version ("""%s""") of the same.

This is a case where the alternative would have been to write (or find a pre-written) Python language parser and figure out how to include and adapt it to my program. The risks are minimal (I'm not worried that student submitted code is going to trick my parser, break out if its jail, delete all my files, send my credit card numbers to Romania and so on)*

*(In part because I was testing them under Linux from an untrusted headless user account).

As here others have said, there are other use cases where you're building code from a template based on input data and need to execute that code (meta programming). You should always be able to accomplish those tasks in another way. However, whenever that alternative entails coding effort that approaches writing a general programming language parser/compiler/interpreter .... then eval may be the better approach.

Is this eval() hackable?

The best answer I can give you is "I don't know". I can't think of anything, but there are lots of people out there a lot smarter than me at breaking things. I hate eval, and think every occurrence of eval is a security hole waiting to happen. You should avoid it at all costs.

If this were me, I'd write:

def login_logger(x, y):
   Do whatever you want here safely

@ratelimit(login_logger)
def login(x, y): ...

Have your @ratelimit call a logging function that takes the exact same arguments as the function it is wrapping.

What's the Difference Between Eval, Exec, and Compile