Why should exec() and eval() be avoided?
There are often clearer, more direct ways to get the same effect. If you build a complex string and pass it to exec, the code is difficult to follow, and difficult to test.
Example: I wrote code that read in string keys and values and set corresponding fields in an object. It looked like this:
for key, val in values:
fieldName = valueToFieldName[key]
fieldType = fieldNameToType[fieldName]
if fieldType is int:
s = 'object.%s = int(%s)' % (fieldName, fieldType)
#Many clauses like this...
exec(s)
That code isn't too terrible for simple cases, but as new types cropped up it got more and more complex. When there were bugs they always triggered on the call to exec, so stack traces didn't help me find them. Eventually I switched to a slightly longer, less clever version that set each field explicitly.
The first rule of code clarity is that each line of your code should be easy to understand by looking only at the lines near it. This is why goto and global variables are discouraged. exec and eval make it easy to break this rule badly.
Why must exec (and not eval) be used for Python import statements?
The problem is that eval evaluates expressions and returns some result, while exec executes statements in some context. import is a statement, while re.match() is an expression.
When to use eval mode rather than exec with Python compile()
Just like the eval
builtin, the compile
eval mode creates a code object that evaluates an expression and returns its result. In contrast, exec mode does not return a result.
>>> exec_code = eval(compile("1 + 2", "<stack overflow>", "exec"))
>>> eval(exec_code) # no result for "exec" mode code
>>> eval_code = eval(compile("1 + 2", "<stack overflow>", "eval"))
>>> eval(eval_code) # some result for "eval" mode code
3
But an expression is a statement.
This is not true. An expression evaluates to a value, a statement does not.
While expression statements mean an expression can be used "as a" statement, that does not make the two equivalent. An expression statement contains an expression; when run, the statement evaluates the expression but discards the result.
What's the difference between eval, exec, and compile?
The short answer, or TL;DR
Basically, eval
is used to evaluate a single dynamically generated Python expression, and exec
is used to execute dynamically generated Python code only for its side effects.
eval
and exec
have these two differences:
eval
accepts only a single expression,exec
can take a code block that has Python statements: loops,try: except:
,class
and function/methoddef
initions and so on.An expression in Python is whatever you can have as the value in a variable assignment:
a_variable = (anything you can put within these parentheses is an expression)
eval
returns the value of the given expression, whereasexec
ignores the return value from its code, and always returnsNone
(in Python 2 it is a statement and cannot be used as an expression, so it really does not return anything).
In versions 1.0 - 2.7, exec
was a statement, because CPython needed to produce a different kind of code object for functions that used exec
for its side effects inside the function.
In Python 3, exec
is a function; its use has no effect on the compiled bytecode of the function where it is used.
Thus basically:
>>> a = 5
>>> eval('37 + a') # it is an expression
42
>>> exec('37 + a') # it is an expression statement; value is ignored (None is returned)
>>> exec('a = 47') # modify a global variable as a side effect
>>> a
47
>>> eval('a = 47') # you cannot evaluate a statement
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
a = 47
^
SyntaxError: invalid syntax
The compile
in 'exec'
mode compiles any number of statements into a bytecode that implicitly always returns None
, whereas in 'eval'
mode it compiles a single expression into bytecode that returns the value of that expression.
>>> eval(compile('42', '<string>', 'exec')) # code returns None
>>> eval(compile('42', '<string>', 'eval')) # code returns 42
42
>>> exec(compile('42', '<string>', 'eval')) # code returns 42,
>>> # but ignored by exec
In the 'eval'
mode (and thus with the eval
function if a string is passed in), the compile
raises an exception if the source code contains statements or anything else beyond a single expression:
>>> compile('for i in range(3): print(i)', '<string>', 'eval')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print(i)
^
SyntaxError: invalid syntax
Actually the statement "eval accepts only a single expression" applies only when a string (which contains Python source code) is passed to eval
. Then it is internally compiled to bytecode using compile(source, '<string>', 'eval')
This is where the difference really comes from.
If a code
object (which contains Python bytecode) is passed to exec
or eval
, they behave identically, excepting for the fact that exec
ignores the return value, still returning None
always. So it is possible use eval
to execute something that has statements, if you just compile
d it into bytecode before instead of passing it as a string:
>>> eval(compile('if 1: print("Hello")', '<string>', 'exec'))
Hello
>>>
works without problems, even though the compiled code contains statements. It still returns None
, because that is the return value of the code object returned from compile
.
In the 'eval'
mode (and thus with the eval
function if a string is passed in), the compile
raises an exception if the source code contains statements or anything else beyond a single expression:
>>> compile('for i in range(3): print(i)', '<string>'. 'eval')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print(i)
^
SyntaxError: invalid syntax
The longer answer, a.k.a the gory details
exec
and eval
The exec
function (which was a statement in Python 2) is used for executing a dynamically created statement or program:
>>> program = '''
for i in range(3):
print("Python is cool")
'''
>>> exec(program)
Python is cool
Python is cool
Python is cool
>>>
The eval
function does the same for a single expression, and returns the value of the expression:
>>> a = 2
>>> my_calculation = '42 * a'
>>> result = eval(my_calculation)
>>> result
84
exec
and eval
both accept the program/expression to be run either as a str
, unicode
or bytes
object containing source code, or as a code
object which contains Python bytecode.
If a str
/unicode
/bytes
containing source code was passed to exec
, it behaves equivalently to:
exec(compile(source, '<string>', 'exec'))
and eval
similarly behaves equivalent to:
eval(compile(source, '<string>', 'eval'))
Since all expressions can be used as statements in Python (these are called the Expr
nodes in the Python abstract grammar; the opposite is not true), you can always use exec
if you do not need the return value. That is to say, you can use either eval('my_func(42)')
or exec('my_func(42)')
, the difference being that eval
returns the value returned by my_func
, and exec
discards it:
>>> def my_func(arg):
... print("Called with %d" % arg)
... return arg * 2
...
>>> exec('my_func(42)')
Called with 42
>>> eval('my_func(42)')
Called with 42
84
>>>
Of the 2, only exec
accepts source code that contains statements, like def
, for
, while
, import
, or class
, the assignment statement (a.k.a a = 42
), or entire programs:
>>> exec('for i in range(3): print(i)')
0
1
2
>>> eval('for i in range(3): print(i)')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print(i)
^
SyntaxError: invalid syntax
Both exec
and eval
accept 2 additional positional arguments - globals
and locals
- which are the global and local variable scopes that the code sees. These default to the globals()
and locals()
within the scope that called exec
or eval
, but any dictionary can be used for globals
and any mapping
for locals
(including dict
of course). These can be used not only to restrict/modify the variables that the code sees, but are often also used for capturing the variables that the exec
uted code creates:
>>> g = dict()
>>> l = dict()
>>> exec('global a; a, b = 123, 42', g, l)
>>> g['a']
123
>>> l
{'b': 42}
(If you display the value of the entire g
, it would be much longer, because exec
and eval
add the built-ins module as __builtins__
to the globals automatically if it is missing).
In Python 2, the official syntax for the exec
statement is actually exec code in globals, locals
, as in
>>> exec 'global a; a, b = 123, 42' in g, l
However the alternate syntax exec(code, globals, locals)
has always been accepted too (see below).
compile
The compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)
built-in can be used to speed up repeated invocations of the same code with exec
or eval
by compiling the source into a code
object beforehand. The mode
parameter controls the kind of code fragment the compile
function accepts and the kind of bytecode it produces. The choices are 'eval'
, 'exec'
and 'single'
:
'eval'
mode expects a single expression, and will produce bytecode that when run will return the value of that expression:>>> dis.dis(compile('a + b', '<string>', 'eval'))
1 0 LOAD_NAME 0 (a)
3 LOAD_NAME 1 (b)
6 BINARY_ADD
7 RETURN_VALUE'exec'
accepts any kinds of python constructs from single expressions to whole modules of code, and executes them as if they were module top-level statements. The code object returnsNone
:>>> dis.dis(compile('a + b', '<string>', 'exec'))
1 0 LOAD_NAME 0 (a)
3 LOAD_NAME 1 (b)
6 BINARY_ADD
7 POP_TOP <- discard result
8 LOAD_CONST 0 (None) <- load None on stack
11 RETURN_VALUE <- return top of stack'single'
is a limited form of'exec'
which accepts a source code containing a single statement (or multiple statements separated by;
) if the last statement is an expression statement, the resulting bytecode also prints therepr
of the value of that expression to the standard output(!).An
if
-elif
-else
chain, a loop withelse
, andtry
with itsexcept
,else
andfinally
blocks is considered a single statement.A source fragment containing 2 top-level statements is an error for the
'single'
, except in Python 2 there is a bug that sometimes allows multiple toplevel statements in the code; only the first is compiled; the rest are ignored:In Python 2.7.8:
>>> exec(compile('a = 5\na = 6', '<string>', 'single'))
>>> a
5And in Python 3.4.2:
>>> exec(compile('a = 5\na = 6', '<string>', 'single'))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
a = 5
^
SyntaxError: multiple statements found while compiling a single statementThis is very useful for making interactive Python shells. However, the value of the expression is not returned, even if you
eval
the resulting code.
Thus greatest distinction of exec
and eval
actually comes from the compile
function and its modes.
In addition to compiling source code to bytecode, compile
supports compiling abstract syntax trees (parse trees of Python code) into code
objects; and source code into abstract syntax trees (the ast.parse
is written in Python and just calls compile(source, filename, mode, PyCF_ONLY_AST)
); these are used for example for modifying source code on the fly, and also for dynamic code creation, as it is often easier to handle the code as a tree of nodes instead of lines of text in complex cases.
While eval
only allows you to evaluate a string that contains a single expression, you can eval
a whole statement, or even a whole module that has been compile
d into bytecode; that is, with Python 2, print
is a statement, and cannot be eval
led directly:
>>> eval('for i in range(3): print("Python is cool")')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<string>", line 1
for i in range(3): print("Python is cool")
^
SyntaxError: invalid syntax
compile
it with 'exec'
mode into a code
object and you can eval
it; the eval
function will return None
.
>>> code = compile('for i in range(3): print("Python is cool")',
'foo.py', 'exec')
>>> eval(code)
Python is cool
Python is cool
Python is cool
If one looks into eval
and exec
source code in CPython 3, this is very evident; they both call PyEval_EvalCode
with same arguments, the only difference being that exec
explicitly returns None
.
Syntax differences of exec
between Python 2 and Python 3
One of the major differences in Python 2 is that exec
is a statement and eval
is a built-in function (both are built-in functions in Python 3).
It is a well-known fact that the official syntax of exec
in Python 2 is exec code [in globals[, locals]]
.
Unlike majority of the Python 2-to-3 porting guides seem to suggest, the exec
statement in CPython 2 can be also used with syntax that looks exactly like the exec
function invocation in Python 3. The reason is that Python 0.9.9 had the exec(code, globals, locals)
built-in function! And that built-in function was replaced with exec
statement somewhere before Python 1.0 release.
Since it was desirable to not break backwards compatibility with Python 0.9.9, Guido van Rossum added a compatibility hack in 1993: if the code
was a tuple of length 2 or 3, and globals
and locals
were not passed into the exec
statement otherwise, the code
would be interpreted as if the 2nd and 3rd element of the tuple were the globals
and locals
respectively. The compatibility hack was not mentioned even in Python 1.4 documentation (the earliest available version online); and thus was not known to many writers of the porting guides and tools, until it was documented again in November 2012:
The first expression may also be a tuple of length 2 or 3. In this case, the optional parts must be omitted. The form
exec(expr, globals)
is equivalent toexec expr in globals
, while the formexec(expr, globals, locals)
is equivalent toexec expr in globals, locals
. The tuple form ofexec
provides compatibility with Python 3, whereexec
is a function rather than a statement.
Yes, in CPython 2.7 that it is handily referred to as being a forward-compatibility option (why confuse people over that there is a backward compatibility option at all),
when it actually had been there for backward-compatibility for two decades.
Thus while exec
is a statement in Python 1 and Python 2, and a built-in function in Python 3 and Python 0.9.9,
>>> exec("print(a)", globals(), {'a': 42})
42
has had identical behaviour in possibly every widely released Python version ever; and works in Jython 2.5.2, PyPy 2.3.1 (Python 2.7.6) and IronPython 2.6.1 too (kudos to them following the undocumented behaviour of CPython closely).
What you cannot do in Pythons 1.0 - 2.7 with its compatibility hack, is to store the return value of exec
into a variable:
Python 2.7.11+ (default, Apr 17 2016, 14:00:29)
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = exec('print(42)')
File "<stdin>", line 1
a = exec('print(42)')
^
SyntaxError: invalid syntax
(which wouldn't be useful in Python 3 either, as exec
always returns None
), or pass a reference to exec
:
>>> call_later(exec, 'print(42)', delay=1000)
File "<stdin>", line 1
call_later(exec, 'print(42)', delay=1000)
^
SyntaxError: invalid syntax
Which a pattern that someone might actually have used, though unlikely;
Or use it in a list comprehension:
>>> [exec(i) for i in ['print(42)', 'print(foo)']
File "<stdin>", line 1
[exec(i) for i in ['print(42)', 'print(foo)']
^
SyntaxError: invalid syntax
which is abuse of list comprehensions (use a for
loop instead!).
Python: Is exec always bad practice and if so why not deprecated
No, eval
and related tools are not always bad by any measure.
There are a number of things that only work well when they are expressed as regular functions with regular, positional or keyword arguments (not magic *args
or **keywords
args). There's no way to dynamically create a function with a desired set of arguments, except with eval
.
For a good example of how this can be used, examine the implementation of collections.namedtuple
. Although it would be possible to create a class factory like that without eval
, but all of the functions it defines, __new__
and _replace
in particular, would have useless help text, and would be a less convenient tool without it. Worse, the non-eval implementation would almost certainly be SLOWER.
Another, more sweeping example of this exact use of eval
is the fine decorator
library, which generalizes this practice in a collection of tools that allow you to dynamically create functions with particular function signatures; it uses eval
internally.
Why is using 'eval' a bad practice?
Yes, using eval
is a bad practice. Just to name a few reasons:
- There is almost always a better way to do it
- Very dangerous and insecure
- Makes debugging difficult
- Slow
In your case you can use setattr instead:
class Song:
"""The class to store the details of each song"""
attsToStore=('Name', 'Artist', 'Album', 'Genre', 'Location')
def __init__(self):
for att in self.attsToStore:
setattr(self, att.lower(), None)
def setDetail(self, key, val):
if key in self.attsToStore:
setattr(self, key.lower(), val)
There are some cases where you have to use eval
or exec
. But they are rare. Using eval
in your case is a bad practice for sure. I'm emphasizing on bad practice because eval
and exec
are frequently used in the wrong place.
Replying to the comments:
It looks like some disagree that eval
is 'very dangerous and insecure' in the OP case. That might be true for this specific case but not in general. The question was general and the reasons I listed are true for the general case as well.
Python 3 - exec() Vs eval() - Expression evaluation
How eval() works different from exec() ?
In your two cases, both eval()
and exec()
do, do the same things. They print the result of the expression. However, they are still both different.
The eval()
function can only execute Python expressions, while the exec()
function can execute any valid Python code. This can be seen with a few examples:
>>> eval('1 + 2')
3
>>> exec('1 + 2')
>>>
>>> eval('for i in range(1, 11): print(i)')
Traceback (most recent call last):
File "<pyshell#45>", line 1, in <module>
eval('for i in range(1, 11): print(i)')
File "<string>", line 1
for i in range(1, 11): print(i)
^
SyntaxError: invalid syntax
>>> exec('for i in range(1, 11): print(i)')
1
2
3
4
5
6
7
8
9
10
>>>
what is the difference between eval and exec in python?
The part of your question about storing the function name can be explained by the fact that this would be equivalent:
def test():
print 'hello world'
func = test
func() # this will call test()
The call to eval()
in your example is no different than a call like:
y = eval('x + 1')
I believe your second question is the same as this one, and the answers might be helpful.
Related Topics
How to Call a Script from Another Script
How to Create a New Column from the Output of Pandas Groupby().Sum()
String Comparison in Python: Is Vs. ==
Multiple Assignment and Evaluation Order in Python
Why Does Substring Slicing With Index Out of Range Work
Store Output of Subprocess.Popen Call in a String
How to Sort a List of Strings Numerically
Generating a Png With Matplotlib When Display Is Undefined
How to Use Pickle to Save a Dict (Or Any Other Python Object)
How to Check If a List Is Empty
Flatten Nested Dictionaries, Compressing Keys
Word Boundary With Words Starting or Ending With Special Characters Gives Unexpected Results
Find the Row Indexes of Several Values in a Numpy Array
How to Use a Variable Inside a Regular Expression
Do Regular Expressions from the Re Module Support Word Boundaries (\B)