Using Python'S Eval() Vs. Ast.Literal_Eval()

Using python's eval() vs. ast.literal_eval()

datamap = eval(input('Provide some data here: ')) means that you actually evaluate the code before you deem it to be unsafe or not. It evaluates the code as soon as the function is called. See also the dangers of eval.

ast.literal_eval raises an exception if the input isn't a valid Python datatype, so the code won't be executed if it's not.

Use ast.literal_eval whenever you need eval. You shouldn't usually evaluate literal Python statements.

ast.literal_eval same behavior as eval

ast.literal_eval docs claims that

Safely evaluate an expression node or a string containing a Python
literal or container display. The string or node provided may only
consist of the following Python literal structures: strings, bytes,
numbers, tuples, lists, dicts, sets, booleans, None and Ellipsis.

custom classes are not supported by that function, as they are not enumerated

python eval vs ast.literal_eval vs JSON decode

I don't really like this attitude on stackoverflow (and elsewhere) telling people without any context that what they are doing is insecure and they shouldn't do it. Maybe it's just a throwaway script to import some data, in that case why not choose the fastest or most convenient way?

In this case, however, json.loads is not only more secure, but also more than 4x faster (depending on your data).

In [1]: %timeit json.loads(data)
10000 loops, best of 3: 41.6 µs per loop

In [2]: %timeit eval(data)
10000 loops, best of 3: 194 µs per loop

In [3]: %timeit ast.literal_eval(data)
1000 loops, best of 3: 269 µs per loop

If you think about it makes sense json is a such more constrained language/format than python, so it must be faster to parse with an optimized parser.

Usage of eval, exec, and ast.literal_eval

eval and exec are for dynamically executing simple Python expressions or more complex statements (respectively).

So, one practical example from the standard library, collections.namedtuple uses exec to dynamically define a __new__ method for the tuple subclass being returned:

# Create all the named tuple methods to be added to the class namespace

s = f'def __new__(_cls, {arg_list}): return _tuple_new(_cls, ({arg_list}))'
namespace = {'_tuple_new': tuple_new, '__name__': f'namedtuple_{typename}'}
# Note: exec() has the side-effect of interning the field names
exec(s, namespace)
__new__ = namespace['__new__']
__new__.__doc__ = f'Create new instance of {typename}({arg_list})'
if defaults is not None:
    __new__.__defaults__ = defaults

This sometimes makes sense, however, this is often abused by inexperience programmers to write unnecessarily dynamic code, e.g. dynamically creating a bunch of variables, when they should have used a list or a dict (or some other container) instead.

When to use ast.literal_eval

When to use it.

ast.literal_eval(input()) would be useful if you expected a list (or something similar) by the user. For example '[1,2]' would be converted to [1,2].

If the user is supposed to provide a number ast.literal_eval(input()) can be replaced with float(input()), or int(input()) if an integer is expected.

---

Performance

Note that premature [micro-]optimization is the root of all evil. But since you asked:

To test the speed of ast.literal_eval(input()) and float(input() you can use timeit.

Timing will vary based on the input given by the user.

Ints and floats are valid input, while anything else would be invalid. Giving 50% ints, 40% floats and 10% random as input, float(input()) is x12 faster.

With 10%, 10%, 80% and float(input()) is x6 faster.

import timeit as tt

lst_size = 10**5

# Set the percentages of input tried by user.
percentages = {'ints': .10,
               'floats': .10,
               'strings': .80}
assert 1 - sum(percentages.values()) < 0.00000001

ints_floats_strings = {k: int(v*lst_size) for k, v in percentages.items()}

setup = """
import ast

def f(x):
    try:
        float(x)
    except:
        pass

def g(x):
    try:
        ast.literal_eval(x)
    except:
        pass

l = [str(i) for i in range({ints})]
l += [str(float(i)) for i in range({floats})]
l += [']9' for _ in range({strings}//2)] + ['a' for _ in range({strings}//2)]
""".format(**ints_floats_strings)

stmt1 = """
for i in l:
    f(i)
"""

stmt2 = """
for i in l:
    g(i)
"""


reps = 10**1
t1 = tt.timeit(stmt1, setup, number=reps)
t2 = tt.timeit(stmt2, setup, number=reps)

print(t1)
print(t2)

print(t2/t1)

Assistance with Python's ast.literal_eval('a_string')

With the context you give in comment, I advice you not to do that:

• you cannot use ast.literal_eval because logging.Debug is not a litteral: you ask a value from a module, and that could have arbitrary side effects
• you should not use eval for a value that is read from an external file.

My advice would be to use a map:

log_levels = { 'DEBUG': logging.DEBUG, ... }

When you later read log_level (say DEBUG) from your config file, you just do:

real_level = log_levels[log_level]

That way you can easily accept case errors with real_level = log_levels[log_level.upper()], and if there is an unexpected value in the config file, you just get a KeyError exception, with no risk of unwanted evaluation.

Parse a zip object using ast.literal_eval() or eval()

The problem was with the change of implementation of zip() function. As zip() in Python2.x used to return list of tuples, which would've worked with ast.literal_eval(). But in Python3.x zip() function returns a lazy iterator, which can't be parsed using either ast.literal_eval or 'eval()'. So, for that reason now I'm sending list(zip()) from views.py to the Jinja template. Which can be sent back to views in request parameters as a list literal.

---

Related Topics