Python List Comprehension - Want to Avoid Repeated Evaluation

Python list comprehension - want to avoid repeated evaluation

A solution (the best if you have repeated value of x) would be to memoize the function f, i.e. to create a wrapper function that saves the argument by which the function is called and save it, than return it if the same value is asked.

a really simple implementation is the following:

storage = {}
def memoized(value):
    if value not in storage:
        storage[value] = f(value)
    return storage[value]

[memoized(x) for x in l if memoized(x)]

and then use this function in the list comprehension. This approach is valid under two condition, one theoretical and one practical. The first one is that the function f should be deterministic, i.e. returns the same results given the same input, and the other is that the object x can be used as a dictionary keys. If the first one is not valid than you should recompute f each timeby definition, while if the second one fails it is possible to use some slightly more robust approaches.

You can find a lot of implementation of memoization around the net, and I think that the new versions of python have something included in them too.

On a side note, never use the small L as a variable name, is a bad habit as it can be confused with an i or a 1 on some terminals.

EDIT:

as commented, a possible solution using generators comprehension (to avoid creating useless duplicate temporaries) would be this expression:

[g(x, fx) for x, fx in ((x,f(x)) for x in l) if fx]

You need to weight your choice given the computational cost of f, the number of duplication in the original list and memory at you disposition. Memoization make a space-speed tradeoff, meaning that it keep tracks of each result saving it, so if you have huge lists it can became costly on the memory occupation front.

How to write a list comprehension without repeating the evaluation of a function?

You can store the values of the calls in a for loop target, if you create an iterable (like a tuple or a list) with just one element in it; that one element would be a tuple with the (f(i), g(i)) result herer:

res = [fi * gi for i in x for fi, gi in [(f(i), g(i))] if fi < gi]

This is, however, not pretty and not very readable.

You'd be far better off using an actual for loop for this:

res = []
for i in x:
    fi, gi = f(i), g(i)
    if fi < gi:
        res.append(fi * gi)

Another option is to produce an intermediate generator expression that calculates f(i) and g(i), then uses the output in a list comprehension:

intermediate = ((f(i), g(i)) for i in x)
res = [fi * gi for fi, gi in intermediate if fi < gi]

Avoid computing the same expression twice in list comprehension

Compute the results beforehand and iterate over them

new_list = [x for x in map(f, old_list) if x !=0]

Moreover, since map computes the result per element when the element is accessed this is just one loop.

List comprehension with duplicated function call

You can use a generator inside the list comprehension:

  [x for x in (el.strip() for el in mylist.split(",")) if x]
#             \__________________ ___________________/
#                                v
#                        internal generator

The generator thus will provide stripped elements, and we iterate over the generator, and only check the truthiness. We thus save on el.strip() calls.

You can also use map(..) for this (making it more functional):

  [x for x in map(str.strip, mylist.split(",")) if x]
#             \______________ ________________/
#                            v
#                           map

But this is basically the same (although the logic of the generator is - in my opinion - better encapsulated).

Avoiding Duplicate Function call in List Comprehension in Python

You can use a nested list comprehension (well, generator expression in this case to avoid actually building a list):

newlns = [i + "\n" for i in (line.strip() for ln in lns) if i]

In fact you really shouldn't even bother to read the file first, just put it in there too: iterating over a file yields its lines.

with open(fname, 'r') as file:
    newlns = [i + "\n" for i in (line.strip() for ln in file) if i]

evaluation of python list comprehensions

A list comprehension is always fully evaluated where it occurs, just like any other expression like a+b. The list comprehension does not "know" that it is inside another list comprehension, so it can't behave any differently on that basis.

If you want to iterate "bit by bit", generating only one item from the list comprehension at a time, then use a generator expression.

Here is a simple comparison:

def inner(x):
    print("inner")
    return x.upper()

def outer(x):
    print("outer")
    return x+"!"

# inner comprehension is a list comprehension
>>> [outer(x) for x in [inner(x) for x in ('a', 'b', 'c')]]
inner
inner
inner
outer
outer
outer
['A!', 'B!', 'C!']

# inner comprehension is a generator comprehension
>>> [outer(x) for x in (inner(x) for x in ('a', 'b', 'c'))]
inner
outer
inner
outer
inner
outer
['A!', 'B!', 'C!']

As keyword in list comprehensions

In python 3.8 you can use walrus operator to do this:

>>> L = [y for x in X if (y := foo(bar(baz(bla(x))))) == 1]

How to save repeated computation in list comprehension in Python?

In 3.8 and above, you can use the walrus operator for this:

sums = [s for k in keyboards for d in drivers if (s := k + d) <= upper_limit]

The performance advantage appears to be slight for this example:

$ python -m timeit "[s for k in range(1000) for d in range(1000) if (s := k + d) <= 1000]"
5 loops, best of 5: 72.5 msec per loop
$ python -m timeit "[k + d for k in range(1000) for d in range(1000) if k + d <= 1000]"
5 loops, best of 5: 75.6 msec per loop

The k + d computation would, presumably, have to be a lot more complex to show a major benefit.

List comprehension to evaluate to true if list contains a value

Did you mean to convert the resulting list to bool()?

a = [0,0,1,1,0,1]
b = bool([i for i in a if i == 1])
print(b)

Mistake in list comprehension

You can't reference the same list inside of a list comprehension.

Thanks @Keith from the comments of the OP for pointing it out.

---

Related Topics