Does Python Support Short-Circuiting

Does Python support short-circuiting?

Yep, both and and or operators short-circuit -- see the docs.

Question about short circuiting with python logical operators

and has a higher precedence than or, so your code is equivalent to this:

d = (False and False) or True
print(d)

e = True or (False and False)
print(e)

For it to work like you expect, you have to change d to this:

d = False and (False or True)

Does Python implement short-circuiting in built-in functions such as min()?

python has to evaluate all values inside the iterable because the languaje evaluate element by element, if you have in your tuple something that is not a number it will trigger an exception when try to perform the % operation. Python can not guess what is inside your list. You can test this by defining a function instead of a lambda and set debug point inside.

def my_mod(x): 
import ipdb; ipdb.set_trace()
return x % 20

then call the function

min((20,11), key = my_mod)

you can do a quick error test case with

min((20,11, "s"), key = my_mod)

It will trigger an exception but first had to evaluate all the previous element in the list.

Does Python's `all` function use short circuit evaluation?

Yes, it short-circuits:

>>> def test():
... yield True
... print('one')
... yield False
... print('two')
... yield True
... print('three')
...
>>> all(test())
one
False

From the docs:

Return True if all elements of the iterable are true (or if the iterable is empty). Equivalent to:

def all(iterable):
for element in iterable:
if not element:
return False
return True

So when it returns False, then the function immediately breaks.

Python short-circuiting of functions

When you did this:

>>> condition = test_a and test_b

you incorrectly expected to get a new function that returns the result test_a(x) and test_b(x). You actually got the evaluation of a Boolean expression:

x and y: if x is false, then x, else y

Since the truth value of both test_a and test_b is True, condition is set to test_b. This is why condition(menu) gives the same result as test_b(menu).

To achieve the expected behavior, do:

>>> def condition(x):
... return test_a(x) and test_b(x)
...

Any/All python short-circuit: Why doesn't the following work?

Your code errors because the expressions have to be evaluated before being passed into the function.

In this context, short-circuiting actually means that as soon as they find a different value, they return and don't bother checking the remaining values. It's not the same kind of short-circuiting that and and or do, which can actually avoid evaluating expressions.

To illustrate, let's use an iterator, which will get consumed:

>>> a = iter([1, 0, 2, 3])
>>> all(a) # Search until a falsy value
False
>>> list(a) # Show what's left in the iterator
[2, 3]

You can see that it consumed 1 and 0, returning on 0 because it's falsy.


Now, if you did want to do lazy evaluation with any and all, you could call lambdas in a generator expression:

>>> any(e() for e in [lambda: 1, lambda: 0, lambda: 2/0])
True
>>> all(e() for e in [lambda: 1, lambda: 0, lambda: 2/0])
False

(Thanks to this question for inspiring this bit.)

Why is short circuiting not working in python's all function?

This doesn't have anything to do with all. You're creating a list literal, which will evaluate its contents before being passed to all:

>>> [a is None, b, "text" in b]

Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: argument of type 'NoneType' is not iterables

If you want this to work as you expect, you need a lazier structure than a list:

a = None
b = None

def lazy(): # A lazy generator
yield a is None
yield b
yield "text" in b

>>> all(lazy())
False

How to stop short circuiting in Python?

if all([a < p.pop(), b < p.pop()])

This creates a list, which will be evaluated in its entirety, and then uses all to confirm that both values are truthy. But this is somewhat obscure and I'd rather suggest you write plain, easy to understand code:

a_within_limit = a < p.pop()
b_within_limit = b < p.pop()
if a_within_limit and b_within_limit:

What's the value of short-circuit of Python?

As I recall, short-circuit is referring to a compiler optimization. As you are already aware, in an AND conditional, the second expression is not evaluated if the first condition is false. After all, what is the point? The overall expression CANNOT be true since the first condition is already false. Similarly, with the OR logic, as soon as a single condition is true, the entire OR expression is true. This boils down to runtime savings. Expressions can be left unevaluated and thus not burn any CPU cycles.

Incidentally, I use OR short-circuits all the time in bash programming. For example, the following expression is useful for running a function if the preceeding condition is false:

[ $# -ge 2 ] || errexit "You have not supplied enough parameters"

In the above example, errexit will be called only if the command line did not have 2 arguments or more. Of course, in my example, I don't care about performance. Rather, I'm using the || short circuit logic as syntactic sugar.

And that's what it boils down to: In a tight loop, where performance matters, you can be somewhat certain that expressions will not be evaluated unnecessarily. But in the example that you described for && to avoid divide by zero, you could have just as easily written that with a nested IF statement. Again, it's a style choice more often than a performance consideration.

All that said, let me answer your questions one at a time:

I can't understand exact meaning of the second paragraph. In C, we can use the &&(logical and) as the following expression: (i != 0) &&
(j / i > 0) to prevent the error of a division by zero. So then, could
I use the expression (i != 0) and ( j / i > 0) in Python as C to get
the same effect? Is my understanding to the passage right?

You are correct.

What's the usage of or as a short-circuit to constructing Boolean expressions as said in the second paragraph ?

As I explained in detail above: performance and syntax sugar ( that is, less typing and shorter expressions; idioms ).

The final question is about the grammar of had the prior test not
succeeded in the second paragraph. I this it should be "an error
condition that can had the prior test not succeeded", am I right?

I agree with you that the statement could be worded better. But when you try to express it, you'll see that it is a difficult thing to do ( in fact, your suggestion is not valid ). Basically, your example of avoiding divide by zero error is a perfect example of what the author is trying to say. Here's my attempt to paraphrase: Short-circuit logic is useful to check pre-conditions of expressions that may generate errors if those conditions are not met.

Does PySpark support the short-circuit evaluation of conditional statements?

I don't think Spark support short-circuit evaluation on conditionals as stated here https://docs.databricks.com/spark/latest/spark-sql/udf-python.html#:~:text=Spark%20SQL%20(including,short-circuiting%E2%80%9D%20semantics.:

Spark SQL (including SQL and the DataFrame and Dataset API) does not guarantee the order of evaluation of subexpressions. In particular, the inputs of an operator or function are not necessarily evaluated left-to-right or in any other fixed order. For example, logical AND and OR expressions do not have left-to-right “short-circuiting” semantics.

Another alternative way would be creating an array of column_one and column_two, then evaluate if the array contains 'red' using SQL EXISTS

data = data.withColumn('is_red', F.expr("EXISTS(array(color_one, color_two), x -> x = 'red')"))
data.show()
+---+---------+---------+------+
| id|color_one|color_two|is_red|
+---+---------+---------+------+
| 1| blue| red| true|
| 2| red| null| true|
| 3| null| green| false|
| 4| yellow| null| false|
| 5| null| red| true|
| 6| null| null| false|
+---+---------+---------+------+


Related Topics



Leave a reply



Submit