Explanation of How Nested List Comprehension Works

Explanation of how nested list comprehension works?

Ah, the incomprehensible "nested" comprehensions. Loops unroll in the same order as in the comprehension.

[leaf for branch in tree for leaf in branch]

It helps to think of it like this.

for branch in tree:
    for leaf in branch:
        yield leaf

The PEP202 asserts this syntax with "the last index varying fastest" is "the Right One", notably without an explanation of why.

List comprehension on a nested list?

Here is how you would do this with a nested list comprehension:

[[float(y) for y in x] for x in l]

This would give you a list of lists, similar to what you started with except with floats instead of strings. If you want one flat list then you would use [float(y) for x in l for y in x].

Understanding nested list comprehension

The short answer is: yes, you are correct in your understanding.

There's only a catch: the way you normally use nested list comprehension in python code is to operate on multidimensional arrays.

A typical example is when you operate on matrices:

>>> matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
>>> [[el - 1 for el in row] for row in matrix]
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

As you can see the "nesting" works by operating on each dimension of the matrix.

In the examples you provided, it seems that ySet [unfortunate name btw, as sets are one of the types provided with python] is just a generic counter, which makes a bit harder to follow what is going on under the hood.

As for your first example:

>>> rows = ([1, 2, 3], [10, 20, 30])
>>> [(min([row[i] for row in rows]),max([row[i] for row in rows])) for i in range(len(rows[0]))]
[(1, 10), (2, 20), (3, 30)]

You might wish to look into the zip built-in function:

>>> zip(rows[0], rows[1])
[(1, 10), (2, 20), (3, 30)]

or for maximum brevity and elegance:

>>> zip(*rows)
[(1, 10), (2, 20), (3, 30)]

HTH!

What is the under the hood reason that we can use nested for loops in list comprehensions

Your first translation should be

li=[]
for item in iterable: 
    li.append( expression )

Your example [s+z for s in iterable_1 for z in iterable_2] is translated as

li=[]
for s in iterable_1:
    for z in iterable_2:
        li.append(s+z)

Congrats, you have discovered ... monads! which are essentially what you've described, generalized nested loops.

Nested loops just produce a plain stream of results. Nested lists, when flattened, also turn into a plain stream of elements. That's the similarity. A lazy append is pretty much like yield.

Each monad type is defined by how it implements its version of the flatMap function, which is a map followed by the flattening of the resulting nested structure. The flattening of the nested structure at each nesting level allows for an arbitrary depth of nesting to be flattened:

M [M (a)]  ==>  M (a)

M [M [M (a)]]  ==>   # flatten the two outer layers first:
                       M [M (a)]  ==>  M (a)
               OR:
               ==>   # flatten the two inner layers first:
                       M [M (a)]  ==>  M (a)

See the difference? There isn't any! Any type that does the above, is a "monad". Like lists.

So it is with loops as well, which can be nested to an arbitrary depth -- two, three, whatever, it doesn't matter. The whole structure is still producing its results one by one, and these are the results which the innermost loop is producing, one by one.

That is the under the hood reason why we can use nested for loops in list comprehensions. Or, saying the same thing in a fancy way, it is because list comprehensions are just like monadic chains of operations (and can be translated as such).

How do nested list comprehensions work in python?

You can write it like so:

def ScalarMult(A,s):
    As = []
    for row in A:
        Arow = []
        for i in row:
            Arow.append(s*i)
        As.append(Arow)
    return As

Taking this one step further gives:

def ScalarMult(A,s):
    As = []
    for row in A:
        As.append([s*i for i in row])
    return As

Taking it two steps further gives:

def ScalarMult(A,s):
    As = [[s*i for i in row] for row in A]
    return As

(i.e. what you started with.)

Hope this makes things clearer.

Python nested list comprehension (accessing nested elements)

If you convert it to a for loop it might be easier to see..?

res = []
for item in matrix_a:
    a, b = item   # a = item[0]; b = item[1]
    res.append(a)

you're basically unpacking the individual items in the list and picking one of them.

Nested list comprehension not the same as nested for loop

Your list comprehension doesn't do anything as it does not run without i already been defined outside the list comprehension.

To achieve what you want (and yes, this is counter intuitive ) you need to do this:

[j  for i in range(1,5) for j in range(0,i)]

This will yield:

[0, 0, 1, 0, 1, 2, 0, 1, 2, 3]

which is the same order as your nested for loops.

List comprehension to access a nested list within a list only in one index

So, if you have something like the following:

distances = [
   ('highway_bost321', [0.0, 10.174343253183386, 10.947746706490813, 7.187637514988234, 7.660483562939873, 10.622402335214636, 10.737785768990813, 10.566917715980832, 10.819389772897063, 12.03400784892136]),
   ('mountain_0871', [10.947746706490813, 0.83758544921875, 0.0, 5.838234191502578, 5.363256367154217, 1.3175048828125, 3.0810546875, 6.634500456993904, 0.78460693359375, 13.91981431878607]),
   ('opencountry_043', [7.660483562939873, 4.668136048210366, 5.363256367154217, 2.8142531243329643, 0.0, 5.347678752303554, 3.236628762987552, 5.377074226549635, 5.634096535101808, 13.405988675235129]),
   ('opencountry_053', [10.622402335214636, 0.73712158203125, 1.3175048828125, 5.876377140065916, 5.347678752303554, 0.0, 3.1134033203125, 6.11476575647307, 1.97711181640625, 13.144871284931902]),
   ('opencountry_123', [10.737785768990813, 3.01690673828125, 3.0810546875, 5.560564920669245, 3.236628762987552, 3.1134033203125, 0.0, 5.71620618616057, 3.09417724609375, 13.597874214619402]),
   ('palace_019', [10.566917715980832, 5.890665007775154, 6.634500456993904, 3.7392389463104037, 5.377074226549635, 6.11476575647307, 5.71620618616057, 0.0, 6.493675414476205, 12.731745772059295]),
   ('volcano_010', [10.819389772897063, 1.294677734375, 0.78460693359375, 5.709877257908828, 5.634096535101808, 1.97711181640625, 3.09417724609375, 6.493675414476205, 0.0, 13.90083238519232]),
   ('waterfall03', [12.03400784892136, 13.17207261956732, 13.91981431878607, 13.095577523116825, 13.405988675235129, 13.144871284931902, 13.597874214619402, 12.731745772059295, 13.90083238519232, 0.0])
]

sample = [
     ('highway_bost321', 0),
     ('mountain_0871', 2),
     ('opencountry_043', 4),
     ('opencountry_053', 5),
     ('opencountry_123', 6),
     ('palace_019', 7),
     ('volcano_010', 8),
     ('waterfall03', 9)
]

then you can use something like:

result = [
    (k, vs[:index] + vs[index+1:]) for (key, index) in samples 
    for k, vs in distances if k == key
]

Which gets you:

[('highway_bost321', [10.174343253183386, 10.947746706490813, 7.187637514988234, 7.660483562939873, 10.622402335214636, 10.737785768990813, 10.566917715980832, 10.819389772897063, 12.03400784892136]),
 ('mountain_0871', [10.947746706490813, 0.83758544921875, 5.838234191502578, 5.363256367154217, 1.3175048828125, 3.0810546875, 6.634500456993904, 0.78460693359375, 13.91981431878607]),
 ('opencountry_043', [7.660483562939873, 4.668136048210366, 5.363256367154217, 2.8142531243329643, 5.347678752303554, 3.236628762987552, 5.377074226549635, 5.634096535101808, 13.405988675235129]),
 ('opencountry_053', [10.622402335214636, 0.73712158203125, 1.3175048828125, 5.876377140065916, 5.347678752303554, 3.1134033203125, 6.11476575647307, 1.97711181640625, 13.144871284931902]),
 ('opencountry_123', [10.737785768990813, 3.01690673828125, 3.0810546875, 5.560564920669245, 3.236628762987552, 3.1134033203125, 5.71620618616057, 3.09417724609375, 13.597874214619402]),
 ('palace_019', [10.566917715980832, 5.890665007775154, 6.634500456993904, 3.7392389463104037, 5.377074226549635, 6.11476575647307, 5.71620618616057, 6.493675414476205, 12.731745772059295]),
 ('volcano_010', [10.819389772897063, 1.294677734375, 0.78460693359375, 5.709877257908828, 5.634096535101808, 1.97711181640625, 3.09417724609375, 6.493675414476205, 13.90083238519232]),
 ('waterfall03', [12.03400784892136, 13.17207261956732, 13.91981431878607, 13.095577523116825, 13.405988675235129, 13.144871284931902, 13.597874214619402, 12.731745772059295, 13.90083238519232])]

But this is highly inefficient. This will be polynomial time. Notice the inner loop, k, vs in distances if k == key which is highly wastefull.

Instead, distances should be a dict,

distances = dict(distances)

then you can do:

{key: distances[key][:index] + distances[key][index+1:] for key, index in samples}

---

Related Topics