Unpacking, Extended Unpacking and Nested Extended Unpacking

Unpacking, extended unpacking and nested extended unpacking

My apologies for the length of this post, but I decided to opt for completeness.

Once you know a few basic rules, it's not hard to generalize them. I'll do my best to explain with a few examples. Since you're talking about evaluating these "by hand," I'll suggest some simple substitution rules. Basically, you might find it easier to understand an expression if all the iterables are formatted in the same way.

For the purposes of unpacking only, the following substitutions are valid on the right side of the = (i.e. for rvalues):

'XY' -> ('X', 'Y')
['X', 'Y'] -> ('X', 'Y')

If you find that a value doesn't get unpacked, then you'll undo the substitution. (See below for further explanation.)

Also, when you see "naked" commas, pretend there's a top-level tuple. Do this on both the left and the right side (i.e. for lvalues and rvalues):

'X', 'Y' -> ('X', 'Y')
a, b -> (a, b)

With those simple rules in mind, here are some examples:

(a,b), c = "XY", "Z"                 # a = 'X', b = 'Y', c = 'Z'

Applying the above rules, we convert "XY" to ('X', 'Y'), and cover the naked commas in parens:

((a, b), c) = (('X', 'Y'), 'Z')

The visual correspondence here makes it fairly obvious how the assignment works.

Here's an erroneous example:

(a,b), c = "XYZ"

Following the above substitution rules, we get the below:

((a, b), c) = ('X', 'Y', 'Z')

This is clearly erroneous; the nested structures don't match up. Now let's see how it works for a slightly more complex example:

(a,b), c, = [1,2],'this'             # a = '1', b = '2', c = 'this'

Applying the above rules, we get

((a, b), c) = ((1, 2), ('t', 'h', 'i', 's'))

But now it's clear from the structure that 'this' won't be unpacked, but assigned directly to c. So we undo the substitution.

((a, b), c) = ((1, 2), 'this')

Now let's see what happens when we wrap c in a tuple:

(a,b), (c,) = [1,2],'this'           # ERROR -- too many values to unpack

Becomes

((a, b), (c,)) = ((1, 2), ('t', 'h', 'i', 's'))

Again, the error is obvious. c is no longer a naked variable, but a variable inside a sequence, and so the corresponding sequence on the right is unpacked into (c,). But the sequences have a different length, so there's an error.

Now for extended unpacking using the * operator. This is a bit more complex, but it's still fairly straightforward. A variable preceded by * becomes a list, which contains any items from the corresponding sequence that aren't assigned to variable names. Starting with a fairly simple example:

a, *b, c = "X...Y"                   # a = 'X', b = ['.','.','.'], c = 'Y'

This becomes

(a, *b, c) = ('X', '.', '.', '.', 'Y')

The simplest way to analyze this is to work from the ends. 'X' is assigned to a and 'Y' is assigned to c. The remaining values in the sequence are put in a list and assigned to b.

Lvalues like (*a, b) and (a, *b) are just special cases of the above. You can't have two * operators inside one lvalue sequence because it would be ambiguous. Where would the values go in something like this (a, *b, *c, d) -- in b or c? I'll consider the nested case in a moment.

*a = 1                               # ERROR -- target must be in a list or tuple

Here the error is fairly self-explanatory. The target (*a) must be in a tuple.

*a, = (1,2)                          # a = [1,2]

This works because there's a naked comma. Applying the rules...

(*a,) = (1, 2)

Since there are no variables other than *a, *a slurps up all the values in the rvalue sequence. What if you replace the (1, 2) with a single value?

*a, = 1                              # ERROR -- 'int' object is not iterable

becomes

(*a,) = 1

Again, the error here is self-explanatory. You can't unpack something that isn't a sequence, and *a needs something to unpack. So we put it in a sequence

*a, = [1]                            # a = [1]

Which is eqivalent to

(*a,) = (1,)

Finally, this is a common point of confusion: (1) is the same as 1 -- you need a comma to distinguish a tuple from an arithmetic statement.

*a, = (1)                            # ERROR -- 'int' object is not 

Now for nesting. Actually this example wasn't in your "NESTED" section; perhaps you didn't realize it was nested?

(a,b), *c = 'XY', 2, 3               # a = 'X', b = 'Y', c = [2,3]

Becomes

((a, b), *c) = (('X', 'Y'), 2, 3)

The first value in the top-level tuple gets assigned, and the remaining values in the top-level tuple (2 and 3) are assigned to c -- just as we should expect.

(a,b),c = 1,2,3                      # ERROR -- too many values to unpack
*(a,b), c = 1,2,3                    # a = 1, b = 2, c = 3

I've already explained above why the first line throws an error. The second line is silly but here's why it works:

(*(a, b), c) = (1, 2, 3)

As previously explained, we work from the ends. 3 is assigned to c, and then the remaining values are assigned to the variable with the * preceding it, in this case, (a, b). So that's equivalent to (a, b) = (1, 2), which happens to work because there are the right number of elements. I can't think of any reason this would ever appear in working code. Similarly,

*(a, *b), c = 'this'                 # a = 't', b = ['h', 'i'], c = 's'

becomes

(*(a, *b), c) = ('t', 'h', 'i', 's')

Working from the ends, 's' is assigned to c, and ('t', 'h', 'i') is assigned to (a, *b). Working again from the ends, 't' is assigned to a, and ('h', 'i') is assigned to b as a list. This is another silly example that should never appear in working code.

Extended tuple unpacking in Python 2

You could define a wrapper function that converts your list to a four tuple. For example:

def wrapper(thelist):
    for item in thelist:
        yield(item[0], item[1], item[2], item[3:])

mylist = [(1,2,3,4), (5,6,7,8)]

for a, b, c, d in wrapper(mylist):
    print a, b, c, d

The code prints:

1 2 3 (4,)
5 6 7 (8,)

What happens in memory while unpacking a collection?

The thing about your picture that's misleading is that it implies that a, b, and c reference slices of list1. If you change list1, though, you will find that a, b, and c aren't affected by that change.

A better way to draw the picture might be to show 4, 8, and 12 separate from list1:

list1-->[ ][ ][ ]
         |  |  |
         V  V  V
         4  8  12
         ^  ^  ^
         |  |  |
         a  b  c

All of the variables are independent of one another, even though some of them (e.g. list1[0] and a) currently point to the same values.

To put it another way: saying a = list1[0] is saying "evaluate list1[0] and assign a to reference whatever that value is right now", which is not the same as saying "make a be an alias for list1[0]".

Extended sequence unpacking in python3

It was a design choice, according to PEP 3132 (with my bold):

A tuple (or list) on the left side of a simple assignment (unpacking is not defined for augmented assignment) may contain at most one expression prepended with a single asterisk (which is henceforth called a "starred" expression, while the other expressions in the list are called "mandatory"). This designates a subexpression that will be assigned a list of all items from the iterable being unpacked that are not assigned to any of the mandatory expressions, or an empty list if there are no such items.

Indeed, the first example in the PEP illustrates your point:

>>> a, *b, c = range(5)
>>> a
0
>>> c
4
>>> b
[1, 2, 3]

Unpacking: [x,y], (x,y), x,y - what is the difference?

There is no difference. Regardless of what kind of syntactic sequence you use, the same byte code is generated.

>>> def f():
...   return 0, 1
...
>>> import dis
>>> dis.dis('[a,b] = f()')
  1           0 LOAD_NAME                0 (f)
              2 CALL_FUNCTION            0
              4 UNPACK_SEQUENCE          2
              6 STORE_NAME               1 (a)
              8 STORE_NAME               2 (b)
             10 LOAD_CONST               0 (None)
             12 RETURN_VALUE
>>> dis.dis('(a,b) = f()')
  1           0 LOAD_NAME                0 (f)
              2 CALL_FUNCTION            0
              4 UNPACK_SEQUENCE          2
              6 STORE_NAME               1 (a)
              8 STORE_NAME               2 (b)
             10 LOAD_CONST               0 (None)
             12 RETURN_VALUE
>>> dis.dis('a, b = f()')
  1           0 LOAD_NAME                0 (f)
              2 CALL_FUNCTION            0
              4 UNPACK_SEQUENCE          2
              6 STORE_NAME               1 (a)
              8 STORE_NAME               2 (b)
             10 LOAD_CONST               0 (None)
             12 RETURN_VALUE

In every case, you simply call f, then use UNPACK_SEQUENCE to produce the values to assign to a and b.

---

Even if you want to argue that byte code is an implementation detail of CPython, the definition of a chained assignment is not. Given

x = [a, b] = f()

the meaning is the same as

tmp = f()
x = tmp
[a, b] = tmp

x is assigned the result of f() (a tuple), not the "list" [a, b].

---

Finally, here is the grammar for an assignment statement:

assignment_stmt ::=  (target_list "=")+ (starred_expression | yield_expression)
target_list     ::=  target ("," target)* [","]
target          ::=  identifier
                     | "(" [target_list] ")"
                     | "[" [target_list] "]"
                     | attributeref
                     | subscription
                     | slicing
                     | "*" target

Arguably, the "[" [target_list] "]" could and should have been removed in Python 3. Such a breaking change would be difficult to implement now, given the stated preference to avoid any future changes to Python on the scale of the 2-to-3 transition.

How to deal with unequal element list to unpack into multiple variables (ValueError: too many values to unpack (expected 4))

Use extended iterable unpacking features:

obtainedDatetime, itemName, itemType, itemRarity, bannerCode, obtainedTimestamp, *rest = item

So, rest will contain a list with the rest of the iterable. If you are not interested in this data, conventionally, we would name it _. So:

obtainedDatetime, itemName, itemType, itemRarity, bannerCode, obtainedTimestamp, *_ = item

why is there a second variable poinbting to map() function and where did the variable student come from

The syntax in the first line is only valid if the map reads a pair of input,
then the first input will be stored in the first variable and the second input will be stored in the second one.
Else, it will give an error.

ngfor unpacking 2 variables at a time

You could convert that array to an object with properties (although I'm guessing you're already starting with an object?)

const keyValues = [[1,2], [4,5], [8,9]]
const obj = Object.fromEntries(keyValues)

There's an Angular pipe that can be used to extract the key value pairs

<div *ngFor="let item of obj | keyvalue">
    {{item.key}}:{{item.value}}
</div>

---

Related Topics