2D List Has Weird Behavor When Trying to Modify a Single Value

2D list has weird behavor when trying to modify a single value

This makes a list with five references to the same list:

data = [[None]*5]*5

Use something like this instead which creates five separate lists:

>>> data = [[None]*5 for _ in range(5)]

Now it does what you expect:

>>> data[0][0] = 'Cell A1'
>>> print data
[['Cell A1', None, None, None, None],
 [None, None, None, None, None],
 [None, None, None, None, None],
 [None, None, None, None, None],
 [None, None, None, None, None]]

Python array weird behavior

When you say [None]*6, a list of six None values is created. When you multiply that by 6, it's six copies of the same list that you get. So modifying one row will also modify all the other rows in the same way.

You can work around this behaviour by doing this (or see this question for alternatives):

arr = [[None]*6 for i in xrange(6)]

List of lists changes reflected across sublists unexpectedly

When you write [x]*3 you get, essentially, the list [x, x, x]. That is, a list with 3 references to the same x. When you then modify this single x it is visible via all three references to it:

x = [1] * 4
xs = [x] * 3
print(f"id(x): {id(x)}")
# id(x): 140560897920048
print(
    f"id(xs[0]): {id(xs[0])}\n"
    f"id(xs[1]): {id(xs[1])}\n"
    f"id(xs[2]): {id(xs[2])}"
)
# id(xs[0]): 140560897920048
# id(xs[1]): 140560897920048
# id(xs[2]): 140560897920048

x[0] = 42
print(f"x: {x}")
# x: [42, 1, 1, 1]
print(f"xs: {xs}")
# xs: [[42, 1, 1, 1], [42, 1, 1, 1], [42, 1, 1, 1]]

To fix it, you need to make sure that you create a new list at each position. One way to do it is

[[1]*4 for _ in range(3)]

which will reevaluate [1]*4 each time instead of evaluating it once and making 3 references to 1 list.

---

You might wonder why * can't make independent objects the way the list comprehension does. That's because the multiplication operator * operates on objects, without seeing expressions. When you use * to multiply [[1] * 4] by 3, * only sees the 1-element list [[1] * 4] evaluates to, not the [[1] * 4 expression text. * has no idea how to make copies of that element, no idea how to reevaluate [[1] * 4], and no idea you even want copies, and in general, there might not even be a way to copy the element.

The only option * has is to make new references to the existing sublist instead of trying to make new sublists. Anything else would be inconsistent or require major redesigning of fundamental language design decisions.

In contrast, a list comprehension reevaluates the element expression on every iteration. [[1] * 4 for n in range(3)] reevaluates [1] * 4 every time for the same reason [x**2 for x in range(3)] reevaluates x**2 every time. Every evaluation of [1] * 4 generates a new list, so the list comprehension does what you wanted.

Incidentally, [1] * 4 also doesn't copy the elements of [1], but that doesn't matter, since integers are immutable. You can't do something like 1.value = 2 and turn a 1 into a 2.

Why does my 2D array have such strange behavior?

I was able to fix my own problem after messing around with the code a lot.

I switched around rows and columns after I realized that elements of the matrix had to be accessed by matrix.data[y][x] instead of matrix.data[x][y]. I found a few places where columns and rows were mixed up, but in a 3x3 matrix, it wouldn't make a difference.

I changed the whole way the struct was created. I stopped using malloc to allocate it first and then fill it in with data and instead did this:

newMatrix {columns, rows, malloc(sizeof(float*) * columns)};
I was using malloc to allocate the struct before because I was making room for the pointer array in my struct, but malloc will allocate room somewhere random and give me the pointer to where it allocated memory. I think I was confused on the line between pointers and arrays.
I also am using newMatrix.data to refer to the pointer array instead of creating the array rowData[rows] and then storing the pointer to row data in newMatrix.data. I think I was confusing pointers and arrays again there.

Because I changed around rows and columns, the pointers in the array matrix.data points to do not point to arrays for the row data, but for the columns now.

matrix.h became this after fixing it:

#pragma once

typedef struct matrix {
  int columns;
  int rows;
  float **data;
} matrix;

matrix createMatrix(int columns, int rows, float initialData[]) {
  //create struct struct                                                                                                                                                                                                                                                             
  matrix newMatrix = {
    columns,
    rows,
    malloc(sizeof(float*) * columns) //create array of pointers and store the pointer to it
  };

  if(newMatrix.data == NULL) puts("pointer array allocation failed");
  memset(newMatrix.data, 0, sizeof(float*) * columns);

  //create initial data if none is given                                                                                                                                                                                                                                        
  if(initialData == NULL) {
    initialData = malloc(sizeof(float) * columns * rows);
    if(initialData == NULL) puts("Array allocation for initial data failed");
    memset(initialData, 0, sizeof(float) * columns * rows);
  }

  //get the elements of each column                                                                                                                                                                                                                                                
  float *columnData;
  int i;
  int j;

  for(i = 0; i < columns; i++) {
    printf("On column: %i\n", i + 1);
    //allocate data to store column data                                                                                                                                                                                                                                        
    columnData = malloc(sizeof(float) * rows); //create array for column and record pointer                                                                                                                                                                                     
    if(columnData == NULL) printf("Array allocation for matrix column %i failed", i + 1);
    memset(columnData, 0, sizeof(float) * rows);
    newMatrix.data[i] = columnData; //store pointer to column data                                                                                                                                                                                                              

    for(j = 0; j < rows; j++) {
      columnData[j] = initialData[(j * columns) + i];
      printf("%f ", newMatrix.data[i][j]);
    }
    printf("\n");
  }

  return newMatrix;
}

void printMatrix(matrix matrix) {
  printf("Confirming pointer to array of pointers:\n%p\n", matrix.data);
  int i;
  puts("Confirming pointers to columns:");
  for(i = 0; i < matrix.columns; i++) {
    printf("%p\n", matrix.data[i]);
  }
  int j;

  for(int i = 0; i < matrix.rows; i++) {
    printf("On row: %i\n", i + 1);
    for(int j = 0; j < matrix.columns; j++) {
      printf("%f ", matrix.data[j][i]);
    }
    printf("\n");
  }
}

Note that I still have yet to deallocate the matrix and to fix the memory leak that is caused when I allocate initialData if NULL is passed, but that wasn't part of the question.

The only line that changed in the main file was:

for(i = 0; i < myMatrix.rows; i++) {

It changed to:

for(i = 0; i < myMatrix.columns; i++) {

In memory, column 1 still is farther away from column 2 than 2 is to 3, and I still don't know why, but it isn't affecting my code. The problem with accessing elements of the matrix was just the product me confusing arrays and pointers and making things messy as I try to fix them.

Modifying 2D array Python

Ah, you have stumbled onto what I thought was a very strange behavior of lists (turns out it makes sense).

basically what is happening is that each sublist is a copy of the previous, and they all point at the same object, meaning that modifying one modifies all of them.

When working with arrays I suggest using numpy

import numpy as np
dp = np.zeros((4,3))

Appending to 2D lists in Python

You haven't created three different empty lists. You've created one empty list, and then created a new list with three references to that same empty list. To fix the problem use this code instead:

listy = [[] for i in range(3)]

Running your example code now gives the result you probably expected:

>>> listy = [[] for i in range(3)]
>>> listy[1] = [1,2]
>>> listy
[[], [1, 2], []]
>>> listy[1].append(3)
>>> listy
[[], [1, 2, 3], []]
>>> listy[2].append(1)
>>> listy
[[], [1, 2, 3], [1]]

---

Related Topics