How Does Multiplication Differ for Numpy Matrix VS Array Classes

how does multiplication differ for NumPy Matrix vs Array classes?

In 3.5, Python finally got a matrix multiplication operator. The syntax is a @ b.

What are the differences between numpy arrays and matrices? Which one should I use?

As per the official documents, it's not anymore advisable to use matrix class since it will be removed in the future.

https://numpy.org/doc/stable/reference/generated/numpy.matrix.html

As other answers already state that you can achieve all the operations with NumPy arrays.

numpy matrix vector multiplication

Simplest solution

Use numpy.dot or a.dot(b). See the documentation here.

>>> a = np.array([[ 5, 1 ,3], 
                  [ 1, 1 ,1], 
                  [ 1, 2 ,1]])
>>> b = np.array([1, 2, 3])
>>> print a.dot(b)
array([16, 6, 8])

This occurs because numpy arrays are not matrices, and the standard operations *, +, -, / work element-wise on arrays.

Note that while you can use numpy.matrix (as of early 2021) where * will be treated like standard matrix multiplication, numpy.matrix is deprecated and may be removed in future releases.. See the note in its documentation (reproduced below):

It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future.

Thanks @HopeKing.

---

Other Solutions

Also know there are other options:

• As noted below, if using python3.5+ and numpy v1.10+, the @ operator works as you'd expect:

  >>> print(a @ b)
  array([16, 6, 8])
  

• If you want overkill, you can use numpy.einsum. The documentation will give you a flavor for how it works, but honestly, I didn't fully understand how to use it until reading this answer and just playing around with it on my own.

  >>> np.einsum('ji,i->j', a, b)
  array([16, 6, 8])
  

• As of mid 2016 (numpy 1.10.1), you can try the experimental numpy.matmul, which works like numpy.dot with two major exceptions: no scalar multiplication but it works with stacks of matrices.

  >>> np.matmul(a, b)
  array([16, 6, 8])
  

• numpy.inner functions the same way as numpy.dot for matrix-vector multiplication but behaves differently for matrix-matrix and tensor multiplication (see Wikipedia regarding the differences between the inner product and dot product in general or see this SO answer regarding numpy's implementations).

  >>> np.inner(a, b)
  array([16, 6, 8])
  
  # Beware using for matrix-matrix multiplication though!
  >>> b = a.T
  >>> np.dot(a, b)
  array([[35,  9, 10],
         [ 9,  3,  4],
         [10,  4,  6]])
  >>> np.inner(a, b) 
  array([[29, 12, 19],
         [ 7,  4,  5],
         [ 8,  5,  6]])
  

• If you have multiple 2D arrays to dot together, you may consider the np.linalg.multi_dot function, which simplifies the syntax of many nested np.dots. Note that this only works with 2D arrays (i.e. not for matrix-vector multiplication).

    >>> np.dot(np.dot(a, a.T), a).dot(a.T)
    array([[1406,  382,  446],
           [ 382,  106,  126],
           [ 446,  126,  152]])
    >>> np.linalg.multi_dot((a, a.T, a, a.T))
    array([[1406,  382,  446],
           [ 382,  106,  126],
           [ 446,  126,  152]])
  

---

Rarer options for edge cases

• If you have tensors (arrays of dimension greater than or equal to one), you can use numpy.tensordot with the optional argument axes=1:

  >>> np.tensordot(a, b, axes=1)
  array([16,  6,  8])
  

• Don't use numpy.vdot if you have a matrix of complex numbers, as the matrix will be flattened to a 1D array, then it will try to find the complex conjugate dot product between your flattened matrix and vector (which will fail due to a size mismatch n*m vs n).

Difference between numpy dot() and Python 3.5+ matrix multiplication @

The @ operator calls the array's __matmul__ method, not dot. This method is also present in the API as the function np.matmul.

>>> a = np.random.rand(8,13,13)
>>> b = np.random.rand(8,13,13)
>>> np.matmul(a, b).shape
(8, 13, 13)

From the documentation:

matmul differs from dot in two important ways.

• Multiplication by scalars is not allowed.
• Stacks of matrices are broadcast together as if the matrices were elements.

The last point makes it clear that dot and matmul methods behave differently when passed 3D (or higher dimensional) arrays. Quoting from the documentation some more:

For matmul:

If either argument is N-D, N > 2, it is treated as a stack of matrices residing in the last two indexes and broadcast accordingly.

For np.dot:

For 2-D arrays it is equivalent to matrix multiplication, and for 1-D arrays to inner product of vectors (without complex conjugation). For N dimensions it is a sum product over the last axis of a and the second-to-last of b

Why is a.dot(b) faster than a@b although Numpy recommends a@b

Your premise is incorrect. You should use larger matrices to measure performance to avoid function calls dwarfing insignificant calculations.

Using Python 3.60 / NumPy 1.11.3 you will find, as explained here, that @ calls np.matmul and both outperform np.dot.

import numpy as np

n = 500
a = np.arange(n**2).reshape(n, n)
b = np.arange(n**2).reshape(n, n)

%timeit a.dot(b)        # 134 ms per loop
%timeit a @ b           # 71 ms per loop
%timeit np.matmul(a,b)  # 70.6 ms per loop

Also note, as explained in the docs, np.dot is functionally different to @ / np.matmul. In particular, they differ in treatment of matrices with dimensions greater than 2.

What is the multiplication operator actually doing with numpy arrays?

It's a little bit complicated and has to do with the concept of broadcasting and the fact that all numpy operations are element wise.

1. a is a 2D array with 1 row and 3 columns and b is a 2D array with 1 column and 3 rows.
2. If you try to multiply them element by element (which is what numpy tries to do if you do a * b because every basic operation except the dot operation is element wise), it must broadcast the arrays so that they match in all their dimensions.
3. Since the first array is 1x3 and the second is 3x1 they can be broadcasted to 3x3 matrix according to the broadcasting rules. They will look like:

a = [[1, 2, 3],
     [1, 2, 3],
     [1, 2, 3]]

b = [[4, 4, 4],
     [5, 5, 5],
     [6, 6, 6]]

And now Numpy can multiply them element by element, giving you the result:

[[ 4,  8, 12],
 [ 5, 10, 15],
 [ 6, 12, 18]]

When you are doing a .dot operation it does the standard matrix multiplication. More in docs

Numpy calculate difference of matrices against all rows in matrix

The numpy solution suggested by blorgon is likely faster, but you can also use scipy.spatial.distance.cdist:

>>> from scipy.spatial.distance import cdist
>>> cdist(a, b)**2
array([[ 18.29  , 112.45  , 308.6765],
       [  7.49  ,  79.65  , 251.0165]])

The problem with this approach is that it takes a square root and then undoes it. The advantage is that it does not use a large intermediate array. You can avoid some intermediates in numpy like this:

>>> diff = b - a[:, np.newaxis]
>>> np.power(diff, 2, out=diff).sum(axis=2)
array([[ 18.29  , 112.45  , 308.6765],
       [  7.49  ,  79.65  , 251.0165]])

---

Related Topics