Numpy Matrix Vector Multiplication

numpy matrix vector multiplication

Simplest solution

Use numpy.dot or a.dot(b). See the documentation here.

>>> a = np.array([[ 5, 1 ,3], 
                  [ 1, 1 ,1], 
                  [ 1, 2 ,1]])
>>> b = np.array([1, 2, 3])
>>> print a.dot(b)
array([16, 6, 8])

This occurs because numpy arrays are not matrices, and the standard operations *, +, -, / work element-wise on arrays.

Note that while you can use numpy.matrix (as of early 2021) where * will be treated like standard matrix multiplication, numpy.matrix is deprecated and may be removed in future releases.. See the note in its documentation (reproduced below):

It is no longer recommended to use this class, even for linear algebra. Instead use regular arrays. The class may be removed in the future.

Thanks @HopeKing.

---

Other Solutions

Also know there are other options:

• As noted below, if using python3.5+ and numpy v1.10+, the @ operator works as you'd expect:

  >>> print(a @ b)
  array([16, 6, 8])
  

• If you want overkill, you can use numpy.einsum. The documentation will give you a flavor for how it works, but honestly, I didn't fully understand how to use it until reading this answer and just playing around with it on my own.

  >>> np.einsum('ji,i->j', a, b)
  array([16, 6, 8])
  

• As of mid 2016 (numpy 1.10.1), you can try the experimental numpy.matmul, which works like numpy.dot with two major exceptions: no scalar multiplication but it works with stacks of matrices.

  >>> np.matmul(a, b)
  array([16, 6, 8])
  

• numpy.inner functions the same way as numpy.dot for matrix-vector multiplication but behaves differently for matrix-matrix and tensor multiplication (see Wikipedia regarding the differences between the inner product and dot product in general or see this SO answer regarding numpy's implementations).

  >>> np.inner(a, b)
  array([16, 6, 8])
  
  # Beware using for matrix-matrix multiplication though!
  >>> b = a.T
  >>> np.dot(a, b)
  array([[35,  9, 10],
         [ 9,  3,  4],
         [10,  4,  6]])
  >>> np.inner(a, b) 
  array([[29, 12, 19],
         [ 7,  4,  5],
         [ 8,  5,  6]])
  

• If you have multiple 2D arrays to dot together, you may consider the np.linalg.multi_dot function, which simplifies the syntax of many nested np.dots. Note that this only works with 2D arrays (i.e. not for matrix-vector multiplication).

    >>> np.dot(np.dot(a, a.T), a).dot(a.T)
    array([[1406,  382,  446],
           [ 382,  106,  126],
           [ 446,  126,  152]])
    >>> np.linalg.multi_dot((a, a.T, a, a.T))
    array([[1406,  382,  446],
           [ 382,  106,  126],
           [ 446,  126,  152]])
  

---

Rarer options for edge cases

• If you have tensors (arrays of dimension greater than or equal to one), you can use numpy.tensordot with the optional argument axes=1:

  >>> np.tensordot(a, b, axes=1)
  array([16,  6,  8])
  

• Don't use numpy.vdot if you have a matrix of complex numbers, as the matrix will be flattened to a 1D array, then it will try to find the complex conjugate dot product between your flattened matrix and vector (which will fail due to a size mismatch n*m vs n).

Numpy Matrix Multiplication with Vectors

The reason why you are getting a scalar because you are multiplying two 1D vectors in numpy, which produces the inner product of 2 vectors. You need to reshape your vector to the shape (3,1), which turns them into a 2D shape and then you get the expected result upon performing the vector multiplication. Check the snippet below

>>> import numpy as np
>>> A = np.array([1,2,3])
>>> B = np.array([4,5,6])
>>> A.shape
(3,)
>>> B.shape
(3,)
>>> AA = A.reshape(3, 1)
>>> BB = B.reshape(3, 1)
>>> AA.shape
(3, 1)
>>> BB.shape
(3, 1)
>>> np.matmul(AA, np.transpose(BB))
array([[ 4,  5,  6],
       [ 8, 10, 12],
       [12, 15, 18]])

Multiplying a matrix with array of vectors in NumPy

There are a few different ways to solve this problem.

Option 1:

The most straightforward is to reshape the array vectors so that it has shape (3, 128 * 128), then call the builtin np.dot function, and reshape the result back to your desired shape.

(Note that the (128, 128) part of the array's shape is not really relevant to the rotation; it's an interpretation that you probably want to make your problem clearer, but makes no difference to the linear transformation you want to apply. Said another way, you are rotating 3-vectors. There are 128 * 128 == 16384 of them, they just happen to be organized into a 3D array like above.)

This approach would look like:

>>> v = vectors.reshape(-1, 3).T
>>> np.dot(rotation_matrix, v).shape
(3, 16384)
>>> rotated = np.dot(rotation_matrix, v).T.reshape(vectors.shape)
>>> rotated.shape == vectors.shape
True

Option 2:

Another method that does not involve any reshaping is to use NumPy's Einstein summation. Einstein summation is very flexible, and takes a while to understand, but its power justifies its complexity. In its simplest form, you "label" the axes that you want to multiply together. Axes that are omitted are "contracted", meaning a sum across that axis is computed. For your case, it would be:

>>> np.einsum('ij,klj->kli', rotation_matrix, vectors).shape
(128, 128, 3)
>>> np.allclose(rotated, np.einsum('ij,klj->kli', rotation_matrix_vectors))
True

Here's a quick explanation of the indexing. We are labeling the axes of the rotation matrix i and j, and the axes of the vectors k, l, and j. The repeated j means those are the axes multiplied together. This is equivalent to right-multiplying the reshaped array above with the rotation matrix (i.e., it's a rotation).

The output axes are labeled kli. This means we're preserving the k and l axes of the vectors. Since j is not in the output labels, there is a summation across that axis. Instead we have the axis i, hence the final shape of (128, 128, 3). You can see above that the dot-product method and the einsum method agree.

It can take a while to wrap your head around Einstein summation, but it is super awesome and powerful. I highly recommend learning more about it, especially if this sort of linear algebra is a common problem for you.

matrix - vector multiplication in python (numpy)

Remember that for a matrix multiplication, the second dimension of the first matrix must be equal to the first dimension of the second one. Therefore, performing a matrix multiplication of a 4x1 vector and a 4x4 matrix is not possible.

What you can do is transpose the vector (using myvector.T) so you get a 1x4 vector and multiply that with your 4x4 matrix.

Alternatively, you could multiply the vector on the right side.

Matrix by Vector multiplication using numpy dot product

The multiplication is possible because v has only one dimension. Numpy considers it as a vector, so as this vector has 2 components, the matrix vector multiplication is allowed.

However, if you force v shape to be 1 row and 2 columns (so 2 dimensions), you will get an (expected) error:

>>> v.reshape(1,2).shape
(1, 2)
>>> np.dot(m, v.reshape(1,2))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<__array_function__ internals>", line 5, in dot
ValueError: shapes (3,2) and (1,2) not aligned: 2 (dim 1) != 1 (dim 0)

In order to have a valid multiplication, you need v to be 2 rows and 1 column (so you need to transpose it):

>>> np.dot(m, v.reshape(1,2).T)
array([[11],
       [17],
       [17]])

And you can see that the shape of the result is (2,1), so 2 rows and one column.

Vectorized Matrix-Vector multiplication along specified axis of multidimensional array

This is where np.einsum shines:

np.einsum('mnij,mnj->mni', A,B)

Check with

np.allclose(np.einsum('mnij,mnj->mni', A,x), b)

returns True

Numpy matrix multiplication between a 2D array and each vector in 3D array

You could use the @ operator:

A @ H.T
 
array([[[ 14,  32,  50],
        [ 32,  77, 122],
        [ 50, 122, 194]],

       [[ 68, 167, 266],
        [ 86, 212, 338],
        [104, 257, 410]]])

How to multiply two vector and get a matrix?

Normal matrix multiplication works as long as the vectors have the right shape. Remember that * in Numpy is elementwise multiplication, and matrix multiplication is available with numpy.dot() (or with the @ operator, in Python 3.5)

>>> numpy.dot(numpy.array([[1], [2]]), numpy.array([[3, 4]]))
array([[3, 4],
       [6, 8]])

This is called an "outer product." You can get it using plain vectors using numpy.outer():

>>> numpy.outer(numpy.array([1, 2]), numpy.array([3, 4]))
array([[3, 4],
       [6, 8]])

Numpy matrix-vector multiplication with complex elements

Borrowing A_p and u_p from @Ivan I'd like to advertise the following much simpler approach that makes use of __matmul__'s inbuilt broadcasting. All we need to do is reverse the order of dimensions:

(u_p.T@A_p.T).T

This gives the same result as Ivan's einsum but is a bit more flexible giving us full broadcasting without us having to figure out the corresponding format string every time.

np.allclose(np.einsum('ijmn,jkmn->ikmn', A_p, u_p),(u_p.T@A_p.T).T)
# True

---

Related Topics