Matrix Multiplication

Matrix Multiplication is a procedure that calculates the components for a new matrix by taking the dot product of a row of the first matrix and a column in the second matrix.

\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \begin{bmatrix} b_{11} & b_{12} \\ b_{21} & b_{22} \end{bmatrix} = \begin{bmatrix} c_{11} & c_{12} \\ c_{21} & c_{22} \end{bmatrix}

  • c_{11} = a_{11}b_{11} + a_{12}b_{21}
  • c_{12} = a_{11}b_{12} + a_{12}b_{22}
  • c_{21} = a_{21}b_{11} + a_{22}b_{21}
  • c_{22} = a_{21}b_{12} + a_{22}b_{22}

Appendix A

Now imagine yourself looking at something that is rank 5 and something that is rank 4 and for every calculation, we select a row from the first thing and we select a column from the second thing and we calculate a dot product.

Appendix B

A rank 3 tensor is a collection of matrices, just like a matrix is a collection of vectors. The multiplication of rank 3 tensors is a pattern of matrix multiplications and each matrix multiplication is a series of the dot product calculations.

The tensor multiplication of rank 3 tensors involves multiple multiplications of rank 2 tensors which is dot product calculations. This sort of thing happens as we go higher–all tensor multiplication boils down to dot product calculations.

Appendix C

What if we did a multiplication of tensor A(ik) by tensor B(ik) instead of the usual A(ij) by B(jk)?

Summation will occur over both indices, i and k. There will no indices remaining after the summation. The result will be a single number.