Tensor Multiplication

We will consider three examples in which all chosen tensors are rank two and can be represented by $\begin{bmatrix} a & b \\ c & d \end{bmatrix}$ where a,b,c,d are values (numbers).

We will tell three different stories with the multiplication of two tensors:

$\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \begin{bmatrix}b_{11} & b_{12} \\ b_{21} & b_{22}\end{bmatrix}$

We will designate tensors with uppercase letters.

First Example

$A_{ij} B_{jk} = C_{ik}$

$C_{ik}$ is simply the result of what is called Matrix Multiplication.

For the Permutations of i,j,k all permutations with the same value for j will end up in a summation and the values of i and k will provide the location in C for that summation.

1,1,1 + 1,2,1 ends up at 1,1

1,1,2 + 1,2,2 ends up at 1,

Second Example

$A_{ij} B_{kl} = C_{ijkl}$

$C_{ijkl}$ is a beast which, if we unleashed it to a physical realm, would require us to represent it in some manner that conveyed four sets of directions. There won’t be any summation. Every permutation of i,j,k,l will result in a calculation that is a product of two factors, $a_{ij} b+{kl}$ . The four values of i,j,k,l will determine the location in C.

Third Example

$A_{ij} B_{ij} = c$

Every permutation will contribute to a single calculation (a summation). The math object c is a value.

Appendix A

We believe the best way to grasp the idea is to put some time into thinking about the computer program that has a for-loop for every index and the for-loops of summed indices are put in a part of the program called the “Summation Arena.” Every index that isn’t involved in a summation has a for-loop that is outside of the summation arena and it helps to determine the location of the result of the calculation.

How we build $C_{ijk}$ is an arbitrary call. In one extreme we always use left-to-right, using nesting to accommodate the need for three structural directions:

{[…][…][…]} {[…][…][…]} {[…][…][…]}

We could go to another extreme and, using nesting, only use the up-and-down direction.

We could use a mixing of both. We haven’t mentioned the possibility of in-and-out since in our opinion, we would want the choice for up-and-down or left-and-right to correspond to contravariant vs. covariant, but we’ve seen drawing on the internet using all three directions.

Finally, there is the option of not building a structure, but simply letting component $c_{ijkl}$ reside in a computer location:

$c[i][j][k][l]$

Appendix B

It might be helpful to introduce the ideas using two matrices with simple numbers;

$\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 2 & 3 \\ 4 & 5 \end{bmatrix}$

For $a_{ij}b_{kl}$ there is no summation and we will end up with 16 components, one for each possible combination of a component in a with a component in b:

(1)(2)
(1)(3)
(1)(4)
(1)(5)
(2)(2)
(2)(3)
(2)(4)
(2)(5)
(3)(2)
(3)(3)
(3)(4)
(3)(5)
(4)(2)
(4)(3)
(4)(4)
(4)(5)

$\begin{bmatrix} 1 & 2 \\ 3 & 4 \end{bmatrix} \begin{bmatrix} 2 & 3 \\ 4 & 5 \end{bmatrix}$

For $a_{ij}b_{ij}$ the math delivers a summation of four products:

(1)(2) + (2)(3) + (3)(4) + (4)(5) = 40

Notice that we get a product for each pair of corresponding components (we pair up $a_{12}$ with $b_{12}$ ).

Appendix C

Caution: we are calling things like left-to-right and up-and-down “directions” rather than “dimensions”. The structure of a vector use one direction but a vector with nine values in it is representing something that is nine-dimensional. Quite a few authors will talk about the dimensions or dimensionality of the structure.

Share this: