Dot Product Magic

If we told you that the vector \begin{bmatrix} 1 \\ 10 \end{bmatrix} was in inches and then asked you what it would be in centimeters we know what you would tell us:

\begin{bmatrix} 2.54 \\ 25.4 \end{bmatrix}

What can we do that will change one vector into another?

\begin{bmatrix} a&b\\c&d \end{bmatrix} \begin{bmatrix} e\\f \end{bmatrix} = \begin{bmatrix} ae + bf \\ ce + df\end{bmatrix}

Now look at something else and notice that it gives the same answer:

\begin{bmatrix} a&b\\c&d \end{bmatrix} \begin{bmatrix} e\\f \end{bmatrix} = \begin{bmatrix} \begin{bmatrix} a & b \end{bmatrix} \cdot \begin{bmatrix} e \\ f \end{bmatrix} \\ \\ \begin{bmatrix} c & d \end{bmatrix} \cdot \begin{bmatrix} e \\ f \end{bmatrix} \end{bmatrix}

  • A dot product always gives a number.
  • A component in a tensor is always a number.

You might see sin( \theta ) and correctly call it a function, but I assure you, it waits and wants to be fed a \theta value like \dfrac {\pi} {2} and then it transfigures into 1.

If the above makes sense then you are asked to scrutinize the math beast below for sensibility:

\begin{bmatrix} \begin{bmatrix} a&b \end{bmatrix} \\ \\ \begin{bmatrix} c&d \end{bmatrix} \end{bmatrix}

The first person sees a matrix and the second person sees a stack of vectors.

If we change our choice of words we csn do the following:

  • We can build a rank 3 tensor from a stack of rank 1 tensors.
  • We can build a rank 3 tensor from a stack of rank 2 tensors.
  • We can build a tank 4 tensor from a stack of rank 3 tensors.

Even though these beasts have more complicated structures, the game is still built around calculating dot products.

It’s not exactly matrix algebra. Remember when you memorized the rule “across first matric, down second matrix?” Did it cross your mind that it might be fun to rebel and do it with down first matrix across second matrix?

Well, if A_{ij}B_{jk} gives the traditional matrix multiplication then A_{ji}B_{kj} will do it your way.

A_{ji}B_{kj} = C_{ki}

“Inner Product” is more general than “Dot Product”. Because of this, for the next part we are going to talk about doing Inner Product, but, like the Dot Product, it’s going to come down to matching components from two equal length lists multiplying “like components” and adding them all together.

Right now, the most general scenario we can imagine is the building of two objects, with both objects being built using vectors and one-forms, and we can do inner product calculations in either direction between the two objects.

Between our two objects we will have indices. In our example they will be {i,j,k,l,m}. Our math objects will be A and B and they will carry indices as follows:

  • A^{ij}_k
  • B^l_{jm}

For example we will say j={1,2,3}

We could pick any letter for the answer and we will use C. The indices on C are controlled by what we have on A and B:

A^{ij}_k B^l_{jm} = C^{il}_{km}

C will need to be a structure that, using up-down, left-right, in-out and nesting, will be a structure with four directions. These will be taken by {i,k,l,m}.

Each component in C will have its location identified by i,k,l,m. For our example, let’s look at the component where i=1, k=2, l=3, m=4. We already said that j runs from 1 to 3. The calculation for this component is as follows:

A(i=1, j=1,k=2)B(j=1,l=3,m=4) + A(i=1,j=2,k=2)B(j=2,l=3,m=4) + A(i=1,j=3,k=2)B(j=3,l=3,m=4)

There’s quite an eyeful in the above calculation. Notice that j changes from 1 to 3 and the other four indices are locked into the numbers of the component we wish to calculate.

Once you feel you have caught the idea of the above, compare it to the simpler example below:

A^i_j B^j_k = C^i_k

After a few seconds you’ll probably notice that in this second example we are asking two tensors to do what we see in matrix multiplication. For someone new to this, the second example might provide more confidence.

What should you take away from all this?

  • Every index found on A and B must be on C.
  • If an index isn’t involved in a summation then it takes its place somewhere in the answer.