13 First Steps in Linear Algebra
In a two-dimensional (or higher) space, movement cannot be expressed by a single number. Imagine yelling at someone “move by 3”. You would need to be a bit more specific and say something like “move 3 steps forward and 2 steps to the right.”
This is where vectors come in. Vectors are ordered lists of numbers, representing coordinates in an \(n\)-dimensional space. These numbers are also called components. For example, the vector \(\mathbf{u}\) can be defined as:
\[ \mathbf{u} = (2, 1) \]
The components of \(\mathbf{u}\) are \(2\) and \(1\). In physics, vectors are used to represent quantities with a magnitude and a direction.
The direction of a vector is where the arrow is pointing to. The magnitude is the length of that arrow. Looking at the vector \(\mathbf{u} = (2, 1)\):
The direction of this vector is roughly North-East, closer to East. A more mathematical description of the direction would be the angle between the \(x\)-axis and the vector (noted \(\theta\)).
The magnitude of the vector is the length of the arrow. This can be calculated with the Pythagorean theorem:
\[ \|\mathbf{u}\| = \sqrt{2^2 + 1^2} = \sqrt{5} \approx 2.24 \]
The examples below will be shown in two dimensions for ease of plotting. The operations studied generalise to any number of dimensions.
Now imagine we also have a vector \(\mathbf{v}\):
\[ \mathbf{u} = (2, 1) \quad \text{and} \quad \mathbf{v} = (-1, 3) \]
13.1 Vector Addition
What would happen if we added these two vectors?
Starting at \((0, 0)\), a point called the origin, you would first move along vector \(\mathbf{u}\):
Then, adding vector \(\mathbf{v}\), you could move from the tip of vector \(\mathbf{u}\) in the direction and with the magnitude of \(\mathbf{v}\):
On the chart above, moving along \(\mathbf{u}\), then along vector \(\mathbf{v}\) brings us to a given point. Going to this point directly from the origin would be the vector defined by \(\mathbf{u} + \mathbf{v}\).
To calculate the vector \(\mathbf{u} + \mathbf{v}\) without having to draw it, you can add the components of the two vectors as follows:
\[ \mathbf{u} + \mathbf{v} = (2, 1) + (-1, 3) = (2 + (-1), \; 1 + 3) = (1, 4) \]
Something you may notice by computing \(\mathbf{v} + \mathbf{u}\) instead of \(\mathbf{u} + \mathbf{v}\) is that you get the same result. This can also be seen geometrically:
This has the fancy name of “commutativity”. You do not need to remember this word. Remembering that \(\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}\) will be enough.
Exercise 13.1 Compute \(\mathbf{a} + \mathbf{b}\) where \(\mathbf{a} = (3, -2)\) and \(\mathbf{b} = (1, 5)\).
13.2 Scalar Multiplication
Can we multiply a vector by a single number like \(2\)? Or \(-1\)? The answer is yes. Think of vectors as a direction and a magnitude.
What should multiplying the vector \(\mathbf{u} = (2, 1)\) by \(2\) do?
To answer this question, let’s go back to the relationship between addition and multiplication. \(2 \times 3\) is equivalent to \(3 + 3\), or \(2 + 2 + 2\).
The same goes for vectors:
\[ 2 \times \mathbf{u} = \mathbf{u} + \mathbf{u} = (2, 1) + (2, 1) = (2 + 2, \; 1 + 1) = (4, 2) \]
To multiply a vector by a number without having to use addition, simply multiply all the components of the vector by this number as follows:
\[ 2 \times \mathbf{u} = 2 \times (2, 1) = (2 \times 2, \; 2 \times 1) = (4, 2) \]
This concept is known as scalar multiplication. In linear algebra, numbers like \(1.2\) or \(-2.54\) are referred to as scalars, i.e. not vectors. On the number line, a scalar is all you need to describe movement.
Exercise 13.2 Compute the following:
- \(3 \times (1, -2)\)
- \(-0.5 \times (4, 6)\)
- \((2, 3) + 2 \times (1, -1)\)
13.3 Vector Subtraction
What about subtraction? Subtraction and addition work in the same way. Computing \(\mathbf{u} - \mathbf{v}\) is equivalent to computing \(\mathbf{u} + (-1) \times \mathbf{v}\).
Intuitively, \((-1) \times \mathbf{v}\), also noted \(-\mathbf{v}\), goes in the opposite direction as \(\mathbf{v}\), keeping the same magnitude.
To find the components of \(-\mathbf{v}\), simply multiply all the components of \(\mathbf{v}\) by \(-1\). Here, we would get:
\[ -\mathbf{v} = (-1 \times (-1), \; -1 \times 3) = (1, -3) \]
Computing \(\mathbf{u} - \mathbf{v}\) as \(\mathbf{u} + (-\mathbf{v})\), we get:
\[ \mathbf{u} - \mathbf{v} = (2, 1) + (1, -3) = (3, -2) \]
13.4 Final Thoughts
We now have the building blocks needed to work with vectors:
- Vector addition: add the components: \((a_1, a_2) + (b_1, b_2) = (a_1 + b_1, \; a_2 + b_2)\)
- Vector subtraction: subtract the components: \((a_1, a_2) - (b_1, b_2) = (a_1 - b_1, \; a_2 - b_2)\)
- Scalar multiplication: multiply each component by the scalar: \(c \times (a_1, a_2) = (c \times a_1, \; c \times a_2)\)
In the next chapter, we will bring vectors and derivatives together with the concept of gradients.
13.5 Solutions
Solution 13.1. Exercise 13.1
\[ \mathbf{a} + \mathbf{b} = (3, -2) + (1, 5) = (3 + 1, \; -2 + 5) = (4, 3) \]
Solution 13.2. Exercise 13.2
\(3 \times (1, -2) = (3 \times 1, \; 3 \times (-2)) = (3, -6)\)
\(-0.5 \times (4, 6) = (-0.5 \times 4, \; -0.5 \times 6) = (-2, -3)\)
First compute \(2 \times (1, -1) = (2, -2)\), then add:
\[ (2, 3) + (2, -2) = (4, 1) \]