Blogs, Linear Algebra

Gram-Schmidt Orthonormalization

Now that we have defined inner product spaces and angles between vectors, we can determine the amount a given vector is in the direction of another vector. As we worked on this, we noted that the process was significantly simplified if we worked with unit vectors. Furthermore, If we wanted to break a vector into components finding its components in different directions, it was helpful it we had started with orthogonal vectors. As such, we will want to be able to find an orthonormal basis of a vector space, if we have generating set of vectors. We used the Gram-Schmidt Orthonormalition process to do this.

Example

Let \(S=span(\left\{(1,1,2),(1,1,1),(0,1,1)\right\})\) be a subspace of \(\mathbb{R}^{3}\) with the inner product defined as the dot product. Find an orthonormal basis for \(S\) using the Gram-Schimdt Orthonormalization process.

First vector

As we work through the Gram-Schmidt Orthonormalization process for this vector space, we will focus on one step at a time. The first step is to find a unit vector in the direction of the first given vector. We will, therefore, let \(\mathbf{v}_{1}=(1,1,2)\). In order to find the unit vector in this direction, we get
\begin{align*}
\mathbf{w}_{1}=\frac{(1,1,2)}{||(1,1,2)||}
\end{align*}
where \(||\mathbf{v}||\) is the norm of \(\mathbf{v}\).

In order to find the norm of \(\mathbf{v}_{1}\), we note that by definition
\begin{align*}
||(1,1,2)||&=\sqrt{(1,1,2)\cdot (1,1,2)} \\
&=\sqrt{1^{2}+1^{2}+2^{2}} \\
&=\sqrt{6}.
\end{align*}
We now find that the first vector in the orthonormal basis is
\begin{align*}
\mathbf{w}_{1}&=\frac{(1,1,2)}{||(1,1,2)||} \\
&=\frac{(1,1,2)}{\sqrt{6}} \\
&=\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right).
\end{align*}

Second Vector

As we work on finding the second vector for our orthonormal basis, we will need to perform two different tasks. First, we will need to make sure that the vector we find is orthogonal to the first vector in our basis. Secondly, we need to ensure that we are left with a unit vector.

Orthogonal

Here, we will start with the second vector in \(S\), \(\mathbf{v}_{2}=(1,1,1)\). In order to find a vector which is orthogonal to \(\mathbf{w}_{1}\), we will take \(\mathbf{v}_{2}\) and subtract off the portion of \(\mathbf{v}_{2}\) in the direction of \(\mathbf{w}_{1}\). This will then leave the portion of \(\mathbf{v}_{2}\) that is orthogonal to \(\mathbf{w}_{1}\). That is, we will define
\begin{align*}
\mathbf{u}_{1}=\mathbf{v}_{2}-\text{proj}_{\mathbf{w}_{1}}(\mathbf{v}_{2}).
\end{align*}

Recall that, in general,
\begin{align*}
\text{proj}_{\mathbf{w}}(\mathbf{v})=\frac{\langle \mathbf{v},\mathbf{w} \rangle}{\langle \mathbf{w}, \mathbf{w} \rangle}\mathbf{w}.
\end{align*}
However, since \(\mathbf{w}_{1}\) is a unit vector, we know that \(\langle \mathbf{w}, \mathbf{w} \rangle=1\), so we can simplify this to,
\begin{align*}
\text{proj}_{\mathbf{w}_{1}}(\mathbf{v}_{2})=\langle \mathbf{v}_{2},\mathbf{w}_{1} \rangle \mathbf{w}.
\end{align*}

We can now find that
\begin{align*}
\text{proj}_{\mathbf{w}_{1}}(\mathbf{v}_{2})&=\langle \mathbf{v}_{2},\mathbf{w}_{1} \rangle \mathbf{w}\\
&=\langle (1,1,1), \left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right)\rangle \left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\left((1,1,1) \cdot \left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right)\right)\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\left(1*\frac{1}{\sqrt{6}}+1*\frac{1}{\sqrt{6}}+\frac{2}{\sqrt{6}}\right)\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\frac{4}{\sqrt{6}}\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\left(\frac{4}{6},\frac{4}{6},\frac{8}{6}\right) \\
&=\left(\frac{2}{3},\frac{2}{3},\frac{4}{3}\right).
\end{align*}

Now that we know the portion of \(\mathbf{v}_{2}\) in the direction of \(\mathbf{w}_{1}\) we can find the portion of \(\mathbf{v}_{2}\) orthogonal to \(\mathbf{w}_{1}\). Here, we get that
\begin{align*}
u_{2}&=(1,1,1)-\left(\frac{2}{3},\frac{2}{3},\frac{4}{3}\right)\\
&=\left(\frac{1}{3},\frac{1}{3},-\frac{1}{3}\right).
\end{align*}

Unit Vector

We have now found the portion of \(\mathbf{v}_{2}\) orthogonal to \(\mathbf{w}_{1}\), however, the norm of the resulting vector is not \(1\). Therefore, we will find a unit vector in this direction. This will be our \(\mathbf{w}_{2}\). Here, we get
\begin{align*}
\mathbf{w}_{2}=\frac{\mathbf{u_{2}}}{||\mathbf{u}_{2}||}.
\end{align*}

As we work finding this, we will find the norm first. Here, we get
\begin{align*}
||\mathbf{u}_{2}||&=||\left(\frac{1}{3},\frac{1}{3},-\frac{1}{3}\right)|| \\
&=\sqrt{\left(\frac{1}{3},\frac{1}{3},-\frac{1}{3}\right) \cdot \left(\frac{1}{3},\frac{1}{3},-\frac{1}{3}\right)\rangle} \\
&=\sqrt{\left(\frac{1}{3}\right)^{2}+\left(\frac{1}{3}\right)^{2}+\left(\frac{-1}{3}\right)^{2}} \\
&=\sqrt{\frac{1}{3}} \\
&=\frac{1}{\sqrt{3}}.
\end{align*}

We, therefore, get that
\begin{align*}
\mathbf{w}_{2}&=\frac{\mathbf{u_{2}}}{||\mathbf{u}_{2}||} \\
&=\frac{\left(\frac{1}{3},\frac{1}{3},-\frac{1}{3}\right)}{\frac{1}{\sqrt{3}}} \\
&=\left(\frac{\sqrt{3}}{3},\frac{\sqrt{3}}{3},-\frac{\sqrt{3}}{3}\right).
\end{align*}

This is, therefore, the second vector in our orthonormal basis of \(S\).

Third Vector

Now that we have the first two vectors we will need to find the third vector in our basis. In order to do this, we will first need to find a vector which is orthogonal to both \(\mathbf{w}_{1}\) and \(\mathbf{w}_{2}\). Then, we will need to find a vector in the same direction of this vector.

Orthogonal

In order to find a vector that is orthogonal both \(\mathbf{w}_{1}\) and \(\mathbf{w}_{2}\), we will start with \(\mathbf{v}_{3}\). We will then subtract off the portions in both the \(\mathbf{w}_{1}\) and \(\mathbf{w}_{2}\) directions. We, therefore, define
\begin{align*}
\mathbf{u}_{3}=\mathbf{v}_{3}-\text{proj}_{\mathbf{w}_{1}}(\mathbf{v}_{3})-\text{proj}_{\mathbf{w}_{2}}(\mathbf{v}_{3}).
\end{align*}

In order to find \(\mathbf{u}_{3}\), we will begin by finding
\begin{align*}
\text{proj}_{\mathbf{w}_{1}}(\mathbf{v}_{3})&=\langle \mathbf{v}_{3}, \mathbf{w}_{1} \rangle \mathbf{w}_{1} \\
&=\left((0,1,1) \cdot \left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right)\right)\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\left(\frac{1}{\sqrt{6}}+\frac{2}{\sqrt{6}}\right)\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\frac{3}{\sqrt{6}}\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right) \\
&=\left(\frac{1}{2},\frac{1}{2},1\right).
\end{align*}

Next, we find that
\begin{align*}
\text{proj}_{\mathbf{w}_{2}}(\mathbf{v}_{3})&=\langle \mathbf{v}_{3}, \mathbf{w}_{2} \rangle \mathbf{w}_{2} \\
&=\left((0,1,1) \cdot \left(\frac{\sqrt{3}}{3},\frac{\sqrt{3}}{3},-\frac{\sqrt{3}}{3}\right)\right)\left(\frac{\sqrt{3}}{3},\frac{\sqrt{3}}{3},-\frac{\sqrt{3}}{3}\right) \\
&=\left( \frac{\sqrt{3}}{3}-\frac{\sqrt{3}}{3} \right) \left(\frac{\sqrt{3}}{3},\frac{\sqrt{3}}{3},-\frac{\sqrt{3}}{3}\right) \\
&=(0,0,0).
\end{align*}

Note that since we got \(\mathbf{v}_{3} \cdot \mathbf{w}_{2}=0\), this tells us that these two were already orthogonal. However, we do still need to subtract off the portion in the \(\mathbf{w}_{1}\) direction. We, therefore, get that
\begin{align*}
\mathbf{u}_{3}&=\mathbf{v}_{3}-\text{proj}_{\mathbf{w}_{1}}(\mathbf{v}_{3})-\text{proj}_{\mathbf{w}_{2}}(\mathbf{v}_{3}) \\
&=(0,1,1)-\left(\frac{1}{2},\frac{1}{2},1\right) \\
&=\left(-\frac{1}{2},\frac{1}{2},0\right).
\end{align*}

Unit Vector

Now that we’ve found a vector orthogonal to both \(\mathbf{w}_{1}\) and \(\mathbf{w}_{2}\), we need to find a vector in the same direction with a norm of \(1\). We will, therefore, let
\begin{align*}
\mathbf{w}_{3}=\frac{\mathbf{u}_{3}}{||\mathbf{u}_{3}||}.
\end{align*}

In order to find this, we then just need to find
\begin{align*}
||\mathbf{u}_{3}||&=\sqrt{\mathbf{u}_{3} \cdot \mathbf{u}_{3}} \\
&=\sqrt{\left(-\frac{1}{2},\frac{1}{2},0\right) \cdot \left(-\frac{1}{2},\frac{1}{2},0\right)} \\
&=\sqrt{\frac{1}{4}+\frac{1}{4}} \\
&=\sqrt{\frac{1}{2}} \\
&=\frac{1}{\sqrt{2}}.
\end{align*}

We are now ready to find that
\begin{align*}
\mathbf{w}_{3}&=\frac{\mathbf{u}_{3}}{||\mathbf{u}_{3}||} \\
&=\frac{\left(-\frac{1}{2},\frac{1}{2},0\right)}{\frac{1}{\sqrt{2}}} \\
&=\left(-\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2},0\right).
\end{align*}

Conclusion

We have now found three orthogonal unit vectors whose span is \(S\). Therefore,
\begin{align*}
B&=\left\{\mathbf{w}_{1},\mathbf{w}_{2},\mathbf{w}_{3}\right\} \\
&=\left\{\left(\frac{1}{\sqrt{6}},\frac{1}{\sqrt{6}},\frac{2}{\sqrt{6}}\right),\left(\frac{\sqrt{3}}{3},\frac{\sqrt{3}}{3},-\frac{\sqrt{3}}{3}\right),\left(-\frac{\sqrt{2}}{2},\frac{\sqrt{2}}{2},0\right)\right\}
\end{align*}
is an orthonormal basis of \(S\).

As always, I hoped this helped you as you studied and that you enjoyed the process. If you did, make sure to share this with anyone else that could use that help and subscribe to our YouTube channel.

We'd love to hear your thoughts!

This site uses Akismet to reduce spam. Learn how your comment data is processed.