definition of Gram-Schmidt orthonormalization of countable subset of vectors space with inner product
Topics
About: vectors space
The table of contents of this article
Starting Context
- The reader knows a definition of inner product on real or complex vectors space.
Target Context
- The reader will have a definition of Gram-Schmidt orthonormalization of countable subset of vectors space with inner product.
Orientation
There is a list of definitions discussed so far in this site.
There is a list of propositions discussed so far in this site.
Main Body
1: Structured Description
Here is the rules of Structured Description.
Entities:
\( F\): \(\in \{\mathbb{R}, \mathbb{C}\}\), with the canonical field structure
\( V\): \(\in \{\text{ the } F \text{ vectors spaces }\}\) with any inner product, \(\langle \bullet, \bullet \rangle: V \times V \to F\)
\( S\): \(\in \{\text{ the countable subsets of } V\}\), \(= \{s_1, s_2, ...\}\)
\(*\widetilde{S}\): \(\in \{\text{ the countable subsets of } V\}\), \(= \{b_1, b_2, ...\}\), as specified below
//
Conditions:
\(b_n\) is defined iteratively:
\(\{\}\) has been already determined from \(\{\}\)
Suppose that \(\{b_1, ..., b_{n - 1}\}\) has been already determined from \(\{s_1, ..., s_{n'}\}\)
When there is the smallest \(\{s_1, ..., s_{n''}\}\) such that \(\{b_1, ..., b_{n - 1}, s_{n''}\}\) is linearly independent, \(b_n := (s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert\), and \(\{b_1, ..., b_n\}\) has been determined from \(\{s_1, ..., s_{n''}\}\)
Otherwise, the iteration is finished, and \(\widetilde{S} := \{b_1, ..., b_{n - 1}\}\)
//
"countable" does not mean that it is infinite: any finite set is countable.
2: Note
Often, \(S\) is supposed to be linearly independent, but this definition does not require \(S\) to be linearly independent.
When \(S\) is linearly independent, each \(\{b_1, ..., b_n\}\) is determined from \(\{s_1, ..., s_n\}\), because \(\{b_1, ..., b_{n - 1}, s_n\}\) is linearly independent, because for \(r^1 b_1 + ... + r^{n - 1} b_{n - 1} + r^n s_n = 0\), as \(r^1 b_1 + ... + r^{n - 1} b_{n - 1}\) is a linear combination of \(\{s_1, ..., s_{n - 1}\}\), \(r^n = 0\) because \(\{s_1, ..., s_n\}\) is linearly independent, so, \(r^1 b_1 + ... + r^{n - 1} b_{n - 1} = 0\), which implies that \(r^j\) s are \(0\), because \(\{b_1, ..., b_{n - 1}\}\) is linearly independent because it is orthonormal, by the proposition that for any vectors space with any inner product, any orthonormal subset is linearly independent.
So, when \(S\) is linearly independent, the cardinality of \(\widetilde{S}\) equals the cardinality of \(S\).
Let us see that \(\widetilde{S}\) is orthonormal, iteratively.
For \(\{b_1\}\), \(\langle b_1, b_1 \rangle = \langle s_{n''} / \Vert s_{n''} \Vert, s_{n''} / \Vert s_{n''} \Vert \rangle = 1 / \Vert s_{n''} \Vert^2 \langle s_{n''}, s_{n''} \rangle = 1 / \Vert s_{n''} \Vert^2 \Vert s_{n''} \Vert^2 = 1\).
So, \(\{b_1\}\) is orthonormal.
For \(\{b_1, b_2\}\), \(\langle b_1, b_2 \rangle = \langle b_1, (s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert \rangle = 1 / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert \langle b_1, (s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j) \rangle = 1 / \Vert s_{n''} - \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_1, s_{n''} \rangle - \overline{\langle s_{n''}, b_1 \rangle} \langle b_1, b_1\rangle) = 1 / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_1, s_{n''} \rangle - \overline{\langle s_{n''}, b_1 \rangle} 1) = 1 / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_1, s_{n''} \rangle - \overline{\langle s_{n''}, b_1 \rangle}) = 1 / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_1, s_{n''} \rangle - \langle b_1, s_{n''} \rangle) = 0\).
\(\langle b_2, b_2 \rangle = \langle (s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert, (s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert \rangle = 1 / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert^2 \langle s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j, s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \rangle = 1 / \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert^2 \Vert s_{n''} - \sum^1_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert^2 = 1\).
So, \(\{b_1, b_2\}\) is orthonormal.
Let us suppose that \(\{b_1, ..., b_{n - 1}\}\) is orthonormal.
For \(\{b_1, ..., b_n\}\), for each \(1 \le l \le n - 1\), \(\langle b_l, b_n \rangle = \langle b_l, (s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert \rangle = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert \langle b_l, s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \rangle = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_l, s_{n''} \rangle - \sum^{n - 1}_{j = 1} \overline{\langle s_{n''}, b_j \rangle} \langle b_l, b_j \rangle) = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_l, s_{n''} \rangle - \sum^{n - 1}_{j = 1} \overline{\langle s_{n''}, b_j \rangle} \delta_{l, j}) = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_l, s_{n''} \rangle - \overline{\langle s_{n''}, b_l \rangle}) = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert (\langle b_l, s_{n''} \rangle - \langle b_l, s_{n''} \rangle) = 0\).
\(\langle b_n, b_n \rangle = \langle (s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert, (s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert \rangle = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert^2 \langle s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j, s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \rangle = 1 / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert^2 \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert^2 = 1\).
So, \(\{b_1, ..., b_n\}\) is orthonormal.
Then, let \(b_l, b_m \in \widetilde{S}\) be any such that \(l \neq m\).
Let \(l \lt m\) without loss of generality.
So, \(b_l, b_m \in \{b_1, ..., b_m\}\).
As \(\{b_1, ..., b_m\}\) is orthonormal, \(\langle b_l, b_m \rangle = 0\).
\(\langle b_l, b_l \rangle = 1\).
Let us see that while each \(\{b_1, ..., b_n\}\) has been determined from \(\{s_1, ..., s_{n''}\}\), each \(s_j \in \{s_1, ..., s_{n''}\}\) is a linear combination of \(\{b_1, ..., b_n\}\).
Let us see that inductively.
While \(\{b_1\}\) has been determined from \(\{s_1, ..., s_{n''}\}\), if \(1 \lt n''\), that means that \(s_1 = 0\), and inevitably, \(n'' = 2\) and \(s_{n''} \neq 0\) (as any set does not allow any duplication), and \(b_1 = (s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j) / \Vert s_{n''} - \sum^{n - 1}_{j = 1} \langle s_{n''}, b_j \rangle b_j \Vert = s_{n''} / \Vert s_{n''} \Vert\), so, \(s_{n''} = \Vert s_{n''} \Vert b_1\) while also \(s_1 = 0\) is a linear combination of \(\{b_1\}\); if \(n'' = 1\), it is just that \(s_{n''} = \Vert s_{n''} \Vert b_1\).
Let us suppose that \(\{b_1, ..., b_{n - 1}\}\) has been determined from \(\{s_1, ..., s_{n''}\}\) and each \(s_j \in \{s_1, ..., s_{n''}\}\) is a linear combination of \(\{b_1, ..., b_{n - 1}\}\).
Let us see that \(\{b_1, ..., b_n\}\) has been determined from \(\{s_1, ..., s_{n'''}\}\) and each \(s_j \in \{s_1, ..., s_{n'''}\}\) is a linear combination of \(\{b_1, ..., b_n\}\).
Each of \(s_1, ... s_{n''}\) is a linear combination of \(\{b_1, ..., b_{n - 1}\}\), so, is a linear combination of \(\{b_1, ..., b_n\}\).
Each of \(s_{n'' + 1}, ..., s_{n''' - 1}\) (if any) is a linear combination of \(\{b_1, ..., b_{n - 1}\}\), so, is a linear combination of \(\{b_1, ..., b_n\}\).
As \(b_n := (s_{n'''} - \sum^{n - 1}_{j = 1} \langle s_{n'''}, b_j \rangle b_j) / \Vert s_{n'''} - \sum^{n - 1}_{j = 1} \langle s_{n'''}, b_j \rangle b_j \Vert\), \(s_{n'''} = \Vert s_{n'''} - \sum^{n - 1}_{j = 1} \langle s_{n'''}, b_j \rangle b_j \Vert b_n + \sum^{n - 1}_{j = 1} \langle s_{n'''}, b_j \rangle b_j\), a linear combination of \(\{b_1, ..., b_n\}\).
So, each \(s_j \in \{s_1, ..., s_{n'''}\}\) is a linear combination of \(\{b_1, ..., b_n\}\).
So, by the induction principle, the claim is confirmed.
