description/proof of that for Euclidean vectors space with Euclidean inner product, nonzero vector, and real positive-definite symmetric matrix, vector that maximizes its inner product with nonzero vector with condition that its bilinear form by matrix is 1 is this
Topics
About: vectors space
The table of contents of this article
Starting Context
- The reader knows a definition of Euclidean inner product on Euclidean vectors space.
- The reader knows a definition of %ring name% matrices space.
- The reader admits the proposition that any positive-definite Hermitian matrix can be transformed to the identity by a unitary matrix multiplied by a positive diagonal matrix from right.
- The reader admits the proposition that for any commutative ring, the inverse of the transpose of any invertible matrix is the transpose of the inverse of the matrix.
- The reader admits the proposition that for any commutative ring, the transpose of the product of any matrices is the product of the transposes of the constituents in the reverse order.
Target Context
- The reader will have a description and a proof of the proposition that for any Euclidean vectors space with the Euclidean inner product, any nonzero vector, and any real positive-definite symmetric matrix, the vector that maximizes its inner product with the nonzero vector with the condition that its bilinear form by the matrix is 1 is this.
Orientation
There is a list of definitions discussed so far in this site.
There is a list of propositions discussed so far in this site.
Main Body
1: Structured Description
Here is the rules of Structured Description.
Entities:
\(\mathbb{R}^n\): \(= \text{ the Euclidean vectors space }\) with the Euclidean inner product, \(\langle \bullet, \bullet \rangle\)
\(v_0\): \(\in \mathbb{R}^n\) such that \(v \neq 0\)
\(M\): \(\in \{\text{ the } n \times n \text{ real positive-definite symmetric matrices }\}\)
\(f\): \(: \mathbb{R}^n \to \mathbb{R}, v \mapsto \langle v, v_0 \rangle\)
\(g\): \(: \mathbb{R}^n \to \mathbb{R}, v \mapsto \langle M v, v \rangle\)
//
Statements:
\(v \in \mathbb{R}^n\) that maximizes \(f (v)\) with the condition that \(g (v) = 1\) is \(v = 1 / \sqrt{{v_0}^t M^{-1} v_0} M^{-1} v_0\)
//
2: Proof
Whole Strategy: Step 1: see that \(f (v) = {v_0}^t v\) and \(g (v) = v^t M v\); Step 2: find an orthogonal matrix multiplied by a positive diagonal matrix from the right, \(T^{-1}\), such that \({T^{-1}}^t M T^{-1} = I\); Step 3: see that \(f (v) = {v_0}^t T^{-1} T v\) and \(g (v) = v^t T^t {T^{-1}}^t M T^{-1} T v = (T v)^t I T v\); Step 4: take \(T v = c {T^{-1}}^t v_0\) and compute \(c\).
Step 1:
By the Euclidean inner product, \(f (v) = \langle v, v_0 \rangle = {v_0}^t v\) and \(g (v) = \langle M v, v \rangle = v^t M v\).
Step 2:
If \(M = I\), obviously, \(v\) will be a scalar multiple of \(v_0\), but that is not necessarily the case.
So, our strategy is to find an orthogonal matrix multiplied by a positive diagonal matrix from the right, \(T^{-1} = U D\), such that \({T^{-1}}^t M T^{-1} = I\), which is possible by the proposition that any positive-definite Hermitian matrix can be transformed to the identity by a unitary matrix multiplied by a positive diagonal matrix from right: we take it to be \(T^{-1}\) instead of \(T\) just for convenience of expressions, which is valid because \(U D\) is invertible.
Step 3:
\(f (v) = {v_0}^t T^{-1} T v\). \({v_0}^t T^{-1} \neq 0^t\), because otherwise, \({v_0}^t = {v_0}^t T^{-1} T = 0^t T = 0^t\), a contradiction.
\(g (v) = v^t {{T^{-1}}^t}^{-1} {T^{-1}}^t M T^{-1} T v = v^t T^t {T^{-1}}^t M T^{-1} T v\), by the proposition that for any commutative ring, the inverse of the transpose of any invertible matrix is the transpose of the inverse of the matrix because \({{T^{-1}}^t}^{-1} = {{T^{-1}}^{-1}}^t = T^t\), \(= v^t T^t I T v = (T v)^t I T v\), by the proposition that for any commutative ring, the transpose of the product of any matrices is the product of the transposes of the constituents in the reverse order.
Step 4:
So, the problem has become of finding the \(T v\) maximizing \({v_0}^t T^{-1} T v\) with the condition that \((T v)^t I T v = 1\).
Obviously, \(T v = c ({v_0}^t T^{-1})^t = c {T^{-1}}^t v_0\) for a \(c \in \mathbb{R}\).
So, \(v = T^{-1} T v = T^{-1} c {T^{-1}}^t v_0 = c T^{-1} {T^{-1}}^t v_0\).
From \({T^{-1}}^t M T^{-1} = I\), \(M T^{-1} = {{T^{-1}}^t}^{-1} {T^{-1}}^t M T^{-1} = {{T^{-1}}^t}^{-1} I = {{T^{-1}}^t}^{-1}\), \(T^{-1} = M^{-1} M T^{-1} = M^{-1} {{T^{-1}}^t}^{-1}\), and \(T^{-1} {T^{-1}}^t = M^{-1} {{T^{-1}}^t}^{-1} {T^{-1}}^t = M^{-1} I = M^{-1}\).
So, \(v = c M^{-1} v_0\).
\(v^t M v = (c M^{-1} v_0)^t M c M^{-1} v_0 = c {v_0}^t {M^{-1}}^t M c M^{-1} v_0 = c^2 {v_0}^t {M^{-1}}^t v_0 = 1\), so, \(c = 1 / \sqrt{{v_0}^t {M^{-1}}^t v_0}\): it needs to be positive, because \({v_0}^t v = {v_0}^t c M^{-1} v_0 = c {v_0}^t M^{-1} v_0 = c {v_0}^t {M^t}^{-1} M^t M^{-1} v_0 = c {v_0}^t {M^{-1}}^t M M^{-1} v_0 = c (M^{-1} v_0)^t M M^{-1} v_0\) where \(0 \le (M^{-1} v_0)^t M M^{-1} v_0\) because \(M\) is positive-definite.
As \(M\) is symmetric, \({M^{-1}}^t = {M^t}^{-1} = M^{-1}\).
So, \(v = 1 / \sqrt{{v_0}^t M^{-1} v_0} M^{-1} v_0\).
