Linear Algebra and Its Applications, 5th Edition
Authors: David C. Lay, Steven R. Lay, Judi J. McDonald
ISBN-13: 978-0321982384
We have solutions for your book!
See our solution for Question 7E from Chapter 6.5 from Lay's Linear Algebra and Its Applications, 5th Edition.
Problem 7E
Chapter:
Problem:
Compute the least-squares error associated with the least squares solution found in Exercise 3. Reference: In Exercises 1–4, find a least-squares solution of Ax = b by (a) constructing the normal equations for and (b) solving for
Step-by-Step Solution
Given Information
We are given with a matrix and a vector \[ A = \left[ \begin{array} { c c } { 1 } & { - 2 } \\ { - 1 } & { 2 } \\ { 0 } & { 3 } \\ { 2 } & { 5 } \end{array} \right] , \mathbf { b } = \left[ \begin{array} { c } { 3 } \\ { 1 } \\ { - 4 } \\ { 2 } \end{array} \right] \] We have to find the least square solution of $Ax=b$.
Step-1: The Normal equations
Compute ${A^T}A$ \[ \begin{aligned} A ^ { T } A & = \left[ \begin{array} { c c c c } { 1 } & { - 1 } & { 0 } & { 2 } \\ { - 2 } & { 2 } & { 3 } & { 5 } \end{array} \right] \left[ \begin{array} { c c } { 1 } & { - 2 } \\ { - 1 } & { 2 } \\ { 0 } & { 3 } \\ { 2 } & { 5 } \end{array} \right] \\ & = \left[ \begin{array} { c c } { 6 } & { 6 } \\ { 6 } & { 42 } \end{array} \right] \end{aligned} \] Compute ${A^T}b$ \[ \begin{aligned} A ^ { T } \mathbf { b } & = \left[ \begin{array} { c c c c } { 1 } & { - 1 } & { 0 } & { 2 } \\ { - 2 } & { 2 } & { 3 } & { 5 } \end{array} \right] \left[ \begin{array} { c } { 3 } \\ { 1 } \\ { - 4 } \\ { 2 } \end{array} \right] \\ & = \left[ \begin{array} { c } { 6 } \\ { - 6 } \end{array} \right] \end{aligned} \] So the normal equation is:
\[ \left[ \begin{array} { c c } { 6 } & { 6 } \\ { 6 } & { 42 } \end{array} \right] \left[ \begin{array} { l } { x _ { 1 } } \\ { x _ { 2 } } \end{array} \right] = \left[ \begin{array} { c } { 6 } \\ { - 6 } \end{array} \right] \]
Step-2: The solution
Compute inverse of ${{A^T}A}$ \[ \begin{aligned} \left( A ^ { T } A \right) ^ { - 1 } & = \left[ \begin{array} { c c } { 6 } & { 6 } \\ { 6 } & { 42 } \end{array} \right] ^ { - 1 } \\ & = \dfrac { 1 } { 252 - 36 } \left[ \begin{array} { c c } { 42 } & { - 6 } \\ { - 6 } & { 6 } \end{array} \right] \\ & = \dfrac { 6 } { 216 } \left[ \begin{array} { c c } { 7 } & { - 1 } \\ { - 1 } & { 1 } \end{array} \right] \\ & = \dfrac { 1 } { 36 } \left[ \begin{array} { c c } { 7 } & { - 1 } \\ { - 1 } & { 1 } \end{array} \right] \end{aligned} \] Solve for x \[ \begin{aligned} \hat { \mathbf { x } } & = \dfrac { 1 } { 36 } \left[ \begin{array} { c c } { 7 } & { - 1 } \\ { - 1 } & { 1 } \end{array} \right] \left[ \begin{array} { c } { 6 } \\ { - 6 } \end{array} \right] \\ & = \dfrac { 1 } { 36 } \left[ \begin{array} { c } { 48 } \\ { - 12 } \end{array} \right] \\ & = \left[ \begin{array} { c } { 4 / 3 } \\ { - 1 / 3 } \end{array} \right] \end{aligned} \] So the least squares solution is:
\[{\bf{x}} = \left[ {\begin{array}{*{20}{l}} {4/3}\\ { - 1/3} \end{array}} \right]\]
Step-3:
\[\begin{array}{l} {\bf{b}} - A{\bf{x}} = \left[ {\begin{array}{*{20}{c}} 3\\ 1\\ { - 4}\\ 2 \end{array}} \right] - \left[ {\begin{array}{*{20}{c}} 1&{ - 2}\\ { - 1}&2\\ 0&3\\ 2&5 \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {\dfrac{4}{3}}\\ { - \dfrac{1}{3}} \end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}} 3\\ 1\\ { - 4}\\ 2 \end{array}} \right] - \left[ {\begin{array}{*{20}{c}} 2\\ { - 2}\\ { - 1}\\ 1 \end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}} 1\\ 3\\ { - 3}\\ 1 \end{array}} \right] \end{array}\]
Step-4: The least square error
\[ \begin{aligned} \| \mathbf { b } - A \mathbf { x } \| & = \sqrt { ( 1 ) ^ { 2 } + ( 3 ) ^ { 2 } + ( - 3 ) ^ { 2 } + ( 1 ) ^ { 2 } } \\ & = \sqrt { 1 + 9 + 9 + 1 } \\ & = \sqrt { 20 } \end{aligned} \] Therefore, the least squares error is:
\[e = 2\sqrt 5 \]
We are given with a matrix and a vector \[ A = \left[ \begin{array} { c c } { 1 } & { - 2 } \\ { - 1 } & { 2 } \\ { 0 } & { 3 } \\ { 2 } & { 5 } \end{array} \right] , \mathbf { b } = \left[ \begin{array} { c } { 3 } \\ { 1 } \\ { - 4 } \\ { 2 } \end{array} \right] \] We have to find the least square solution of $Ax=b$.
Step-1: The Normal equations
Compute ${A^T}A$ \[ \begin{aligned} A ^ { T } A & = \left[ \begin{array} { c c c c } { 1 } & { - 1 } & { 0 } & { 2 } \\ { - 2 } & { 2 } & { 3 } & { 5 } \end{array} \right] \left[ \begin{array} { c c } { 1 } & { - 2 } \\ { - 1 } & { 2 } \\ { 0 } & { 3 } \\ { 2 } & { 5 } \end{array} \right] \\ & = \left[ \begin{array} { c c } { 6 } & { 6 } \\ { 6 } & { 42 } \end{array} \right] \end{aligned} \] Compute ${A^T}b$ \[ \begin{aligned} A ^ { T } \mathbf { b } & = \left[ \begin{array} { c c c c } { 1 } & { - 1 } & { 0 } & { 2 } \\ { - 2 } & { 2 } & { 3 } & { 5 } \end{array} \right] \left[ \begin{array} { c } { 3 } \\ { 1 } \\ { - 4 } \\ { 2 } \end{array} \right] \\ & = \left[ \begin{array} { c } { 6 } \\ { - 6 } \end{array} \right] \end{aligned} \] So the normal equation is:
\[ \left[ \begin{array} { c c } { 6 } & { 6 } \\ { 6 } & { 42 } \end{array} \right] \left[ \begin{array} { l } { x _ { 1 } } \\ { x _ { 2 } } \end{array} \right] = \left[ \begin{array} { c } { 6 } \\ { - 6 } \end{array} \right] \]
Step-2: The solution
Compute inverse of ${{A^T}A}$ \[ \begin{aligned} \left( A ^ { T } A \right) ^ { - 1 } & = \left[ \begin{array} { c c } { 6 } & { 6 } \\ { 6 } & { 42 } \end{array} \right] ^ { - 1 } \\ & = \dfrac { 1 } { 252 - 36 } \left[ \begin{array} { c c } { 42 } & { - 6 } \\ { - 6 } & { 6 } \end{array} \right] \\ & = \dfrac { 6 } { 216 } \left[ \begin{array} { c c } { 7 } & { - 1 } \\ { - 1 } & { 1 } \end{array} \right] \\ & = \dfrac { 1 } { 36 } \left[ \begin{array} { c c } { 7 } & { - 1 } \\ { - 1 } & { 1 } \end{array} \right] \end{aligned} \] Solve for x \[ \begin{aligned} \hat { \mathbf { x } } & = \dfrac { 1 } { 36 } \left[ \begin{array} { c c } { 7 } & { - 1 } \\ { - 1 } & { 1 } \end{array} \right] \left[ \begin{array} { c } { 6 } \\ { - 6 } \end{array} \right] \\ & = \dfrac { 1 } { 36 } \left[ \begin{array} { c } { 48 } \\ { - 12 } \end{array} \right] \\ & = \left[ \begin{array} { c } { 4 / 3 } \\ { - 1 / 3 } \end{array} \right] \end{aligned} \] So the least squares solution is:
\[{\bf{x}} = \left[ {\begin{array}{*{20}{l}} {4/3}\\ { - 1/3} \end{array}} \right]\]
Step-3:
\[\begin{array}{l} {\bf{b}} - A{\bf{x}} = \left[ {\begin{array}{*{20}{c}} 3\\ 1\\ { - 4}\\ 2 \end{array}} \right] - \left[ {\begin{array}{*{20}{c}} 1&{ - 2}\\ { - 1}&2\\ 0&3\\ 2&5 \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {\dfrac{4}{3}}\\ { - \dfrac{1}{3}} \end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}} 3\\ 1\\ { - 4}\\ 2 \end{array}} \right] - \left[ {\begin{array}{*{20}{c}} 2\\ { - 2}\\ { - 1}\\ 1 \end{array}} \right]\\ = \left[ {\begin{array}{*{20}{c}} 1\\ 3\\ { - 3}\\ 1 \end{array}} \right] \end{array}\]
Step-4: The least square error
\[ \begin{aligned} \| \mathbf { b } - A \mathbf { x } \| & = \sqrt { ( 1 ) ^ { 2 } + ( 3 ) ^ { 2 } + ( - 3 ) ^ { 2 } + ( 1 ) ^ { 2 } } \\ & = \sqrt { 1 + 9 + 9 + 1 } \\ & = \sqrt { 20 } \end{aligned} \] Therefore, the least squares error is:
\[e = 2\sqrt 5 \]