# Tuesday, January 25 :::{.remark} - Inner products: - $\inp{f}{f} \geq 0$ and $\inp{f}{f} = 0 \iff f=0$ - $\inp{f}{g+h} = \inp{f}{g} + \inp{f}{h}$ - $\inp{f}{g} = \bar{\inp{g}{f}}$ - $\inp{\alpha f}{g} = \alpha \inp{f}{g}$ - A pre-Hilbert space is an inner-product space. - Example: $\inp f g \da \int_\Omega w(x) f(x) \bar{g(x)}\dx$ for $f,g\in C(\Omega)$, where $w$ is any weighting function. - Example: $\inp f g = \sum f_i \bar{g_i}$ for $f,g\in \CC^n$. - Inner products induce norms: $\norm{f} \da \sqrt{\inp f f}$ - Orthogonality: write $f\perp g$ iff $\inp f g = 0$ and $S^\perp = \ts{f\in \mch \st f\perp s \, \forall s\in S}$. - Definition of a Hilbert space: a pre-Hilbert space complete with respect to the norm induced by its inner product. - Recall $C(\Omega)$ with $\inp f g \da \int_\Omega f\bar{g}$ is not complete, thus not a Hilbert space. - Example: a common optimization problem, $\mathrm{argmin} \norm{x}$ such that $Ax=0$. ::: :::{.theorem title="Cauchy-Schwarz"} \[ \abs{\inp{f}{g}}\leq \norm{f} \norm{g} .\] ::: :::{.proof title="?"} Use $\inp f g = \abs f \abs g \cos \theta_{fg}$ where $\abs{ \cos \theta } \leq 1$. - Assume $g\neq 0$, then STS $\inp{f}{g\over \norm g} \leq \norm f$. - Use\[ 0 &\leq \norm{f - \inp f g g}^2 \\ &= \inp{f - \inp f g g}{f - \inp f g g} \\ &= \inp f f - \inp f g \inp g f - \bar{\inp f g} \inp f g + \inp f g \bar{\inp f g } \\ &= \norm{f}^2 - \abs{ \inp f g}^2 .\] ::: :::{.theorem title="Closest approximations"} Let $f\in M\leq \mch$ be a closed (and thus complete) subspace of a Hilbert space $\mch$. Then there is a unique element $g$ in $\mch$ closest to $M$ in the norm. ::: :::{.proof title="?"} Let $d\da \dist(f, M)$, choose a sequence $g_n \in M$ such that $\norm{f-g_n}\to d$, which is possible since $d = \inf_{g\in M}\norm{f-g}$. Apply the parallelogram law to write \[ \norm{g_n - g_m}^2 &= \norm{(g_n - f) - (g_m - f) }^2 \\ &= 2\norm{g_n - f}^2 + 2\norm{g_m - f}^2 - 4\norm{ {1\over 2} (g_n + g_m) - f}^2 \\ &\leq 2\norm{g_n - f}^2 + 2\norm{g_m - f}^2 - 4d^2 \\ &\leq 2d^2 + 2d^2 - 4d^2 \\ &= 0 ,\] so the $g_n$ are Cauchy. Here we've used that ${1\over 2}(g_n + g_m) = m \in M$ since $M$ is a subspace, and $\norm{m-f} \geq d$. Since $M$ is complete, $g_n\to g\in M$ and moreover $\norm{f-g} = d$. For uniqueness, if $\norm{f-g'} = d$ then \[ \norm{ f - {1\over 2} (g+g')}^2 = d^2 - \norm{g-g'}^2 < d^2 \qquad \contradiction .\] ::: :::{.theorem title="Projection theorem"} Let $M\leq \mch$ be a closed subspace of a Hilbert space. Then $M^\perp \leq \mch$ is closed, and $\mch = M \oplus M^\perp$. In the decomposition $f= g+h$, in fact $g\in M$ is the closest approximation to $f$ in $M$, making this decomposition unique. ::: :::{.proof title="?"} STS $\mch = M + M^\perp$ by the exercises. If $f\in M$, take $f=g+h$ where $g=f$ and $h=0$, so suppose $f\not\in M$. Let $g = \mathrm{argmin} \dist(f, M) \in M$, and we claim $f-g\in M^\perp$, so $\inp {f-g}{h} = 0$ for any $h\in M$. For all $h\in M$ and $\alpha > 0$, we have $g + \alpha h\in M$, so \[ \norm{f - g}^2 &\leq \norm{f - (g+\alpha h)}^2 \\ &= \norm{f-g}^2 -2\Re\alpha \inp{h}{f-g} + \alpha^2 \norm{h}^2 ,\] so \[ 2\Re \alpha \inp h {f-g} \leq \alpha^2 \norm{h}^2 \implies 2\Re \inp h {f-g} \leq \alpha \norm{h}^2 \convergesto{\alpha\to 0} 0 ,\] so $\Re\inp{h}{f-g} = 0$. Similarly $\Im\inp{h}{f-g} = 0$. ::: :::{.exercise title="?"} Show $S^\perp$ is closed for any $S\in \mch$, and in fact $S^\perp = \spanof_\CC(\cl_\mch(S))^\perp$, and if $f\in S \intersect S^\perp$ then $f=0$. ::: :::{.exercise title="?"} Prove the parallelogram law \[ \norm{f-g}^2 + \norm{f+g}^2 = 2\norm{f}^2 + 2\norm{g}^2 .\] :::