# Thursday November 7 

## Bessel

Let $H$ be a Hilbert space, then we have 

**Theorem (Bessel's inequality)**:

If $\theset{u_n}$ is orthonormal in $H$, then for any $x \in H$ we have equation 0
$$
\sum_n \abs{\inner{x}{u_n}}^2 \leq \norm{x}^2,
$$

or equivalently $\theset{ \inner{x}{u_n} } \in \ell^2 \NN$.

*Proof:*

We have

\begin{equation}
0 \leq \norm{
x - \sum_{n=1}^N \inner{x}{u_n} u_n
}^2 =
\norm{x}^2 - \sum_{n=1}^N \abs{
\inner{x}{u_n}
}^2 \forall N
.\end{equation}


**Remark (Characterization of Basis):**

TFAE:

- $$\mathrm{span} \theset{u_n} = H,$$ i.e. $u_n$ is a basis.

- $$\sqrt{ \sum_n \abs{ \inner{x}{u_n}  }^2  } = \norm{x} \forall x\in H,$$ 
    i.e. **Parseval's identity**

- $$\lim_{N\to\infty} \norm{ x - \sum_n^N \inner{x}{u_n}  } = 0,$$ 
  
    i.e. the Fourier series converges in $H$.

Recall the Riesz-Fischer theorem: 

If $\theset u_n$ is orthonormal in $H$ and $\theset a_n \in \ell^2(\NN)$, then 
$$
\exists x\in H \text{ such that } a_n = \inner{x}{u_n} \text{ and } \norm{x}^2 = \sum_n \abs{a_n}^2.$$

Moreover, the map $x \mapsto \hat{x}(u) \definedas \inner{x}{u_n}$ maps $H$ onto $\ell^2(\NN)$ surjectively.

*Remark*: 
This $x$ is only unique if $\theset u_n$ is *complete*, i.e. $\inner{y}{u_n} = 0 \quad \forall n \implies y = 0$.

*Proof:*
Let $S_N \definedas \sum_{n=1}^N a_n u_n$.

Then $S_N$ is Cauchy, so

\begin{align*}
\norm{S_N - S_M}^2 &= \norm{ \sum_{n=M+1}^N a_n u_n }^2 \\
&= \sum_{n=M+1}^N  \norm{a_n u_n}^2 \quad\quad\text{by Pythagoras since the $u_n$ are orthogonal} \\
&= \sum_{n=M+1}^n \abs{a_n} \to 0
,\end{align*}

since $\sum \abs{a_n} < \infty$ implies that the sum is Cauchy.


Since $H$ is complete, $S_N \to x$ for some $x\in H$.

We now need to argue that $a_n = \inner{x}{u_n}$.

If $N \geq n$, then we have the identity 
$$
\abs{\inner{x}{u_n} - a_n} = \abs{\inner{x}{u_n} - \inner{S_N}{u_n}  } = \abs{ \inner{x - S_N}{u_n} } \leq \norm{x - S_N} \to 0.
$$

> Note: should be able to translate this to statements about epsilons almost immediately!

But then equation 1 holds in the limit as $N \to \infty$, which establishes equation 0. $\qed$

*Proof of characterization of basis:*

$1 \implies 2$: 
Let $\varepsilon > 0, x\in H, \inner{x}{u_n} = 0$ for all $n$. 
We will attempt to show that $\norm{x} < \varepsilon$, so $x = 0$.

By (1), there is a $y\in \mathrm{span}\theset{u_n}$ such that $\norm{x - y}< \varepsilon$.
But then $\inner{x}{y} = 0$, so 
$$
\norm{x}^2 = \inner{x}{x} = \inner{x}{x-y} \leq \norm{x}{x-y} \leq \varepsilon \norm {x} \to 0
.$$ 
$\qed$

> Note: $\inner{x}{x} = \inner{x}{x} - \inner{x}{y} = \inner{x}{x-y}$ since $\inner{x}{y} = 0$.

$2 \implies 3$: 
By Bessel, we have $\theset{\inner{x}{u_n}  } \in \ell^2 \NN$, and we know that its norm is bounded by $\norm{x}$.

By Riesz-Fischer, there exists a $y\in H$ such that $\inner{y}{u_n} = \inner{x}{u_n}$ and $\norm{y} = \sqrt{ \sum \abs{\inner{x}{u_n}}^2  }$.

By completeness, we get $x=y$. 
$\qed$

## Existence of Bases

- Every Hilbert space has an orthonormal basis (possibly uncountable)

- $H$ *separable* Hilbert space $\iff$ $H$ has a *countable* basis (separable = countable dense subset).


Some examples of orthonormal bases:

- 
$$
\ell^2 \NN: \quad u_n(k) = \vector e_n \definedas \begin{cases} 1 & n=k \\ 0 & \text{otherwise}\end{cases}
$$

- 
$$
L^2([0,1]):\quad e_n(x) \definedas e^{2\pi i n x}
.$$
  
  Normed: by Cauchy-Schwarz, but need to show it's complete. Can use the fact that $L^1$ is complete.
  
  Note that 
  $$
  \inner{f}{e_n} = \int_0^1 f(x) e^{-2\pi i n x} ~dx
  ,$$ 
  which is exactly the Fourier coefficient.

## $L^2$ is Complete (Sketch)

*Sketch proof that $L^2([0, 1])$ is complete:*

Note that $L^2([0, 1]) \subseteq L^1([0, 1])$, since 
$$
f\in L^2 \implies \int_0^1 \abs{f} \cdot 1 ~dx \leq \sqrt{\int_0^1 \abs{f}^2}
$$ by Cauchy-Schwarz. 
This also shows that $\norm{f}_1 \leq \norm{f}_2$.

Let $f_n$ be Cauchy in $L_2$.
Then $f_n$ is Cauchy in $L^1$, and since $L^1$ is complete, there is a subsequence converging to $f$ almost everywhere.

By Fatou,
$$
\int \liminf_k \abs{f_{n_j} - f_{n_k}}^2 \leq \liminf \int \abs{f_{n_j} - f_{n_k}}
.$$

But the LHS goes to $\int \abs{f_{n_j} - f}$ and the RHS is $\norm{f_{n_j} - f_{n_k}} \to 0$, so less than $\varepsilon$ if $j$ is big enough.
So $f_{n_j} \mapsvia{L^2} f$ in $L^2$ as $j\to\infty$, and thus $f_n \to f\in L^2$ as $n\to\infty$.

## Unitary Maps

**Definition:**
Let $U: H_1 \to H_2$ such that $\inner{Ux}{Uy} = \inner{x}{y}$, i.e. $U$ preserves angles, and we say $U$ is *unitary*.

Then $\norm{Ux} = \norm{x}$, i.e. $U$ is an *isometry*.

Every unitary map is an isometry. 
If $U$ is surjective, this implication can be reversed.

For example, taking the Fourier transform yields 
$$
\sum \abs{\hat{f}(u)}^2 = \norm{f}_2^2 = \int\abs{f}^2 \text{ and } \sum \hat{f}(u) \overline{\hat{g}(u)} = \int f \overline{g}
.$$

**A corollary of Riesz-Fischer:**
If $\theset u-N$ is an orthonormal basis in $H$, then the map $x \mapsto \hat{x}(u) \definedas \inner{x}{u_n}$ is a *unitary* map from $H$ to $\ell^2$.

So all Hilbert spaces are unitarily equivalent to $\ell^2 \NN$.


![Image](figures/2019-11-07-12:22.png)\

> Subspaces in Hilbert spaces don't have to be closed, but orthogonal complements are always closed! See homework problem.