This course: solving \(Lf=g\) for \(L\) a linear operator, in analogy to solving \(Ax=b\) in matrices. References:

• Hutson-Pym-Cloud, Applications of Functional Analysis and Operator Theory
• Reed-Simon, Methods of Modern Mathematical Physics
• Brezis, Functional Analysis, Sobolev Spaces, and PDEs

The issue when passing to infinite-dimensional vector spaces: the topology matters. E.g. the closure of the unit ball is closed and bounded and thus compact in finite dimensions, but this may no longer be true in \({\mathbb{R}}^\infty\) or \({\mathbb{C}}^\infty\). Recall that a Banach space is a complete normed space, and is further a Hilbert space if the norm is induced by an inner product. See the textbook for a review of vector spaces, metric spaces, norms, and inner products.

Linear subspaces are subspaces that contain zero, as opposed to affine subspaces. An example is \(C_0([0, L]; {\mathbb{R}}) \leq C([0, L]; {\mathbb{R}})\), the subspace of bounded continuous functionals on \([0, L]\) which vanish at the endpoints. For any subset \(S \subseteq V\), write \([S]\) or \(\mathop{\mathrm{span}}S\) for the linear span of \(S\): all finite linear combinations of elements in \(S\).

Limits of finite operators are compact. The classical example: set \((A_N)_{i, i} = {1\over i}\), so \(A_N = \operatorname{diag}\qty{{1}, {1\over 2}, {1\over 3}, \cdots, {1\over N}}\). Then \(\operatorname{Spec}A_N = \left\{{1\over n}\right\}_{n\leq N}\), but \(A\mathrel{\vcenter{:}}=\lim_N A_N\) is an operator with \(0\in \mkern 1.5mu\overline{\mkern-1.5mu\operatorname{Spec}(A)\mkern-1.5mu}\mkern 1.5mu\) as an accumulation point. Exercise: what is \(\ker A\)? Is it nontrivial?

Recall Holder’s inequality: \begin{align*} {\left\lVert {fg} \right\rVert}_1 \leq {\left\lVert {f} \right\rVert}_p \cdot {\left\lVert {g} \right\rVert}_q ,\end{align*} Schwarz’s inequality \begin{align*} {\left\lvert {{\left\langle {f},~{g} \right\rangle}} \right\rvert} \leq {\left\lVert {f} \right\rVert} {\left\lVert {g} \right\rVert} \qquad {\left\lVert {f} \right\rVert} \mathrel{\vcenter{:}}=\sqrt{{\left\langle {f},~{f} \right\rangle}} ,\end{align*} and Minkowski’s inequality \begin{align*} {\left\lVert {f+g} \right\rVert}_p \leq {\left\lVert {f} \right\rVert}_p + {\left\lVert {g} \right\rVert}_p .\end{align*}

A nice proof of Cauchy-Schwarz:

My notes:

• \(K \subseteq {\mathcal{H}}\) is complete iff \(K^\perp = 0\).
• Bessel: for \(f\in {\mathcal{H}}\) write \(f_n \mathrel{\vcenter{:}}={\left\langle {f},~{e_n} \right\rangle} e_n\), then \({\left\lVert {(f_n)} \right\rVert}_{\ell^2({\mathbb{C}})} \leq {\left\lVert {f} \right\rVert}_{{\mathcal{H}}}^2\).
• Best estimate: for any other sequence \((c_n) \in \ell^2({\mathbb{C}}), {\left\lVert {f - \sum c_n e_n} \right\rVert} \geq {\left\lVert {f - \sum f_n e_n} \right\rVert}\).
• For \(\left\{{e_n}\right\}\) orthonormal, \((c_n) \in \ell^2({\mathbb{C}}) \iff \sum c_n e_n\) converges. If the series converges, it can be rearranged.
• Differentiating through an integral:

• Parseval, Plancherel, and Fourier inversion:

Last time: any norm yields a metric: \(d(f, g) \mathrel{\vcenter{:}}={\left\lVert {f-g} \right\rVert}\).

• Open/closed balls: \(B_r(f) \mathrel{\vcenter{:}}=\left\{{x{~\mathrel{\Big\vert}~}{\left\lVert {f-x} \right\rVert} < r}\right\}\).
• Bounded subsets: contained in some ball of finite radius.
• \({\operatorname{diam}}S = \inf_{r, f} {\operatorname{diam}}B_r(f)\) is the diameter of the smallest ball containing \(S\).
• \(d(f, S) \mathrel{\vcenter{:}}=\inf_{x\in S} {\left\lVert {f-x} \right\rVert}\).
• \(V = {\mathbb{R}}^n\) with \({\left\lVert {f} \right\rVert}_2^2 \mathrel{\vcenter{:}}=\sum_{k\leq n} f_i^2\), \(\mkern 1.5mu\overline{\mkern-1.5muB\mkern-1.5mu}\mkern 1.5mu_0(1)\) is a metric space but not a vector space.
• For \(L^2\), there are unique least squares projections, but uniqueness may fail for \(L^1\).
  • Counterexample: take a line \(M = \left\{{{\left[ {\alpha, \alpha} \right]}}\right\}\) in \({\mathbb{R}}^2\) of angle \(\pi/4\) with respect to the \(x{\hbox{-}}\)axis and consider \(f\mathrel{\vcenter{:}}={\left[ {[} \right]}0, 1]\). Then for \(g\mathrel{\vcenter{:}}=(\alpha, \alpha)\), \({\left\lVert {f-g} \right\rVert} = {\left\lvert {1-\alpha} \right\rvert} + {\left\lvert {\alpha} \right\rvert} \geq {\left\lvert {1-\alpha + \alpha} \right\rvert} = 1\), and the minimizer occurs for any \(\alpha \in [0, 1]\).
  • Similar issues may happen for \(L^\infty\) – but \(L^1, L^\infty\) have sharper tails than \(L^2\), so this can be useful e.g. in image problems.
• If limits of sequences \((f_n)\) exist, i.e. \({\left\lVert {f_n - f_m} \right\rVert}\to 0\), then the limiting function \(f_n\to f\) is unique by the triangle inequality.
• Example from last time: \(\operatorname{diag}\qty{ 1, {1\over 2}, \cdots, {1\over n}} \to A\) a compact self-adjoint operator with \(\operatorname{Spec}A = \left\{{1\over n}\right\}_{n\geq 0}\).
  • What is \(\ker A\)? Note that \(0\in \sigma(A)\), where \(\sigma(A)\) is the set where \((A-I\lambda)^{-1}\) is not defined. It turns out \(\ker A = \left\{{0}\right\}\).
• Defining closures of subsets: for \(S \subseteq V\), say \(f\in \mkern 1.5mu\overline{\mkern-1.5muS\mkern-1.5mu}\mkern 1.5mu\) iff there exists a sequence of not necessarily distinct points \(f_n \in S\) with \(f_n\to f\).
  • Say \(S_1 \subseteq S_2 \subseteq V\) is closed in \(S_2\) iff \(S_1 = C \cap S_2\) for some \(C\) closed in \(V\). The closure of \(S_1\) in \(S_2\) is \({ \operatorname{cl}} _V(S_1) \cap S_2\).
• A set that is neither open nor closed: \(X \mathrel{\vcenter{:}}=[a, b] \cap{\mathbb{Q}}\), and \({{\partial}}X = [a,b] \supseteq X\) is actually larger.
• Recall the little \(\ell^p\) norms: \({\left\lVert {(f_n)} \right\rVert}_{p} \mathrel{\vcenter{:}}=\qty{ \sum {\left\lvert {f_n} \right\rvert}^p }^{1\over p}\) and \({\left\lVert {(f_n)} \right\rVert}_{\infty} \mathrel{\vcenter{:}}=\sup_n {\left\lvert {f_n} \right\rvert}\).

Last time:

• \({\left\lVert {f} \right\rVert}_{\ell^p} = \qty{\sum {\left\lvert {f_n} \right\rvert} ^p}^{1\over p}\)
• \({\left\lVert {f} \right\rVert}_{\ell^ \infty} = \sup_{n} {\left\lvert {f_n} \right\rvert}\).

Today:

• \(\ell_p = \left\{{f \mathrel{\vcenter{:}}=(f_n) {~\mathrel{\Big\vert}~}{\left\lVert {f} \right\rVert}_{\ell^p} < \infty }\right\}\).

• Example: set \(f^k \mathrel{\vcenter{:}}=(0, 0, \cdots, 1, 1, \cdots)\) with zeros for the first \(k-1\) entires and ones for all remaining entries. Then \(f^k_i \overset{k\to\infty}\longrightarrow 0\) for each fixed component at index \(i\). So \(f^k \to (0)\) component-wise, but \({\left\lVert {f_k} \right\rVert}_{\ell^\infty} = 1\) for every \(k\), so this doesn’t converge in \(\ell_\infty\).

• Recall the \({\varepsilon}{\hbox{-}}\delta\) and limit definitions of continuity.

• Recall the definition of uniform continuity.

• For \(\Omega \subseteq {\mathbb{R}}^n\), write \(C(\Omega)\) for the \({\mathbb{R}}{\hbox{-}}\)vector space of continuous bounded functionals \(f: \Omega\to {\mathbb{C}}\) with the norm \({\left\lVert {f} \right\rVert}_{L^\infty} = \sup_{x\in \Omega} {\left\lvert {f(x)} \right\rvert}\).

• Define \(C^k(\Omega, {\mathbb{C}}^m)\) to be functions with \(k\) continuous partial derivatives which are bounded, and set \(C^\infty(\Omega, {\mathbb{C}}^m) = \cap_{k\geq 0} C^k(\Omega, {\mathbb{C}}^m)\). Define a norm \({\left\lVert {f} \right\rVert}_{C^k} = \sum_{j\leq k} \sup_{x\in \Omega} {\left\lvert {f^{(j)} (x)} \right\rvert}\).

• Take \(g(x) = 2-x^2\) and consider \({\mathbb{B}}_{1\over 2}(g)\) in \(C[0, 1]\) with \({\left\lVert {{-}} \right\rVert}_{L^\infty}\).

• Show that convergent implies Cauchy-convergent using the triangle inequality.

• Lipschitz with \({\left\lvert {c} \right\rvert} < 1\) implies Cauchy.

• Lemma 1.4.2: \({\left\lVert {f_{n+k} - f_n} \right\rVert} \leq q^n (1-q)^{-1} {\left\lVert {f_1 - f_0} \right\rVert}\) for all \(k\geq 0\). Use \begin{align*} {\left\lVert {f_{n+k} - f_n} \right\rVert} = {\left\lVert {\sum_{j=1}^k (f_{n+j} - f_{n+j-1} ) } \right\rVert} \leq {\left\lVert {f_1 - f_0} \right\rVert} \sum_{j=1}^k q^{n+j-1} \overset{n\to\infty}\longrightarrow 0 .\end{align*}

• Counterexample, not all normed spaces are complete: take \(V = C[-1, 1]\) with \({\left\lVert {f} \right\rVert}_{L^1} \mathrel{\vcenter{:}}=\int_{-1}^1 {\left\lvert {f(x) } \right\rvert}\,dx\). Define a sequence of functions \((f_n)\):

  

  Check that \({\left\lVert {f_{n+1} - f_n} \right\rVert}_{L^1} \leq q {\left\lVert {f_n - f_{n-1}} \right\rVert}_{L^1}\), and pointwise \(f_n \to \chi_{[-1, 0]}\) which is discontinuous and not in \(C[-1, 1]\).

• Banach spaces: complete normed vector spaces.

• Series:

  • Convergent: \(f \mathrel{\vcenter{:}}=\lim_N \sum_{n\leq N} f_n \in V\).
  • Absolute convergence: \(\sum {\left\lVert {f_n} \right\rVert} < \infty\).
  • In a Banach space, absolutely convergent series can be rearranged.

• Theorem: A normed space is complete iff absolute convergence \(\implies\) convergence. Proof:

  • Step 1: show that every Cauchy sequence has a convergent subsequence.
  • Set \(a_n \mathrel{\vcenter{:}}=\sup_{m > n} {\left\lVert {f_n - f_m} \right\rVert}\), then Cauchy implies \(a_n\to 0\) in \({\mathbb{R}}\).
  • Get a convergent subsequence \(a_{n_j} \leq j^{-2}\).
  • Set \(g_j \mathrel{\vcenter{:}}= f_{n_j} - f_{n_{j+1}}\), then \(g\mathrel{\vcenter{:}}=\sum g_j\) absolutely converges, say to \(g\).
  • Note \(f_{n_i} - f_{n_{i+1}} = \sum_{j=1}^i g_j\), so the subsequence \((f_{n_j})\) is convergent.
  • Step 2: use this to show that the original sequence \((f_n)\) converges.
  • Set \(f = \lim f_n\), then \({\left\lVert {f_n - f} \right\rVert} \leq {\left\lVert {f_n - f_{n_i}} \right\rVert} + {\left\lVert {f_{n_i} - f} \right\rVert}\), using Cauchy for the first \({\varepsilon}\) and the convergent subsequence for the second.

Some theorems that hold in Hilbert spaces but not necessarily Banach spaces:

Absolute continuity:

\(L_p^{\mathsf{loc}}\):

\(\ker L = 0\) may not be sufficient to guarantee bijectivity in infinite dimensions:

Boundedness:

Bounded iff continuous:

Last time: complete iff absolutely convergent implies convergent. Today: wrapping up some results on Banach and Hilbert spaces, skipping a review of \(L^p\) spaces, and starting on operators next week.

Note that \(S \mathrel{\vcenter{:}}=(0, 1]\) is open and not complete, but \({ \operatorname{cl}} _{\mathbb{R}}(S) = [0, 1]\) is both closed and complete. Generalizing:

Extending to vector-valued functions: for \(\Omega \subseteq {\mathbb{R}}^n\), take \(\mathbf{x} = {\left[ {x_1, \cdots, x_n} \right]}\) and \(F = {\left[ {f_1, \cdots, f_m} \right]}: {\mathbb{C}}^n\to {\mathbb{C}}^m\). Then there is a Banach space \begin{align*} X = C^(\Omega, {\mathbb{C}}^m), \qquad {\left\lVert {f} \right\rVert}_{C_1(\Omega)} \mathrel{\vcenter{:}}=\sum_{i\leq m} \sup_{x\in \Omega} {\left\lvert {f(x)} \right\rvert} + \sum_{i\leq m, j\leq n} \sup_{x\in \Omega} {\left\lvert {{\frac{\partial f_i}{\partial x_j}\,}(x) } \right\rvert} .\end{align*} Similarly define \(L^p(\Omega)\), noting that \({\left\lVert {f} \right\rVert}_{L^\infty(\Omega)}\) is the essential supremum.

Recall that for \(S_1 \subseteq S_2 \subseteq B\), \(S_1\) is dense in \(S_2\) iff \({ \operatorname{cl}} _{S_2}(S_1) = S_2\). Recall Weierstrass’ theorem: for \(\Omega \subseteq {\mathbb{R}}^n\) is closed and bounded and write \({\mathcal{O}}\mathrel{\vcenter{:}}={\mathbb{R}}[x_1, \cdots, x_n]\) for the polynomials in \(n\) variables. Then \({\mathcal{O}}\subseteq C(\Omega)\), and \({ \operatorname{cl}} _{C(\Omega)} {\mathcal{O}}= C(\Omega)\), i.e. \({\mathcal{O}}\) is a dense subspace in the \(L^\infty\) norms. In fact, piecewise linear functions are dense.

Norms are equivalent iff \(c_1 {\left\lVert {f} \right\rVert}_a \leq {\left\lVert {f} \right\rVert}_b\leq c_2 {\left\lVert {f} \right\rVert}_a\) for some constants \(c_i\). All norms on \({\mathbb{R}}^n\) (resp. \({\mathbb{C}}^n\)) are equivalent.

Note that a basis for a norm can be used as a basis with respect to an equivalent norm in finite dimensions. In infinite dimensions this may not hold – e.g. for Fourier series, \(\left\{{e_k(x)}\right\}_{k\in {\mathbb{Z}}}\) is not a basis for \(C[0, 2\pi]\) with the sup norm.

• Inner products:
  • \({\left\langle {f},~{f} \right\rangle} \geq 0\) and \({\left\langle {f},~{f} \right\rangle} = 0 \iff f=0\)
  • \({\left\langle {f},~{g+h} \right\rangle} = {\left\langle {f},~{g} \right\rangle} + {\left\langle {f},~{h} \right\rangle}\)
  • \({\left\langle {f},~{g} \right\rangle} = \mkern 1.5mu\overline{\mkern-1.5mu{\left\langle {g},~{f} \right\rangle}\mkern-1.5mu}\mkern 1.5mu\)
  • \({\left\langle {\alpha f},~{g} \right\rangle} = \alpha {\left\langle {f},~{g} \right\rangle}\)
• A pre-Hilbert space is an inner-product space.
• Example: \({\left\langle {f},~{g} \right\rangle} \mathrel{\vcenter{:}}=\int_\Omega w(x) f(x) \mkern 1.5mu\overline{\mkern-1.5mug(x)\mkern-1.5mu}\mkern 1.5mu\,dx\) for \(f,g\in C(\Omega)\), where \(w\) is any weighting function.
• Example: \({\left\langle {f},~{g} \right\rangle} = \sum f_i \mkern 1.5mu\overline{\mkern-1.5mug_i\mkern-1.5mu}\mkern 1.5mu\) for \(f,g\in {\mathbb{C}}^n\).
• Inner products induce norms: \({\left\lVert {f} \right\rVert} \mathrel{\vcenter{:}}=\sqrt{{\left\langle {f},~{f} \right\rangle}}\)
  • Orthogonality: write \(f\perp g\) iff \({\left\langle {f},~{g} \right\rangle} = 0\) and \(S^\perp = \left\{{f\in {\mathcal{H}}{~\mathrel{\Big\vert}~}f\perp s \, \forall s\in S}\right\}\).
• Definition of a Hilbert space: a pre-Hilbert space complete with respect to the norm induced by its inner product.
• Recall \(C(\Omega)\) with \({\left\langle {f},~{g} \right\rangle} \mathrel{\vcenter{:}}=\int_\Omega f\mkern 1.5mu\overline{\mkern-1.5mug\mkern-1.5mu}\mkern 1.5mu\) is not complete, thus not a Hilbert space.
• Example: a common optimization problem, \(\mathrm{argmin} {\left\lVert {x} \right\rVert}\) such that \(Ax=0\).

Notes:

• Bessel: \begin{align*} \sum_n {\left\lvert { {\left\langle {f},~{\phi_n} \right\rangle} } \right\rvert} \leq {\left\lVert {f} \right\rVert}^2,\qquad {\left\lVert {f} \right\rVert} \mathrel{\vcenter{:}}=\sqrt{{\left\langle {f},~{f} \right\rangle}} .\end{align*}
  • Prove using the fact that \begin{align*} 0\leq {\left\lVert {f - \sum_{n\leq m} {\left\langle {f},~{\phi_n} \right\rangle} \phi_n } \right\rVert}^2 = {\left\lVert {f} \right\rVert}^2 - \sum_{n\leq m} {\left\lvert {{\left\langle {f},~{\phi_n} \right\rangle}} \right\rvert}^2 .\end{align*}
• Best fit: \begin{align*} {\left\lVert {f - \sum_{n\leq m} c_n \phi_n } \right\rVert} \geq {\left\lVert {f - \sum_{n\leq m} {\left\langle {f},~{\phi_n} \right\rangle} \phi_n } \right\rVert} ,\end{align*} so define projections \(P_M(f) \mathrel{\vcenter{:}}=\sum {\left\langle {f},~{\phi_n} \right\rangle} \phi_n\) for \(\mathop{\mathrm{span}}\left\{{\phi_n}\right\} = M\).
  • Prove using \begin{align*} {\left\lVert {f - \sum_{n\leq m} c_n \phi_n } \right\rVert} &= {\left\lVert {f} \right\rVert}^2 - \sum_{n\leq m} {\left\lvert {{\left\langle {f},~{\phi_n} \right\rangle}} \right\rvert}^2 + \sum_{n\leq m} {\left\lvert {{\left\langle {f},~{\phi_n} \right\rangle} - c_n } \right\rvert}^2 \\ &\geq {\left\lVert {f} \right\rVert}^2 - \sum_{n\leq m} {\left\lvert {{\left\langle {f},~{\phi_n} \right\rangle}} \right\rvert}^2 .\end{align*}
• Hilbert spaces are separable: have countable dense subsets.
  • \(\ell^\infty({\mathbb{Z}})\) is not separable.
• For \(\left\{{\phi_n}\right\}\) orthonormal and \(\left\{{c_n}\right\}\) scalars, \(\sum c_n \phi_n\) is convergent iff \(\left\{{a_n}\right\}\in \ell^2({\mathbb{Z}})\), so \(\sum{\left\lvert {c_n} \right\rvert}^2<\infty\). If it converges, it can be rearranged, and \begin{align*} {\left\lVert {\sum c_n \phi_n} \right\rVert}^2 = {\left\lVert {\left\{{c_n}\right\}} \right\rVert}_{\ell^2({\mathbb{Z}})}^2 = \sum {\left\lvert {c_n} \right\rvert}^2 .\end{align*}
  • To prove, use that \({\left\lVert { \sum_{i\leq n\leq j}c_n \phi_n} \right\rVert}^2 = \sum_{i\leq n\leq j}{\left\lvert {c_n} \right\rvert}^2\), so the sequence \(\left\{{S_m}\right\}_{m\geq 0}\) where \(S_m \mathrel{\vcenter{:}}=\sum_{n\leq m} c_n\phi_n\) is Cauchy since \(\sum{\left\lvert {c_n} \right\rvert}^2\) converges.
  • If \(g = \sum c_n \phi_n\) and \(f = \sum c_{m_n} \phi_{m_n}\) is a rearrangement, if \(\left\{{c_n}\right\}\in \ell^2\) then \({\left\lVert {f} \right\rVert}^2 = {\left\lVert {g} \right\rVert}^2 = \sum {\left\lvert {c_n} \right\rvert}^2\). Then \({\left\lVert {f-g} \right\rVert}^2 = {\left\lVert {f} \right\rVert}^2 + {\left\lVert {g} \right\rVert}^2 - 2\Re{\left\langle {f},~{g} \right\rangle}\), but \({\left\langle {f},~{g} \right\rangle} = \sum {\left\lvert {c_n} \right\rvert}^2\), so \({\left\lVert {f-g} \right\rVert} = 0\).
• If \(K = \left\{{\phi_n}\right\} \subseteq {\mathcal{H}}\) is a proper subset, so \(\mathop{\mathrm{span}}\left\{{\phi_n}\right\}\neq {\mathcal{H}}\), write \(P_K(f) = \sum {\left\langle {f},~{\phi_n} \right\rangle}\phi_n\). Then \(P_K(f) = 0\) if \(f\in { \operatorname{cl}} (K)^\perp\), \(P_K(f) = f\) if \(f\in { \operatorname{cl}} (K)\), and if \(f\not \in K\), since \({ \operatorname{cl}} (K)\leq {\mathcal{H}}\) is closed then there exists a \(g\in { \operatorname{cl}} (K)\) where \({\left\lVert {f-g} \right\rVert} = \operatorname{dist}(f, K)\). Write \(f = h + (f-g) \in { \operatorname{cl}} (K) \oplus { \operatorname{cl}} (K)^\perp\), so \(f = P_K f + (I-P_K)f\).
• Recall complete subspaces of Banach spaces are closed.
• Next theorem: every separable Hilbert space admits an orthonormal basis.

Note \({\left\langle {f},~{\phi_n} \right\rangle}\) is the \(n\)th Fourier coefficient \(\widehat{f}(\xi) = \sum {\left\langle {f},~{\phi_n} \right\rangle}\phi_n(\xi)\), and Parseval says \({\left\lVert {f} \right\rVert}^2 = {\left\lVert {\widehat{f}} \right\rVert}^2\).

Notes:

• Assume \({\mathcal{H}}\) is a separable Hilbert space: there exists a countable set of vectors \(\left\{{v_i}\right\}_{i\in {\mathbb{Z}}}\) which span a subspace that is dense in \({\mathcal{H}}\), so \({ \operatorname{cl}} (\mathop{\mathrm{span}}\left\{{v_i}\right\}) = {\mathcal{H}}\).
• Complete subspaces: \(M\leq {\mathcal{H}}\) with \(M^\perp = 0\).
• Show that for any \(S \subseteq {\mathcal{H}}\), \(S^\perp\) is closed in \({\mathcal{H}}\), \(S^\perp = \qty{{ \operatorname{cl}} \mathop{\mathrm{span}}S}^\perp\), and \(f\in S \cap S^\perp \implies f=0\).
• If \(K = \left\{{\phi_k}\right\}_{k\in {\mathbb{Z}}}\) is an orthonormal set in \({\mathcal{H}}\), then TFAE
  • \(K \leq {\mathcal{H}}\) is a complete subspace, so \(K^\perp = 0\), i.e. \({\left\langle {f},~{\phi_k} \right\rangle} = 0\) for all \(k\) implies \(f=0\).
  • \({ \operatorname{cl}} \mathop{\mathrm{span}}K = {\mathcal{H}}\), so every \(f\in {\mathcal{H}}\) is the limit of a sequence of vectors from \(\mathop{\mathrm{span}}K\).
  • \(K\) is an orthonormal basis
  • Parseval: equality in Bessel, i.e. \({\left\lVert {f} \right\rVert} = \sum {\left\lvert { {\left\langle {f},~{\phi_k} \right\rangle} } \right\rvert}^2\)
• Lemma: if \(M, N\leq {\mathcal{H}}\) with \(\dim M < \dim N\), then \(M^\perp \cap N \neq 0\).
  • Without loss of generality assume \(\dim N = n + 1\) where \(n\mathrel{\vcenter{:}}=\dim M\), take a basis \(\left\{{\psi_k}\right\}_{k\leq n+1}\) for \(N\).
  • Try to find \(f\in N\) with \(f\perp M\), i.e. coefficients \(\left\{{b_i}\right\}_{i\leq n}\) with \(\sum b_i \psi_i \perp \phi_k\) for every \(\phi_k\) basis elements of \(M\).
  • This is a linear system of \(n\) equations in \(n+1\) unknowns, so it has a nontrivial solution.
• Theorem, orthonormal bases are stable: if \(\left\{{\phi_k}\right\}\) is an orthonormal basis and \(\left\{{\psi_k}\right\}\) is an orthonormal system, if \(\sum {\left\lVert {\phi_k - \psi_k} \right\rVert}^2 < \infty\) then \(\left\{{\psi_k}\right\}\) is a basis.
  • Assume note, then find a \(\psi_0 \in {\mathcal{H}}\) with \({\left\lVert {\psi_0} \right\rVert} = 1\) and \({\left\langle {\psi_0},~{\psi_j} \right\rangle} = 0\) for all \(j\).
  • Choose \(N\gg 1\) so that \(\sum_{k\geq N} {\left\lVert {\psi_k - \phi_k} \right\rVert} < 1\).
  • Use previous lemma to produce \(w\in \mathop{\mathrm{span}}\left\{{\psi_0, \psi_1,\cdots, \psi_N}\right\}\) with \(w\neq 0\) and \(w\perp \phi_j\) for all \(j\leq N\).
  • Note \(w\perp \mathop{\mathrm{span}}\left\{{\psi_n}\right\}_{n > N}\).
  • Apply Parseval: \begin{align*} 0 &< {\left\lVert {w} \right\rVert}^2 \\ &= \sum_{n\geq 1} {\left\lvert {{\left\langle {w},~{\phi_n} \right\rangle}} \right\rvert}^2 \\ &= \sum_{n\geq N+1}{\left\lvert {{\left\langle {w},~{\phi_n} \right\rangle}} \right\rvert}^2 \\ &= \sum_{n\geq N+1} {\left\lvert {{\left\langle {w},~{\phi_n - \psi_n} \right\rangle}} \right\rvert}^2 \\ &\leq {\left\lVert {w} \right\rVert}^2 \sum_{n\geq N+1}{\left\lVert {\phi_n - \psi_n} \right\rVert}^2 \\ &< {\left\lVert {w} \right\rVert}^2 \cdot 1 ,\end{align*} where we’ve used that \({\left\langle {w},~{\psi_n} \right\rangle} = 0\) for \(n\geq N\). \(\contradiction\)
• \({\mathcal{H}}\) admits a countable orthonormal basis iff \({\mathcal{H}}\) is separable.
  • \(\implies\): clear, since the basis is countable, and every element is a limit of partial sums against the basis.
  • \(\impliedby\): Gram-Schmidt.
    • \(h_1 = \psi_1\) and \(\phi_1 = h_1/{\left\lVert {h_1} \right\rVert}\)
    • \(h_n = \psi_n - \sum_{1\leq k\leq n-1} {\left\langle {\psi_k},~{\phi_k} \right\rangle} \phi_k\) and normalize.
• Exercise: a closed subspace of a separable Hilbert space is separable.
• Linearly isometric inner product spaces: \(E\sim F\) iff there is a map \(A: E\twoheadrightarrow F\) with
  • \(A(au + bv) = aAu + bAv\)
  • \({\left\lVert {Au} \right\rVert}_F = {\left\lVert {u} \right\rVert}_E\)
• Theorem: if \({\mathcal{H}}_1, {\mathcal{H}}_2\) are infinite dimensional separable Hilbert spaces, then \({\mathcal{H}}_1 \sim {\mathcal{H}}_2\). Thus for any Hilbert space \({\mathcal{H}}\) over \({\mathbb{C}}\), \({\mathcal{H}}\sim \ell^2({\mathbb{C}})\).
  • Pick orthonormal bases \(\left\{{\phi_k}\right\} \subseteq {\mathcal{H}}_1, \left\{{\psi_k}\right\} \subseteq {\mathcal{H}}_2\).
  • For \(u\in {\mathcal{H}}_1\), define \(Au \mathrel{\vcenter{:}}=\sum {\left\langle {u},~{\phi_k} \right\rangle}\psi_k\), which converges – this will be the linear isometry, and satisfies condition (i).
  • Check \({\left\lVert {Au} \right\rVert}_{{\mathcal{H}}_2}^2 = \sum_{k\geq 1}{\left\lvert {{\left\langle {u},~{\phi_k} \right\rangle}} \right\rvert}^2 = {\left\lVert {u} \right\rVert}_{{\mathcal{H}}_2}\), which is condition (ii).
  • Check \(A\) is surjective: for \(y\in {\mathcal{H}}_2\), write \(y = \sum_{k} {\left\langle {y},~{\psi_k} \right\rangle} \psi_k = Av\) for \(v\mathrel{\vcenter{:}}=\sum_k {\left\langle {y},~{\psi_k} \right\rangle}\phi_k \in {\mathcal{H}}_1\).
• Non-separable spaces: look at almost-periodic functions.
  • E.g. \(\sum_{k\leq n} c_k \exp(i\lambda_k t)\) for \(\lambda_k \in {\mathbb{R}}\).

Motivating question: when is an operator equation solvable? Today: relation between boundedness and continuity for linear operators. Nonlinear operators next week.

• A map of vector spaces \(V\to W\) is a linear map defined on some domain \(D(A) \subseteq V\), where \(D(A)\) need not equal \(V\).
  • Notation: \(A(f) = Af = g\).
  • \(Af \subseteq W\) is the image of \(f\), and \(R(A) \mathrel{\vcenter{:}}=\left\{{Af{~\mathrel{\Big\vert}~}f\in D(A)}\right\} \subseteq W\) is the range. Preimages of \(S \subseteq W\): \(A^{-1}(S) = \left\{{f{~\mathrel{\Big\vert}~}f\in D(A) \text{ and } Af\in S}\right\}\).
• We distinguish operators with different domains, e.g. \(Af \mathrel{\vcenter{:}}= f'\) can be the formula for distinct operators \(A_1, A_2\) where \(D(A_1) = C^\infty[0, 1] \subseteq C^0[0, 1]\) or \(D(A_2) = C^1[0, 1] \subseteq C^0[0, 1]\), so \(A_1\neq A_2\).
  • I.e. \(A_1 = A_2 \iff D(A_1) = D(A_2)\) and \(A_1 f = A_2 f\) for all \(f\in D(A_1)=D(A_2)\).
  • If \(A_1 f = A_2 f\) with \(D(A_1) \subseteq D(A_2)\), say \(A_2\) is an extension of \(A_1\). The extension is proper iff \(D(A_1)\neq D(A_2)\).
• Example operator: the Laplace equation \(\Delta f= g\) where \(\Delta= {\partial}_{xx} + {\partial}_{yy}\). We can take domains \(g\in C[0, 1], L^2[0, 1], H^2[0, 1] = \left\{{f\in L^2(0, 1) {~\mathrel{\Big\vert}~}{\partial}_{xx} f, {\partial}_{yy} f\in L^2(0, 1) }\right\}\).
  • Why domains matter: boundary conditions affect what eigenfunctions you get. Examples where \(A_1\neq A_2\):
  • Dirichlet boundary conditions: \(\Delta f = g, { \left.{{f}} \right|_{{{{\partial}}\Omega}} } = 0\). The relevant solution spaces is \(D(A_1) = \left\{{\phi\in C^2[0,1]^2 {~\mathrel{\Big\vert}~}{ \left.{{\phi }} \right|_{{{{\partial}}\Omega}} } = 0}\right\}\) for \(A_1 \phi \mathrel{\vcenter{:}}=\Delta\phi\).
  • Neumann boundary conditions: \(\Delta f = g, { \left.{{ {\frac{\partial f}{\partial \mathbf{n}}\,}}} \right|_{{{{\partial}}\Omega}} } = 0\), i.e. there is no flux across the boundary. The relevant solution space is \(D(A_2) = \left\{{\psi \in C^2[0,1]^2{~\mathrel{\Big\vert}~}{ \left.{{ {\frac{\partial \psi}{\partial \mathbf{n}}\,} }} \right|_{{{{\partial}}\Omega}} } = 0 }\right\}\) for \(A_2 \psi \mathrel{\vcenter{:}}=\Delta\psi\).
• Injectivity: for \(A: V\to W\), for every \(g\in R(A)\) there is exactly one \(f\in D(A)\) with \(Af=g\). Equivalently for linear operators, \(Af = 0 \implies f=0\).
• Surjectivity: \(R(A) = W\).
• Example: \(A\mathrel{\vcenter{:}}= x\mapsto \sin(x)\) regarded as a function \(A:{\mathbb{R}}\to {\mathbb{R}}\) is neither surjective nor injective: \(R(A) = [-1, 1] \subsetneq {\mathbb{R}}\), and \(\sin(\pi {\mathbb{Z}}) = 0\).
• Linearity: for \(Lf = g\), \(L\) is linear if \(L(af + bg) = aLf + bLg\).

Last time:

• Continuous operators are bounded:
  • If \({\left\lVert {Lf_n} \right\rVert} = 1\) and \({\left\lVert {f_n} \right\rVert} \to 0\), check \(\lim (Lf_n) = L(\lim f_n) = L0 = 0\).
  • Take norms to contract \({\left\lVert {Lf_n} \right\rVert} = 1\).

For \({\mathcal{L}}\in L({\mathbb{C}}^\infty, {\mathbb{C}}^\infty)\) defined by \((Lf)_i = \sum_{j\geq 11} \alpha_{ij} f_j\) for \(j\geq 1\), one needs a notion of convergence of the coordinates \(\alpha_{ij}\) in order for \({\mathcal{L}}\) to be bounded. A sufficient condition is \(m\mathrel{\vcenter{:}}=\sup_i \sum_{j\geq 1} {\left\lvert {\alpha_{ij}} \right\rvert} < \infty\).

Note that if \({\mathcal{L}}: \ell^p\to \ell^p\), then \({\left\lVert {L} \right\rVert} \leq {\left\lVert \left\lVert {\alpha} \right\rVert\right\rVert}_p\) for \(p\in [1, \infty)\) and for \(p=\infty\) this is an equality.