Normed & Banach Spaces: Completeness, Compactness & the Hahn-Banach Theorem

A Banach space is a complete normed vector space — completeness ensures that Cauchy sequences converge, a property that makes iterative algorithms well-defined in infinite dimensions. Without it, a sequence of approximations can look convergent yet have no limit inside the space, breaking fixed-point arguments and the guarantees that underlie optimization and approximation theory.

Concepts

You have been working with vectors and measuring their size with the Euclidean norm all along. But the Euclidean norm is just one choice — $\ell^1$ (sum of absolute values), $\ell^\infty$ (largest absolute value), and the whole $\ell^p$ family are equally valid norms, each inducing a different geometry and different unit balls. Banach spaces generalize this to infinite dimensions with one critical additional requirement: completeness. Without it, sequences that "should" converge have no limit in the space, and the iterative arguments that underlie optimization and approximation theory break down entirely.

Normed Vector Spaces

A normed vector space is a pair $(X, \|\cdot\|)$ where $X$ is a vector space over $\mathbb{R}$ (or $\mathbb{C}$) and $\|\cdot\|: X \to [0,\infty)$ satisfies three axioms:

1. Positive definiteness: $\|x\| \geq 0$, and $\|x\| = 0$ if and only if $x = 0$
2. Homogeneity: $\|\alpha x\| = |\alpha|\,\|x\|$ for all scalars $\alpha$
3. Triangle inequality: $\|x + y\| \leq \|x\| + \|y\|$

Every norm induces a metric $d(x,y) = \|x - y\|$, giving a topology on $X$ in which open balls are the unit of convergence.

Lp Norms and Classical Inequalities

For a sequence $x = (x_1, x_2, \ldots)$ and $1 \leq p < \infty$, the $\ell^p$ norm is

$\|x\|_p = \left(\sum_i |x_i|^p\right)^{1/p}$

The $\ell^\infty$ norm is $\|x\|_\infty = \sup_i |x_i|$. For functions, the $L^p([a,b])$ norm replaces the sum with an integral: $\|f\|_p = \bigl(\int_a^b |f(x)|^p\,dx\bigr)^{1/p}$.

Hölder's inequality. For conjugate exponents $1/p + 1/q = 1$ with $p, q \geq 1$:

$\|fg\|_1 \leq \|f\|_p\,\|g\|_q$

In finite dimensions this reads $\sum_i |x_i y_i| \leq \|x\|_p\|y\|_q$. The case $p = q = 2$ is the Cauchy-Schwarz inequality.

Minkowski's inequality is the triangle inequality for $L^p$: $\|f+g\|_p \leq \|f\|_p + \|g\|_p$. It follows from Hölder applied to $(f+g)^{p-1}$.

The three axioms of a norm are not arbitrary choices — they encode exactly what is needed for distance to behave like distance: positivity ensures distinct points are distinguishable, homogeneity ensures scaling is consistent, and the triangle inequality ensures that detours don't help. Hölder's and Minkowski's inequalities are the proofs that these axioms survive when you raise vectors to powers, turning a metrically well-behaved space into an arithmetically useful one.

Banach Spaces: Definition and Examples

A Banach space is a normed vector space that is complete: every Cauchy sequence converges to an element of the space. Formally, if $\|x_n - x_m\| \to 0$ as $n,m \to \infty$, there exists $x \in X$ with $\|x_n - x\| \to 0$.

Four Foundational Theorems

Hahn-Banach Theorem. Let $Y \subset X$ be a subspace and $f: Y \to \mathbb{R}$ a bounded linear functional with $\|f\|_Y \leq M$. Then $f$ extends to $\tilde{f}: X \to \mathbb{R}$ with $\|\tilde{f}\|_X = \|f\|_Y$. This is the fundamental existence result for dual functionals.

Open Mapping Theorem. If $T: X \to Y$ is a surjective bounded linear operator between Banach spaces, then $T$ is an open map. Corollary: a bijective bounded linear operator between Banach spaces has a bounded inverse.

Closed Graph Theorem. A linear operator $T: X \to Y$ between Banach spaces is bounded if and only if its graph $\{(x, Tx) : x \in X\}$ is closed in $X \times Y$.

Uniform Boundedness Principle (Banach-Steinhaus). Let $\{T_\alpha\}$ be a family of bounded linear operators from a Banach space $X$ to a normed space $Y$. If $\sup_\alpha \|T_\alpha x\| < \infty$ for every $x \in X$, then $\sup_\alpha \|T_\alpha\| < \infty$.

Dual Spaces

The dual space $X^* = B(X, \mathbb{R})$ consists of all bounded linear functionals $f: X \to \mathbb{R}$, with norm $\|f\|_{X^*} = \sup_{\|x\| \leq 1} |f(x)|$. It is always a Banach space regardless of whether $X$ is.

Riesz representation for $L^p$. For $1 < p < \infty$ and $1/p + 1/q = 1$, every bounded functional on $L^p$ has the form $f \mapsto \int g f\,dx$ for a unique $g \in L^q$, giving an isometric isomorphism $(L^p)^* \cong L^q$. The dual of $\ell^1$ is $\ell^\infty$ by the same argument applied termwise.

Worked Example

Proving Hölder's Inequality for Finite Sequences

For $a, b \geq 0$ and $1/p + 1/q = 1$, Young's inequality states $ab \leq a^p/p + b^q/q$. Apply it to normalized terms $a = |x_i|/\|x\|_p$ and $b = |y_i|/\|y\|_q$:

Multiplying through by $\|x\|_p\|y\|_q$ gives $\sum_i |x_i y_i| \leq \|x\|_p\|y\|_q$.

The Dual of $\ell^1$ Is $\ell^\infty$

Every bounded linear functional $f: \ell^1 \to \mathbb{R}$ satisfies $f(x) = \sum_i a_i x_i$ where $a_i = f(e_i)$ and $e_i$ is the $i$-th standard basis vector. Boundedness gives $|a_k| = |f(e_k)| \leq \|f\|\,\|e_k\|_1 = \|f\|$ for each $k$, so $(a_k) \in \ell^\infty$ with $\|(a_k)\|_\infty \leq \|f\|$. Conversely any $(a_k) \in \ell^\infty$ defines a bounded functional via this formula with $\|f\| = \|(a_k)\|_\infty$. The Hahn-Banach theorem guarantees functionals defined on subspaces extend to all of $\ell^1$, completing the isometric isomorphism $(\ell^1)^* \cong \ell^\infty$.

Closed Graph Theorem: Why Completeness Is Essential

Consider the identity $T = \mathrm{id}: (C([0,1]), \|\cdot\|_{L^1}) \to (C([0,1]), \|\cdot\|_\infty)$. Its graph is closed: if $f_n \to f$ in $L^1$ and $f_n \to g$ uniformly, then $f = g$ a.e. so the graph closes up. Yet $T$ is not bounded: take $f_n(x) = \min(nx, 1)$ truncated to a triangle of height $1$ and base $1/n$ — then $\|f_n\|_{L^1} = 1/2$ while $\|f_n\|_\infty = 1$ stays constant, but scaling to a spike gives $\|f_n\|_\infty / \|f_n\|_{L^1} \to \infty$. The closed graph theorem does not apply because the domain $(C([0,1]), \|\cdot\|_{L^1})$ is not complete — it is not a Banach space.

Connections

Where Your Intuition Breaks

The Uniform Boundedness Principle says: if a family of bounded linear operators is pointwise bounded (each operator is bounded on each individual input), then the operators are uniformly bounded (there is a single constant that bounds all of them). The surprising direction is the contrapositive: if no such uniform bound exists, there must be a single input $x$ where the operators are unbounded. The existence of such an $x$ is proved non-constructively via the Baire category theorem — the set of "bad" inputs is a residual set (countable intersection of dense open sets), which in a complete metric space is non-empty. This is why completeness of the domain is essential: the Baire argument fails in incomplete spaces, and there exist examples of pointwise-bounded families that are not uniformly bounded when the domain is not Banach.