Bounded Linear Operators: Adjoints, Spectrum & Compact Operators

Bounded linear operators between Hilbert spaces generalize matrices to infinite dimensions — their spectrum characterizes invertibility and dynamical behavior, while compact operators form a tractable subclass with a complete spectral theory that mirrors finite-dimensional eigendecomposition.

Concepts

A matrix acting on $\mathbb{R}^n$ can be diagonalized (if symmetric) or decomposed by singular values. Bounded linear operators generalize this to infinite-dimensional function spaces, where the question of when an operator is "diagonalizable" — and what its eigenvalues look like — becomes substantially more subtle. The spectrum of an operator replaces the eigenvalue list, but in infinite dimensions it can contain points that are neither eigenvalues nor their absence: a continuous spectrum where the operator is injective but not surjective, and a residual spectrum where the range is not even dense.

Bounded Linear Operators

A map $T: X \to Y$ between normed spaces is linear if $T(\alpha x + \beta y) = \alpha Tx + \beta Ty$. It is bounded if

$\|T\| = \sup_{\|x\| \leq 1} \|Tx\|_Y < \infty$

Boundedness is equivalent to continuity for linear maps. The space $B(X, Y)$ of bounded linear operators is itself a Banach space under this operator norm, and $B(X, X)$ is a Banach algebra under composition.

Adjoint. For $T \in B(H, K)$ between Hilbert spaces, the adjoint $T^*: K \to H$ is the unique operator satisfying $\langle Tx, y \rangle_K = \langle x, T^*y \rangle_H$ for all $x \in H$, $y \in K$. Self-adjoint operators satisfy $T = T^*$ and are the infinite-dimensional analogue of symmetric matrices.

The Spectrum

For $T \in B(X)$ (a bounded operator on a Banach space) and $\lambda \in \mathbb{C}$, the operator $T - \lambda I$ may fail to be invertible for three distinct reasons:

• Point spectrum $\sigma_p(T)$: $T - \lambda I$ is not injective — $\lambda$ is an eigenvalue
• Continuous spectrum $\sigma_c(T)$: $T - \lambda I$ is injective with dense range but not surjective (no inverse)
• Residual spectrum $\sigma_r(T)$: $T - \lambda I$ is injective but range is not dense

The spectrum $\sigma(T) = \sigma_p \cup \sigma_c \cup \sigma_r$ is always closed in $\mathbb{C}$ and non-empty. The resolvent set $\rho(T) = \mathbb{C} \setminus \sigma(T)$ is where $(T - \lambda I)^{-1}$ exists and is bounded.

For self-adjoint operators on Hilbert spaces, $\sigma(T) \subset \mathbb{R}$ and $\sigma_r(T) = \emptyset$ — the spectrum consists only of eigenvalues and continuous spectrum.

The three-part decomposition of the spectrum is forced by the definition of invertibility: an operator fails to be invertible if it is not injective (point spectrum), not surjective (residual or continuous spectrum depending on density of range). In finite dimensions, injective implies surjective for square maps, so this distinction collapses to eigenvalues vs. not. In infinite dimensions, injectivity and surjectivity decouple, creating the richer spectral structure — and the spectral type determines the dynamics of the associated differential operator.

Spectral Theorem for Compact Self-Adjoint Operators

An operator $T \in B(H)$ is compact if it maps bounded sets to precompact sets (sets whose closure is compact). Equivalently, if $x_n \rightharpoonup x$ weakly, then $Tx_n \to Tx$ in norm. Compact operators can be approximated by finite-rank operators.

Spectral theorem for compact self-adjoint operators. Let $T: H \to H$ be a compact self-adjoint operator on a Hilbert space. Then:

1. Every nonzero $\lambda \in \sigma(T)$ is an eigenvalue with finite-dimensional eigenspace
2. Eigenvalues are real and form a sequence $\lambda_1, \lambda_2, \ldots$ with $|\lambda_1| \geq |\lambda_2| \geq \cdots \to 0$
3. The eigenvectors $\{e_k\}$ corresponding to distinct eigenvalues are orthogonal and form a complete ONB for $\overline{\mathrm{ran}(T)}$
4. The spectral expansion holds: $Tx = \sum_k \lambda_k \langle x, e_k \rangle e_k$

The only possible accumulation point of eigenvalues is $0$. If $H$ is infinite-dimensional, $0$ must lie in the spectrum (but may not be an eigenvalue).

Hilbert-Schmidt Operators

An operator $T: L^2(X) \to L^2(X)$ defined by an integral kernel $k: X \times X \to \mathbb{R}$,

$Tf(x) = \int_X k(x, y)\,f(y)\,dy$

is Hilbert-Schmidt if $\|k\|_{L^2}^2 = \int\!\int |k(x,y)|^2\,dx\,dy < \infty$. Every Hilbert-Schmidt operator is compact.

Singular Value Decomposition for Operators

For a compact operator $T: H \to K$, the operator $T^*T: H \to H$ is compact and self-adjoint with non-negative eigenvalues. The singular values are $\sigma_k = \sqrt{\lambda_k(T^*T)}$, arranged in decreasing order $\sigma_1 \geq \sigma_2 \geq \cdots \to 0$. The operator SVD is

$Tf = \sum_k \sigma_k \langle f, v_k \rangle u_k$

where $\{v_k\}$ is an ONB for $\overline{\mathrm{ran}(T^*)}$ (right singular vectors) and $\{u_k\}$ is an ONB for $\overline{\mathrm{ran}(T)}$ (left singular vectors). Truncating this expansion at $r$ terms gives the best rank-$r$ approximation to $T$.

Fredholm alternative. For a compact operator $T$ and $\lambda \neq 0$, either $(T - \lambda I)$ is invertible (unique solution for every right-hand side), or the equation $(T - \lambda I)x = 0$ has a nonzero solution (eigenvalue). There is no third option — this is the exact analogue of the Fredholm alternative for matrices.

Worked Example

The Integral Operator Is Hilbert-Schmidt and Compact

Let $k: [0,1]^2 \to \mathbb{R}$ be square-integrable, and define $Tf(x) = \int_0^1 k(x,y)f(y)\,dy$ on $L^2([0,1])$.

Boundedness: By Cauchy-Schwarz, $|Tf(x)|^2 \leq \int_0^1 |k(x,y)|^2\,dy \cdot \|f\|^2$. Integrating over $x$:

$\|Tf\|^2 \leq \|k\|_{L^2}^2 \|f\|^2$

so $\|T\| \leq \|k\|_{L^2}$. Thus $T$ is bounded and Hilbert-Schmidt.

Compactness: Write $k(x,y) = \sum_{j,l} c_{jl}\,e_j(x)e_l(y)$ in any ONB $\{e_j\}$ of $L^2$. Truncating to $N$ terms gives a finite-rank operator $T_N$, and $\|T - T_N\| \leq \|k - k_N\|_{L^2} \to 0$ as $N \to \infty$. Since finite-rank operators are compact and the compact operators form a closed set under norm limits, $T$ is compact.

Eigenvalues via Separation of Variables (Separable Kernel)

For a separable kernel $k(x,y) = \phi(x)\psi(y)$, the operator $Tf(x) = \phi(x)\int_0^1 \psi(y)f(y)\,dy = \phi(x)\langle f, \overline{\psi}\rangle$ is rank-1. Its only nonzero eigenvalue is $\lambda = \langle \phi, \overline{\psi}\rangle = \int_0^1 \phi(y)\psi(y)\,dy$ with eigenvector $\phi/\|\phi\|$. The finite-rank SVD truncation applied to a general $k$ extends this idea: the $r$-term truncation is the sum of $r$ rank-1 operators, each from a pair of singular vectors.

Finite-Rank SVD Approximation

If $T$ is Hilbert-Schmidt with singular values $\sigma_1 \geq \sigma_2 \geq \cdots$, the best rank-$r$ approximation in operator norm is $T_r = \sum_{k=1}^r \sigma_k \langle \cdot, v_k\rangle u_k$, with approximation error $\|T - T_r\| = \sigma_{r+1}$. This is the operator analogue of matrix truncated SVD and justifies low-rank kernel approximations (Nyström method) in machine learning.

Connections

Where Your Intuition Breaks

The spectral theorem for compact self-adjoint operators looks like eigendecomposition, and in a sense it is — but the zero eigenvalue plays a special role that has no finite-dimensional analogue. In infinite dimensions, a compact operator $T$ must have $0$ in its spectrum even if $0$ is not an eigenvalue ($Tx = 0$ has only the trivial solution). This is because a compact operator cannot be invertible on an infinite-dimensional space: if it were, the unit ball would be compact (image of a bounded set under a bounded invertible map), but Riesz's theorem says the unit ball in an infinite-dimensional normed space is never compact. The "invisibility" of $0$ in the point spectrum does not mean $T$ is invertible — it means the failure of invertibility is more subtle than the existence of a kernel.