Continuity Theorem

(Levy's Continuity Theorem)

X(n)⇝Xif and only ifϕX(n)(t)→ϕX(t),∀t∈Rd.$${\mathit{X}}^{(n)}⇝\mathit{X}\text{if and only if}{\varphi }_{{\mathit{X}}^{(n)}}(\mathit{t})\to {\varphi }_{\mathit{X}}(\mathit{t}),\mathrm{\forall }\mathit{t}\in {\mathbb{R}}^d.$$

Proof. We assume that X(n)⇝X${\mathit{X}}^{(n)}⇝\mathit{X}$. Since exp(it⊤X)=cost⊤X+isint⊤X$\exp (i{\mathit{t}}^{\mathrm{\top }}\mathit{X})=\cos {\mathit{t}}^{\mathrm{\top }}\mathit{X}+i\sin {\mathit{t}}^{\mathrm{\top }}\mathit{X}$ we have that ϕ$\varphi$ is continuous and bounded as a function of X$\mathit{X}$, which together with implication 1⇒3$1\Rightarrow 3$ implies the pointwise convergence of the characteristic function.

Conversely, we assume that ∀t∈Rd,ϕX(n)(t)→ϕX(t)$\mathrm{\forall }\mathit{t}\in {\mathbb{R}}^d,{\varphi }_{{\mathit{X}}^{(n)}}(\mathit{t})\to {\varphi }_{\mathit{X}}(\mathit{t})$ and show that for any continuous function g$g$that is zero outside a bounded and closed set, we have E(g(X(n)))→E(g(X))$\mathsf{E}(g({\mathit{X}}^{(n)}))\to \mathsf{E}(g(\mathit{X}))$. Using the Portmanteau theorem, this implies that X(n)⇝X${\mathit{X}}^{(n)}⇝\mathit{X}$. Since g$g$ is continuous on a compact set, it is uniformly continuous and we can select for all ϵ>0$ϵ>0$ a δ>0$\delta >0$ such that ∥x−y∥<δ$\Vert \mathit{x}-\mathit{y}\Vert <\delta$ implies |g(x)−g(y)|<ϵ$|g(\mathit{x})-g(\mathit{y})|<ϵ$.

Denoting by Z$\mathit{Z}$ a N(0,σ2I)$N(\mathbf{0},{\sigma }^{2}I)$ random vector that is independent of X$\mathit{X}$ and the sequence X(n)${\mathit{X}}^{(n)}$, we have

|E(g(X(n)))−E(g(X))|=|E(g(X(n)))−E(g(X))+E(g(X(n)+Z))−E(g(X(n)+Z))+E(g(X+Z))−E(g(X+Z))|≤|E(g(X(n)))−E(g(X(n)+Z))|+|E(g(X(n)+Z))−E(g(X+Z))|+|E(g(X+Z))−E(g(X))|.$$\begin{array}{rl}& |\mathsf{E}(g({\mathit{X}}^{(n)}))-\mathsf{E}(g(\mathit{X}))|\\ & =|\mathsf{E}(g({\mathit{X}}^{(n)}))-\mathsf{E}(g(\mathit{X}))+\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))\\ & -\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))+\mathsf{E}(g(\mathit{X}+\mathit{Z}))-\mathsf{E}(g(\mathit{X}+\mathit{Z}))|\\ & \le |\mathsf{E}(g({\mathit{X}}^{(n)}))-\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))|+|\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))-\mathsf{E}(g(\mathit{X}+\mathit{Z}))|\\ & +|\mathsf{E}(g(\mathit{X}+\mathit{Z}))-\mathsf{E}(g(\mathit{X}))|.\end{array}$$

The first term above is bounded by 2ϵ$2ϵ$ since for σ$\sigma$ sufficiently small

|E(g(X(n)))−E(g(X(n)+Z))|≤E(|g(X(n)))−E(g(X(n)+Z))|I(∥Z∥≤δ)+E(|g(X(n)))−E(g(X(n)+Z))|I(∥Z∥>δ)≤E(ϵ)+2(supw|g(w)|)P(∥Z∥>δ)≤2ϵ.$$\begin{array}{rl}|\mathsf{E}(g({\mathit{X}}^{(n)}))-\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))|& \le \mathsf{E}(|g({\mathit{X}}^{(n)}))-\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))|I(\Vert \mathit{Z}\Vert \le \delta )\\ & +\mathsf{E}(|g({\mathit{X}}^{(n)}))-\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))|I(\Vert \mathit{Z}\Vert >\delta )\\ & \le \mathsf{E}(ϵ)+2\left(\underset{\mathit{w}}{sup}|g(\mathit{w})|\right)\mathsf{P}(\Vert \mathit{Z}\Vert >\delta )\\ & \le 2ϵ.\end{array}$$

The third term above is also bounded by 2ϵ$2ϵ$ due to a similar argument. It remains to show that the second term converges to zero: E(g(X(n)+Z))→E(g(X+Z))$\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))\to \mathsf{E}(g(\mathit{X}+\mathit{Z}))$. We will then have that |E(g(X(n)))−E(g(X))|→0$|\mathsf{E}(g({\mathit{X}}^{(n)}))-\mathsf{E}(g(\mathit{X}))|\to 0$, implying that E(g(X(n)))→E(g(X)$\mathsf{E}(g({\mathit{X}}^{(n)}))\to \mathsf{E}(g(\mathit{X})$ (for all continuous functions g$g$that are zero outside a bounded and closed set), which together with the Portmanteau theorem implies X(n)⇝X${\mathit{X}}^{(n)}⇝\mathit{X}$.

We show below that E(g(X(n)+Z))→E(g(X+Z))$\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))\to \mathsf{E}(g(\mathit{X}+\mathit{Z}))$. We have

E(g(X(n)+Z))=1(2π−−√σ)d∬g(x+z)exp(−z⊤z/(2σ2))dzdFX(n)=1(2π−−√σ)d∬g(u)exp(−(u−x)⊤(u−x)/(2σ2))dudFX(n)=1(2π−−√σ)d∬g(u)∏j=1dexp(−(uj−xj)22σ2)dudFX(n)=1(2π−−√σ)d∬g(u)∏j=1dσ2π−−√∫exp(itj(uj−xj)−σ2t2j/2)dtjdudFX(n)=1(2π)d∭g(u)exp(it⊤(u−x)−σ2t⊤t/2)dtdudFX(n)=1(2π)d∬g(u)exp(it⊤u−σ2t⊤t/2)ϕX(n)(−t)dtdu,(*)$$\begin{array}{rl}\text{(*)}& \mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))& =\frac{1}{(\sqrt{2\pi }\sigma {)}^d}\iint g(\mathit{x}+\mathit{z})\exp (-{\mathit{z}}^{\mathrm{\top }}\mathit{z}/(2{\sigma }^2))d\mathit{z}d{F}_{{\mathit{X}}^{(n)}}& =\frac{1}{(\sqrt{2\pi }\sigma {)}^d}\iint g(\mathit{u})\exp (-(\mathit{u}-\mathit{x}{)}^{\mathrm{\top }}(\mathit{u}-\mathit{x})/(2{\sigma }^2))d\mathit{u}d{F}_{{\mathit{X}}^{(n)}}\\ & =\frac{1}{(\sqrt{2\pi }\sigma {)}^d}\iint g(\mathit{u})\prod _{j=1}^d\exp \left(-\frac{(u_j-x_j{)}^2}{2{\sigma }^2}\right)d\mathit{u}d{F}_{{\mathit{X}}^{(n)}}\\ & =\frac{1}{(\sqrt{2\pi }\sigma {)}^d}\iint g(\mathit{u})\prod _{j=1}^d\frac{\sigma }{\sqrt{2\pi }}\int \exp \left(i{t}_j(u_j-x_j)-{\sigma }^2{t}_j^2/2\right)\\ & d{t}_{j}d\mathit{u}d{F}_{{\mathit{X}}^{(n)}}\\ & =\frac{1}{(2\pi {)}^d}\iiint g(\mathit{u})\exp \left(i{\mathit{t}}^{\mathrm{\top }}(\mathit{u}-\mathit{x})-{\sigma }^2{\mathit{t}}^{\mathrm{\top }}\mathit{t}/2\right)d\mathit{t}d\mathit{u}d{F}_{{\mathit{X}}^{(n)}}\\ & =\frac{1}{(2\pi {)}^d}\iint g(\mathit{u})\exp \left(i{\mathit{t}}^{\mathrm{\top }}\mathit{u}-{\sigma }^2{\mathit{t}}^{\mathrm{\top }}\mathit{t}/2\right){\varphi }_{{\mathit{X}}^{(n)}}(-\mathit{t})d\mathit{t}d\mathit{u},\end{array}$$

where u=x+z$\mathit{u}=\mathit{x}+\mathit{z}$. Note that we used Lemma 8.7.1 in the fourth equality and Proposition 8.7.2 in the last equality.

Since g$g$ is continuous and non-zero only on a closed and bounded set, g(u)$g(\mathit{u})$ may be made into a distribution by adding a constant to it and dividing by a constant. This implies that E(g(X(n)+Z))$\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))$may be considered as an expectation over a two random vectors U$\mathit{U}$ having density c(g(u)+b)$c(g(\mathit{u})+b)$ and T$\mathit{T}$have a Gaussian density. The argument of that expectation is the bounded function exp(it⊤u)ϕX(n)(−t)$\exp \left(i{\mathit{t}}^{\mathrm{\top }}\mathit{u}\right){\varphi }_{{\mathit{X}}^{(n)}}(-\mathit{t})$, and so by the dominated convergence theorem for random variables (Proposition 8.3.1)

1(2π)d∬g(u)exp(it⊤u−σ2t⊤t/2)ϕX(n)(−t)dtdu→1(2π)d∬g(u)exp(it⊤u−σ2t⊤t/2)ϕX(−t)dtdu.$$\begin{array}{c}\frac{1}{(2\pi {)}^d}\iint g(\mathit{u})\exp \left(i{\mathit{t}}^{\mathrm{\top }}\mathit{u}-{\sigma }^2{\mathit{t}}^{\mathrm{\top }}\mathit{t}/2\right){\varphi }_{{\mathit{X}}^{(n)}}(-\mathit{t})d\mathit{t}d\mathit{u}\hfill \\ \hfill \to \frac{1}{(2\pi {)}^d}\iint g(\mathit{u})\exp \left(i{\mathit{t}}^{\mathrm{\top }}\mathit{u}-{\sigma }^2{\mathit{t}}^{\mathrm{\top }}\mathit{t}/2\right){\varphi }_{\mathit{X}}(-\mathit{t})d\mathit{t}d\mathit{u}.\end{array}$$

Repeating the derivation in Equation (*) with X$\mathit{X}$ substituting X(n)${\mathit{X}}^{(n)}$ we see that

E(g(X(n)+Z))=1(2π)d∬g(u)exp(it⊤u−σ2t⊤t/2)ϕX(−t)dtdu,$$\begin{array}{}\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))=\frac{1}{(2\pi {)}^d}\iint g(\mathit{u})\exp \left(i{\mathit{t}}^{\mathrm{\top }}\mathit{u}-{\sigma }^2{\mathit{t}}^{\mathrm{\top }}\mathit{t}/2\right){\varphi }_{\mathit{X}}(-\mathit{t})d\mathit{t}d\mathit{u},\end{array}$$

implying that E(g(X(n)+Z))→E(g(X+Z))$\mathsf{E}(g({\mathit{X}}^{(n)}+\mathit{Z}))\to \mathsf{E}(g(\mathit{X}+\mathit{Z}))$.

Note that Levy's continuity theorem above is similar to Proposition 2.4.2. The former equates convergence in distribution to convergence of characteristic functions. The latter equates convergence in distribution to convergence of the moment generating functions. An advantage of Levy's theorem is that in many cases the moment generating function does not exist, while the characteristic function always exist.

The following result shows a way to prove multivariate convergence in distribution using a variety of unvariate convergence results.