Wiener Khinchin Theorem

Abstract

A rigorous proof of the Wiener-Khinchin theorem, which establishes the fundamental relationship between the autocorrelation function and the spectral power density of a random process. This theorem is central to signal processing and statistical analysis, providing the mathematical foundation for understanding how random signals distribute their energy across different frequencies. The proof demonstrates the elegant connection between time-domain and frequency-domain representations of stochastic processes.

Keywords

Wiener Khinchin Theorem, spectral analysis, random processes

\(\require{cancel}\) \(\def\oot{\frac{1}{2}}\)

Consider a random variable \(x(t)\) which evolves with time. The auto correlation function is defined as: \[\begin{eqnarray} C(\tau)=\langle x(t) x(t+\tau)\rangle. \label{eq:autoc} \end{eqnarray}\] The Fourier transform of \(C(\tau)\) is defined as \[\begin{eqnarray} \hat C(\omega)=\int_{-\infty}^\infty d \tau e^{-i\omega \tau}C(\tau). \label{eq:autocF} \end{eqnarray}\]

Let us define the truncated Fourier transform of \(x(t)\) as \[\begin{eqnarray} \hat x_T(\omega)=\int_{-\frac{T}{2}}^{\frac{T}{2}} dt x(t) e^{-i\omega t}, \label{eq:xt} \end{eqnarray}\] and the truncated spectral power density as \[\begin{eqnarray} S_T(\omega)=\frac{1}{T}\langle |\hat x_T(\omega)|^2\rangle. \label{eq:swT} \end{eqnarray}\] The spectral power density is the limiting case of \(S_T(\omega)\):

\[\begin{eqnarray} S(\omega)=\lim_{T\to \infty}S_T(\omega)=\lim_{T\to \infty}\frac{1}{T}\langle |\hat x_T(\omega)|^2\rangle. \label{eq:sw} \end{eqnarray}\]

The Wiener-Khinchin Theorem states that if the limit in Eq. \(\ref{eq:sw}\) exists, then the spectral power density is the Fourier transform of the the auto correlation function, i.e., the following equality holds: \[\begin{eqnarray} S(\omega)=\int_{-\infty}^\infty d \tau e^{-i\omega \tau}C(\tau). \label{eq:WautocF} \end{eqnarray}\]

We start from the average of \(|\hat x_T(\omega)|^2\) \[\begin{eqnarray} |\hat x_T(\omega)|^2&=&\int_{-\frac{T}{2}}^{\frac{T}{2}} \int_{-\frac{T}{2}}^{\frac{T}{2}}dt' dt \langle x(t') x(t) \rangle e^{-iw(t'-t)}\nonumber\\ &=&\int_{-\frac{T}{2}}^{\frac{T}{2}} \int_{-\frac{T}{2}}^{\frac{T}{2}}dt' dt C(t'-t)e^{-i\omega(t'-t)}. \label{eq:WautocF2} \end{eqnarray}\] Note that \(C(t'-t)e^{-i\omega(t'-t)}\) depends only on the difference of the parameters.

The key insight is that we can change variables from \((t, t')\) to \((u, v)\) where \(u = t' - t\) and \(v = t' + t\). This transformation maps the square integration domain to a diamond-shaped domain, and since the integrand depends only on \(u = t' - t\), the integration over \(v\) gives the height of the integration region.

We want to compute the integral \(I=\int_{\frac{-T}{2}}^{\frac{T}{2}}\int_{\frac{-T}{2}}^{\frac{T}{2}} dt' dt f(t'-t)\).

The argument of the function begs for a change of coordinates:

\[\begin{eqnarray} u=t'-t, \quad \text{and} \quad v=t+t' \label{eq:trans}, \end{eqnarray}\] and the associated inverse transform reads: \[\begin{eqnarray} t'=\frac{u+v}{2}, \quad \text{and} \quad t=\frac{v-u}{2}. \label{eq:transi} \end{eqnarray}\]

This transformation will rotate and scale the integration domain as shown in Figure 1.

Figure 1: The integration domain in the \(t-t'\) domain (left) and \(u-v\) domain(right). Since there is no \(v\) dependence, \(v\) integration gives the height of the green and blue slices.

The equation of the top boundary on the right can be written as \(v=T-u\), and on the left as $ v= T+u$. We can actually combine them as \(v=T-|u|\). We can do the same analysis for the lower boundaries to see that the height of the slices at a given \(u\) is \(2(T-|u|)\). This will help us easily integrate \(v\) out as follows: \[\begin{eqnarray} I&=&\int_{\frac{-T}{2}}^{\frac{T}{2}}\int_{\frac{-T}{2}}^{\frac{T}{2}} dt' dt f(t'-t) =\iint_{S_{u,v}}\left|\frac{\partial(t,t')}{\partial(u,v)}\right| dv du f(u)\nonumber\\ &=&\int_{-T}^T 2(T-|u|) \times\frac{1}{2} dv du f(u)=\int_{-T}^T du f(u)(T-|u|) \label{eq:transeq}, \end{eqnarray}\] where \(\left|\frac{\partial(t,t')}{\partial(u,v)}\right|=\frac{1}{2}\) is the determinant of the Jacobian matrix associated with the transformation in Eq. \(\ref{eq:transi}\).

Therefore, setting \(u=\tau\), we get \[\begin{eqnarray} |\hat x_T(\omega)|^2&=& \int_{-T}^{T} d\tau e^{-i\omega\tau} C(\tau)(T-|\tau|). \label{eq:WautocFF2} \end{eqnarray}\] Taking the average we have the required result: \[\begin{eqnarray} S(\omega)&=&\lim_{T\to \infty}S_T(\omega)=\lim_{T\to \infty}\frac{1}{T}\langle |\hat x_T(\omega)|^2\rangle\nonumber\\ &=& \lim_{T\to \infty}\frac{1}{T}\int_{-T}^{T} d\tau e^{-i\omega\tau} C(\tau)(T-|\tau|)=\int_{-\infty}^{\infty} d\tau e^{-i\omega\tau} C(\tau), \label{eq:sww} \end{eqnarray}\] which completes the proof.