ECE Ph.D. Qualifying Exam

Communication, Networking, Signal and Image Processing (CS)

Question 1: Probability and Random Processes

August 2001



3. (30 Points)

Let $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ be a sequence of random variables that are not necessarily statistically independent, but that each have identical mean $ \mu $ and variance $ \sigma^{2} $ . Let $ \mathbf{Y}_{1},\cdots,\mathbf{Y}_{n},\cdots $ be a sequence of random variable with $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}. $

(a)

Given that $ \mathbf{X}_{1},\cdots,\mathbf{X}_{n},\cdots $ are uncorrelated, determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.

(b)

Given that the covariance between $ \mathbf{X}_{j} $ and $ \mathbf{X}_{k} $ is given by
$ cov\left(\mathbf{X}_{j},\mathbf{X}_{k}\right)=\begin{cases} \begin{array}{lll} \sigma^{2} \text{, for }j=k\\ r\sigma^{2} \text{, for }\left|j-k\right|=1\\ 0 \text{, elsewhere, } \end{array}\end{cases} $
where $ -1\leq r\leq1 $ , determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.


Share and discuss your solutions below.


Solution 1

(a)

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}. $ 

$ E\left[\mathbf{Y}_{n}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\mu. $

$ E\left[\mathbf{Y}_{n}^{2}\right]=E\left[\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}\mathbf{X}_{k}\mathbf{X}_{l}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\mathbf{X}_{k}\mathbf{X}_{l}\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\mathbf{X}_{k}\right]E\left[\mathbf{X}_{l}\right] $$ =\frac{1}{n}\left(\mu^{2}+\sigma^{2}\right)+\frac{n\left(n-1\right)}{n^{2}}\mu^{2}=\frac{1}{n}\mu^{2}+\frac{1}{n}\sigma^{2}+\mu^{2}-\frac{1}{n}\mu^{2} $$ =\frac{\sigma^{2}}{n}+\mu^{2}. $

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\mathbf{Y}_{n}^{2}\right]-2E\left[\mathbf{Y}_{n}\right]\mu+\mu^{2}=\frac{\sigma^{2}}{n}+\mu^{2}-2\mu\cdot\mu+\mu^{2}=\frac{\sigma^{2}}{n}. $ $ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $

(b)

Given that the covariance between $ \mathbf{X}_{j} $ and $ \mathbf{X}_{k} $ is given by
$ cov\left(\mathbf{X}_{j},\mathbf{X}_{k}\right)=\begin{cases} \begin{array}{lll} \sigma^{2} \text{, for }j=k\\ r\sigma^{2} \text{, for }\left|j-k\right|=1\\ 0 \text{, elsewhere, } \end{array}\end{cases} $
where $ -1\leq r\leq1 $ , determine whether or not $ \left\{ \mathbf{Y}_{n}\right\} $ converges to $ \mu $ in the mean square sense.

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\left|\frac{1}{n}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right|^{2}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}. $

$ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{1}{n}\sigma^{2}+\frac{2\left(n-1\right)}{n^{2}}r\sigma^{2}\right)=0. $

Thus, $ \mathbf{Y}_{n} $ converges in the mean square sense to $ \mu $ .


Solution 2

(a)

$ E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=E\left[\left|\frac{1}{n}\sum_{k=1}^{n}\left(\mathbf{X}_{k}-\mu\right)\right|^{2}\right]=\frac{1}{n^{2}}\sum_{k=1}^{n}\sum_{l=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)\left(\mathbf{X}_{l}-\mu\right)\right] $$ =\frac{1}{n^{2}}\sum_{k=1}^{n}E\left[\left(\mathbf{X}_{k}-\mu\right)^{2}\right]+\frac{1}{n^{2}}\underset{k\neq l}{\sum_{k=1}^{n}\sum_{l=1}^{n}}E\left[\mathbf{X}_{k}-\mu\right]E\left[\mathbf{X}_{l}-\mu\right] $$ =\frac{1}{n^{2}}\cdot n\cdot\sigma^{2}+\frac{1}{n^{2}}\cdot n\left(n-1\right)\cdot0^{2}=\frac{\sigma^{2}}{n}. $

$ \lim_{n\rightarrow\infty}E\left[\left|\mathbf{Y}_{n}-\mu\right|^{2}\right]=\lim_{n\rightarrow\infty}\left(\frac{\sigma^{2}}{n}\right)=0. $


Alumni Liaison

Basic linear algebra uncovers and clarifies very important geometry and algebra.

Dr. Paul Garrett