Communication, Networking, Signal and Image Processing (CS)

Question 1: Probability and Random Processes

August 2005

1. (30 Points)

Assume that $\mathbf{X}$ is a binomial distributed random variable with probability mass function (pmf) given by $p_{n}\left(k\right)=\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n$ where $0<p<1$ .

(a)

Find the characteristic function of $\mathbf{X}$ . (You must show how you derive the characteristic function.)

$\Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{n}e^{i\omega k}\cdot\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}=\sum_{k=0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)\left(p\cdot e^{i\omega}\right)^{k}\left(1-p\right)^{n-k} $$=\left(p\cdot e^{i\omega}+1-p\right)^{n}. (b) Compute the standard deviation of \mathbf{X} . E\left[\mathbf{X}\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=n\left(pe^{i\omega}+1-p\right)^{n-1}\cdot pe^{i\omega}\Bigl|_{i\omega=0}=np. E\left[\mathbf{X}^{2}\right]=\frac{d^{2}}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=\frac{d}{di\omega}npe^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}$$ =np\left[\frac{d}{di\omega}e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}\right] $$=np\left[e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}+e^{i\omega}(n-1)\left(pe^{i\omega}+1-p\right)^{n-2}\cdot pe^{i\omega}\Bigl|_{i\omega=0}\right]$$ =np\left[1+\left(n-1\right)p\right]=np+n\left(n-1\right)p^{2}.$

$Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=np+n\left(n-1\right)p^{2}-\left(np\right)^{2}=np+n^{2}p^{2}-np^{2}-n^{2}p^{2}=np\left(1-p\right).$

$\therefore\sigma_{\mathbf{X}}=\sqrt{np\left(1-p\right)}.$

(c)

Find the value or values of $k$ for which $p_{n}\left(k\right)$ is maximum, and express the answer in terms of $p$ and $n$ . Give the most complete answer to this question that you can.

$\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}=\frac{\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}}{\left(\begin{array}{c} n\\ k-1 \end{array}\right)p^{k-1}\left(1-p\right)^{n-k+1}}=\frac{\frac{n!}{\left(n-k\right)!k!}p^{k}\left(1-p\right)^{n-k}}{\frac{n!}{\left(n-k+1\right)!\left(k-1\right)!}p^{k-1}\left(1-p\right)^{n-k+1}}$$=\frac{n!}{\left(n-k\right)!k!}\cdot\frac{\left(n-k+1\right)!\left(k-1\right)!}{n!}\cdot\frac{p}{1-p}=\frac{n-k+1}{k}\cdot\frac{p}{1-p}.$

$\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}$ is monotonically descreasing function of $k$ . Hence, $p_{n}\left(k\right)$ is maximized by the largest $k$ such that $\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}\geq1$ .$\left(n-k+1\right)p\geq k\left(1-p\right)\Longrightarrow np-kp+p\geq k-kp\Longrightarrow\left(n+1\right)p\geq k$. $k$ should be the integer. Thus, $k=\left\lfloor \left(n+1\right)p\right\rfloor$ maximize $p_{n}\left(k\right)$ . If $p_{n}\left(k\right)=p_{n}\left(k-1\right)$ , both $k=\left(n+1\right)p$ and $k=\left(n+1\right)p-1$ maximize $p_{n}\left(k\right)$ .

## Alumni Liaison

Correspondence Chess Grandmaster and Purdue Alumni

Prof. Dan Fleetwood