| Line 3: | Line 3: | ||
'''1. (30 Points)''' | '''1. (30 Points)''' | ||
| − | Assume that <math>\mathbf{X}</math> is a binomial distributed random variable with probability mass function (pmf) given by <math>p_{n}\left(k\right)=\left(\begin{array}{c} | + | Assume that <math class="inline">\mathbf{X}</math> is a binomial distributed random variable with probability mass function (pmf) given by <math class="inline">p_{n}\left(k\right)=\left(\begin{array}{c} |
n\\ | n\\ | ||
k | k | ||
| − | \end{array}\right)p^{k}\left(1-p\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n,</math><math> where <math>0<p<1</math> . | + | \end{array}\right)p^{k}\left(1-p\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n,</math><math class="inline"> where <math class="inline">0<p<1</math> . |
'''(a)''' | '''(a)''' | ||
| − | Find the characteristic function of <math>\mathbf{X}</math> . (You must show how you derive the characteristic function.) | + | Find the characteristic function of <math class="inline">\mathbf{X}</math> . (You must show how you derive the characteristic function.) |
| − | <math>\Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{n}e^{i\omega k}\cdot\left(\begin{array}{c} | + | <math class="inline">\Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{n}e^{i\omega k}\cdot\left(\begin{array}{c} |
n\\ | n\\ | ||
k | k | ||
| Line 18: | Line 18: | ||
n\\ | n\\ | ||
k | k | ||
| − | \end{array}\right)\left(p\cdot e^{i\omega}\right)^{k}\left(1-p\right)^{n-k}</math><math>=\left(p\cdot e^{i\omega}+1-p\right)^{n}.</math> | + | \end{array}\right)\left(p\cdot e^{i\omega}\right)^{k}\left(1-p\right)^{n-k}</math><math class="inline">=\left(p\cdot e^{i\omega}+1-p\right)^{n}.</math> |
'''(b)''' | '''(b)''' | ||
| − | Compute the standard deviation of <math>\mathbf{X}</math> . | + | Compute the standard deviation of <math class="inline">\mathbf{X}</math> . |
| − | <math>E\left[\mathbf{X}\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=n\left(pe^{i\omega}+1-p\right)^{n-1}\cdot pe^{i\omega}\Bigl|_{i\omega=0}=np.</math> | + | <math class="inline">E\left[\mathbf{X}\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=n\left(pe^{i\omega}+1-p\right)^{n-1}\cdot pe^{i\omega}\Bigl|_{i\omega=0}=np.</math> |
| − | <math>E\left[\mathbf{X}^{2}\right]=\frac{d^{2}}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=\frac{d}{di\omega}npe^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}</math><math>=np\left[\frac{d}{di\omega}e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}\right]</math><math>=np\left[e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}+e^{i\omega}(n-1)\left(pe^{i\omega}+1-p\right)^{n-2}\cdot pe^{i\omega}\Bigl|_{i\omega=0}\right]</math><math>=np\left[1+\left(n-1\right)p\right]=np+n\left(n-1\right)p^{2}.</math> | + | <math class="inline">E\left[\mathbf{X}^{2}\right]=\frac{d^{2}}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=\frac{d}{di\omega}npe^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}</math><math class="inline">=np\left[\frac{d}{di\omega}e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}\right]</math><math class="inline">=np\left[e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}+e^{i\omega}(n-1)\left(pe^{i\omega}+1-p\right)^{n-2}\cdot pe^{i\omega}\Bigl|_{i\omega=0}\right]</math><math class="inline">=np\left[1+\left(n-1\right)p\right]=np+n\left(n-1\right)p^{2}.</math> |
| − | <math>Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=np+n\left(n-1\right)p^{2}-\left(np\right)^{2}=np+n^{2}p^{2}-np^{2}-n^{2}p^{2}=np\left(1-p\right).</math> | + | <math class="inline">Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=np+n\left(n-1\right)p^{2}-\left(np\right)^{2}=np+n^{2}p^{2}-np^{2}-n^{2}p^{2}=np\left(1-p\right).</math> |
| − | <math>\therefore\sigma_{\mathbf{X}}=\sqrt{np\left(1-p\right)}.</math> | + | <math class="inline">\therefore\sigma_{\mathbf{X}}=\sqrt{np\left(1-p\right)}.</math> |
'''(c)''' | '''(c)''' | ||
| − | Find the value or values of <math>k</math> for which <math>p_{n}\left(k\right)</math> is maximum, and express the answer in terms of <math>p</math> and <math>n</math> . Give the most complete answer to this question that you can. | + | Find the value or values of <math class="inline">k</math> for which <math class="inline">p_{n}\left(k\right)</math> is maximum, and express the answer in terms of <math class="inline">p</math> and <math class="inline">n</math> . Give the most complete answer to this question that you can. |
| − | <math>\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}=\frac{\left(\begin{array}{c} | + | <math class="inline">\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}=\frac{\left(\begin{array}{c} |
n\\ | n\\ | ||
k | k | ||
| Line 42: | Line 42: | ||
n\\ | n\\ | ||
k-1 | k-1 | ||
| − | \end{array}\right)p^{k-1}\left(1-p\right)^{n-k+1}}=\frac{\frac{n!}{\left(n-k\right)!k!}p^{k}\left(1-p\right)^{n-k}}{\frac{n!}{\left(n-k+1\right)!\left(k-1\right)!}p^{k-1}\left(1-p\right)^{n-k+1}}</math><math>=\frac{n!}{\left(n-k\right)!k!}\cdot\frac{\left(n-k+1\right)!\left(k-1\right)!}{n!}\cdot\frac{p}{1-p}=\frac{n-k+1}{k}\cdot\frac{p}{1-p}.</math> | + | \end{array}\right)p^{k-1}\left(1-p\right)^{n-k+1}}=\frac{\frac{n!}{\left(n-k\right)!k!}p^{k}\left(1-p\right)^{n-k}}{\frac{n!}{\left(n-k+1\right)!\left(k-1\right)!}p^{k-1}\left(1-p\right)^{n-k+1}}</math><math class="inline">=\frac{n!}{\left(n-k\right)!k!}\cdot\frac{\left(n-k+1\right)!\left(k-1\right)!}{n!}\cdot\frac{p}{1-p}=\frac{n-k+1}{k}\cdot\frac{p}{1-p}.</math> |
| − | <math>\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}</math> is monotonically descreasing function of <math>k</math> . Hence, <math>p_{n}\left(k\right)</math> is maximized by the largest <math>k</math> such that <math>\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}\geq1</math> .<math>\left(n-k+1\right)p\geq k\left(1-p\right)\Longrightarrow np-kp+p\geq k-kp\Longrightarrow\left(n+1\right)p\geq k</math>. <math>k</math> should be the integer. Thus, <math>k=\left\lfloor \left(n+1\right)p\right\rfloor</math> maximize <math>p_{n}\left(k\right)</math> . If <math>p_{n}\left(k\right)=p_{n}\left(k-1\right)</math> , both <math>k=\left(n+1\right)p</math> and <math>k=\left(n+1\right)p-1</math> maximize <math>p_{n}\left(k\right)</math> . | + | <math class="inline">\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}</math> is monotonically descreasing function of <math class="inline">k</math> . Hence, <math class="inline">p_{n}\left(k\right)</math> is maximized by the largest <math class="inline">k</math> such that <math class="inline">\frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}\geq1</math> .<math class="inline">\left(n-k+1\right)p\geq k\left(1-p\right)\Longrightarrow np-kp+p\geq k-kp\Longrightarrow\left(n+1\right)p\geq k</math>. <math class="inline">k</math> should be the integer. Thus, <math class="inline">k=\left\lfloor \left(n+1\right)p\right\rfloor</math> maximize <math class="inline">p_{n}\left(k\right)</math> . If <math class="inline">p_{n}\left(k\right)=p_{n}\left(k-1\right)</math> , both <math class="inline">k=\left(n+1\right)p</math> and <math class="inline">k=\left(n+1\right)p-1</math> maximize <math class="inline">p_{n}\left(k\right)</math> . |
'''2. (30 Points)''' | '''2. (30 Points)''' | ||
| − | Let <math>\mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots</math> be a sequence of binomially distributed random variables, with <math>\mathbf{X}_{n}</math> having probability mass function <math>p_{n}\left(k\right)=\left(\begin{array}{c} | + | Let <math class="inline">\mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots</math> be a sequence of binomially distributed random variables, with <math class="inline">\mathbf{X}_{n}</math> having probability mass function <math class="inline">p_{n}\left(k\right)=\left(\begin{array}{c} |
n\\ | n\\ | ||
k | k | ||
| − | \end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n,</math> where <math>0<p_{n}<1</math> for all <math>n=1,2,3,\cdots</math> . Show that if <math>np_{n}\rightarrow\lambda\text{ as }n\rightarrow\infty,</math> then the random sequence <math>\mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots</math> converges in distribution to a Poisson random variable having mean <math>\lambda</math> . | + | \end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n,</math> where <math class="inline">0<p_{n}<1</math> for all <math class="inline">n=1,2,3,\cdots</math> . Show that if <math class="inline">np_{n}\rightarrow\lambda\text{ as }n\rightarrow\infty,</math> then the random sequence <math class="inline">\mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots</math> converges in distribution to a Poisson random variable having mean <math class="inline">\lambda</math> . |
| − | <math>\Phi_{\mathbf{X}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}_{n}}\right]=\sum_{k=0}^{n}e^{i\omega k}\left(\begin{array}{c} | + | <math class="inline">\Phi_{\mathbf{X}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}_{n}}\right]=\sum_{k=0}^{n}e^{i\omega k}\left(\begin{array}{c} |
n\\ | n\\ | ||
k | k | ||
| Line 59: | Line 59: | ||
n\\ | n\\ | ||
k | k | ||
| − | \end{array}\right)\left(e^{i\omega}p_{n}\right)^{k}\left(1-p_{n}\right)^{n-k}</math><math>=\left(e^{i\omega}p_{n}+1-p_{n}\right)^{n}=\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}.</math> | + | \end{array}\right)\left(e^{i\omega}p_{n}\right)^{k}\left(1-p_{n}\right)^{n-k}</math><math class="inline">=\left(e^{i\omega}p_{n}+1-p_{n}\right)^{n}=\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}.</math> |
| − | Since <math>np_{n}\rightarrow\lambda</math> as <math>n\rightarrow\infty</math> , <math>p_{n}\rightarrow\lambda/n</math> . <math>\lim_{n\rightarrow\infty}\Phi_{\mathbf{X}_{n}}\left(\omega\right)=\lim_{n\rightarrow\infty}\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}=\lim_{n\rightarrow\infty}\left(1-\frac{\lambda}{n}\left(e^{i\omega}-1\right)\right)^{n}=e^{\lambda\left(e^{i\omega}-1\right)}.</math> | + | Since <math class="inline">np_{n}\rightarrow\lambda</math> as <math class="inline">n\rightarrow\infty</math> , <math class="inline">p_{n}\rightarrow\lambda/n</math> . <math class="inline">\lim_{n\rightarrow\infty}\Phi_{\mathbf{X}_{n}}\left(\omega\right)=\lim_{n\rightarrow\infty}\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}=\lim_{n\rightarrow\infty}\left(1-\frac{\lambda}{n}\left(e^{i\omega}-1\right)\right)^{n}=e^{\lambda\left(e^{i\omega}-1\right)}.</math> |
| − | <math>\because\lim_{n\rightarrow\infty}\left(1+\frac{x}{n}\right)^{n}=e^{x}.</math> | + | <math class="inline">\because\lim_{n\rightarrow\infty}\left(1+\frac{x}{n}\right)^{n}=e^{x}.</math> |
| − | <math>e^{\lambda\left(e^{i\omega}-1\right)}</math> is the characteristic function of a Poisson random variable with mean <math>\lambda</math> . Thus, as <math>n\rightarrow\infty</math> , <math>\mathbf{X}_{n}</math> converges in distribution to a Poisson random variable with mean <math>\lambda</math> . | + | <math class="inline">e^{\lambda\left(e^{i\omega}-1\right)}</math> is the characteristic function of a Poisson random variable with mean <math class="inline">\lambda</math> . Thus, as <math class="inline">n\rightarrow\infty</math> , <math class="inline">\mathbf{X}_{n}</math> converges in distribution to a Poisson random variable with mean <math class="inline">\lambda</math> . |
'''3. (40 Points)''' | '''3. (40 Points)''' | ||
| − | Consider a homogeneous Poisson point process with rate <math>\lambda</math> and points (event occurrence times) <math>\mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots</math> . | + | Consider a homogeneous Poisson point process with rate <math class="inline">\lambda</math> and points (event occurrence times) <math class="inline">\mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots</math> . |
'''(a)''' | '''(a)''' | ||
| − | Derive the pdf <math>f_{k}\left(t\right)</math> of the <math>k</math> -th point <math>\mathbf{T}_{k}</math> for arbitrary <math>k</math> . | + | Derive the pdf <math class="inline">f_{k}\left(t\right)</math> of the <math class="inline">k</math> -th point <math class="inline">\mathbf{T}_{k}</math> for arbitrary <math class="inline">k</math> . |
• The cdf is | • The cdf is | ||
| − | <math>F_{\mathbf{T}_{k}}\left(t\right)=P\left(\mathbf{T}_{k}\leq t\right)=P\left(\text{at least }k\text{ points within }t\right)=\sum_{j=k}^{\infty}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}</math><math>=1-P\left(\mathbf{T}_{k}>t\right)=1-P\left(\text{less than }k\text{ points within }t\right)=1-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}.</math> | + | <math class="inline">F_{\mathbf{T}_{k}}\left(t\right)=P\left(\mathbf{T}_{k}\leq t\right)=P\left(\text{at least }k\text{ points within }t\right)=\sum_{j=k}^{\infty}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}</math><math class="inline">=1-P\left(\mathbf{T}_{k}>t\right)=1-P\left(\text{less than }k\text{ points within }t\right)=1-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}.</math> |
• The pdf by differentiating the cdf | • The pdf by differentiating the cdf | ||
| − | <math>isf_{\mathbf{T}_{k}}\left(t\right)=\frac{dF_{\mathbf{T}_{k}}}{dt}=-\sum_{j=0}^{k-1}\frac{\left(-\lambda\right)e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot j\left(\lambda t\right)^{j-1}\cdot\lambda}{j!}</math><math>=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{j\left(\lambda t\right)^{j-1}}{j!}=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{\left(\lambda t\right)^{j-1}}{\left(j-1\right)!}</math><math>=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=0}^{k-2}\frac{\left(\lambda t\right)^{j}}{j!}=\lambda e^{-\lambda t}\frac{\left(\lambda t\right)^{k-1}}{\left(k-1\right)!}.</math> | + | <math class="inline">isf_{\mathbf{T}_{k}}\left(t\right)=\frac{dF_{\mathbf{T}_{k}}}{dt}=-\sum_{j=0}^{k-1}\frac{\left(-\lambda\right)e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot j\left(\lambda t\right)^{j-1}\cdot\lambda}{j!}</math><math class="inline">=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{j\left(\lambda t\right)^{j-1}}{j!}=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{\left(\lambda t\right)^{j-1}}{\left(j-1\right)!}</math><math class="inline">=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=0}^{k-2}\frac{\left(\lambda t\right)^{j}}{j!}=\lambda e^{-\lambda t}\frac{\left(\lambda t\right)^{k-1}}{\left(k-1\right)!}.</math> |
– This is Erlang distribution. | – This is Erlang distribution. | ||
| Line 85: | Line 85: | ||
'''(b)''' | '''(b)''' | ||
| − | What kind of distribution does <math>\mathbf{T}_{1}</math> have? | + | What kind of distribution does <math class="inline">\mathbf{T}_{1}</math> have? |
| − | • If <math>k=1</math> , then <math>f_{\mathbf{T}_{1}}\left(t\right)=\lambda e^{-\lambda t}</math> . Thus <math>\mathbf{T}_{1}</math> is a exponential random variable with parameter <math>\lambda</math> . | + | • If <math class="inline">k=1</math> , then <math class="inline">f_{\mathbf{T}_{1}}\left(t\right)=\lambda e^{-\lambda t}</math> . Thus <math class="inline">\mathbf{T}_{1}</math> is a exponential random variable with parameter <math class="inline">\lambda</math> . |
'''(c)''' | '''(c)''' | ||
| − | What is the conditional pdf of <math>\mathbf{T}_{k}</math> given <math>\mathbf{T}_{k-1}=t_{0}</math> , where <math>t_{0}>0</math> ? (You can give the answer without derivation if you know it.) | + | What is the conditional pdf of <math class="inline">\mathbf{T}_{k}</math> given <math class="inline">\mathbf{T}_{k-1}=t_{0}</math> , where <math class="inline">t_{0}>0</math> ? (You can give the answer without derivation if you know it.) |
• The conditional cdf is | • The conditional cdf is | ||
| − | <math>F_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(\mathbf{T}_{k}\leq t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(N\left(t_{k},t_{0}\right)\geq1\right)</math><math>=1-P\left(N\left(t_{k},t_{0}\right)=0\right)=1-e^{-\lambda\left(t_{k}-t_{0}\right)}.</math> | + | <math class="inline">F_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(\mathbf{T}_{k}\leq t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(N\left(t_{k},t_{0}\right)\geq1\right)</math><math class="inline">=1-P\left(N\left(t_{k},t_{0}\right)=0\right)=1-e^{-\lambda\left(t_{k}-t_{0}\right)}.</math> |
• The conditional pdf by differentiating the conditional cdf is | • The conditional pdf by differentiating the conditional cdf is | ||
| − | <math>\therefore f_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=\lambda e^{-\lambda\left(t_{k}-t_{0}\right)}.</math> | + | <math class="inline">\therefore f_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=\lambda e^{-\lambda\left(t_{k}-t_{0}\right)}.</math> |
'''(d)''' | '''(d)''' | ||
| − | Suppose you have a random number generator that produces independent, identically distributed (i.i.d. ) random variables <math>\mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots</math> that are uniformaly distributed on the interval <math>\left(0,1\right)</math> . Explain how you could use these to simulate the Poisson points <math>\mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots</math> describe above. Provide as complete an explanation as possible. | + | Suppose you have a random number generator that produces independent, identically distributed (i.i.d. ) random variables <math class="inline">\mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots</math> that are uniformaly distributed on the interval <math class="inline">\left(0,1\right)</math> . Explain how you could use these to simulate the Poisson points <math class="inline">\mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots</math> describe above. Provide as complete an explanation as possible. |
• This problem is similar to [[ECE600 QE 2003 August|QE 2003 August]] Problem 2. | • This problem is similar to [[ECE600 QE 2003 August|QE 2003 August]] Problem 2. | ||
Revision as of 07:32, 1 December 2010
7.10 QE 2005 August
1. (30 Points)
Assume that $ \mathbf{X} $ is a binomial distributed random variable with probability mass function (pmf) given by $ p_{n}\left(k\right)=\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n, $$ where <math class="inline">0<p<1 $ .
(a)
Find the characteristic function of $ \mathbf{X} $ . (You must show how you derive the characteristic function.)
$ \Phi_{\mathbf{X}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}}\right]=\sum_{k=0}^{n}e^{i\omega k}\cdot\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}=\sum_{k=0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)\left(p\cdot e^{i\omega}\right)^{k}\left(1-p\right)^{n-k} $$ =\left(p\cdot e^{i\omega}+1-p\right)^{n}. $
(b)
Compute the standard deviation of $ \mathbf{X} $ .
$ E\left[\mathbf{X}\right]=\frac{d}{di\omega}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=n\left(pe^{i\omega}+1-p\right)^{n-1}\cdot pe^{i\omega}\Bigl|_{i\omega=0}=np. $
$ E\left[\mathbf{X}^{2}\right]=\frac{d^{2}}{d\left(i\omega\right)^{2}}\Phi_{\mathbf{X}}\left(\omega\right)\Bigl|_{i\omega=0}=\frac{d}{di\omega}npe^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0} $$ =np\left[\frac{d}{di\omega}e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}\Bigl|_{i\omega=0}\right] $$ =np\left[e^{i\omega}\left(pe^{i\omega}+1-p\right)^{n-1}+e^{i\omega}(n-1)\left(pe^{i\omega}+1-p\right)^{n-2}\cdot pe^{i\omega}\Bigl|_{i\omega=0}\right] $$ =np\left[1+\left(n-1\right)p\right]=np+n\left(n-1\right)p^{2}. $
$ Var\left[\mathbf{X}\right]=E\left[\mathbf{X}^{2}\right]-\left(E\left[\mathbf{X}\right]\right)^{2}=np+n\left(n-1\right)p^{2}-\left(np\right)^{2}=np+n^{2}p^{2}-np^{2}-n^{2}p^{2}=np\left(1-p\right). $
$ \therefore\sigma_{\mathbf{X}}=\sqrt{np\left(1-p\right)}. $
(c)
Find the value or values of $ k $ for which $ p_{n}\left(k\right) $ is maximum, and express the answer in terms of $ p $ and $ n $ . Give the most complete answer to this question that you can.
$ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}=\frac{\left(\begin{array}{c} n\\ k \end{array}\right)p^{k}\left(1-p\right)^{n-k}}{\left(\begin{array}{c} n\\ k-1 \end{array}\right)p^{k-1}\left(1-p\right)^{n-k+1}}=\frac{\frac{n!}{\left(n-k\right)!k!}p^{k}\left(1-p\right)^{n-k}}{\frac{n!}{\left(n-k+1\right)!\left(k-1\right)!}p^{k-1}\left(1-p\right)^{n-k+1}} $$ =\frac{n!}{\left(n-k\right)!k!}\cdot\frac{\left(n-k+1\right)!\left(k-1\right)!}{n!}\cdot\frac{p}{1-p}=\frac{n-k+1}{k}\cdot\frac{p}{1-p}. $
$ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)} $ is monotonically descreasing function of $ k $ . Hence, $ p_{n}\left(k\right) $ is maximized by the largest $ k $ such that $ \frac{p_{n}\left(k\right)}{p_{n}\left(k-1\right)}\geq1 $ .$ \left(n-k+1\right)p\geq k\left(1-p\right)\Longrightarrow np-kp+p\geq k-kp\Longrightarrow\left(n+1\right)p\geq k $. $ k $ should be the integer. Thus, $ k=\left\lfloor \left(n+1\right)p\right\rfloor $ maximize $ p_{n}\left(k\right) $ . If $ p_{n}\left(k\right)=p_{n}\left(k-1\right) $ , both $ k=\left(n+1\right)p $ and $ k=\left(n+1\right)p-1 $ maximize $ p_{n}\left(k\right) $ .
2. (30 Points)
Let $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots $ be a sequence of binomially distributed random variables, with $ \mathbf{X}_{n} $ having probability mass function $ p_{n}\left(k\right)=\left(\begin{array}{c} n\\ k \end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}\;,\qquad k=0,1,2,\cdots,n, $ where $ 0<p_{n}<1 $ for all $ n=1,2,3,\cdots $ . Show that if $ np_{n}\rightarrow\lambda\text{ as }n\rightarrow\infty, $ then the random sequence $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots $ converges in distribution to a Poisson random variable having mean $ \lambda $ .
$ \Phi_{\mathbf{X}_{n}}\left(\omega\right)=E\left[e^{i\omega\mathbf{X}_{n}}\right]=\sum_{k=0}^{n}e^{i\omega k}\left(\begin{array}{c} n\\ k \end{array}\right)p_{n}^{k}\left(1-p_{n}\right)^{n-k}=\sum_{k=0}^{n}\left(\begin{array}{c} n\\ k \end{array}\right)\left(e^{i\omega}p_{n}\right)^{k}\left(1-p_{n}\right)^{n-k} $$ =\left(e^{i\omega}p_{n}+1-p_{n}\right)^{n}=\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}. $
Since $ np_{n}\rightarrow\lambda $ as $ n\rightarrow\infty $ , $ p_{n}\rightarrow\lambda/n $ . $ \lim_{n\rightarrow\infty}\Phi_{\mathbf{X}_{n}}\left(\omega\right)=\lim_{n\rightarrow\infty}\left(1-p_{n}\left(e^{i\omega}-1\right)\right)^{n}=\lim_{n\rightarrow\infty}\left(1-\frac{\lambda}{n}\left(e^{i\omega}-1\right)\right)^{n}=e^{\lambda\left(e^{i\omega}-1\right)}. $
$ \because\lim_{n\rightarrow\infty}\left(1+\frac{x}{n}\right)^{n}=e^{x}. $
$ e^{\lambda\left(e^{i\omega}-1\right)} $ is the characteristic function of a Poisson random variable with mean $ \lambda $ . Thus, as $ n\rightarrow\infty $ , $ \mathbf{X}_{n} $ converges in distribution to a Poisson random variable with mean $ \lambda $ .
3. (40 Points)
Consider a homogeneous Poisson point process with rate $ \lambda $ and points (event occurrence times) $ \mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots $ .
(a)
Derive the pdf $ f_{k}\left(t\right) $ of the $ k $ -th point $ \mathbf{T}_{k} $ for arbitrary $ k $ .
• The cdf is $ F_{\mathbf{T}_{k}}\left(t\right)=P\left(\mathbf{T}_{k}\leq t\right)=P\left(\text{at least }k\text{ points within }t\right)=\sum_{j=k}^{\infty}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!} $$ =1-P\left(\mathbf{T}_{k}>t\right)=1-P\left(\text{less than }k\text{ points within }t\right)=1-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}. $
• The pdf by differentiating the cdf $ isf_{\mathbf{T}_{k}}\left(t\right)=\frac{dF_{\mathbf{T}_{k}}}{dt}=-\sum_{j=0}^{k-1}\frac{\left(-\lambda\right)e^{-\lambda t}\cdot\left(\lambda t\right)^{j}}{j!}-\sum_{j=0}^{k-1}\frac{e^{-\lambda t}\cdot j\left(\lambda t\right)^{j-1}\cdot\lambda}{j!} $$ =\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{j\left(\lambda t\right)^{j-1}}{j!}=\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=1}^{k-1}\frac{\left(\lambda t\right)^{j-1}}{\left(j-1\right)!} $$ =\lambda e^{-\lambda t}\sum_{j=0}^{k-1}\frac{\left(\lambda t\right)^{j}}{j!}-\lambda e^{-\lambda t}\sum_{j=0}^{k-2}\frac{\left(\lambda t\right)^{j}}{j!}=\lambda e^{-\lambda t}\frac{\left(\lambda t\right)^{k-1}}{\left(k-1\right)!}. $
– This is Erlang distribution.
(b)
What kind of distribution does $ \mathbf{T}_{1} $ have?
• If $ k=1 $ , then $ f_{\mathbf{T}_{1}}\left(t\right)=\lambda e^{-\lambda t} $ . Thus $ \mathbf{T}_{1} $ is a exponential random variable with parameter $ \lambda $ .
(c)
What is the conditional pdf of $ \mathbf{T}_{k} $ given $ \mathbf{T}_{k-1}=t_{0} $ , where $ t_{0}>0 $ ? (You can give the answer without derivation if you know it.)
• The conditional cdf is
$ F_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(\mathbf{T}_{k}\leq t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=P\left(N\left(t_{k},t_{0}\right)\geq1\right) $$ =1-P\left(N\left(t_{k},t_{0}\right)=0\right)=1-e^{-\lambda\left(t_{k}-t_{0}\right)}. $
• The conditional pdf by differentiating the conditional cdf is
$ \therefore f_{\mathbf{T}_{k}}\left(t_{k}|\mathbf{T}_{k-1}=t_{0}\right)=\lambda e^{-\lambda\left(t_{k}-t_{0}\right)}. $
(d)
Suppose you have a random number generator that produces independent, identically distributed (i.i.d. ) random variables $ \mathbf{X}_{1},\mathbf{X}_{2},\cdots,\mathbf{X}_{n},\cdots $ that are uniformaly distributed on the interval $ \left(0,1\right) $ . Explain how you could use these to simulate the Poisson points $ \mathbf{T}_{1},\mathbf{T}_{2},\cdots,\mathbf{T}_{n},\cdots $ describe above. Provide as complete an explanation as possible.
• This problem is similar to QE 2003 August Problem 2.
