| Line 1: | Line 1: | ||
| − | 2.3 Weak law of large numbers | + | =2.3 Weak law of large numbers= |
| − | Let <math>\left\{ \mathbf{X}_{n}\right\}</math> be a sequence of <math>i.i.d.</math> random variables with mean <math>\mu</math> and variance <math>\sigma^{2}</math> . Define <math>\mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k},\quad n=1,2,\cdots</math> . Then for any <math>\epsilon>0</math> , <math>p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|>\epsilon\right\} \right)\rightarrow0</math> as <math>n\rightarrow\infty </math>(convergence in probability). | + | Let <math class="inline">\left\{ \mathbf{X}_{n}\right\}</math> be a sequence of <math class="inline">i.i.d.</math> random variables with mean <math class="inline">\mu</math> and variance <math class="inline">\sigma^{2}</math> . Define <math class="inline">\mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k},\quad n=1,2,\cdots</math> . Then for any <math class="inline">\epsilon>0</math> , <math class="inline">p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|>\epsilon\right\} \right)\rightarrow0</math> as <math class="inline">n\rightarrow\infty </math>(convergence in probability). |
| − | <math>\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.</math> | + | <math class="inline">\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.</math> |
Proof | Proof | ||
| − | <math>E\left[\mathbf{Y}_{n}\right]=E\left[\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\frac{1}{n}\cdot\left(n\mu\right)=\mu.</math> | + | <math class="inline">E\left[\mathbf{Y}_{n}\right]=E\left[\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\frac{1}{n}\cdot\left(n\mu\right)=\mu.</math> |
| − | <math>Var\left[\mathbf{Y}_{n}\right]</math> | + | <math class="inline">Var\left[\mathbf{Y}_{n}\right]</math> |
By the Chebyshev inequality, | By the Chebyshev inequality, | ||
| − | <math>p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|\geq\epsilon\right\} \right)\leq\frac{\sigma^{2}}{n\epsilon^{2}}\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow0.</math> | + | <math class="inline">p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|\geq\epsilon\right\} \right)\leq\frac{\sigma^{2}}{n\epsilon^{2}}\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow0.</math> |
| − | <math>\Longrightarrow\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.\blacksquare </math> | + | <math class="inline">\Longrightarrow\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.\blacksquare </math> |
Note | Note | ||
| Line 23: | Line 23: | ||
Note | Note | ||
| − | There are also stronger forms of the law of large numbers. Strong one uses coveriance <math>\left(a.e.\right)</math> as well as weak one uses coveriance <math>\left(p\right)</math> . | + | There are also stronger forms of the law of large numbers. Strong one uses coveriance <math class="inline">\left(a.e.\right)</math> as well as weak one uses coveriance <math class="inline">\left(p\right)</math> . |
---- | ---- | ||
Latest revision as of 11:39, 30 November 2010
2.3 Weak law of large numbers
Let $ \left\{ \mathbf{X}_{n}\right\} $ be a sequence of $ i.i.d. $ random variables with mean $ \mu $ and variance $ \sigma^{2} $ . Define $ \mathbf{Y}_{n}=\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k},\quad n=1,2,\cdots $ . Then for any $ \epsilon>0 $ , $ p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|>\epsilon\right\} \right)\rightarrow0 $ as $ n\rightarrow\infty $(convergence in probability).
$ \mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty. $
Proof
$ E\left[\mathbf{Y}_{n}\right]=E\left[\frac{1}{n}\sum_{k=1}^{n}\mathbf{X}_{k}\right]=\frac{1}{n}\sum_{k=1}^{n}E\left[\mathbf{X}_{k}\right]=\frac{1}{n}\cdot\left(n\mu\right)=\mu. $
$ Var\left[\mathbf{Y}_{n}\right] $
By the Chebyshev inequality,
$ p\left(\left\{ \left|\mathbf{Y}_{n}-\mu\right|\geq\epsilon\right\} \right)\leq\frac{\sigma^{2}}{n\epsilon^{2}}\longrightarrow\left(n\rightarrow\infty\right)\longrightarrow0. $
$ \Longrightarrow\mathbf{Y}_{n}\longrightarrow\left(p\right)\longrightarrow\mu\text{ as }n\longrightarrow\infty.\blacksquare $
Note
You can show this is true as long as the mean exists. The variance need not exist. Proof for this is harder and not responsible for this.
Note
There are also stronger forms of the law of large numbers. Strong one uses coveriance $ \left(a.e.\right) $ as well as weak one uses coveriance $ \left(p\right) $ .
