m (Protected "ECE600 F13 rv definition mhossain" [edit=sysop:move=sysop]) |
|||
| (14 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
[[Category:ECE600]] | [[Category:ECE600]] | ||
[[Category:Lecture notes]] | [[Category:Lecture notes]] | ||
| + | |||
| + | [[ECE600_F13_notes_mhossain|Back to all ECE 600 notes]]<br/> | ||
| + | [[ECE600_F13_Statistical_Independence_mhossain|Previous Topic: Statistical Independence]]<br/> | ||
| + | [[ECE600_F13_rv_distribution_mhossain|Next Topic: Random Variables: Distributions]] | ||
| + | ---- | ||
| + | [[Category:ECE600]] | ||
| + | [[Category:probability]] | ||
| + | [[Category:lecture notes]] | ||
| + | [[Category:slecture]] | ||
<center><font size= 4> | <center><font size= 4> | ||
| − | '''Random Variables and Signals''' | + | [[ECE600_F13_notes_mhossain|'''The Comer Lectures on Random Variables and Signals''']] |
</font size> | </font size> | ||
| + | |||
| + | [https://www.projectrhea.org/learning/slectures.php Slectures] by [[user:Mhossain | Maliha Hossain]] | ||
| + | |||
<font size= 3> Topic 5: Random Variables: Definition</font size> | <font size= 3> Topic 5: Random Variables: Definition</font size> | ||
</center> | </center> | ||
| − | + | ---- | |
---- | ---- | ||
| Line 24: | Line 36: | ||
'''Definition''' <math>\quad</math> Given any F ⊂ ''S'' and G ⊂ ''R'', the '''image of F under <math>f</math>''' is<br/> | '''Definition''' <math>\quad</math> Given any F ⊂ ''S'' and G ⊂ ''R'', the '''image of F under <math>f</math>''' is<br/> | ||
| − | <center><math> f(F)=\{a\in\mathcal R:\;a=f(\omega)\; | + | <center><math> f(F)=\{a\in\mathcal R:\;a=f(\omega)\;\mbox{for some}\;\omega\in F\}</math></center> |
and the '''inverse image of G under <math>f</math>''' is <br/> | and the '''inverse image of G under <math>f</math>''' is <br/> | ||
<center><math> f^{-1}(G)=\{\omega\in\mathcal S:\;f(\omega) \in G\}</math></center> | <center><math> f^{-1}(G)=\{\omega\in\mathcal S:\;f(\omega) \in G\}</math></center> | ||
| Line 33: | Line 45: | ||
* The inverse image, <math>f^{-1}</math>(G), is not the same as the inverse function <math>f^{-1}</math>(a), a ∈ ''R''. | * The inverse image, <math>f^{-1}</math>(G), is not the same as the inverse function <math>f^{-1}</math>(a), a ∈ ''R''. | ||
*<math>f(\omega)</math>, <math>\omega</math> ∈ ''S'', is an '''element''' of ''R''; <math>f</math>(F), F ⊂ ''S'', is a '''subset''' of ''R''. | *<math>f(\omega)</math>, <math>\omega</math> ∈ ''S'', is an '''element''' of ''R''; <math>f</math>(F), F ⊂ ''S'', is a '''subset''' of ''R''. | ||
| − | * <math>f^{-1}</math>(a), a ∈ ''R'', is an element of ''S''; <math>f^{-1}</math>(G), G ⊂ ''R'', is a | + | * <math>f^{-1}</math>(a), a ∈ ''R'', is an element of ''S''; <math>f^{-1}</math>(G), G ⊂ ''R'', is a subset of ''S''. |
*The inverse function <math>f^{-1}</math>(a) ∀a ∈ ''R'', may not exist. The inverse image <math>f^{-1}</math>(G), G ⊂ ''R'', always exists. | *The inverse function <math>f^{-1}</math>(a) ∀a ∈ ''R'', may not exist. The inverse image <math>f^{-1}</math>(G), G ⊂ ''R'', always exists. | ||
| − | '''Definition of Random Variable''' (Beta version)< | + | '''Definition of Random Variable''' (Beta version)<br/> |
Given a probability space (''S,F'',P), a '''random variable''' is a function from ''S'' to the real line, <br/> | Given a probability space (''S,F'',P), a '''random variable''' is a function from ''S'' to the real line, <br/> | ||
| − | <center><math> | + | <center><math>X:\mathcal S\rightarrow\mathbb R</math></center> |
| − | [[Image:rv_definition_fig1.png|400px|thumb|left|Fig 1: Mapping elements of ''S'' to the real number line]] | + | <center>[[Image:rv_definition_fig1.png|400px|thumb|left|Fig 1: Mapping elements of ''S'' to the real number line]]</center> |
In order to assign probabilities to events related to X, we will define a new probability space ('''R''',B('''R'''),P<math>_x</math>), representing an "experiment" where the value of X is the outcome. Since <math>X(\omega)</math> ∈ '''R''' ∀<math>\omega</math> ∈ ''S'', our sample space is '''R'''. we always use the Borel field as the event space in this case. | In order to assign probabilities to events related to X, we will define a new probability space ('''R''',B('''R'''),P<math>_x</math>), representing an "experiment" where the value of X is the outcome. Since <math>X(\omega)</math> ∈ '''R''' ∀<math>\omega</math> ∈ ''S'', our sample space is '''R'''. we always use the Borel field as the event space in this case. | ||
| − | What should P< | + | What should P<math>_X</math> be? We need to define P<math>_X</math>(A) ∀ A ∈ '''R''', where P<math>_X</math>(A)is the probability of event A in out new probability space. Since we can view the value of X as our outcome in this space, A occurs if X ∈ A, so P<math>_X</math>(A) is the probability that <math>X(\omega)</math> ∈ A. Now, <br/> |
<center><math>\begin{align} | <center><math>\begin{align} | ||
\{X(\omega)\in A\} &= \{ \omega\in\mathcal S:\;X(\omega)\in A \} \\ | \{X(\omega)\in A\} &= \{ \omega\in\mathcal S:\;X(\omega)\in A \} \\ | ||
| Line 52: | Line 64: | ||
\end{align}</math></center> | \end{align}</math></center> | ||
but this requires that <br/> | but this requires that <br/> | ||
| − | <center><math> | + | <center><math>X^{-1}(A) \in \mathcal F \quad \forall A\in B(\mathbb R)</math></center> |
This leads us to <br/> | This leads us to <br/> | ||
| Line 65: | Line 77: | ||
* an indicator function <math>I_A(\omega)</math> for some A ∈ B('''R''') | * an indicator function <math>I_A(\omega)</math> for some A ∈ B('''R''') | ||
* the limit of a sequence of measurable function: i.e. for <math>X_n</math> measurable for all n = 1,2,...<br/> | * the limit of a sequence of measurable function: i.e. for <math>X_n</math> measurable for all n = 1,2,...<br/> | ||
| − | <center><math>X(\omega) = lim_{n\rightarrow\infty}X_n(\omega)</math></center> | + | <center><math>X(\omega) = \lim_{n\rightarrow\infty}X_n(\omega)</math></center> |
*A countable sum of measurable functions | *A countable sum of measurable functions | ||
| − | An example of a non measurable function: let ''S''='''R''' and F ⊂ ''S'' where F = ''P''('''R''')-B('''R'''), i.e. the set difference between the power set of '''R''' and its Borel field. | + | Note that if a function belongs to one or more of the four categories listed above then we can say that the function is measurable but it is not a necessary condition (if but not iff). |
| + | |||
| + | An example of a non measurable function: let ''S''='''R''' and F ⊂ ''S'' where F = ''P''('''R''')-B('''R'''), i.e. the set difference between the power set of '''R''' and its Borel field. So F ∉ B('''R'''). Then, <math>X(\omega) = I_F(\omega)</math> is not measurable. To see this, note that <br/> | ||
<center><math>X(\omega) = | <center><math>X(\omega) = | ||
\begin{cases} | \begin{cases} | ||
| Line 84: | Line 98: | ||
'''Notation''' | '''Notation''' | ||
<center><math>P_X(A) = P(X\in A) = P(\{\omega \in \mathcal S:\; X(\omega)\in A\})</math></center> | <center><math>P_X(A) = P(X\in A) = P(\{\omega \in \mathcal S:\; X(\omega)\in A\})</math></center> | ||
| − | |||
| − | In practice, how do we find P(X ∈ A)? the answer depends on whether X ia a discrete or continuous random variable, which we discuss next. | + | So A is an event in the Borel field in the probability soace ('''R''',B('''R'''),P<math>_X</math>). But {X ∈ A} = {<math>\omega</math> ∈ ''S'': X(<math>\omega</math>) ∈ A} is an event in ''F''. |
| + | |||
| + | In practice, how do we find P(X ∈ A)? the answer depends on whether X ia a discrete or continuous random variable, which we discuss next in [[ECE600_F13_rv_distribution_mhossain|Distributions]]. | ||
| Line 106: | Line 121: | ||
---- | ---- | ||
| − | [[ECE600_F13_notes_mhossain|Back to all ECE 600 notes]] | + | [[ECE600_F13_notes_mhossain|Back to all ECE 600 notes]]<br/> |
| + | [[ECE600_F13_Statistical_Independence_mhossain|Previous Topic: Statistical Independence]]<br/> | ||
| + | [[ECE600_F13_rv_distribution_mhossain|Next Topic: Random Variables: Distributions]] | ||
Latest revision as of 12:11, 21 May 2014
Back to all ECE 600 notes
Previous Topic: Statistical Independence
Next Topic: Random Variables: Distributions
The Comer Lectures on Random Variables and Signals
Topic 5: Random Variables: Definition
Random Variables
So far, we have been dealing with sets. But as engineers, we work with numbers, variables, vectors, functions, signals, etc. We will spend the rest of the course learning about random variables, vectors, sequences, and processes, since these usually serve us better as engineers in practice. However, we will link each of these topics to a probability space, since probability spaces tie everything together.
Informally, a random variable, often abbreviated to rv, is a function mapping the sample space S to the reals. However, we will restrict this function to lie within a certain class of functions.
First we present some definitions that will be needed.
Definition $ \quad $ Given two spaces S and R, an R-valued function or mapping $ f $:S→R assigns each element in S a value in R, so
Definition $ \quad $ Given any F ⊂ S and G ⊂ R, the image of F under $ f $ is
and the inverse image of G under $ f $ is
Note the following
- $ f $(F) is the set of all values in R that are mapped to by some element in F
- $ f^{-1} $(G) is the set of all values in S that map to some value in G
- The inverse image, $ f^{-1} $(G), is not the same as the inverse function $ f^{-1} $(a), a ∈ R.
- $ f(\omega) $, $ \omega $ ∈ S, is an element of R; $ f $(F), F ⊂ S, is a subset of R.
- $ f^{-1} $(a), a ∈ R, is an element of S; $ f^{-1} $(G), G ⊂ R, is a subset of S.
- The inverse function $ f^{-1} $(a) ∀a ∈ R, may not exist. The inverse image $ f^{-1} $(G), G ⊂ R, always exists.
Definition of Random Variable (Beta version)
Given a probability space (S,F,P), a random variable is a function from S to the real line,
In order to assign probabilities to events related to X, we will define a new probability space (R,B(R),P$ _x $), representing an "experiment" where the value of X is the outcome. Since $ X(\omega) $ ∈ R ∀$ \omega $ ∈ S, our sample space is R. we always use the Borel field as the event space in this case.
What should P$ _X $ be? We need to define P$ _X $(A) ∀ A ∈ R, where P$ _X $(A)is the probability of event A in out new probability space. Since we can view the value of X as our outcome in this space, A occurs if X ∈ A, so P$ _X $(A) is the probability that $ X(\omega) $ ∈ A. Now,
but this requires that
This leads us to
Random Variable Definition 1
Given (S,F,P), a random variable is a mapping X:S→R with property that
X:S→R satisfying this property is called a Borel measurable function. With this property, we can let
It can be shown that X(.) is measurable if it is:
- continuous
- an indicator function $ I_A(\omega) $ for some A ∈ B(R)
- the limit of a sequence of measurable function: i.e. for $ X_n $ measurable for all n = 1,2,...
- A countable sum of measurable functions
Note that if a function belongs to one or more of the four categories listed above then we can say that the function is measurable but it is not a necessary condition (if but not iff).
An example of a non measurable function: let S=R and F ⊂ S where F = P(R)-B(R), i.e. the set difference between the power set of R and its Borel field. So F ∉ B(R). Then, $ X(\omega) = I_F(\omega) $ is not measurable. To see this, note that
and consider A={1} ∈ B(R).
Then
We will use definition 1 of a random variable in this class. Then we can let
Notation
So A is an event in the Borel field in the probability soace (R,B(R),P$ _X $). But {X ∈ A} = {$ \omega $ ∈ S: X($ \omega $) ∈ A} is an event in F.
In practice, how do we find P(X ∈ A)? the answer depends on whether X ia a discrete or continuous random variable, which we discuss next in Distributions.
References
- M. Comer. ECE 600. Class Lecture. Random Variables and Signals. Faculty of Electrical Engineering, Purdue University. Fall 2013.
Questions and comments
If you have any questions, comments, etc. please post them on this page
Back to all ECE 600 notes
Previous Topic: Statistical Independence
Next Topic: Random Variables: Distributions
