Difference between revisions of "ECE600 F13 rv distribution mhossain" - Rhea

Revision as of 08:22, 1 October 2013

How do we find , compute and model P(x ∈ A) for a random variable X for all A ∈ B(R)? We use three different functions:

1. the cumulative distribution function (cdf)
2. the probability density function (pdf)
3. the probability mass function (pmf)

We will discuss these in this order, although we could come at this discussion in a different way and a different order and arrive at the same place.

Definition $ \quad $ The cumulative distribution function (cdf) of X is defined as

Notation $ \quad $ Normally, we write this as

So $ F_X(x) $ tells us P$ P_X(A) $ if A = (-∞,x] for some real x.
What about other A ∈ B(R)? It can be shown that any A ∈ B(R) can be written as a countable sequence of set operations (unions, intersections, complements) on intervals of the form (-∞,x$ _n $], so can use the probability axioms to find $ P_X(A) $ from $ F_X $ for any A ∈ B(R). This is not how we do things in practice normally. This will be discussed more later.

Can an arbitrary function $ F_X, $ be a valid cdf? No, it cannot.
Properties of a valid cdf:
1.

This is because

and

2. For any $ x_1,x_2 $ ∈ R such that $ x_1<x_2 $,

i.e. $ F_X(x) $ is a non decreasing function.

3. $ F_X $ is continuous from the right , i.e.

Proof: First, we need some results from analysis and measure theory:
(i) For a sequence of sets, $ A_1, A_2,... $, if $ A_1 $ ⊃ $ A_2 $ ⊃ ..., then

(ii) If $ A_1 $ ⊃ $ A_2 $ ⊃ ..., then

(iii) We can write $ F_X(x^+) $ as

Now let

Then

4. $ P(X>x) = 1-F_X(x) $ for all x ∈ R

5. If $ x_1 < x_2 $, then

6. $ P(\{X=x\})= F_X(x) - F_X(x^-) $, where

The Probability Density Function

Definition $ \quad $ The probability density function (pdf) of a random variable X is the derivative of the cdf of X,

at points where $ F_x $ is differentiable.
From the Fundamental Theorem of Calculus, we then have that

Important note: the cdf $ F_X $ might not be differentiable everywhere. At points where $ F_X $ is not differentiable, we can use the Dirac delta function to defing $ f_x $.

Definition $ \quad $ The Dirac Delta Function $ \delta(x) $ is the function satisfying the properties:
1.

2.

If $ F_X $ is not differentiable at a point, use $ \delta(x) $ at that point to represent $ f_X $.

Why do we do this? Consider the step function $ u(x) $, which is discontinuous and thus not differentiable at $ x=0 $. This is a common type of discontinuity we see in cdfs. The derivative of $ u(x) $ is defined as

This limit does not exist at $ x=0 $

Let's look at the function

It looks like this:

Fig 1: g(x) for h>0

For any x ≠ 0, we have that

for small enough h.
Also, ∀ $ \epsilon $<0,

So, in the limit, the function g(x) has the properties of the $ \delta $-function as h tends to 0. A similar argument can be made for h<0.
So this is why it is sometimes written that

Since we will only work with non-differentiable functions that have step discontinuities as cdfs, we write

with the understanding that $ d/dx $ is not necessarily the traditional definition of the derivative.

Properties of the pdf:
1. (proof)

2. (proof)

3. (proof) if $ x_1<x_2 $</center>, then

Some notes:

• We introduced the concept of a pdf in our discussion of probability spaces. We could have defined the pdf of a random variable X as a function $ f_X $ satisfying properties 1 and 2 above, and then define $ F_X $ in terms of $ f_X $.
• f_X(x) is not a probability for a fixed x, it gives us instead the "probability density", so it must be integrated to give us the probability.
• In practice, to compute probabilities of random variable X, we normally use
  

References

• M. Comer. ECE 600. Class Lecture. Random Variables and Signals. Faculty of Electrical Engineering, Purdue University. Fall 2013.

Back to all ECE 600 notes