Difference between revisions of "Bayes theorem S13" - Rhea

Revision as of 23:12, 13 March 2013

Bayes' Theorem

by Maliha Hossain

 keyword: probability, Bayes' Theorem, Bayes' Rule 

INTRODUCTION

Bayes' Theorem (or Bayes' Rule) allows us to calculate P(A|B) from P(B|A) given that P(A) and P(B) are also known, where A and B are events. In this tutorial, we will derive Bayes' Theorem and illustrate it with a few examples.

Note that this tutorial assumes familiarity with conditional probability and the axioms of probability.

 Contents
- Bayes' Theorem
- Proof
- Example 1
- Example 2
- Example 3
- References

Bayes' Theorem

Let $ B_1, B_2, ..., B_n $ be a partition of the sample space $ S $, i.e. $ B_1, B_2, ..., B_n $ are mutually exclusive events whose union equals the sample space S. Suppose that the event $ A $ occurs. Then, by Bayes' Theorem, we have that

$ P[B_j|A] = \frac{P[A|B_j]P[B_j]}{P[A]}, j = 1, 2, . . . , n $

Bayes' Theorem is also often expressed in the following form:

$ P[B_j|A] = \frac{P[A|B_j]P[B_j]}{\sum_{k=1}^n P[A|B_k]P[B_k]} $

Proof

We will now derive Bayes'e Theorem as it is expressed in the second form, which simply takes the expression one step further than the first.

Let $ A $ and $ B_j $ be as defined above. By definition of the conditional probability, we have that

$ P[A|B_j] = \frac{P[A\cap B_j]}{P[B_j]} $

Multiplying both sides with $ B_j $, we get

$ P[A\cap B_j] = P[A|B_j]P[B_j] $

Using the same argument as above, we have that

$ P[B_j|A] = \frac{P[B_j\cap A]}{P[A]} $

$ \Rightarrow P[B_j\cap A] = P[B_j|A]P[A] $

Because of the commutativity property of intersection, we can say that

$ P[B_j|A]P[A] = P[A|B_j]P[B_j] $

Dividing both sides by $ P[A] $, we get

$ P[B_j|A] = \frac{P[A|B_j]P[B_j]}{P[A]} $

Finally, the denominator can be broken down further using the theorem of total probability so that we have the following expression

$ P[B_j|A] = \frac{P[A|B_j]P[B_j]}{\sum_{k=1}^n P[A|B_k]P[B_k]} $