## Bayes' Theorem

by Maliha Hossain, proud Member of the Math Squad.

keyword: probability, Bayes' Theorem, Bayes' Rule

INTRODUCTION

Bayes' Theorem (or Bayes' Rule) allows us to calculate P(A|B) from P(B|A) given that P(A) and P(B) are also known. In this tutorial, we will derive Bayes' Theorem and illustrate it with a few examples. After going over the examples, if you have any questions or if you find any mistakes please leave me a comment at the end of the relevant section.

Note that this tutorial assumes familiarity with conditional probability and the axioms of probability. If you interested in the derivation of the conditional distributions for continuous and discrete random variables, you may wish to go over Professor Mary Comer's notes on the subject.

 Contents
- Bayes' Theorem
- Proof
- Example Problems
- References


## Bayes' Theorem

Let $B_1, B_2, ..., B_n$ be a partition of the sample space $S$, i.e. $B_1, B_2, ..., B_n$ are mutually exclusive events whose union equals the sample space S. Suppose that the event $A$ occurs. Then, by Bayes' Theorem, we have that

$P[B_j|A] = \frac{P[A|B_j]P[B_j]}{P[A]}, j = 1, 2, . . . , n$

Bayes' Theorem is also often expressed in the following form:

$P[B_j|A] = \frac{P[A|B_j]P[B_j]}{\sum_{k=1}^n P[A|B_k]P[B_k]}$

## Proof

We will now derive Bayes' Theorem as it is expressed in the second form, which simply takes the expression one step further than the first.

Let $A$ and $B_j$ be as defined above. By definition of the conditional probability, we have that

$P[A|B_j] = \frac{P[A\cap B_j]}{P[B_j]}$

Multiplying both sides with $B_j$, we get

$P[A\cap B_j] = P[A|B_j]P[B_j] \$

Using the same argument as above, we have that

\begin{align} P[B_j|A] & = \frac{P[B_j\cap A]}{P[A]} \\ \Rightarrow P[B_j\cap A] &= P[B_j|A]P[A] \end{align}

Because of the commutativity property of intersection, we can say that

$P[B_j|A]P[A] = P[A|B_j]P[B_j] \$

Dividing both sides by $P[A]$, we get

$P[B_j|A] = \frac{P[A|B_j]P[B_j]}{P[A]}$

Finally, the denominator can be broken down further using the theorem of total probability so that we have the following expression

$P[B_j|A] = \frac{P[A|B_j]P[B_j]}{\sum_{k=1}^n P[A|B_k]P[B_k]}$

## References

• Alberto Leon-Garcia, Probability, Statistics, and Random Processes for Electrical Engineering, Third Edition