AMS 131: Introduction to Probability Theory (Spring 2013)

Chapter 2: Conditional Probability

Denition of conditional probability: Consider a sample space, S, with a probability measure Pr(A)
dened for events A in S. Then, the conditional probability, Pr(A | B), of event A given event B, where
Pr(B) > 0, is dened as Pr(A | B) = Pr(A B)/Pr(B).
Note that conditional probability denes a proper probability measure, that is, the set function
{Pr(A | B) : A S}, for an event B with Pr(B) > 0, satises the three probability axioms. Therefore,
all results and properties for (unconditional) probability can be extended to conditional probability.
Multiplication rule for conditional probabilities: Consider events A
1
, ..., A
n
such that
Pr(A
1
A
2
... A
n1
) > 0. Then,
Pr(A
1
A
2
... A
n
) = Pr(A
1
)Pr(A
2
| A
1
)Pr(A
3
| A
1
A
2
) Pr(A
n
| A
1
... A
n1
).
Independent events: Two events A and B are independent if learning that one of them has occurred
does not change the probability of the other. Hence, events A and B, with Pr(A) > 0 and Pr(B) > 0,
are independent if by denition Pr(A | B) = Pr(A) and Pr(B | A) = Pr(B). Equivalently, events A and
B are independent if and only if Pr(A B) = Pr(A)Pr(B).
In general, k( 2) events A
1
, ..., A
k
are independent if by denition for any j = 2, ..., k and any subset
A
i1
, ..., A
ij
of size j, we have
Pr(A
i1
... A
ij
) = Pr(A
i1
) Pr(A
ij
).
Considering an event B with Pr(B) > 0, events A
1
, ..., A
k
are conditionally independent given B if by
denition for any j = 2, ..., k and any subset A
i1
, ..., A
ij
of size j,
Pr(A
i1
... A
ij
| B) = Pr(A
i1
| B) Pr(A
ij
| B).
Bayes Theorem
Consider a set of k events, B
1
, ..., B
k
, that partition the sample space, that is, B
1
, ..., B
k
are disjoint
and

k
i=1
B
i
= S. (In applications, the events that form the partition are chosen such that a source of
uncertainty is reduced if we learn which of the B
i
has occurred.) Moreover, assume that we can specify
or obtain the unconditional probability of each B
i
and the conditional probability of some event A given
each event B
i
. Then, Bayes theorem provides a rule for revising the probability for each B
i
in light
of the information contained in A, that is, a rule to compute the conditional probability of each B
i
given A.
Law of total probability: Assume that events B
1
, ..., B
k
form a partition of the sample space S,
and that Pr(B
j
) > 0 for j = 1, ..., k. Then, for any event A in S,
Pr(A) =
k

j=1
Pr(B
j
)Pr(A | B
j
).
Bayes theorem: Let the events B
1
, ..., B
k
form a partition of the sample space S such that Pr(B
j
) > 0
for j = 1, ..., k, and consider an event A such that Pr(A) > 0. Then, for i = 1, ..., k,
Pr(B
i
| A) =
Pr(B
i
)Pr(A | B
i
)

k
j=1
Pr(B
j
)Pr(A | B
j
)
.