Module 1.

Introduction to Probability Theory

Student Notes 1.3: Conditional Probability and Bayes Theorem

Conditional Probability

We would like to consider the probability of an event, say A, under the condition that : event B
has occurred.
Does putting such condition affect the probability?
Suppose A be the event that a randomly chosen student get grade A in Probability and statistics
course and B be the event that a randomly chosen student has CGPA < 5.
P(A) = Probaility that a randomly chosen student get grade A in Probability and statistics course.
P(A | B) = Probability of event A given CGPA < 5.
Clearly we see that with the given condition that event B has occurred i.e. if the CGPA < 5 then
probability of occurrence of event A is less. Thus we see that P(A) is not same as P(A | B). We
need to consider the probability P(A | B) separately. This probability is called conditional
probability.

Definition: For the events A, B in a sample space S, with P(B) ≠ 0, the conditional probability of
P( A B)
A given B, is defined by P( A∨B)= .
P(B)
Alternately, we are considering the probability of A when the sample space is restricted to B.

Example : Let A be the event : a randomly selected tube light lasts for 2 years.
If we want to consider probability that a randomly selected tube light lasts for 2 years given it
is produced by Philips.
This is P(A|B) where B is event : a randomly chosen tube light is produced by Philips.
Equivalently this probability is also the probability that a randomly chosen tube light produced
by Philips lasts for 2 years.

General Multiplication Rule

1. For 2 event A, B: P (A  B)  P (A| B) P(B), if P(B)  0.
2. For 3 events A, B, C: P (A  B  C) = P (A | B  C)P(B |C)P(C), provided P(B  C)  0.
Proof: P((A  B  C)= P(A  D) (D = B  C)
= P(A | D)P(D)
=P(A | B  C)P(B  C)
= P(A | B  C)P(B | C)P(C) (as P( B  C)  0 we have P(C)  0).
3. For n events A1, A2, …., An with P (A1  A2  …… An-1)  0
P (A1  A2  …..  An) = P(A1)P(A2|A1)P(A3| A1  A2)…..P(An| (A1  ….  An-1)).
Proof: Using mathematical proof follows.

Example: The probability that a new airport will get an award for its design is 0.16, the
probability that it will get an award for the efficient use of materials is 0.24, and the probability
that it will get both awards is 0.11. Find the probabilities that the airport will get the design
award given that
(a) It got award for the efficient use of materials.
(b) It did not get the award for the efficient use of materials.
Solution: Let the events be A : the airport gets award for design and B : the airport gets award
for efficient use of materials.
We know : P(A) 0.16, P(B)= 0.24, P(A  B) = 0.11.
(a) We want to find P(A | B).
P ( A B ) 0.11 11
P( A∨B)= = = .
P ( B ) 0.24 24
P ( A BC ) P ( A )−P( A B) 5
= .
C
(b) The required probability : P( A∨B )= =
P(B ) C
1−P ( B ) 76
Here A  BC = A – (A  B) and since A  B C A, we have P(A – (A  B) )= P(A) - P(A  B).

Example: 4 cards are drawn one by one without replacement from ordinary pack of playing
cards. Find the probability that all the 4 cards are spades.
Solution: Let Ai be the event that ith card is a spade i = 1, 2, 3, 4.
Required Probability =P(A1  A2  A3  A4)
=P(A1)P(A2 | A1)P(A3 |A1  A2)P(A4 | A1  A2  A3)
= 13/ 52 x 12/51 x 11/50 x 10/49 = 11/4165.
P(A1) = Probability that first card chosen is spade = (no of spades )/ total no of cards = 13/52.
P(A2 | A1) = Probability that second card chosen is spade it is given that first card chosen is
spade = (no of spade cards remaining)/ (total no of cards remaining)= 12/51
Similarly P(A3 |A1  A2) = 11/50 and P(A4 | A1  A2  A3) = 10/49.

Independence

Definition: Let A, B be events with P(B) 0. We say A is independent of B if P (A I B)  P (A)

Theorem: Let A, B be events with P(A)  0 and P(B) 0.
Then P (A  B) = P(A)P(B) if and only if P (A | B) = P(A) if and only if P (B | A) = P(B).
Proof: P (A  B) = P(A)P(B) if and only if P (A  B)/ P(B) = P(A) if and only if P (A | B) = P(A).
P (A  B) = P(A)P(B) if and only if P (A  B)/ P(A) = P(B) if and only if P (B | A) = P(B).
Thus, if P(A)  0 and P(B)  0, then A is independent of B if and only if B is independent of A.
We only say that A and B are independent.
In practice, we apply independence of events A and B when occurrence of A has no effect on
occurrence of B and vice versa.
For example:
Sampling with replacement: draw a card from a deck of cards. Put it back and shuffle well.
Then draw a card again. The events A : 1st card is a spade and B : 2nd card is a spade are
independent. Here P(B) = P(B | A) = 13/52
Sampling without replacement: draw a card from a deck of cards. Do not put card back. Then
draw a card again. The events A : 1st card is a spade and B : 2nd card is a spade are not
independent. Here P(B) ≠ P(B | A) = 12/51.

Theorem: Let A, B be independent events. Then

(i) A, BC are independent (ii) AC, B are independent (iii) AC, BC are independent.
Proof: (i) Suppose A, B be independent events. Then P (A  B) = P(A)P(B).
P (A  BC) = P(A) – P(A  B) = P(A) - P(A)P(B) = P(A)[1 - P(B)] = P(A)P(BC).
(ii) and (iii) cab be proved similarly.

Bayes’ Theorem
Total Probability Rule
Let S = B1  B2 ……… Bn, where B1 ,B2 ,…….., Bn are mutually exclusive events of the sample
space S with P(Bi)  0 for all i = 1,2,…..,n. Let A be an event with P(A)  0.
n
Then P( A)=∑ P ( Bi ) P ( A|B i ¿ ¿.
i=1

Proof: A =A  S = A  (B1  B2 ……… Bn)= (A  B1)  (A  B2) …….( A  Bn)

As ( A Bi ) ( A B j ) C Bi B j=ɸ, hence A  Bi , i = 1,2….n are mutually exclusive events, P(A) = P(A 
B1) + P(A  B2) + ………….+ (A  Bn)
= P(B1)P(A/B1) + P(B2)P(A/B2)+……..+ P(Bn)P(A/Bn), using multiplication theorem of probability.

Bayes’ Theorem
Let S = B1  B2 ……… Bn, where B1 ,B2 ,…….., Bn are mutually exclusive events of the sample
space S with P(Bi)  0 for all i = 1,2,…..,n. Let A be an event with P(A)  0.
P ( B i| A ¿=P ( A|B i ¿ P(Bi ) n ¿ ¿
Then P B P ( A| B ¿ .
∑ ( i) i
i=1

P (B i A) ¿
P ( B i| A ¿= =P ( A|B i ¿ P( Bi ) ¿
Proof: P( A) n
.
∑ P ( B i ) P ( A|Bi ¿
i=1
Remark : In the Bayes theorem we see that if all probabilities P(A | B i) are known then we can
calculate the reverse probabilities P(Bi | A).

Prior and Posterior Probabilities

Any manager, dealing with a situation of uncertainty, can assign probabilities to the possible
outcomes. These probabilities may be a combination of subjective and objective probabilities.
These probabilities are based on historical data and reports.
These initial probabilities are termed as prior probabilities.
Later new information is received – Survey or Product Test or some stray event.
This new information is factored in to generate the posterior probabilities.
Bayes’ theorem is the tool for revising the prior probabilities.
The probability model may be continuously updated as new data flow in!

Cause effect relationship

The conditional probability P(A|B) may be interpreted as the probability that B is the cause of A
(equivalently A is the effect of B).
Thus Bayes’ theorem reverses the roles of causes and effect.

There are also disadvantages to using Bayesian analysis: It does not tell you how to select a
prior. There is no correct way to choose a prior. Bayesian inferences require skills to translate
subjective prior beliefs into a mathematically formulated prior. If you do not proceed with
caution, you can generate misleading results. It can produce posterior distributions that are
heavily influenced by the priors. From a practical point of view, it might sometimes be difficult
to convince subject matter experts who do not agree with the validity of the chosen prior. It
often comes with a high computational cost, especially in models with a large number of
parameters.

Example on conditional probability and Baye’s theorem:

Example 1: Urn 1 contains 2 white, 2 black balls, urn 2 contains 1 white, 3 black balls and urn 3
contains 3 white, 1 black ball. One urn is chosen at random and one ball is chosen at random
from the chosen urn. Find the probability that
(a) A white ball is chosen.
(b) If a white ball is chosen, find the probability that urn 3 is chosen.
Solution: Let B1 be the event that ith urn is chosen where i= 1,2,3 and let A be the event that a
white ball is chosen.
(a) P(B1) = P(B2) = P(B3) = 1/3 since we can chose any urn out of 3 in 3 way and we can choose I
urn in 1 way.
P(A|B1) = Probability of choosing a white ball when it is known that urn 1 has been already
chosen = 2/4, since we can choose any ball out of 4 in 4 ways where a white ball can be chosen
in 2 ways. Similarly P(A|B2) = 1/4, P(A|B3) = 3/4.
Hence by theorem on total probability
P(A) = P(B1)P(A|B1) + P(B2)P(A|B2) + P(B3)P(A|B3) =1/3 [1/2 +1/4 +3/4 ] = 1/2.
(b) P(B3|A)= P(B3)P(A|B3) / P(A) = (1/3 . 3/4) / (1/2) = 1/2.

Example 2: In a bolt producing factory, machine A is producing 30%, machine B is producing

30% and machine C is producing 40% items. Machine A is producing 2% defective items, B and C
are producing 4% and 5% defective items respectively of theirs total production. One bolt is
chosen at randim form a large lot of bolts produced by this factory.
(a) What is the probability that this chosen bolt is defective?
(b) If this chosen bolt is defective, find the probability that this bolt is produce by machine C.
Solution: Let B1 ,B2 ,B3 be the events that the chosen bolt is produced by machines A,B,C
respectively. Let D be the event that chosen bolt is defective.
P(B1) = 0.3, P(B2) =0 .3, P(B3) = 0.4, P(D|B1) = 0.02
P(D|B2) = 0.04, P(B3) = 0.05
(a) Using theorem on total probability, we have
Required Probability = P(D) = P(B1)P(D|B1) + P(B2)P(D|B2) + P(B3)P(D|B3)
= 0.3 + .02 +0 .3 + 0.04 + 0.4 + .05 = 0.038
(b) Required Probability = P(B3 | D) = [P(B3)P(D|B3) ]/ P(D) = (0.4 x0 .05 /0 .038) = 10/19

Learning Outcome:
Conditional Probability, Independent Events, Multiplicative Law.
Prior and Posterior Probabilities, total probability rule, Bayes’ Theorem, Applications of Bayes
theorem.

Questions to consider:
Q1. Define conditional probability?
Q2. What are independent events?
Q3. What is general multiplicative law?
Q4. A company has 140 employees, of which 30 are supervisors. Eighty of the employees are
married, and 20% of the married employees are supervisors. If a company employee is
randomly selected, what is the probability that the employee is married and is a supervisor?
Q5. What are prior and posterior probabilities?
Q6. What is total probability rule?
Q7. What is Baye’s theorem?
Q8. What are some of the difficulties in the Bayesian approach?
Q9. What are applications of Bayes theorem?
Q10. When do we use Bayes theorem?