Lecture 2

Probability Theory

DR. MAHA HASSANEIN

Spring 2024 DR. MAHA A. HASSANEIN 1

Random Experiment
Random Experiment : a procedure with a
set of well-defined possible outcomes

Sample space : the set of all possible

outcomes Ω

Event: a subset of the sample space

Spring 2024 DR. MAHA A. HASSANEIN 2

Disjoint Events A and B
Disjoint (mutually exclusive) events A and B
Events A & B that has no elements in common
The intersection is the Null Event ( the empty set)

𝐴∩𝐵 =𝜙

Spring 2024 DR. MAHA A. HASSANEIN 3

The Classical Probability Concept

In a random experiment, the probability of

successful output is the ratio of the number of
successful outputs n to the number of equally-likely
possible outputs N in the sample space S

𝑛
𝑃 𝐴 =
𝑁

Spring 2024 DR. MAHA A. HASSANEIN 4

Counting Principle
Fundamental Principle of Counting
If an experiment can be formed in k operations,
the first in 𝑛1 ways, the second 𝑛2 ways, and so
on until the kth operation, then together the k
operations can be formed in
𝑛1 × 𝑛2 × ⋯ … × 𝑛𝑘
Review : Types of Counting ( Permutations,
Combinations , with replacement)

Spring 2024 DR. MAHA A. HASSANEIN 5

Example 5
If 3 Balls are selected at random from a box
containing 4 Red , 3 Black balls. What is the
Probability that 2 Red and 1 Black are selected

Spring 2024 DR. MAHA A. HASSANEIN 6

The Axioms of Probability
Axiom 1 0≤𝑃 𝐴 ≤1 for any A in S

Axiom 2 𝑃(𝑆) = 1

Axiom 3 If A and B are disjoint events in S , then

𝑃 𝐴𝑈𝐵 = 𝑃 𝐴 + 𝑃 𝐵
Generalization of Axiom 3: If 𝐴1 , 𝐴2 , … , 𝐴𝑛 are mutually exclusive
events in Sample space , then 𝐴𝑖 ∩ 𝐴𝑗 = 𝜙 for all 𝑖 ≠ 𝑗
𝑛

𝑃 𝐴1 ∪ 𝐴2 … ∪ 𝐴𝑛 = ෍ 𝑃(𝐴𝑖 )
𝑖=1

Spring 2024 DR. MAHA A. HASSANEIN 7

Basic Probability Rules
For events A,B in S
Theorem 1. 𝑃(𝜙) = 0

Theorem 2. 𝑃(𝐴) = 1 − 𝑃(𝐴)

Theorem 3. If 𝐴 ⊆ 𝐵, 𝑡ℎ𝑒𝑛 𝑃(𝐴) ≤ 𝑃(𝐵 )

Theorem 4. If 𝐴, 𝐵 𝑖𝑛 𝑆 , then 𝑃(𝐴 ∩ 𝐵 ) = 𝑃(𝐴) − 𝑃(𝐴 ∩ 𝐵 )

Theroem 5. 𝑃(𝐴 ∪ 𝐵 ) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵 )

Prove the above.

Spring 2024 DR. MAHA A. HASSANEIN 8

 Example 2
Bits transmitted along a transmission line, if the
probability of errors detected during transmission
is 0.08 , errors during reception is 0.05 and errors
during both transmission and reception is 0.03.
What is the probability that a bit transmitted has
i. Either transmission or reception errors
ii. Neither transmission nor reception errors
iii. At most one error
iv. At least one error

Spring 2024 DR. MAHA A. HASSANEIN 9

Generalization of Theorem 5
𝑃 (𝐴 ∪ 𝐵 ∪ 𝐶 ) = 𝑃 (𝐴 ) + 𝑃 (𝐵 ) + 𝑃 (𝐶 ) − 𝑃 (𝐴 ∩ 𝐵 ) −
𝑃 (𝐴 ∩ 𝐶 ) − 𝑃 (𝐵 ∩ 𝐶 ) + 𝑃 (𝐴 ∩ 𝐵 ∩ 𝐶 )

B
A

Spring 2024 DR. MAHA A. HASSANEIN 10

Example
In a sample of 900 person in a town , the following information is given
in table.

Employed Not Employed Total

Male 460 40 500
Female 140 260 400
600 300 900
Find the following : P(M), P(F), P(E), P(E’),
𝑃 𝑀 ∪ 𝐸 , 𝑃 𝑀 ∩ 𝐸’ , 𝑃 𝐹 ∪ 𝐸 , 𝑃 𝐹 ∩ 𝐸’ ,
Draw the Venn Diagram of Experiment

Spring 2024 DR. MAHA A. HASSANEIN 11

Conditional Probability
Conditional probability is the probability of an event occurring, given
that another event has already occurred. If the event of interest is A and
event B has already occurred, the conditional probability of A given B is
usually written as P(A|B).

𝑃(𝐴 ∩ 𝐵)
𝑃(𝐴|𝐵) = , 𝑃(𝐵) ≠ 0
𝑃(𝐵)

In this context B is the sure event

Spring 2024 DR. MAHA A. HASSANEIN 12

 Properties of Conditional
Probability
FOR ANY EVENTS E,F⊆ 𝑺 WE HAVE

𝑃 𝐸𝑆 𝑃 𝐸
𝑃 𝜑𝑆 0
𝑃 𝜑𝐸 0
𝑃 𝑆𝐸 1
𝑃(𝐹′|𝐸) 1 − 𝑃(𝐹|𝐸)

Can you describe the meaning of each in words?

Spring 2024 DR. MAHA A. HASSANEIN 13

 Theorems related to conditional
probability
1. 𝐴1 ⊆ 𝐴2 , 𝑡ℎ𝑒𝑛 𝑃(𝐴1 |𝐵) ≤ 𝑃(𝐴2 |𝐵)

2. 𝑃(𝐴|𝐵) = 1 − 𝑃(𝐴|𝐵)

3. 𝑃 (𝐴1 ∪ 𝐴2 )|𝐵 = 𝑃(𝐴1 |𝐵) + 𝑃(𝐴2 |𝐵) − 𝑃 (𝐴1 ∩ 𝐴2 )|𝐵

𝑃(𝐴)−𝑃(𝐴∩𝐵)
4. 𝑃 (𝐴|𝐵) =
1−𝑃(𝐵)

•Ex. Show that

Spring 2024 DR. MAHA A. HASSANEIN 14

 Example
Given a box with 3 Red and 4 Black balls. Two balls are selected at random one
after the other a) Find the probability that the 2nd selected ball is red given that
the 1st is red
Note Events
Ri= “the ith ball selected out of the box is red “
Bi= “the ith ball selected out of the box is black “
Find the probability that the :
a) 2nd selected ball is Red given that the 1st is red
b) 2nd selected ball is black given that the 1st is red
c) 2nd selected ball is black given that the 1st is black

Spring 2024 DR. MAHA A. HASSANEIN 15

Example
In a sample of 900 person in a town , the following information is given
in table.

Employed Not Employed Total

Male 460 40 500
Female 140 260 400
600 300 900
Find
𝑃 𝑀 𝐸 , 𝑃 𝐹 𝐸 , 𝑃 𝑀 𝐸 ′ , 𝑃 𝑀′ 𝐸 ′ , 𝑃 𝑀𝑈𝐸 ′ , 𝑃(𝑀′|𝐸)

Spring 2024 DR. MAHA A. HASSANEIN 16

The Multiplication Rule
From the definition of conditional probability

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴|𝐵)𝑃(𝐵), 𝑃(𝐵) ≠ 0

𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐵|𝐴)𝑃(𝐴), 𝑃(𝐴) ≠ 0
◦ In case of independent events A and B , we have

𝑃 𝐴|𝐵 = 𝑃 𝐴
𝑃 𝐵|𝐴 = 𝑃 𝐵
Tree Diagram

Spring 2024 DR. MAHA A. HASSANEIN 17

 General Multiplication Rule

Let 𝐴1 , 𝐴2 , … , 𝐴𝑛 be any events in S, then

𝑃(𝐴1 ∩ 𝐴2 ∩ … 𝐴𝑛 ) = 𝑃(𝐴1 )𝑃(𝐴2 |𝐴1 )𝑃(𝐴3 |𝐴1 ∩ 𝐴2 ) …

Spring 2024 DR. MAHA A. HASSANEIN 18

Example
Given a Box with 3 Red and 4 Black balls. Three balls are selected at
random one after the other without replacement. Find probability that
a) the first ball selected is red , the second is black and the third is
red
b) The third ball is red

Events
𝑅𝑖 = “the ith ball selected out of the box is red “
𝐵𝑖 = “the ith ball selected out of the box is black “
Use Tree Diagram

Spring 2024 DR. MAHA A. HASSANEIN 19

 Example 8
A box contain 15 fuse of which 5 are defective. If 3 fuses are selected at
random , one after the other, what is the probability that the fuses are
1. All defective
2. The 1st defective, 2nd and 3rd nondefective
3. The 1st defective and the 2nd defective
4. The 3rd defective. Write down the subset of the sample space
5. At least two are defective
6. At most one is defective

Use Tree Diagram

𝐴𝑛𝑠. (1)5.4.3/15.14.13 , 2 5.10.9/15.14.13 , (3)5.4/15.14, 4
Spring 2024 DR. MAHA A. HASSANEIN 20
Bayes Rule probability of
the probability
before the
evidence is
the evidence, considered
given the belief (hypothesis)
is true
PRIOR
LIKELIHOOD

𝑃 𝐵|𝐴 𝑃(𝐴)
𝑃 𝐴𝐵 =
𝑃(𝐵)

POSTERIOR
(updated EVIDENCE
probability after probability of the
the evidence is evidence, under
considered) any circumstance

Spring 2024 DR. MAHA A. HASSANEIN 21

 Definition. Partition of a Set
𝐸1 , 𝐸2 , … , 𝐸𝑛 is a partition of a set E if 𝐸2
1. 𝐸𝑖 ⊆ 𝐸 𝐸𝑛
2. 𝐸𝑖 ∩ 𝐸𝑗 = 𝜙 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑖 ≠ 𝑗
3. 𝑛𝑖=1 𝐸𝑖 = 𝐸 𝐸1
….

Examples of partitions. (board)

Total probability theory 𝐸
Let 𝐸1 , 𝐸2 , … , 𝐸𝑛 be a partition of a sample space S; such that
𝑃(𝐸𝑖 ) ≠ 0.If 𝐹 is an event in S , then
𝑛
𝑃 (𝐹 ) = 𝑖=1 𝑃 (𝐸𝑖 ). 𝑃(𝐹|𝐸𝑖 )

Consider the case n=3. (board)

Spring 2024 DR. MAHA A. HASSANEIN 22

Bayes' Theorem
◦ Let 𝐸1 , 𝐸2 , … , 𝐸𝑛 be a partition of a sample space S;
such that 𝑃(𝐸𝑖 ) ≠ 0. If 𝐹 is an event in S , 𝑃(𝐹) ≠0 , then

𝑃 (𝐸𝑘 ).𝑃(𝐹|𝐸𝑘 )
𝑃(𝐸𝑘 |𝐹 ) = 𝑛
𝑖=1 𝑃 (𝐸𝑖 ).𝑃(𝐹|𝐸𝑖 )

Simply derived from the definition of conditional probability

Spring 2024 DR. MAHA A. HASSANEIN 23

 Example
Three machines 𝑀1 , 𝑀2 𝑎𝑛𝑑 𝑀3 produce 50%, 30% & 20% ,
respectively, of the total number of products in a factory. The
percentage of defective products of each is 3%, 4% & 5% , respectively.
If a product is selected at random, what is the probability that it is
defective? Find the probability that the selected defective product is
produced by 𝑀1 . If we want to discard one of the machines; which one
to choose? Why?

Ans. .037, .405,

𝑃 𝑀2 𝐷 = 32.4; 𝑃 𝑀3 𝐷 = 0.27 → 𝑑𝑖𝑠𝑐𝑎𝑟𝑑 𝑀1
Spring 2024 DR. MAHA A. HASSANEIN 24
Example
A device that detects faulty products, detects a fault with probability
90%, if it exists, detects a fault with probability 1% if it doesn’t exist. In a
lot containing 0.1% faulty products. What is the probability that :
i. The product has a fault if a product is selected at random and the
device detects a fault?
Ii. The product doesn’t have a fault given the device detects one?
Iii. The product has a fault given the device doesn’t detect it ?

Ans. 9%, , 91%

Spring 2024 DR. MAHA A. HASSANEIN 26
Independent Events
Two events 𝐴, 𝐵 are independent events if and only if

𝑃 𝐴 ∩ 𝐵 = 𝑃 𝐴 . 𝑃(𝐵)
𝐴1 , 𝐴2 , … , 𝐴𝑛 are Independent events if and only if

𝑃(𝐴1 ∩ 𝐴2 ∩ … ∩ 𝐴𝑛 ) = 𝑃(𝐴1 )𝑃(𝐴2 ) … 𝑃(𝐴𝑛 )

Spring 2024 DR. MAHA A. HASSANEIN 27

More on Independent Events
If 𝐴, 𝐵 are independent events then
 𝐴′, 𝐵 are independent→ 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴). 𝑃(𝐵)
 𝐴, 𝐵′ are independent→ 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴). 𝑃(𝐵)
 𝐴′, 𝐵' are independent→ 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴). 𝑃(𝐵)

𝐴1 , 𝐴2 , … , 𝐴𝑛 are Independent events then

𝑃(𝐴1 ∩ 𝐴2 ∩ … ∩ 𝐴𝑛 ) = 𝑃(𝐴1 )𝑃(𝐴2 ) … 𝑃(𝐴𝑛 )

Spring 2024 DR. MAHA A. HASSANEIN 28

Example 9
The probability that a missile A hits a target is ¾ and missile B hits is
2/3.
Find the Probability that :
i) Both hits the target
ii) At least one hits the target
iii) Neither of them hits the target
iv) Exercise. Exactly one hits the target

Sol. ½, 11/12 , 1/12

Spring 2024 DR. MAHA A. HASSANEIN 29

 Union of Independent events
If 𝐴, 𝐵 are independent events then
𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴)𝑃(𝐵) ….(1)
= 1 − 𝑃(𝐴 ∪ 𝐵) ……….. (2)

= 1 − 𝑃(𝐴)𝑃(𝐵) …………(3)

If A, B, and C are independent events then

P( A  B  C )  P( A) P( B) P(C )  P( B) P(C )  P(C )

 1  P( A) P( B) P(C )

Spring 2024 DR. MAHA A. HASSANEIN 30

 Text book
◦ Chapter 3 - Sections 3.1 - 3.7

References

Spring 2024 DR. MAHA A. HASSANEIN 31