Walpole

Ch 02:
1

Probability
Probability & &
Statistics
Statistics for
for
Engineers
Engineers & &
Scientists,
Scientists, by
by
Walpole,
Walpole, Myers,
Myers,
Myers
Myers && Ye
Ye
~
~
Chapter
Chapter 22 Notes
Notes
Class notes for ISE 201

San Jose State University

Steve Kennedy

Spring 2007
Probability Intro Walpole
Ch 02:
2
 The sample space S of an experiment is the
set of all possible outcomes.
• We must understand the sample space in order to
determine the probability of each outcome occurring.
• The sample space is a set, the domain of the
probability function.
• Each probability value, p, is a real number 0  p  1.
 Important: The sum of the probabilities for
all elements in the sample space always
equals 1.
• Why is this important?
• This fact allows us to check our answers.
 Properly enumerating the sample space is
key to correctly calculating probabilities.
• A tree diagram is sometimes useful.
Spring 2007
Events Walpole
Ch 02:
3
 An event is a subset of a sample space (E  S)
• Note that both S and  are events as well.
 Sample spaces can be continuous or discrete.
• What is a continuous vs. discrete sample space?
 Example: Life in years of a component. S = ?
• S = {t | t  0} => “all values of t such that t  0”
• A = component fails before the end of the fifth year.
• A = {t | t < 5}.
 Example: Flip a coin three times. S = ?
• S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}
 Event A = 1st flip is heads.
• A = {HHH, HHT, HTH, HTT}

Spring 2007
Event/Set Operations Walpole
Ch 02:
4
 The complement of an event A?
• The set of all elements of S not in A. Denoted A’.
• A = 1st flip is heads. A’ = first flip is not heads.
 The intersection of two events A and B?
• The set of all elements in both A and B. Denoted A  B.
• B = 2nd or 3rd flip, but not both, are heads.
• B = {HHT, HTH, THT, TTH}. A  B = ?
• A  B = {HHT, HTH}
 Two events are mutually exclusive if…?
•A  B = 
 The union of two events, A and B?
• The set of elements in either A or B. A  B = ?
• A  B = {HHH, HHT, HTH, HTT , THT, TTH}.

Spring 2007
Venn Diagrams Walpole
Ch 02:
5
 Venn Diagrams show various events
graphically, and are sometimes helpful in
understanding set theory problems.
 Standard set theory results hold:
•A   = ?
•A   = ?
• A  A’ = ?
• A  A’ = ?
• S’ = ?
• ’ = ?
• (A’)’ = ?
• (A  B)’ = ?
• (A  B)’ = ?
• (A  B)’=A’  B’, (A  B)’=A’  B’

Spring 2007
Intuitive Sample Point Counting Walpole
Ch 02:
6
 If one operation can be performed in n1
ways, and for each way, a second can be
performed in n2 ways, then the two can be
performed a total of n1 n2 ways.
• For three operations?
• n1 n2 n3.
• How many passwords of length 5 need to be checked
by a password hacking program if only lower case
letters are used?
• 265 = 11,881,376.
 This is called sampling with replacement.
 A tree diagram can be used to enumerate all
of the options.

Spring 2007
Permutation Orderings Walpole
Ch 02:
7
 A permutation is an ordering of a set or
subset of objects.
 The number of distinct orderings of n items?
• n items can go in the first position.
• Once the first item is fixed, n-1 items can go in the 2 nd
position.
• Then n-2 items in the third position, etc.
• Number of orderings is n (n-1) (n-2) … 1, or n!
• Remember that 1! = 0! = 1.
 This is called sampling without replacement.
• Once we use a value, it can’t be used again.

Spring 2007
Permutations of Partial Orderings Walpole
Ch 02:
8
 Suppose that we will give 3 different awards
to three students out of a class of 60 students.
How many ways can the awards be given.
• What if the problem was slightly different and one student
could win all three awards?
• This would be with replacement.
• The number is 603 = 216,000
 Without replacement, 60 students could get
the first award, then 59 students are eligible
for the 2nd and 58 for the third, or 60*59*58 =
205,320.
 In general, the number of permutations of n
things taken r at a time is written nPr = n! / (n
– r)!
• 60! / 57! = 60*59*58.

Spring 2007
Other Permutations Walpole
Ch 02:
9
 Circular permutations: n distinct objects
arranged in a circle.
• The position of the first object could be anywhere.
• If all objects moved one position clockwise, it’s still the
same permutation.
• Fix the first object anywhere on the circle, then n-1
objects can go to the left, n-2, next, etc.
• The number of circular permutations of n objects is (n -
1)!.
• Another way to look at it: the set of all permutations is
n!. For each starting value, there are n orderings that
are identical (moving the same ordering around the
circle). So the total number of different orderings is n!/n
= (n-1)!
• For example, for the ordering 35142 for n = 5.
• There are 5 identical orderings (35142, 51423, 14235,
42351, 23514).
Spring 2007
Permutations with Identical Objects Walpole
Ch 02:
10
 If some objects are identical, with n1 of type
1, n2 of type 2, …, nk of type k, and n = n1 +
n2 + … + nk, the number of distinct
permutations is n! / (n1! n2! … nk!)
• Example, 3 items of type 1, 1 of type 2 gives 4
permutations.
• 1112, 1121, 1211, and 2111.
• For each of the four, there would be 3! orderings if the
1’s were distinct (say a, b, and c).
• For example, 1121 would be ab2c, ac2b, ba2c, bc2a,
ca2b, and cb2a. Or, if all items are distinct, there
would be 4*3! Or 4! orderings.
• Three items being identical reduces the number of
permutations by a factor of 3!, so we divide by 3!.

Spring 2007
Arranging n Objects Into r Cells Walpole
Ch 02:
11
 Partitioning n distinct objects into r cells or
subsets, each of a given fixed size, where the
ordering of objects within a cell doesn’t matter.
 Example divide 5 items into two cells, one of size
3 and one of size 2.
• {(1,2,3), (4,5)}, {(1,2,4), (3,5)}, {(1,2,5), (3,4)}, {(1,3,4),
(2,5)}, {(1,3,5), (2,4)}, {(1,4,5), (2,3)}, {(2,3,4), (1,5)},
{(2,3,5), (1,4)}, {(2,4,5), (1,3)}, {(3,4,5), (1,2)}
 There are n! total possible orderings, but n1! In
the 1st cell, and n2! in the 2nd cell, etc., are
identical.
 In general, the number of distinct combinations
of n distinct objects into r cells, with n1 items in
the 1st cell, n2 in the 2nd, …, and nr in the rth cell is
n! / (n1! n2!…nr!).

Spring 2007
Combinations of n Items Taken r at a Walpole
Ch 02:
Time 12
 To review, there are how many permutations of n
items taken r at a time if each ordering is distinct?
• n (n-1) (n-2) ... (n-r+1), or
• nPr = n! / (n – r)! .
• For any given set of r items, there are r! possible orderings.
 So what if the order of the r items doesn’t matter?
• Divide nPr by r! to get the number of distinct outcomes.
 The number of combinations of n items taken r at
a time, where order doesn’t matter, is nCr = n! /
(r!(n-r)!).
 Example: How many distinct poker hands of 5
cards each can be dealt using a deck of 52 cards?
• 52C5 = 52! / (5!47!) = 52*51*50*49*48 / 5*4*3*2*1 =
2,598,960.

Spring 2007
Probability of an Event Walpole
Ch 02:
13
 For now, we only consider discrete sample
spaces.
 Each point in a sample space is assigned a
weight or probability value. The higher the
probability, the more likely that outcome is to
occur.
 The probability of an event A is the sum of
the probabilities of the individual points in A.
Then,
• 0  P(A)  1
• P() = ?
• P(S) = ?
• If two events are mutually exclusive, (which means?)
• (that they have no points in common, or A  B = ),
• then P(A  B) = P(A) + P(B).

Spring 2007
Relative Frequency for Probability Walpole
Ch 02:
14
 If an experiment has N different equally
likely outcomes, and n outcomes correspond
to event A, then P(A) = ?.
• P(A) = n / N.

Spring 2007
Additive Probability Rules Walpole
Ch 02:
15
 We already know that if A and B are
mutually exclusive, P(A  B) = P(A) + P(B).
 Also, if more than 2 events are mutually
exclusive, the probability of the union of all
of those events is the sum of all of the
individual probabilities.
 What is P(A  B) if A and B are not mutually
exclusive?
• Can use a Venn diagram to show this case.
• The sample points in P(A  B) are double counted.
• So P(A  B) = ?
• P(A  B) = P(A) + P(B) - P(A  B).
 Given that we know P(A), what is P(A’)?
• P(A’) = 1 – P(A).

Spring 2007
Conditional Probability Walpole
Ch 02:
16
 Conditional probability, written P(B|A), is
the probability of “B, given A”, the
probability that B occurs, given that we
know that A has occurred.
• Look at a Venn diagram of A and B.
• Suppose all sample points are equally likely.
• P(B) = (# of outcomes in B)/(total # of outcomes in S)
• P(B|A) = ?
• P(B|A) = (# of outcomes in A  B)/(total # of outcomes
in A)
• or, P(B|A) = P(A  B) / P(A) (as long as P(A) > 0).

Spring 2007
Conditional Probability Example Walpole
Ch 02:
17
 Example. For the following population of 900
people:
• Employed Unemployed Total
• Male 360 140 500
• Female 240 160 400
• Total 600 300 900
 If a person is selected at random from this
group,
• P(E) = ?
• P(M) = ?
• P(E  M) = ?
• P(E|M) = ?
• P(M|E) = ?
• Answers: P(E)=600/900=2/3, P(M)=500/900=5/9,
P(E  M)=360/900=2/5, P(E|M)=360/500=18/25, P(M|
E)=360/600=3/5.

Spring 2007
Independence Walpole
Ch 02:
18
 Conditional probability helps us update the
probability of an event given additional
information.
 Suppose P(B|A) = P(B). What does this tell
us?
• Whether A occurs or not, the probability of B occurring
doesn’t change.
 If P(B|A) = P(B), then A and B are
independent.
• Can show that if P(B|A) = P(B) is true, then P(A|B) = P(A)
is always also true.
 From the above, and the definition of
conditional probability, if A and B are
independent,
• P (A  B ) = P(A) P(B)

Spring 2007
Multiplicative Rules Walpole
Ch 02:
19
 Rearranging the conditional probability
formula, if both A and B can occur, then
P(A  B) = P(B|A) P(A)
Or, the probability of both A and B occurring
equals the probability of B given A times the
probability of A.
 Note that it is also true that
P(A  B) = P(A|B) P(B)
 If (and only if) events A and B are
independent, then from the above formula,
we have
P(A  B) = P(A) P(B)
 For more than two independent events,
multiply all of the probabilities together.

Spring 2007
Theorem of Total Probability Walpole
Ch 02:
20
 Suppose the sample space S can be partitioned
into events A1, A2, and A3. What does this
mean?
• A1, A2, and A3 are disjoint and between them cover all of S.
 Then the probability of an event B occurring can
be calculated using conditional probabilities
given that either A1 or A2 or A3 occurred.
 P(B) = P(B|A1)P(A1) + P(B|A2)P(A2) + P(B|
A3)P(A3). In words, this means?
• The probability that B and A1 occur + the probability that B
and A2 occur + the probability that B and A3 occur.
 This rule, called the theorem of total
probability, or the rule of elimination, holds for
any partitioning of S.

Spring 2007
Bayes Rule Walpole
Ch 02:
21
 Here we can calculate reverse conditional
probabilities.
 Using an example from an earlier slide:
• Employed Unemployed Total
• Male 360 140 500
• Female 240 160 400
• Total 600 300 900
 First we show how to use the theorem of total
probability to calculate P(M).
• S can be partitioned into mutually exclusive events E and U.
• Then P(M) = P(M|E)P(E) + P(M|U)P(U)
• Here P(M) = (360/600)(600/900) + (140/300)(300/900) =

2/5 + 7/45 = 5/9 (as expected).

Spring 2007
Bayes Theorem Walpole
Ch 02:
22
 Bayes Theorem can be used to adjust the
probability of a partition subset of S given
additional information.
• In the example suppose we know that the person selected is
a male (M). Then what is the probability that the person
selected is employed?
• As before, P(E|M) = P(E  M)/P(M), by the definition of
conditional probability.
• By Bayes Theorem,
P(E|M) = _ P(M|E)P(E) _
P(M|E)P(E) + P(M|U)P(U)
• In the example,
P(E|M) = (360/600)(600/900) / (2/5 + 7/45) = 18/25
• P(E) was 2/3. If we know M occurred, P(E|M) becomes 18/25.
 P(E) is called the prior (or "a priori"), and P(E|M)
is called the posterior (or "a posteriori")
probabilities.

Spring 2007