Ch9, p.

Hypothesis testing
• What is hypothesis testing?
Question 7.1 (What is a hypothesis testing question?)

Ch9, p.2

Question 7.2

Definition 7.1 (null and alternative hypotheses, simple and composite hypotheses, TBp.331,332,334)

Question 7.3 (asymmetry between H0 and HA)

 Ch9, p.3

Example 7.1 (some null and alternative hypotheses, TBp.329, 334)

Ch9, p.4
 Ch9, p.5

Ch9, p.6

• Neyman-Pearson paradigm --- concept and procedure of hypothesis testing

Definition 7.2 (test, rejection region, acceptance region, test statistic, TBp. 331)

decisions hypotheses

S AR Ω0 Ω
RR ΩA
decisions
Note. Two types of errors accept H0 reject H0
may be incurred in applying hypo- H0 is true Type I error
this paradigm theses H is true Type II error
A
 Ch9, p.7

Definition 7.3 (type I error, significance level, type II error, power, TBp. 331)

Question 7.4 (dilemma between type I and type II errors)

S AR S AR
RR RR

Ch9, p.8

Example 7.2 (cont. Ex. 7.1 item 1 (LNp.3), TBp.330-331)

H0 HA

H 0:
p=0.5

HA:
p=0.7
 Ch9, p.9

Ch9, p.10

Question 7.5
 Ch9, p.11

Definition 7.4 (test statistic, null distribution, critical value, TBp. 331 & 334)

hypotheses
Q1: How to find a
good test statistic? S Ω0 ΩA Ω

Q2: How to find the S T = t0

critical value?
AR
RR

Ch9, p.12

Example 7.3 (cont. Ex. 7.2)

Question 7.6 (difficulty with level-α tests)

pdf of T under H0 (null distribution)

 Ch9, p.13

Area = α

Area = α

Ch9, p.14

α=

tp-value =

Definition 7.5 (p-value)

Example 7.4 (cont. Ex. 7.2, LNp.8)

 Ch9, p.15

7 8 9 10

Question 7.7 (how to determine H0 and HA)

Some guidelines. (TBp. 335-336)

 Reading: textbook, 9.1, 9.2, 9.2.1, 9.2.2; Further reading: Hogg et al., 5.5

Ch9, p.16

• Neyman Pearson Lemma --- most powerful tests

Question 7.8

Definition 7.6 (uniformly most powerful test, TBp. 336)

• most powerful test for simple vs. simple hypotheses

Definition 7.7 (likelihood ratio, TBp.329)
 Ch9, p.17

Theorem 7.1 (Neyman-Pearson lemma, TBp. 332)

Ch9, p.18

Example 7.7 (cont. Ex. 7.2 (LNp.8), TBp. 329, 330)

Example 7.8 (TBp. 333)

 Ch9, p.19

Question 7.9

• UMP examples for testing certain composite hypotheses

Ch9, p.20
Example 7.9 (cont. Ex. 7.8 (LNp.18), TBp.336)

Example 7.10 (cont. Ex. 7.9, TBp.336)

 Ch9, p.21

Ch9, p.22

Definition 7.7 (test function, nonrandomized test, randomized test)

Theorem 7.2 (significance level, power, and test function)

 Ch9, p.23
• UMP tests for one-parameter exponential family

Theorem 7.3 (UMP test for one-parameter exponential family, one-sided hypothesis)

Ch9, p.24

Question 7.10

Example 7.11 (UMP for i.i.d. Bernoulli)

AR
S RR
 Ch9, p.25

 Reading: textbook, 9.1, 9.2, 9.2.3; Further reading: Roussas, 13.1, 13.2, 13.3

• UMPU tests (for one-parameter exponential family)

Question 7.11

Ch9, p.26

Definition 7.8 (unbiased test)

Theorem 7.4 (UMPU tests for one-parameter exponential family, two-sided hypothesis)

(1)
 Ch9, p.27

Example 7.12 (UMPU test for normal mean)

Ch9, p.28
 Ch9, p.29

Example 7.13 (UMPU test for normal variance)

Ch9, p.30

Definition 7.9 (monotone likelihood ratio)

 Ch9, p.31

Theorem 7.5

Theorem 7.6

 Further reading: Roussas, 13.4, 13.5

• (Generalized) likelihood ratio tests

Question 7.12

Ch9, p.32
Definition 7.10 (generalized likelihood ratio test, TBp. 339)
 Ch9, p.33

Example 7.14 (GLR tests for normal mean with known variance, two-sided, TBp.339-340)

Ch9, p.34

Example 7.15 (GLR tests for normal mean with unknown variance, two-sided)
 Ch9, p.35

Question 7.13

Ch9, p.36

Theorem 7.7 (large sample theory for null distribution of GLR statistics, TBp.341)

 Reading: textbook, 9.4; Further reading: Roussas, 13.7

• Application of GLR test I --- tests for multinomial distribution, goodness-of-fit tests
Example 7.16 (GLR test for multinomial distribution)
 Ch9, p.37

Ch9, p.38

Question 7.14 (examine distribution assumption in statistical modeling  goodness of fit)

 Ch9, p.39

Example 7.17 (GLR tests for goodness-of-fit, TBp.341-342)

Ch9, p.40
 Ch9, p.41

Question 7.15

Example 7.18 (Pearson’s Chi-square test, TBp.342-343)

Ch9, p.42
 Ch9, p.43

Remarks.

Example 7.19 (Hardy-Weinberg Equilibrium, TBp.343-344, or Ex.6.15, LN, Ch8, p.24)

Ch9, p.44
 Ch9, p.45

Example 7.20 (Bacterial Clumps, TBp. 344-345)

Ch9, p.46

Example 7.21 (Fisher’s reexamination of Mendel’s data, TBp. 345-346)

 Ch9, p.47

Ch9, p.48

 Reading: textbook, 9.5

• Application of GLR test II --- Poisson dispersion test

Question 7.16
 Ch9, p.49

Example 7.22 (GLR test for Poisson dispersion, TBp. 347-348)

Ch9, p.50

Example 7.23 (Asbestos Fibers, Poisson dispersion test, TBp. 348)

 Ch9, p.51

Example 7.24 (Bacterial Clumps, Poisson dispersion test, TBp. 348-349)

Question 7.17

Ch9, p.52

 Reading: textbook, 9.6

• Some concerns about hypothesis testing

0 µ

．
．
 Ch9, p.53

Note: this is why we prefer not to say

“accept H0”,
but rather
“sample size is not large enough to reject H0”,
or
“fail to reject H0”

Ch9, p.54

• duality of confidence intervals and hypothesis tests

Example 7.25 (Normal mean, TBp. 337)
 Ch9, p.55

T (test statistic)

confidence interval

Tobs

θ
θ0

acceptance
region of
H0: θ= θ0 vs.
HA : θ≠θ 0

Theorem 7.8 (TBp. 338)

Ch9, p.56

Theorem 7.9 (TBp. 338)

Note.
 Ch9, p.57

Example 7.26 (Normal variance)

 Reading: textbook, 9.3

Ch9, p.58
 Ch9, p.59

Ch9, p.60

• Test for normality

Question 7.18
“X1 , . . . , Xn are i.i.d. from N(µ, σ 2 )” is a very popular assumption in statis-
tical analysis. How can we test the goodness of fit to the normal distribution?

² Pearson’s chi-square (or GLR) test for goodness of fit. In order

for the limiting distribution to be chi-square, the parameters must be
estimated from the grouped data. (Chernoff and Lehmann, 1954).
² Departures from normality often takes the form of skewness or asymmetry.
² A goodness-of-fit test can be based on the coefficient of skewness for
the sample:
1 n 3
n i=1 (Xi ¡ X)
b1 =
s3
Reject the null hypothesis that X1 , ¢¢¢, Xn is a sample from a Normal
distribution for large values of jb1j.
² Symmetric distributions can depart from normality by being heavy-tailed
or light-tailed or too peaked or too flat in the center. Thesis forms of
departures may be detected by coefficient of kurtosis for the sample:
 Ch9, p.61
1 n 4
n i=1 (Xi ¡ X)
b2 = .
s4
² Linearity of the normal probability plot can be measured by the correlation
coefficient r of the normal quantiles and the ordered observations. The
test rejects for small values of r.
² Sampling distributions of b1 , b2 , and r under normal model can be ap-
proximated by simulation.
 Reading: textbook, 9.9

• Some graphical methods for accessing goodness of fit

Recall: the 3rd step in the procedure of statistics --- Data Analysis
• numerical methods
 estimation (point estimation & interval estimation)
 hypothesis testing
 accurate conclusion

 but only based on one (or several) numbers

• graphical methods
 offer intuitive perception and various information

 (could bring) divergent conclusions

Ch9, p.62
• Hanging rootograms
Definition 7.11 (hanging histogram, rootograms, chi-gram TBp. 349-352)
Plot differences between the histogram (i.e., observed counts, Oi ) and ex-
pected counts Ei :

Figure (a): histogram – roughly bell shaped

To examine carefully, if the jth interval is [xj−1 , xj ], the probability
that an observation falls in it is p̂j = Φ ((xj ¡ x̄)/σ̂) ¡ Φ ((xj−1 ¡ x̄)σ̂)
under the normal model (µ, σ estimated by x̄ ,σ̂).
 Ch9, p.63

Figure (b): hanging histogram of the differences:

nj (observed count) ¡ n̂j (expected count) , where n̂j = np̂j
.
V ar(nj ¡ n̂j ) = V ar(nj ) (ignore variability from n̂j )
#
2
1 1
= npj (1 ¡ pj ) = n ¡ pj ¡
4 2
¼ npj (when pj is small)

=) the cell variances are unequal

Theorem 7.10 (variance stabilizing transformation, TBp.351)
Let X be a random variable with mean µ and variance σ 2 = σ 2(µ). Let
Y = f (X), then
V ar(Y ) ¼ V ar(X)[f ′ (µ)]2 = σ 2(µ) ¢f ′ (µ)2
If f is chosen so that σ 2(µ)f ′ (µ)2 is constant, the variance of Y will not
depend on µ. Such an f is called a variance stabilizing transformation.
p
Figure (c): hanging rootogram showing nj ¡ n̂j .
Applying the variance stabilizing transformation to this case:
under H0 (i.e., the model is correct),

Ch9, p.64

E(nj ) = npj = µ
V ar(nj ) ¼ npj = σ 2 (µ)
p
Then µ[f ′ (µ)]2 is constant if f (x) = x and
p p
E( nj ) ¼ npj
p
V ar( nj ) ¼ 1/4 .

Note that in Figure (c), the deviations in the center were down weighted
and deviations in the tails were emphasized.
Figure (d): hanging chi-gram plot, a plot of the components of Pearsons
chi-square statistic: nj ¡ n̂j
n̂j
since, neglecting the variability in n̂j , V ar(nj ¡ n̂j ) ¼ npj = n̂j , so
nj ¡ n̂j
V ar ¼ 1.
n̂j

Question : What more information can you obtain from the plots than a
goodness-of-fit test?
 Reading: textbook, 9.7
 Ch9, p.65
• Probability Plots
Question 7.19
² histogram , pdf/pmf ² ? , cdf
Definition 7.12 (empirical cumulative distribution function, TBp. 378)
² x1 , x2 , . . . , xn : a batch of numbers ) a realization of random variables
X1 , . . . , Xn that are i.i.d. with cdf F .
² empirical cumulative distribution function (ecdf ) is
1
Fn(x) = (#xi · x) ) an estimate of F (x) = P (X · x)
n
² Let x(1) · x(2) · ¢¢¢· x (n) be the ordered numbers of x1 , . . . , xn , then

 0, if x < x(1)
k
Fn(x) = n
, if x(k) · x < x(k+1) , where k = 1, 2, . . . , n ¡ 1

1, if x ¸ x(n)
n
1
or Fn(x) = I(−∞,x] (xi )
n i=1
where I is the indicator function:
1 if Xi · x
I(−∞,x](Xi ) =
0 if Xi > x

Ch9, p.66

² I(−∞,x](Xi ), i = 1, 2, . . . , n are independent Bernoulli random variables.

Thus, nFn (x) is a B(n, F (x)) random variable.
1
E[Fn (x)] = F (x); V ar[Fn(x)] = F (x)[1 ¡ F (x)]
n
Fn (x) unbiased, has maximum variance at the median of F .
Question 7.20
How to examine by graph whether Fn (x) is similar to F (x)?
 Ch9, p.67

Definition 7.13 (probability plot, TBp. 352-354)

² Suppose that it is hypothesized that X follows a certain distribution
F . Given an i.i.d. sample X1 , . . . , Xn » F , plot
k
F (X(k) ) versus
n+1
or equivalently
k
X(k) versus F −1 .
n+1
² Note that we are plotting the ordered observations (which may be
viewed as the observed or empirical quantiles) versus the quantiles of
the hypothetical distribution.
² a rationale of probability plot: If X is a continuous random variable
with a strictly increasing c.d.f. F . Then Y = F (X) has a Uniform(0,1)
distribution
P (Y · y) = P (F (X) · y) = P (X · F −1 (y)) = F F −1 (y) = y,
for 0 < y < 1, and
k
E(Y(k) ) = E(F (X(k) )) = .
n+1

Ch9, p.68

² probability plot for location-scale family. Suppose F is of the

form x¡ µ
Fµ,σ (x) = G ,
σ
where µ is the location parameter and σ is the scale parameter. Plot
X(k) ¡ µ k
against G−1
σ n+1
or plot k
X(k) against G−1 .
n+1
If the model were correct, the result would be approximately a straight
line: k
X(k) ¼ σG−1 + µ.
n+1
k
A slight modification uses E(X(k) ) instead of G−1 n+1 . But it can be
argued that

−1 k k
E(X(k) ) ¼ Fµ,σ ¼ σG−1 +µ
n+1 n+1
so this modification yields very similar results to the original procedure.
 Ch9, p.69

² probability plot for grouped data Suppose the grouping gives

t0 , ¢¢¢, tm for the bin boundaries of the histogram. In the interval
[ti−1, ti ) there are Oi counts, i = 1, ¢¢¢, m. Denote the cumulative
j
frequencies by Nj = i=1 Oi . We plot
Nj
tj versus G−1 , j = 1, ¢¢¢, m.
n+1

² Caution
— Probability plots are by nature monotone increasing. Some experi-
ence is necessary in gauging “straightness”
— Simulations are very useful in sharpening one’s judgment.
 Reading: textbook, 9.8, 10.2.1