Takashi Yamano
Fall Semester 2009
Lecture Notes on Advanced Econometrics
Lecture 10: GLS, WLS, and FGLS
Generalized Least Square (GLS)
So far, we have been dealing with heteroskedasticity under OLS framework. But if we
knew the variance-covariance matrix of the error term, then we can make a
heteroskedastic model into a homoskedastic model.
As we defined before
E (uu ) = 2 = .
Define further that
1 = P P
P is a n x n matrix
Pre-multiply P on a regression model
Py = PX + Pu
or
~y = X~ + u~
In this model, the variance of u~ is
E (u~u~ ) = E ( Puu P ) = PE (uu ) P = P 2 P = 2 PP = 2 I
Note that PP = I , because define PP = A , then P PP = P A . By the definition of
P, 1P = P A , thus P = P A . Therefore, A must be I.
Because E (u~u~ ) = 2 I , the model satisfies the assumption of homoskedasticity. Thus,
we can estimate the model by the conventional OLS estimation.
Hence,
~~ ~
= ( X X ) 1 X ~y
= ( X P PX ) 1 X P Py
= ( X 1 X ) 1 X 1 y
is the efficient estimator of . This is called the Generalized Least Square (GLS)
~
estimator. Note that the GLS estimators are unbiased when E (u~ | X ) = 0 . The variance
of GLS estimator is
) = 2 ( X~ X~ ) 1 = 2 ( X 1 X ) 1 .
var(
Note that, under homoskedasticity, i.e., 1 =I, GLS becomes OLS.
The problem is, as usual, that we dont know 2 or . Thus we have to either assume
or estimate empirically. An example of the former is Weighted Least Squares
Estimation and an example of the later is Feasible GLS (FGLS).
Weighted Least Squares Estimation (WLS)
Consider a general case of heteroskedasticity.
Var(ui) = i2 = 2 i .
Then,
E (uu ) = 2
1 0
0
2
0 0
0
0
= 2 , thus 1 =
11 0
1
0 2
0 0
0
.
n1
Because of 1 = P P , P is a n x n matrix whose i-th diagonal element is 1 / i . By
pre-multiplying P on y and X, we get
y1 / 1
y2 / 2
y* = Py =
y /
n
n
and
1 / 1 x11 / 1 ... x1k / 1
1 / 2 x 21 / 2 ... x 2 k / 2
X * = PX =
.
1 / x / ... x /
n
n1
n
nk
n
The OLS on y* and X * is called the Weighted Least Squares (WLS) because each
variable is weighted by i . The question is: where can we find i ?
Feasible GLS (FGLS)
Instead of assuming the structure of heteroskedasticity, we may estimate the structure of
heteroskedasticity from OLS. This method is called Feasible GLS (FGLS). First, we
instead of .
from OLS, and, second, we use
estimate
1 X ) 1 X
1 y
FGLS = ( X
There are many ways to estimate FGLS. But one flexible approach (discussed in
Wooldridge page 277) is to assume that
var(u | X ) = u 2 = 2 exp( 0 + 1 x1 + 2 x 2 + ... + k x k )
By taking log of the both sides and using u 2 instead of u 2 , we can estimate
log(u 2 ) = 0 + 1 x1 + 2 x 2 + ... + k x k + e .
The predicted value from this model is g i = log(u 2 ) . We then convert it by taking the
exponential into i = exp( g i ) = exp(log(u 2 ) ) = u 2 . We now use WLS with weights
1 / or 1 / u 2 .
i
Example 1
. * Estimate the log-wage model by using WAGE1.dta with WLS
. * Weight is educ
.
.
.
.
.
.
.
* Generate weighted varaibles
gen w=1/(educ)^0.5
gen wlogwage=logwage*w
gen wfemale=female*w
gen weduc=educ*w
gen wexper=exper*w
gen wexpsq=expsq*w
. * Estimate weighted least squares (WLS) model
. reg wlogwage weduc wfemale wexper wexpsq w, noc
Source |
SS
df
MS
---------+-----------------------------Model | 113.916451
5 22.7832901
Residual | 7.12253755
519 .013723579
---------+-----------------------------Total | 121.038988
524 .230990435
Number of obs
F( 5,
519)
Prob > F
R-squared
Adj R-squared
Root MSE
=
524
= 1660.16
= 0.0000
= 0.9412
= 0.9406
= .11715
-----------------------------------------------------------------------------wlogwage |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
---------+--------------------------------------------------------------------
weduc |
.080147
.006435
12.455
0.000
.0675051
.0927889
wfemale | -.3503307
.0354369
-9.886
0.000
-.4199482
-.2807133
wexper |
.0367367
.0045745
8.031
0.000
.0277498
.0457236
wexpsq | -.0006319
.000099
-6.385
0.000
-.0008264
-.0004375
w |
.4557085
.0912787
4.992
0.000
.2763872
.6350297
------------------------------------------------------------------------------
End of Example 1
Example 2
. * Estimate reg
. reg logwage educ female exper expsq
(Output omitted)
. predict e, residual
. gen logesq=ln(e*e)
. reg logesq educ female exper expsq
(output omitted)
. predict esqhat
(option xb assumed; fitted values)
. gen omega=exp(esqhat)
.
.
.
.
.
.
.
* Generate weighted varaibles
gen w=1/(omega)^0.5
gen wlogwage=logwage*w
gen wfemale=female*w
gen weduc=educ*w
gen wexper=exper*w
gen wexpsq=expsq*w
. * Estimate Feasible GLS (FGLS) model
. reg wlogwage weduc wfemale wexper wexpsq w, noc
Source |
SS
df
MS
---------+-----------------------------Model | 31164.1981
5 6232.83962
Residual | 2060.77223
519 3.97065941
---------+-----------------------------Total | 33224.9703
524
63.406432
Number of obs
F( 5,
519)
Prob > F
R-squared
Adj R-squared
Root MSE
=
524
= 1569.72
= 0.0000
= 0.9380
= 0.9374
= 1.9927
-----------------------------------------------------------------------------wlogwage |
Coef.
Std. Err.
t
P>|t|
[95% Conf. Interval]
---------+-------------------------------------------------------------------weduc |
.0828952
.0069779
11.880
0.000
.0691868
.0966035
wfemale | -.2914609
.0349884
-8.330
0.000
-.3601971
-.2227246
wexper |
.0376525
.004497
8.373
0.000
.0288179
.0464872
wexpsq | -.0006592
.0001008
-6.540
0.000
-.0008573
-.0004612
w |
.3848487
.0950576
4.049
0.000
.1981038
.5715936
------------------------------------------------------------------------------
End of Example 2
