BECO2001 Econometrics I, 2024/2025-1
SIMPLE REGRESSION MODEL: SPECIFICATION
(Textbook: Chapter 2, Section 2.1, Section 2.2 (part), Section 2.4c, Section 2.5 (part))
Regression analysis
We are interested in “explaining Y in terms of X”, or in “studying how Y varies with changes
in X”. Regression analysis is used for estimating the population mean value of the dependent
variable, Y, on the basis of the known or fixed values of the explanatory variable(s), X.
Consider the bivariate, or two-variable, simple linear regression model.
Regression Function
Population regression function (PRF)
In the population, the random variable Y is a variable of interest. Its values depend on
another variable X, according to economic theory/economic reasoning. Thus, the conditional
pdf of Y, fY|X(y|x), describes how the values of Y are distributed over the population, given a
specific value for X. The conditional mean of Y, E(Y|X) = Y, is the population mean of Y,
given X. Also, the conditional variance of Y, Var(Y|X) = Y2, measures the dispersion of Y
about its mean, given X. The population parameters Y and Y2 give valuable information
about the population under consideration.
1
PRF:
The PRF states the functional relationship between the “expected value of Y given Xi” and
“Xi”:
E(𝑌|𝑋𝑖) = 𝜇𝑖 = 𝑓(𝑋𝑖) = 𝛽0 + 𝛽1𝑋𝑖 [economic (mathematical) model]
𝛽0 is the intercept parameter, whereas 𝛽1 is the slope parameter. These 's are unknown
fixed population parameters. Regression analysis concerns the estimation of the values of
these unknown parameters on the basis of observations on Y and X.
In a linear regression model, there is linearity in the parameters 's, but not necessarily in the
variables. A function is said to be linear in the parameter if it is not governed by any
mathematical function, appears with a power of 1 only, and is not multiplied or divided by
any other parameter.
Sample Regression Function (SRF)
In practice, only a randomly selected sample of Y values corresponding to some fixed X 's is
available. The PRF is estimated from the sample data, and the estimated function is called the
𝑌̂𝑖 = 𝛽̂ 0 + 𝛽1̂ 𝑋
SRF, given by:
̂
where 𝑌̂ 𝑖 is an estimator of E(Y|Xi), 𝐸 (𝑌 | 𝑋 𝜄 ) , and 𝛽̂𝑘 is an estimator of k, k = 0, 1.
�,
�
2
Because of sampling fluctuations, different SRFs are obtained from different samples.
Individual values of Y given Xi cluster around E(𝑌|𝑋𝑖) = 𝛽0 + 𝛽1𝑋𝑖 . In other
Stochastic specification of regression function
consists of a systematic/deterministic/nonrandom/nonstochastic component, E(𝑌|𝑋𝑖) = 𝛽0
words, Y
+
𝛽1𝑋𝑖, and a nonsystematic/nondeterministic/random/stochastic component, Yi - E(𝑌|𝑋𝑖)
= ui, called the random (or stochastic) error (or disturbance) term. Thus, the stochastic
𝑌𝑖 = E(𝑌|𝑋𝑖) + 𝑢𝑖 = 𝛽0 + 𝛽1𝑋𝑖 + 𝑢𝑖 [econometric model]
specification of the PRF is:
𝑌𝑖 = 𝑌̂𝑖 + 𝑢̂ 𝑖 + 𝛽̂ + 𝑢̂ 𝑖 ,
The stochastic specification of the SRF is:
= 𝛽̂ 𝑋
where 𝑢̂ 𝑖 denotes the residual 0 1 𝑖
term.
Components of the random error term (𝑢𝑖):
1. Excluded / omitted variables
(a) Vagueness of theory about other variables affecting Y.
(b) Unavailability of data on other variables affecting Y.
(c) Parsimony in the case of small joint influence of other variables.
2. Errors in variables due to measurement errors and/or poor proxy variables.
3. Wrong functional form between X and Y.
4. Intrinsic randomness of individual Y 's.
sampling fluctuations, the SRF is at best an approximation of the PRF. 𝑌̂ 𝑖 may
Regression analysis amounts to estimating the PRF on the basis of the SRF. Because of
overestimate or underestimate the true E(𝑌|𝑋𝑖).
3
𝑌𝑖̂ = 𝛽̂0 +1𝛽̂
𝑋
𝑖
E(𝑌|𝑋𝑖) = 𝛽0 +
𝛽1𝑋𝑖
Assumptions of a linear regression model
1. Linearity in the parameters of k, k = 0, 1
2. Sample variation in the regressor X: 0 < Var(X) <
3. Nature of regressor
Zero covariance between error term and random regressor
[Nonrandom (fixed in repeated sampling, or known) regressor]
Cov(Xi, ui) = 0 (by definition)
4. Zero expected value of error term
E(ui| Xi) = 0 (by properties of conditional expectations)
E(ui) = E(ui| Xi) = 0 (by law of iterated expectation)
5. Homoscedasticity (or equal variance) of error term
Var(ui|Xi) = E(ui2|Xi) – [E(ui|Xi)]2 = E(ui2|Xi) = 2 (by definition)
Var(ui) = Var(ui|Xi) = 2 (by properties of conditonal mean and conditional variance)
(The situation in which Var(ui|Xi) = i2 is known as heteroscedasticity.)
4
6. No autocorrelation (or serial correlation) among error terms
Cov(ui, uj|Xi, Xj) = E[(ui| Xi)(uj| Xj)] - E(ui| Xi)E(uj| Xj) = E[(ui| Xi)(uj| Xj)] = 0, i ≠ j
(by definition)
It can be shown that Cov(ui, uj|Xi, Xj) = Cov(ui, uj) = 0.
The stronger version of this assumption is that the error terms are independent (with
random sampling for example).
7. Error terms normally distributed
(not needed for point estimation, needed for interval estimation and hypothesis testing).
𝑢
|𝑋 𝑖𝑖𝑑 2
𝑖 𝑖
⬚
𝑜𝑟⬚ 𝑢𝑖 ~ 𝑁(0, 𝜎 )
