The Gauss Markov Theorem

The Gauss-Markov Theorem is telling us that in a regression model, where the expected value of our error terms is zero, E(\epsilon_{i}) = 0 and variance of the error terms is constant and finite \sigma^{2}(\epsilon_{i}) = \sigma^{2} \textless \infty and \epsilon_{i} and \epsilon_{j} are uncorrelated for all i and j the least squares estimator b_{0} and b_{1} are unbiased and have minimum variance among all unbiased linear estimators. Note that there might be biased estimator which have a even lower variance.

You can find the exact proof of the Gauss-Markov Theorem here.

A short reminder of what estimator for b_{0} and b_{1} look like:

(1) b_{1}=\frac{\sum(\textbf{X}_{i}-\bar{\textbf{X}})(\textbf{Y}_{i}-\bar{\textbf{Y}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}} = \sum \textbf{k}_{i}\textbf{Y}_{i}, \textbf{k}_{i}=\frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}}

(2) b_{0}=\bar{Y}-b_{1}\bar{\textbf{X}}

The exact derivation of the least squares estimator in matrix notation can be found here.

The variance estimator for b_{1} looks the following way:

(3) \sigma^{2}(b_{1})=\sigma^{2}(\sum \textbf{k}_{i}\textbf{Y}_{i})=\sum \textbf{k}_{i}^{2} \sigma^{2}(\textbf{Y}_{i})=\sigma^{2}(\frac{1}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}})

The Gauss-Markov Theorem is telling us that b_{1} has minimum variance among all unbiased linear estimators of the form

(4) \hat{\beta}_{1} = \sum c_{i}Y_{i}

The Gauss-Markov Theorem is further telling us the this estimator must be unbiased. Under the assumption of unbiasedness we know that

(5) E(\hat{\beta}_{1}) = \sum c_{i}E(Y_{i})

(6) E(\hat{\beta}_{1}) = \sum c_{i}E(\beta_{0}+\beta_{1}\textbf{X}_{i})

(7) E(\hat{\beta}_{1}) = \beta_{0} \sum c_{i} + \beta_{1} \sum c_{i} \textbf{X}_{i} = \beta_{1}

(8) E(\hat{\beta}_{1}) =\beta_{1}

If it holds that E(\hat{\beta}_{1}) =\beta_{1} we know that c_{i} must have certain properties/characteristics, i.e. it means that the assumption of unbiasedness imposes some restrictions on the c_{i}.

You can find the exact proof of the Gauss-Markov Theorem here.

Advertisements
This entry was posted in Econometrics, Mathematical Proofs, Statistic and tagged , , , , . Bookmark the permalink.

4 Responses to The Gauss Markov Theorem

  1. Pingback: Assumptions of classical linerar Regressionmodels | Economic Theory Blog

  2. Pingback: Proof Gauss Markov Theorem | Economic Theory Blog

  3. Pingback: Robust Standard Errors | Economic Theory Blog

  4. Pingback: Clustered Standard Errors | Economic Theory Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s