## The Gauss Markov Theorem

The Gauss-Markov Theorem is telling us that in a regression model, where the expected value of our error terms is zero, $E(\epsilon_{i}) = 0$ and variance of the error terms is constant and finite $\sigma^{2}(\epsilon_{i}) = \sigma^{2} \textless \infty$ and $\epsilon_{i}$ and $\epsilon_{j}$ are uncorrelated for all $i$ and $j$ the least squares estimator $b_{0}$ and $b_{1}$ are unbiased and have minimum variance among all unbiased linear estimators. Note that there might be biased estimator which have a even lower variance.

You can find the exact proof of the Gauss-Markov Theorem here.

A short reminder of what estimator for $b_{0}$ and $b_{1}$ look like:

(1) $b_{1}=\frac{\sum(\textbf{X}_{i}-\bar{\textbf{X}})(\textbf{Y}_{i}-\bar{\textbf{Y}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}} = \sum \textbf{k}_{i}\textbf{Y}_{i}, \textbf{k}_{i}=\frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}}$

(2) $b_{0}=\bar{Y}-b_{1}\bar{\textbf{X}}$

The exact derivation of the least squares estimator in matrix notation can be found here.

The variance estimator for $b_{1}$ looks the following way:

(3) $\sigma^{2}(b_{1})=\sigma^{2}(\sum \textbf{k}_{i}\textbf{Y}_{i})=\sum \textbf{k}_{i}^{2} \sigma^{2}(\textbf{Y}_{i})=\sigma^{2}(\frac{1}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}})$

The Gauss-Markov Theorem is telling us that $b_{1}$ has minimum variance among all unbiased linear estimators of the form

(4) $\hat{\beta}_{1} = \sum c_{i}Y_{i}$

The Gauss-Markov Theorem is further telling us the this estimator must be unbiased. Under the assumption of unbiasedness we know that

(5) $E(\hat{\beta}_{1}) = \sum c_{i}E(Y_{i})$

(6) $E(\hat{\beta}_{1}) = \sum c_{i}E(\beta_{0}+\beta_{1}\textbf{X}_{i})$

(7) $E(\hat{\beta}_{1}) = \beta_{0} \sum c_{i} + \beta_{1} \sum c_{i} \textbf{X}_{i} = \beta_{1}$

(8) $E(\hat{\beta}_{1}) =\beta_{1}$

If it holds that $E(\hat{\beta}_{1}) =\beta_{1}$ we know that $c_{i}$ must have certain properties/characteristics, i.e. it means that the assumption of unbiasedness imposes some restrictions on the $c_{i}$.

You can find the exact proof of the Gauss-Markov Theorem here.