Proof Gauss Markov Theorem

From a previous posts on the Gauss Markov Theorem and OLS we know that the assumption of unbiasedness must full fill the following condition

(1) $E(\hat{\beta}_{1}) = \beta_{0} \sum c_{i} + \beta_{1} \sum c_{i} \textbf{X}_{i} = \beta_{1}$

which means that $\sum c_{i} = 0$ and $\sum c_{i} \textbf{X}_{i} = 1$ .

Looking at the estimator of the variance for $\beta_{1}$

(2) $\sigma^{2}(\hat{\beta}_{1})= \sum c_{i}^{2}\sigma^{2}(\textbf{Y}_{i})=\sigma^{2} \sum c_{i}^{2}$

tells us that the estimator put additional restrictions on the $c_{i}$ ‘s.

To continue the proof we define $c_{i} = k_{i}+d_{i}$ , where $k_{i}$ are the constants we already defined above. $d_{i}$ on the other hand are arbitrary constants. What we are basically saying is that $c_{i}$ is composed of a constant $k_{i}$ (see above for definition) and an arbitrary constant $d_{i}$ . Let’s continue by plugging in the definition $c_{i} = k_{i}+d_{i}$ in the equation for the variance estimator for $\hat{\beta_{1}}$ .

(3) $\sigma^{2}(\hat{\beta}_{1})= \sum c_{i}^{2}\sigma^{2}(\textbf{Y}_{i})$

(4) $\sigma^{2}(\hat{\beta}_{1})=\sigma^{2} \sum c_{i}^{2}$

(5) $\sigma^{2}(\hat{\beta}_{1})=\sigma^{2} \sum (k_{i}+d_{i})^{2}$

(6) $\sigma^{2}(\hat{\beta}_{1})=\sigma^{2} (\sum k_{i}^{2}+2\sum k_{i}d_{i}+\sum d_{i}^{2})$

Note that Equation 6 is much more than just a transformation, it tells us that

(7) $\sigma^{2} \sum k_{i}^{2} = \sigma^{2}(b_{1})$

Now, why do we need equation 7? Well, it shows us the relationship between $\sigma^{2}(\hat{\beta_{1}})$ and $\sigma^{2}(b_{1})$ . It basically shows, that $\sigma^{2}(\hat{\beta_{1}})$ is related to $\sigma^{2}(b_{1})$ plus some extra “stuff”.

Finally, what we need to do to terminate the proof is to show that some of this extra stuff ( $\sum k_{i}d_{i}$ is actually zero.

(8) $\sum k_{i}d_{i} = \sum k_{i}(c_{i}-k_{i})$

(9) $\sum k_{i}d_{i} = \sum k_{i}c_{i} - \sum k_{i}^{2}$

(10) $\sum k_{i}d_{i} = \sum c_{i} \frac{X_{i} -\bar{X}}{\sum (X_{i} -\bar{X})^{2}} - \frac{1}{\sum(X_{i}-\bar{X})^{2}}$

(11) $\sum k_{i}d_{i} = \frac{\sum c_{i}X_{i} - \sum c_{i}\bar{X}}{\sum (X_{i} -\bar{X})^{2}} - \frac{1}{\sum(X_{i}-\bar{X})^{2}}$

(12) $\sum k_{i}d_{i} = \frac{\sum c_{i}X_{i} - \bar{X}\sum c_{i}}{\sum (X_{i} -\bar{X})^{2}} - \frac{1}{\sum(X_{i}-\bar{X})^{2}}$

And as we know is $\sum c_{i} = 0$ and $\sum c_{i} \textbf{X}_{i} = 1$ , which, when plugging into equation 21, shows us that the some of the extra “stuff” is actually zero. For the sake of completeness

(13) $\sum k_{i}d_{i} = \frac{1 - \bar{X}0}{\sum (X_{i} -\bar{X})^{2}} - \frac{1}{\sum(X_{i}-\bar{X})^{2}}$

(14) $\sum k_{i}d_{i} = \frac{1}{\sum (X_{i} -\bar{X})^{2}} - \frac{1}{\sum(X_{i}-\bar{X})^{2}}=0$

(15) $\sum k_{i}d_{i} = 0$

Having proofed that $\sum k_{i}d_{i} = 0$ reduces equation 15 to

(16) $\sigma^{2}(\hat{\beta}_{1})=\sigma^{2} (\sum k_{i}^{2}+\sum d_{i}^{2})$

(17) $\sigma^{2}(\hat{\beta}_{1})=\sigma^{2} (\sum k_{i}^{2}) + \sigma^{2} (\sum d_{i}^{2})$

Now we will show that $\sigma^{2} (\sum k_{i}^{2}) = \sigma^{2}(b_{1})$ . Note that from equation 3 we know that

(18) $\sigma^{2}(b_{1}) =\sigma^{2}(\frac{1}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}})$

and from equation 1 we know that

(19) $\textbf{k}_{i}=\frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}}$

this lead us to

(20) $\textbf{k}_{i}^{2}=\frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}} \frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}}$

(21) $\sum \textbf{k}_{i}^{2}= \sum \frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}} \frac{(\textbf{X}_{i}-\bar{\textbf{X}})}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}} = \frac{1}{\sum(\textbf{X}_{i}-\bar{\textbf{X}})^{2}}=b_{1}$