Category Archives: Mathematical Proofs

Derivation of the Least Squares Estimator for Beta in Matrix Notation – Proof Nr. 1

In the post that derives the least squares estimator, we make use of the following statement:

\frac{\partial b'X'Xb}{\partial b} =2X'Xb

This post shows how one can prove this statement. Let’s start from the statement that we want to prove:

\frac{\partial \hat{\beta}'X'X\hat{\beta}}{\partial \hat{\beta}}=2 X'X \hat{\beta}'

Continue reading Derivation of the Least Squares Estimator for Beta in Matrix Notation – Proof Nr. 1

Advertisements

Unbiased Estimator of Sample Variance – Vol. 2

Lately I received some criticism saying that my proof (link to proof) on the unbiasedness of the estimator for the sample variance strikes through its unnecessary length. Well, as I am an economist and love proofs which read like a book, I never really saw the benefit of bowling down a proof to a couple of lines. Actually, I hate it if I have to brew over a proof for an hour before I clearly understand what’s going on. However, in order to satisfy the need for mathematical beauty, I looked around and found the following proof which is way shorter than my original version.

Continue reading Unbiased Estimator of Sample Variance – Vol. 2

The Gauss Markov Theorem

When studying the classical linear regression model, one necessarily comes across the Gauss-Markov Theorem. The Gauss-Markov Theorem is a central theorem for linear regression models. It states different conditions that, when met, ensure that your estimator has the lowest variance among all unbiased estimators. More formally, Continue reading The Gauss Markov Theorem

What is an indirect proof?

In economics, especially in theoretical economics, it is often necessary to formally prove your statements. Meaning to show your statements are correct in a logical way. One possible way of showing that your statements are correct is by providing an indirect proof. The following couple of lines try to explain the concept of indirect proof in a simple way.

Continue reading What is an indirect proof?

Proof of Unbiasedness of Sample Variance Estimator

Proof of Unbiasness of Sample Variance Estimator

(As I received some remarks about the unnecessary length of this proof, I provide shorter version here)

In different application of statistics or econometrics but also in many other examples it is necessary to estimate the variance of a sample. The estimator of the variance, see equation (1) is normally common knowledge and most people simple apply it without any further concern. The question which arose for me was why do we actually divide by n-1 and not simply by n? In the following lines we are going to see the proof that the sample variance estimator is indeed unbiased. Continue reading Proof of Unbiasedness of Sample Variance Estimator