In many scientific fields, such as economics, political science and electrical engineering, ordinary least squares (OLS) or linear least squares is the standard method to analyze data. In my eyes, every scientist, data analyst or informed person should have a minimal understanding of this method, in order to understand, interpret and judge the validity of results presented in studies and reports. This page provides an overview and introduction of the ordinary least squares estimator, sometimes also referred to as the classical regression model (CLRM).

 

What does the OLS estimator do?

In data analysis, we use OLS for estimating the unknown parameters in a linear regression model. The goal is minimizing the differences between the collected observations in some arbitrary dataset and the responses predicted by the linear approximation of the data. We can express the estimator by a simple formula. You can find the general OLS formula and its exact mathematical derivation here.

 

Why exactly OLS?

Generally, we are looking for the best estimator when analyzing your data. In data analysis the best estimator is refer to as BLUE (best linear unbiased estimator). The Gauss-Markov theorem shows that, if your data fulfill certain requirements, OLS is the best linear unbiased estimator available, i.e. that OLS is BLUE. The mathematical proof that OLS is the best linear unbiased estimator when our data fulfill all required assumptions can be found here.

 

When to use OLS?

The OLS estimator is consistent when the Gauss-Markov assumptions (sometimes called OLS assumptions or assumptions of the CLRM) are met. You can find an overview and a more profound discussion of these assumptions here. In summary, the OLS estimator requires that the explanatory variables are exogenous and there is no perfect multicollinearity. Furthermore, OLS is optimal in the class of linear unbiased estimators when the errors are homoscedastic and serially uncorrelated. Under these conditions, the method of OLS provides minimum-variance mean-unbiased estimation when the errors have finite variances. Under the additional assumption that the errors be normally distributed, OLS is the maximum likelihood estimator.