Serving the Quantitative Finance Community

 
User avatar
musicgold1
Topic Author
Posts: 1
Joined: March 25th, 2008, 3:40 pm

Assumptions behind the OLS regression model?

May 11th, 2009, 3:00 pm

Hi,In many statistics textbooks I read the following text: “A models based on ordinary linear regression equation models Y, the dependent variable, as a normal random variable, whose mean is linear function of the predictors, b0 + b1*X1 + ... , and whose variance is constant. While generalized linear models extend the linear model in two ways. First, assumption of linearity in the parameters is relaxed, by introducing the link function. Second, error distributions other than the normal can be modeled.”My Statistics teacher never bothered to explain these things to us. He started the regression lesson with the equation Y = b0 + b1 * X1, and an example based on the Weight and Height relation. He never talked about these assumptions about normality and the variance. As a result for quite some time, I treated this equation was an identity, similar to Assets= Liability + Equity. I have never understood what difference those underlying assumptions make. Can anybody please explain me why these assumptions are required for this model, and what happens to the result of this model if these assumptions are violated? Thanks,MG.
 
User avatar
brophypt
Posts: 0
Joined: November 27th, 2007, 6:27 pm

Assumptions behind the OLS regression model?

May 11th, 2009, 4:16 pm

I recommend, Gujarati, D.N., 2003, Basic Econometrics, International Edition, McGraw Hill,
 
User avatar
bilbo1408
Posts: 0
Joined: August 3rd, 2007, 12:50 pm

Assumptions behind the OLS regression model?

May 11th, 2009, 4:53 pm

I second that. The answer to your question takes up about 500 pages in that book.
 
User avatar
quantmeh
Posts: 0
Joined: April 6th, 2007, 1:39 pm

Assumptions behind the OLS regression model?

May 11th, 2009, 7:06 pm

actually, OLS parameter estimates will work even in non-normal error distributions, the standard confidence interval formulas wont work though
 
User avatar
brianjd
Posts: 0
Joined: April 29th, 2004, 11:25 pm

Assumptions behind the OLS regression model?

May 12th, 2009, 4:06 am

QuoteCan anybody please explain me why these assumptions are required for this model, and what happens to the result of this model if these assumptions are violated? First off, for your question to make sense you need, e.g., "to produce consistent/unbiased/efficient parameter estimates" to follow "...for this model". So, certain model assumptions will produce desirable properties for your estimates of the true underlying population parameters (assuming we're Frequentists of course)One thing that took me a while to wrap my head around and is important to understand before talking about the statistics: OLS is just a dumb, line-fitting method. There's no stats involved! Draw a scatter plot of some height vs weight data. Draw the best (in the sense of minimizing the sum of squared residuals) line through the data. There you go. That's OLS. Not much to it. The complicated statistical stuff comes when you assume that your data is actually a sample from some true population. Then, the parameters you calculate via OLS are no longer just some dumb parameters that produce the best line fit but are also interpreted as estimates of some true, underlying population parameters. Anyway, so what's the answer? As other people in this thread alluded, that is usually the subject of an entire introductory econometrics course. My favorite friendly, intuition-providing text is "A Guide to Econometrics" by Kennedy and favorite intro course text is "Introductory Econometrics" by Wooldridge. The latter is expensive but can be found free (illegally of course) on torrent networks. I could go through and explain each assumption and what happens when they're violated...but Kennedy (starting pg. 48-49, and then chapters 6-11 (each one dedicated to a violation of a different assumption)) does this so well that there's really no point. Take a look at his book. One warning: I've never seen two econometrics texts that list the same set of assumptions for the classical linear regression model (to be consistent/unbiased/efficient/etc.). You can arrive at these desirable qualities of the parameter estimates by strengthening some assumptions and weakening others. If you can understand the implications of different authors' assumptions and how they relate then you're a real econometrics stud.