Linear Regression

savvy

New member
Joined
Jun 18, 2026
Messages
0
Reaction score
0
Can someone explain why we use estimates of the parameter values as oppose to the actual values? And how is using estimates different from a sample, or is it the same given that it still remains to be a parameter and not statistic? Thanks in advance.
 
when making inferences about a population, say all males, you need to take a sample. you cannot get a "true" population parameter unless you observe every male on earth. hence, you need to get a statistic from a sample of the population.

statistics are from samples and parameters are from the population. parameters are supposed to be estimated since they are unobservable. when you say "estimate" it is an estimate of the entire population and not an estimate of the sample that you've just taken.

hope this helps
 
" we never observe the population parameter values b0 and b1 in a regression model. Instead, we observe only estimates (b1 & b0), which are estimates of the population parameter values..." p.519 (CFAI)

- I still don't have a clear understanding of the above statement and why we use estimates if "actual" values are available

- Also, can we claim that the estimates used are the most optimal?
 
Population (all males on the earth): average height is a descriptive statistic. It's easy to define and easy to calculate if we can measure all males.

Sample (subset of the population, for example, random 1,000 males). If we calculate average height of the sample and the sample is representative (we are not measuring 1,000 basketball players in a summer camp) it is a random variable that converges to the average height of all males on the earth as we increase the sample size.
 
We don't know what the actual parameters are but we can estimate them using some kind of optimization algorithm that minimizes error. Typically least mean square optimization. Error function is equal to the sum of squared deviations of the series from its linear regression for a given set of parameters (alpha, beta) or (alpha, beta1, beta2, ..., betan). Minimization of the error function gives the estimates that are least mean square optimal.
 
savvy Wrote:
-------------------------------------------------------

[...]
> - I still don't have a clear understanding of the
> above statement and why we use estimates if
> "actual" values are available

No, actual values are not available unless you are able to survey all males on earth and take their height. That would be very difficult and costly.

Because we cannot survey all males, we get a sample of them, and assume that their height is an estimate of the height of all males. I can't think of a simpler way to explain it.
 
^sorry, i was referring to the quote from the book. All your examples make perfect sense but its the formula itself that gives me the problem.

Let me try again. when solving for the dependent variable Y we set it equal to the intercept, b0, plus a slope coefficient, b1, times the independent variable, X, plus an error term.

When we compute for a line that best fits the observation the formula is the Sum of independent variable minus, b0(hat), mines slope coefficient, b1(hat), times the independent variable , X, squared.

- in the second formula we use hats over the regression coefficients to indicate estimated values... the book states that " we never observe the population parameter values b0 and b1 in a regression model. Instead, we observe only estimates (b1 & b0), which are estimates of the population parameter values..." p.519 (CFAI)

I understand that the first formula is used to plot the points on the graph and the second formula is used to fit a line through the plots allowing us to derive inferential observation. But, the book doesn't make it clear as to why we use estimates of the regression coefficients for the "line formula" when it is not necessary for the "Y-dependent" formula? I hope this question sounds reasonably clear and please feel free to correct me if anything above is wrong -thanks
 
I suppose that the "Y-dependent" formula is " when solving for the dependent variable Y we set it equal to the intercept, b0, plus a slope coefficient, b1, times the independent variable, X, plus an error term.", yes?

So in this formula, we are suggesting a model of the data. For example, I am saying that Y = lifetime income and X = years of education and that Y = b0 + b1* X + error. At this point I know nothing about b0 or b1 or anything about that error term. I have merely suggested a model that might help me either understand the relationship or predict lifetime income when given years of education. Probably I believe that b1 > 0 so that for each additional year of education someone gets, their lifetime earnings go up by, say, $50000. If I found that, I could sell my analysis to public television who could then have Anna Nicole Smith say that "If you want to have money, stay in schooool" (forgetting for a moment that she is dead). My next step would be to gather data, estimate b0 and b1 from the data (and I would call my estimates b0-hat, b1-hat), and see if the data suggest that my model is right.

The "population values" for b0 and b1 are unknowable. Even if you could sample all trhe people in the world, your equation is broader than that. It asks about all the people who are or could have been or will be and what would happen to them if they had more or less education. God knows b0 and b1 and your mission is to estimate a truth known only to Him (every once in a while, God sees this as hubris and throws a statistician into the lake of fire. It usually doesn't happen to CFA candidates).

The cool thing is that even though I have this very vast unobservable universe of possibilities that I am trying to make inferences about, if I do my sampling right and I make some possibly defensible assumptions about that error term, I can make reasonable good inferences that depend only on my sample size.
 
Joey, I have read many of your posts, and just love your sense of humor!!!
 
Anna's Mom? I'm sorry if that reference to your daughter offended you, Mrs Tabers.
 
Actually, my comment was meant very sincerely. I have always appreciated a quick wit.

My Anna is only 5 years old, and if she ever even thinks about pulling any of the stunts that the "other Anna" was famous for, she will promptly be locked in her room until she is 30!!!
 
so the wise owl appears at dawn... thanks.
 
Back
Top