Sample Size Effect on Type1 and Type2 Error

Y

Young_Prof

New member
Joined
Jun 18, 2026
Messages
0
Reaction score
0
Looks like an earlier thread was deleted that contained a this discussion, hopefully this one will fair better...

There was a debate about whether increasing the sample size will increase or decrease the probability of a type1 and/or type2 error. A lot of people seemed to think that increasing the sample size would decrease the probability of a type1 and type2 error.

Holding all else constant, I don't think that changing the sample size will change the probability of either type of error; I think that only changing the confidence level will change the probability of each error. Any opinions?
 
I have tried to explain it from my view before and haven't done real well, but I did look it up in both CFAI text and Schweser and it reduces both types of errors.

Maybe JDV would lend some knowledge...
 
Directly from the Quant text:

"The only way to reduce the probabilities of both types of errors simultaneously is to increase the sample size, n"
 
http://www.intuitor.com/statistics/T1T2Errors.html

diagrams of alpha and beta.

alpha is probability of a false positive ,beta is prob. of a false negative.
atleast intuitively, only way to reduce both is to have a larger sample.

also alpha * beta =k =constant. (not sure why i want to state this,but here it is)
 
hueion Wrote:
-------------------------------------------------------
> Directly from the Quant text:
>
> "The only way to reduce the probabilities of both
> types of errors simultaneously is to increase the
> sample size, n"

I guess I'll chalk that up to being something that Schweser skipped over. I believe it, since its in the Quant text, but I still don't agree...
 
hueion Wrote:
-------------------------------------------------------
> Directly from the Quant text:
>
> "The only way to reduce the probabilities of both
> types of errors simultaneously is to increase the
> sample size, n"


I completely agree unless another possibility is to use a more powerful test.
 
Intuitively, I agree more n, less error.
But, mathematically, Type I error is determined by alpha, level of significance, which does not relate to n. So, I could not see Type I is related to the size.
 
well,

minus 1 for me on the exam
plus 1 for CFAI for getting something schweser skipped.
 
willowlau Wrote:
-------------------------------------------------------
> Intuitively, I agree more n, less error.
> But, mathematically, Type I error is determined by
> alpha, level of significance, which does not
> relate to n. So, I could not see Type I is
> related to the size.


I agree with you completely. I won't argue w/the CFA text, but it doesn't make to me. I can see how type II would decrease, though.
 
There's a narrow semantic issue here - alpha is the MAXIMUM probability of a Type I error. The P-value of the test is the probabilty of a type I error on a particular test.
 
ozzy609 Wrote:
-------------------------------------------------------
> willowlau Wrote:
> --------------------------------------------------
> -----
> > Intuitively, I agree more n, less error.
> > But, mathematically, Type I error is determined
> by
> > alpha, level of significance, which does not
> > relate to n. So, I could not see Type I is
> > related to the size.
>
>
> I agree with you completely. I won't argue w/the
> CFA text, but it doesn't make to me. I can see
> how type II would decrease, though.

Nice to know I'm not alone. Pretty crappy question, I'd say....
 
I don't know the question, but here are the error types and the effects of sample size:

Type 1 Error (the typical error you are worried about): This is when you say that there is a significant difference (or that something is statistically significant) and it turns out that there isn't. The larger the sample size, the smaller the difference has to be before you can say something is statistically significant. The alpha level is basically the probability of a wrong answer (expressed as a proportion) that you are willing to tolerate in your analysis. The alpha level is not affected by your sample size, but the size of the confidence interval at a particular alpha level gets smaller the greater your N.

Type 1 errors are typically described as "rejecting the null hypothesis when the null hypothesis is true." Personally, I don't find this description very intuitive, since it's hard to remember what is the null hypothesis and whether you want it to be true or not (until you have some practice). I think of Type 1 Errors as "False Positives" - you say there is something significant, but there isn't

Type 2 Error: This is a "False Negative" - null hypothesis is not true, but you don't reject it. In other words, you say that there is no significant relationship when, in fact, there is. The probability of this is called "the power of the test," and the effect of a larger sample size is that the range of numbers where this outcome can happen gets smaller. Effectively, this probability shrinks with a larger sample size. The smaller your chance of a type 1 error, the larger your chance of a type 2 error, but the relationship is not completely inverse, this is because there is also the possibility that your hypothesis testing actually came to the correct conclusion (i.e. neither a false positive nor a false negative, but actually something correct), and the probability of this tends to increase with greater sample size.



Edited 1 time(s). Last edit at Monday, June 11, 2007 at 08:18PM by bchadwick.
 
You must log in or register to reply here.
Back
Top