Several times I’ve taught a course in Problem Solving and Decision Making. As part of that course, I give the students a test: they’re to write down 90% confidence intervals for ten numbers that they probably don’t know. The numbers are, for example:
- Martin Luther King’s age when he died
- The gestation period of an Indian elephant
- The diameter of the moon, in miles
- The weight of an unloaded 747
- and so on
I tell them to make sure that they give me 90% confidence intervals, not 100% confidence intervals (e.g., for the diameter of the moon, a range of 0 miles to 93 million miles would be a 100% confidence interval). Obviously, if they think that they know the number well, their interval will be narrow; if they think that they don’t know the number well, their interval will be wide.
If they’ve done this accurately, you would expect that the number of intervals that contain the actual number would be about 9 (= 90% of 10).
In fact, most individuals get 4 – 6 intervals containing the actual number: they’re obviously too confident in their skills at creating confidence intervals.
(What’s really interesting is that when I create intervals for the class – using the lowest minimum and the highest maximum for each interval – the actual numbers lie in those intervals usually only about 70% to 80% of the time: collectively, they’re still overconfident.)
Clearly, the students don’t believe that they have any control over the age at which Martin Luther King died.
That’s the difference.