Writing difficult questions is part of the assessment process. I see a lot of students in my classes who Hoover up all the easy points but miss anything that’s not straightforward. They invariably respond with “I really knew the material, BUT…”. It’s my experience that anything to the left of the BUT can be ignored.whatsyourgovt wrote:
Test on a more difficult subject, expand the LOS, but don’t fail someone who has put in more than enough hours and understands the subjects simply because the question(s) are written in a vague and ambigious manner. As we see now, what this does is places more of an importance on gaming the test as opposed to outright knowledge of the material.
Unfortunately, once you’re out of the classroom, many problems aren’t straightforward, and must be solved in a limited time. So, I think it’s a fair approach.
It would be different if it were a 10-question exam. But over 120 questions, there will be enough variation of scores that CFAI can discriminate between candidates. The people who really know it cold do well on the exam, and the ones who don’t, don’t. The question is what happens to the ones that are in some band around the bubble.
I know someone who was fairly high up at CFAI and was involved heavily with curriculum and testing issues - they spend a LOT of time trying to write questions that are both of the appropriate level of dicciculty and do a good job of separating the sheep from the goats.