“I don’t think so, and I think people make the issue too easy for themselves. If you increase the time then you’ll also pass people who aren’t qualified just because they were able to figure things out in the moment.”
Ah, so here we’re coming to a fundamental disagreement. These exams aren’t multiple choice. You can’t just guess and check the choices until you get a right answer. If you’re someone who didn’t memorize the formula for the credibility of a claims free year from Bailey and Simon, but instead spent 10 minutes deriving it from first principles, I don’t consider you any less of a qualified actuary than the person who memorized the flash card and took 20 practice exams and knows how to answer it just by glancing at the table (I’m the latter one btw). Knowing how to answer that question faster and quicker isn’t the same thing as being the more qualified actuary.
"Assume the CAS wants an exam of difficulty where 40% pass. They accept some % of the group is sufficiently qualified but don’t know who. All the complaints posit that we’re passing lots who we shouldn’t and missing a bunch who we should. How can we change the exams to keep the % passing constant while more effectively sorting.
Just opening the floodgates doesn’t make your exam more effective, it just increases your type 1 error."
So a couple of points here. I think your premise if flawed. The CAS doesn’t even claim this to be the case. Why should there be an arbitrary 40% pass rate? Why are you assuming a coherent rank-ordering strategy of “qualified actuary” even exists? In the extreme case where everyone who is sitting for exam 9, having passed 9 other very difficult actuarial exams already, is actually qualified (not that extreme when you think about it), why would you want to rank-order them and only pass the top 40%? Isn’t it pretty obvious that any such rank-ordering will be fairly arbitrary? i.e. if they are all qualified actuaries, we can rank order them based on their ability to solve algebra problems very quickly, and result in a given 40% cohort. But we can also rank order them on their language and grammar usage (which is even more relevant for a professional actuary) and come up with another rank ordering. Or we can rank order them on their ability to solve complex logical puzzles, and have yet a 3rd ordering. Neither of these 3 methods are any better at differentiating what the test is allegedly a measure of, is the person a qualified actuary.
By constraining yourself to a set pass mark you are necessarily making an arbitrary choice that results in a worse ordering. Making your test actually be about what it’s supposed to be doesn’t increase your risk of type 1 errors. Sticking with an arbitrary ~40% pass rate and testing algebra speed does increase your type 1 errors though.
As an example: Last years test had a really dumb problem where we had to solve a system of 3 equations to get the answer. Some people did this quickly. Some people did this slowly. Then we get to the last question of the exam. Imagine the person who did this quickly gets to the last question of the exam and says “Wow, I have no idea how to answer that question, I don’t even understand it.” Whereas the person who does it slowly runs out of time before they even read it, but had they had time they could have answered it for full points. The CAS then looks at their exam statistics and says “wow, question 20 sure was difficult, so many people didn’t even attempt it, I guess we’ll set the MQC relatively low for that one.” The fast algebra guy might score higher, despite not being fully qualified. Any time you’re testing on some other variable you’re going to get these kind of errors.