I can confirm. I was pretty certain that I had failed the exam because of this question. Thankfully, I ended up scraping together enough points to still get the pass.
Hi, I’m an education and exam volunteer, and I’m interested in exam philosophy. I’ve just browsed this thread.
Well, when you had to do all the calculations by hand, that sounds like a huge waste of time. But now that you have a spreadsheet available, that sounds like a terrific question. Blindly following some method without adapting it to the data you actually have is a pretty common failure mode for inexperienced actuaries, and it is a really dangerous failure mode, because you think you understand something about the problem, but you are wrong.
While I’m sure there have been grading failures of that type, I can tell you that as a grader I have seen answers I didn’t expect and decided they were also right and gave them full credit.
Umm… seriously? I do algebra all the time as an actuary. I tell people that being an actuary is a terrific way to monetize being really good at high school algebra.
1 Like
That exam 8 question was when we had to do it by hand…
I get that. And I agree that the exams are often too long. I hope that moving to a spreadsheet will help with that in a lot of ways. Including allowing more questions similar to what you describe.
It’s a constant struggle to get exams to fit in the time constraints, and it’s a struggle that often isn’t successful, imo.
1 Like
I don’t write upper levels, but you’d be surprised how much time we spend reviewing the questions even for multiple choice. And if there’s even any ambiguity with interpretation, we likely throw the Q out/give free credit.
@Lucy, from your experience, was it common for them to make across the board adjustments to the mqc, passmark etc. based on how candidates performed? For example if a question was badly worded or the exam was too long. Take us inside what the process would hypothetically look like now (a week or so before results). What are they potentially doing with the diagnostics (hypothetically of course)? My question isn’t directed at this sitting. And of course, not asking you to reveal anything confidential.
As I understand it, there isn’t any “across the board” adjustments. If a question was found to require an adjustment to the MQC score (I’ve never heard of an “upward” adjustment, FWIW), then the MCQ threshold is just changed; but no candidate’s score is changed–including for the item in question. But that change could result in someone going from a (barely) not-passing to (barely) passing. But this latter isn’t reviewed, it just happens.
As for the “diagnostics” . . . it is likely the scenario that something “odd” (however that’s viewed by the CAS Board) was seen and asked about. The Exam Chair may have insight and can answer right away . . . or they may not have thought that “oddity” initially important and now has to spend a bit of time looking at things to answer the Board’s question.
And given that many companies is still processing year-end results and trying to get Opinions drafted and ready . . . that could add some additional time for everyone to respond.
The MQC score starts as a score for each item, and the numbers are added up.
I know that if a question is found to be defective, or harder than expected, or… there can be an adjustment to the MQC for that item, which changes the MQC overall.
I assume there is some sort of overall adjustment when the exam is too long, but I’m not familiar with any details. I will say, based on public information plus my observations as an exam proctor, that exams that look too long when I proctored them tended to have fewer candidates pass than exams that looked to be the right length. So my bias is that whatever adjustment they’ve made for “exam was too long” is smaller than what the lack of time costs candidates. I suppose if your philosophy is that you care more about making sure each passing candidate has demonstrated competency than about fairness to candidates this is the right choice. Once an exam is broken (for instance, by being too long) there isn’t any way to fix it that’s fair to all the stakeholders.
Yes. Graders and exam mucketymucks volunteered assuming that they needed to have a certain amount of time free to deal with exams in November. When that got moved to Dec/Jan, some of those people no doubt ran into unexpected conflicts with other priorities. I expect that’s added something to the processing time.
I will say that not releasing the exams is really intended to build a question bank for better / more frequent CBT exams.
That’s different from the inability to differentiate qualified vs. not qualified actuaries. [Discussion on what a “qualified” actuary in a context other than SAOs omitted.] I might amend the statement to say that the CAS exam system does a poor job of differentiating those who really know the material and can properly apply it from those who know the material but can’t properly apply it. What we have now is a system designed to cater to those who came through actuarial science programs and are good at test-taking and math … which is great, but it’s not what the actuarial field traditionally represents.
Don’t worry, though, because … unicorns.
You made me reread the CAS strategic plan and there’s literally nothing in there showing what the CAS is going to do, it’s just a 13 page document on the benefit of being/having unicorns.
1 Like
Anyway if they are taking extra time, determining where they can give candidates the benefit of the doubt or lower the overall mqc due to various reasons (technical issues, unclear questions, etc) than it is definitely worth it for them taking the extra time.
My fears is that they are being stricter on the pass marks and now reviewing all the details so they get comfortable with it.
And getting back to transparency not releasing the pass marks. I feel is not being transparent.
Hopefully they at least release the pass percentages.
I imagine we’ll see the pass rates in mid-February or so. I really doubt they publish the actual pass marks though.
Do you understand their logic for not posting it? Are they afraid if they post the pass mark for 9 is 72% people will complain? I don’t get it.
Yeh I don’t see why they’d publish one and not the other.
If they’re trying to keep the exams completely confidential they probably want the pass mark to be confidential too. On the other hand as they build a question bank it’s possible the pass mark can vary wildly and they might avoid publishing it so they don’t upset people.
1 Like
Are you saying the following they have an initial pass mark on Exam 9 of 70%… but when they get better at calibrating the questions the pass mark gets raised three years from now.
So when someone gets largely a similar test to what we took they end up having to need to get a higher pass mark?
No, I’m saying this is a new process and the pass mark could potentially vary wildly and they may not want to publish it to avoid upsetting people. I’m just speculating though. I think wanting to completely avoid publishing it because it’s part of the confidentiality of the exam is more likely.
I’m not saying I agree with it, I just could see the CAS using this logic.