The MQC score starts as a score for each item, and the numbers are added up.
I know that if a question is found to be defective, or harder than expected, or… there can be an adjustment to the MQC for that item, which changes the MQC overall.
I assume there is some sort of overall adjustment when the exam is too long, but I’m not familiar with any details. I will say, based on public information plus my observations as an exam proctor, that exams that look too long when I proctored them tended to have fewer candidates pass than exams that looked to be the right length. So my bias is that whatever adjustment they’ve made for “exam was too long” is smaller than what the lack of time costs candidates. I suppose if your philosophy is that you care more about making sure each passing candidate has demonstrated competency than about fairness to candidates this is the right choice. Once an exam is broken (for instance, by being too long) there isn’t any way to fix it that’s fair to all the stakeholders.
Yes. Graders and exam mucketymucks volunteered assuming that they needed to have a certain amount of time free to deal with exams in November. When that got moved to Dec/Jan, some of those people no doubt ran into unexpected conflicts with other priorities. I expect that’s added something to the processing time.
I will say that not releasing the exams is really intended to build a question bank for better / more frequent CBT exams.
That’s different from the inability to differentiate qualified vs. not qualified actuaries. [Discussion on what a “qualified” actuary in a context other than SAOs omitted.] I might amend the statement to say that the CAS exam system does a poor job of differentiating those who really know the material and can properly apply it from those who know the material but can’t properly apply it. What we have now is a system designed to cater to those who came through actuarial science programs and are good at test-taking and math … which is great, but it’s not what the actuarial field traditionally represents.
You made me reread the CAS strategic plan and there’s literally nothing in there showing what the CAS is going to do, it’s just a 13 page document on the benefit of being/having unicorns.
Anyway if they are taking extra time, determining where they can give candidates the benefit of the doubt or lower the overall mqc due to various reasons (technical issues, unclear questions, etc) than it is definitely worth it for them taking the extra time.
My fears is that they are being stricter on the pass marks and now reviewing all the details so they get comfortable with it.
If they’re trying to keep the exams completely confidential they probably want the pass mark to be confidential too. On the other hand as they build a question bank it’s possible the pass mark can vary wildly and they might avoid publishing it so they don’t upset people.
Are you saying the following they have an initial pass mark on Exam 9 of 70%… but when they get better at calibrating the questions the pass mark gets raised three years from now.
So when someone gets largely a similar test to what we took they end up having to need to get a higher pass mark?
No, I’m saying this is a new process and the pass mark could potentially vary wildly and they may not want to publish it to avoid upsetting people. I’m just speculating though. I think wanting to completely avoid publishing it because it’s part of the confidentiality of the exam is more likely.
I’m not saying I agree with it, I just could see the CAS using this logic.
End of day the passing candidates will be set in stone when they release results, so pass percentages and pass marks only really make people upset. Especially the low ones when the CAS has to write a letter how candidates just couldn’t cut it that sitting.
They have historically passed like 50% on exam 9. I have a hard time thinking they will do that this time. If it goes really low like exam 8, I think people will have the right to be upset. However, it’s not going to change anything.
I think the reason to not release the pass mark relates to what amp said earlier about confidentiality.
If a pass mark is released, a candidate knows approximately what % they got correct. If the plan is to reuse questions in a question bank, the reasons feel fairly straightforward as to why the CAS wouldn’t want candidates to know if they got questions right or not.
Naw, i don’t see any benefit to preventing candidates from knowing what they got right and wrong. Especially if they failed and need to learn more. But I don’t see any particular benefit to publishing the pass mark when the questions aren’t published, either. Who will do what with it?
I expect that when they have a bank of vetted questions, some questions will be known to be harder than others, and the pass mark might wobble around a little depending on whether they used more “harder” or “easier” questions in a given exam. But it probably will be reasonably stable, because they will have the data to make it so.
Publishing pass mark ensures transparency. If they don’t, what is to prevent them from setting it to 90? If it is set unreasonably high the industry will not wish to fund exam days, membership dues and exam and fees. It’s a form of governance for an organization that answers to no one.
