Okay. I actually care less about the pass mark and more about the pass percentage anyway.
End of day the passing candidates will be set in stone when they release results, so pass percentages and pass marks only really make people upset. Especially the low ones when the CAS has to write a letter how candidates just couldn’t cut it that sitting.
Why?
They have historically passed like 50% on exam 9. I have a hard time thinking they will do that this time. If it goes really low like exam 8, I think people will have the right to be upset. However, it’s not going to change anything.
I think the reason to not release the pass mark relates to what amp said earlier about confidentiality.
If a pass mark is released, a candidate knows approximately what % they got correct. If the plan is to reuse questions in a question bank, the reasons feel fairly straightforward as to why the CAS wouldn’t want candidates to know if they got questions right or not.
Naw, i don’t see any benefit to preventing candidates from knowing what they got right and wrong. Especially if they failed and need to learn more. But I don’t see any particular benefit to publishing the pass mark when the questions aren’t published, either. Who will do what with it?
I expect that when they have a bank of vetted questions, some questions will be known to be harder than others, and the pass mark might wobble around a little depending on whether they used more “harder” or “easier” questions in a given exam. But it probably will be reasonably stable, because they will have the data to make it so.
Publishing pass mark ensures transparency. If they don’t, what is to prevent them from setting it to 90? If it is set unreasonably high the industry will not wish to fund exam days, membership dues and exam and fees. It’s a form of governance for an organization that answers to no one.
Why would they WANT to set it at 90%? Serious question. What nefarious goal would that serve?
I’ve graded exams. I think some exams are too hard. But I’ve never heard of a case where tptb were surprised that candidates did better than expected, let alone unhappy at such a thing. The usual grading problem is, “oh shit, they scored much lower than we expected”.
1 Like
When you eventually become a fellow and have the heart to volunteer, you’ll understand why it’ll never be 90.
1 Like
Maybe you’re right. With 1 exam to go, I am probably not qualified to pass judgements on lack of transparency. Please forgive this subpar human. 90 percent was an extreme example to show a point that lack of disclosure opens grounds for moral hazard.
Now to be fair, as a grader, how much say do you have in pass mark. If the intent is really to manufacture qualified professionals, then we should be able to learn from our mistakes and develop. Telling me that I did poorly on question 5 serves no purpose. I can’t improve on that fact alone. How can i make an appeal explaining that my alternative answer may also be correct? It seems more like a barrier to entry than anything. And it is awfully frustrating that after all the invested time, money, sacrifices, we are entitled only to what CAS decides. And they can change anything without so much as a proper explanation. At no point are the candidates consulted. At no point are concerns answered or addressed.
1 Like
I’m a writer, not a grader, and probably not for your particular exam. However, I trust that the CAS has the same level of scrutiny throughout.
Here’s what I can say based on my experience: all appeals are taken very seriously. If there’s even any doubt, the benefit is usually given to the student, not to the writer.
Is this not the case for all professional credentials? Also…did you actually pay for the exam out of pocket?
- I did pay for some. Regardless of who pays for exam, you cannot argue that candidates make big sacrifices. I do feel that the attitude has been, “don’t speak unless spoken to,” “don’t challenge authority.” I mean, one could argue I am unprofessional for questioning motives and procedures. Maybe I should stay quiet. I am not alone in saying that the process is not transparent. What has been done to address this?
- It is not so with all professions. I’ve said this before, a qualified candidate should not feel lucky to pass.
- I do believe that it is now impossible to appeal. I cannot remember at all what each question asked. Unless I see them, I can’t form an argument.
- I do believe that entering candidates are at a greater disadvantage. Syllabus will evolve, questions will adapt more to computer structure as spreadaheets allow for a different kind of testing. With non disclosure, how can we practice?
- As a writer, do you feel that exams from 5 years ago were just as challenging, in particular to integrated questions? What a mind game when you are 45 minutes into an exam and you finished 1 out of 20+ questions. A huge question, but still, you don’t know what’s coming next, cause there is no reading period this time around. You also have a new computer to get used to (kind of like someone handing you a brand new calculator you never used before).
- Then you also feel incredibly dehydrated after sitting in a mask with no water for a prolonged period.
1 Like
- yes, of course candidates work hard. But candidates have a voice. There’s a candidate liaison committee that is definitely heard.
And I’m not suggesting you shut up. I’m suggesting that you are assuming malice when none is there, and you should try to put yourself in the position of the examiners if you want a clearer perspective of what’s going on.
The prior process was extremely transparent but suffered other flaws. For example, it was all too common for an exam to be too long, and there’s no real way to fix that after the fact. But by reusing questions the CAS can construct an exam knowing how long it will take the typical candidate to complete.
You are unfortunately caught in the middle, suffering the cost of lack of transparency without the gain of higher quality exams. But there’s no way to get from here to there without that. And hopefully the transition will be brief.
-
my father used to write medical board exams. They had all the same problems we have with setting appropriate exams. Most professions are less exam-based, but i don’t think we are that unusual as exam-based professions go.
-
it was not impossible to appeal, but it’s now too late. I graded this year, and the item I graded was appealed. The grading was modified based on the appeals. So some candidates remembered the items well enough to appeal.
The CAS has never accepted appeals of the form “look again”. Appeals have always had to be based on flaws in the question or flaws in the grading rubric. And most successful appeals were always about the question.
I am very sad that we have lost the ability to get feedback on flaws in the grading rubric. The people who chose to reuse questions believe that there will be enough benefit from that to make up for the loss.
I believe the exam chairs spent more time looking at statistics like, “did candidates who did well overall do better or worse on this question than the average candidate?” as an attempt to make up for that loss. I expect there will also be sample questions and sample answers published over time.
-
As mentioned above, i expect that as new questions are developed sample questions will be published. But this year’s exam was pretty much written before the CAS pivoted to computer based testing. I guess we’ll find out what happens in the the future, but that wasn’t really an issue this year.
-
I’ve answered that before. Yes, individual items are certainly harder than they were in the past. There are also much better resources to study than there were in the past. Passing percents are higher than when I sat for those “easier” exams, so I’m not convinced the overall process is harder.
I’m old. There was no reading period in my day. Standard advice was, “if you are stuck, move on, and return to that question time permitting”. I can’t imagine no one gave you that advice this year.
And I thought you had a chance to play around with the interface? Not as much time to do so as I would have liked. But candidates I work with did spend some time checking it out.
- having to wear a mask while sitting for exams sucks, but it’s hardly the fault of the CAS.
How are you Tigger 209? I thought the most wonderful thing about tiggers is that there’s only one?
-
I did write before, and the answer was, unfortunately we can’t disclose any more info. We’ll factor things in. How? Do you adjust passing grade? Do you adjust just candidate’s mark? How do you distinguish between someone who was adversely affected by computer equipment and left questions blank vs someone whi didn’t know how to answer? To do that, you have to let marker read the whole exam.
I am pointing out potential for malice when there is no accountability and reporting. Even charities disclose information on operations. -
We are caught in the middle. And it is a really bad time too. It has been a tedious year for candidates.
-
I’ll be honest, i came out of exam not remembering half the questions and second guessing what I remembered. I could not possibly construct any sound arguments on items i thought ambiguous. And wasted a tonne of time stating my assumptions, probably way too many assumptions. But i feared deduction of marks.
-
Spreadsheet, yes. But sticky mouse, different keyboard, delay in screen refreshing are all very individual to test centers. The set up is not like at home. I really don’t know how u can improve if I did not pass.
-
I do not blame CAS. I view masks as a positive and take the situation very seriously. I practiced with a mask on computer at home. But I do think that CAS did not factor this into exam length. I had plenty of time to spare at home. It is always the case that there is less time in actual setting, but not this much less.
-
A tally of total points would have been nice too. I did waste a minute or two adding them up to gage how much each question is worth. It is easier to flip through paper pages to see what’s on exam. On computer you waste 5-20 seconds on screen refresh.
Asking for information is different from providing feedback. I said candidates have a voice. I don’t deny that the new method is less transparent than the prior method. It’s no less transparent than the SATs or many other high-stakes tests, though.
I was not part of this decision-making process, and i have very mixed feelings about it. But it undeniably has benefits, and i am hopeful they will outweigh the costs.
The CAS has concrete plans to work with the vendors to improve the process whether or not you, personally, pass. If you feel the hardware you used was outside the normal range of working hardware, i recommend you complain to Pierson Vue. (And i would have recommended you complain immediately after sitting, not weeks later.) Mice and keyboard are cheap and i suspect the staff doesn’t personally test them. If it’s time to replace one, it’s surely helpful for customers to mention it.
As for things like a different refresh rate… I recommend you go into the exam recognizing that there will be minor differences from your home setup. And hey, at least you don’t have to write everything out longhand.
Do you honestly think that wearing a mask made you work slower? That seems extremely unlikely to me. Especially if you practiced with a mask on. I think it’s far more likely that you worked slower on new-to-you problems than on problems you had done many times, and that the review materials were specifically geared to help you with.
And again, one of the potential benefits of reusing questions in a CBT environment is that going forward, the exam committee ought to be far better at making the test the right length. Admittedly, that will take a couple cycles. But it’s one of the largest potential benefits i see to this new model.
1 Like
You edited and added more stuff as i typed. 
Yes, that sucks. And it’s been a shitty year for everyone, too.
1 Like
Anecdotal, since this is not the usual experience.
Almost every exam I took, pass or fail, I finished at least 30 minutes to an hour early. Something about the adrenaline just makes me write faster in an exam room setting.
Of course, I also embraced the common exam taking strategies: skip problems I don’t know the answers outright to, never write in complete sentences, etc.
That’s was my experience, too, although I’d say 15-30 minutes for me. In particular, i hated being done and locked in a room full of stressed people. So i almost always passed in my paper and left before the 15 minute lockdown at the end.
But i know that’s not the typical experience.