Neither exams or examiners’ reports will be published going forward, beginning with Fall 2020 exams:
https://www.casact.org/cms/index.cfm?fa=viewArticle&articleID=4836
I find this to be initially terrifying.
Uggh. So, not only will you not have examples of what types of questions they will be asking in the future, you won’t know what type of responses they are looking for, nor will you have any idea whether the solution they are grading against is actually correct.
Yeah, I want to hear the explanation, but this sounds pretty unfortunate to me.
WTF.
Will they at least provide sample exam questions and desired responses? Because I can’t see this doing well in building trust with candidates.
[As an SOA person, I say – yeah, keep doing that. Easier for us to take over the world!]
It sounds like the explanation boils down to:
- We can’t write quality questions, so we want to be able to reuse the ones we do manage to create.
- We can’t figure out how hard the questions we write are, so this will let us use them against multiple cohorts for benchmarking.
1 Like
They said they will provide sample questions and responses as new material is added to the syllabus.
1 Like
If they never did this to begin with, I don’t see a big deal. I’m sure there are societies that don’t publish anything. But this is definitely going backwards, and it’s not a good look.
1 Like
The CFA Institute also decided in the last year to stop releasing level III exams and answers.
This would be foolish. First, obviously many people have to re-take CAS exams.
Second, it’s a small world. People will discuss the problems. Later cohorts would be expecting these questions, so wouldn’t be comparable to earlier ones.
The CFA Institute does not have to worry about a hostile takeover by a rival org. (Yet)
I think it will take on the form of exams 1 & 2 eventually, which have a base set of questions that have been the same for many years now (I think they add a few every once in awhile?). My recollection was that those were decent enough to prepare for the actual exam, though that was nearly a decade ago. The examiners reports are what really carry the most value I think. Getting inside the graders’ heads has been the single most valuable resource imo.
1 Like
Exams 5 & 7 are likely to have a core set of content that won’t change significantly over shorter time periods (say, 5 years).
Exam 6 has some core content that won’t likely change (history of insurance, general accounting principles), but has some material that might be relevant now in terms of recent history, but after 3 or 4 years, that may very well change.
All three of the above Exams also have “core” reference materials (vs. a collection of professional articles) that are intended to not change (perhaps see updates, though).
Exams 8 and 9 are the ones that I see that the “core” material may well be very small and attending more to “current trends” in the market. YMMV.
So glad I’m done with exams, this sounds horrible to me
How will we know if questions are defective going forward? It’s hard to report defective questions if you don’t have a copy of the question. Will they simply go off of how many people got it right?
1 Like
The same way you report defective questions on the SAT - you don’t…
If CAS 9 questions were as straightforward as SAT questions, I wouldn’t have a problem with that. But we have a long history of defective questions and question writers not understanding the material, or assuming knowledge that isn’t on the syllabus.
Yes I agree exam writers need to be more well versed in the material. Part of the issue is they’re volunteers, and come from a variety of background (professors, actuaries, fellows doing non actuarial work) and may not be super up to date on syllabus changes.
I expect there to be far fewer defective questions since they’re now building a reusable problem bank.
What’s going to happen in this context is that two, maybe three, questions will be “new” problems to be assessed for quality. So, if there are 100 points available on the Exam and 6 of those points are from “new” problems, then the passing mark would be determined based on the other 94 points available.
Yeah it seems like giving them the ability to pilot test questions will ultimately lead to higher quality exams.
Also, candidates will be able to flag questions as defective during the exam. So when there are defective questions the CAS will see it get flagged by candidates who understand the material well enough to know the question is defective. This is something that wasn’t really possible with paper exams.
It also seems like the risk of cheating is pretty low if the problem bank is sufficiently large.
My main concern is that candidates will be less prepared because they won’t have access to a representative set of sample exams.
My main concern is that this makes it REALLY hard to change the syllabus. It’s probably fine for Exam 5, which is can probably remain stable. But the fellowship exams ought to grow and change with the times.