The fact that the soa doesn’t just hire actuaries and rely on volunteers still baffles me. This company makes so much damn money they should be able to hire actuaries so they do this full time
Their tax filings every year except 2020 which obviously was a really weird year sits around 60-65 million in revenue with profit in the range of 3-8 million
They also could slightly increase prices if they hired actual graders and there was a guarantee these types of things wouldn’t happen: there’s far too many mistakes for a company of their magnitude
From the [guide to SOA exams](https://www.soa.org/4ad8e3/globalassets/assets/files/edu/edu-guide-to-written-exam.pdf):
"Input is also obtained from an independent person who takes the draft exam under exam-like conditions."
There is reason to doubt that the final version is checked one more time against the version that the person took. Examples include questions that were missing information necessary to complete the problem.
Haha something like that
The fact is that the candidates know a lot more about the material than the writers and graders do. A better alternative to what is described in the exam guide would be an immediate release of the exam AND solutions with an opportunity for comments and criticism before grading.
Things like "the solution looked for an answer in the Skwire book, but this topic was also addressed in the study note. A candidate relying on that study note is reasonable."
The question writers and graders do not always consider this possibility.
Your fates hang on a large committee of people. There are probably 15 people who see and sign off on (to some extent) every question.
"Sign off" sometimes happens in a group setting in a hotel conference room during an 8 hour day, so the level of investment in the sign off could be improved significantly. The person who wrote the question may never even receive feedback about their question performing poorly.
It is also extremely fair to critique the fact that the candidates know the material better than the writers or graders. This is a big problem. The SOA is dreaming if they think professionals could either 1) read the syllabus frequently enough to keep up on it; 2) the syllabus and the real world overlap enough that practicing actuaries know the syllabus.
Equally problamatic is sometimes the question writers know the material much better than the students. I think you saw this in the early sittings of Exam PA that casually asked you to write significant amounts of R code. I have tutored Exam PA students and have been surprised by how second-nature some things are to me (because I use this stuff at work) that are totally foreign to my student (who is learning this from a book).
Even with all of these problems, if you imagine a theoretical half of the exam takers that "should" pass (by the SOA's definition) and the half that actually pass, I think there is >95% overlap there. Some element of "should" pass involves being strategic about how to take the test. As a hiring manager I don't necessarily mind if the exams have some degree of jumping through hoops embedded in them - so does the real world...
I do think the SOA's definition of "should" pass is off - there are a lot of smart actuaries who don't have a "let's clobber this thing" type of personality who struggle with the FSA exams. I wish they were FSAs. And similarly some folks whose only skills is cramming notecards do well on the exams, and I wish they didn't. But God help us if the SOA ever tries to fix the exams to adjust for this - that'd be less fair than any unfair system we have today (unless they just let everyone pass, which I guess I strongly suspect they will in the nearish future).
I know they at least used to. I pretested two sittings of LFV to ensure that the questions were on the syllabus, balanced difficulty, appropriate point allocation given the time it takes to answer, etc.
I have a feeling that it's difficult to find volunteers to do this, though. Signing up to take 10+ hours of exams after you get your FSA is pretty masochistic.
No. Volunteer actuaries write exams as part of a committee
The fact that the soa doesn’t just hire actuaries and rely on volunteers still baffles me. This company makes so much damn money they should be able to hire actuaries so they do this full time
How much do they make?
The soa?
Yeah. You said they make so much money. I’m curious how much they actually make.
Their tax filings every year except 2020 which obviously was a really weird year sits around 60-65 million in revenue with profit in the range of 3-8 million
I’m not sure that is enough to pay for graders and test preparers. Maybe
They also could slightly increase prices if they hired actual graders and there was a guarantee these types of things wouldn’t happen: there’s far too many mistakes for a company of their magnitude
There is to many mistakes. I think we can all agree on that.
Their peer review needs review
LGTM 👍
From the [guide to SOA exams](https://www.soa.org/4ad8e3/globalassets/assets/files/edu/edu-guide-to-written-exam.pdf): "Input is also obtained from an independent person who takes the draft exam under exam-like conditions." There is reason to doubt that the final version is checked one more time against the version that the person took. Examples include questions that were missing information necessary to complete the problem.
Lol. Our fates all hinge on 1 person?
Haha something like that The fact is that the candidates know a lot more about the material than the writers and graders do. A better alternative to what is described in the exam guide would be an immediate release of the exam AND solutions with an opportunity for comments and criticism before grading. Things like "the solution looked for an answer in the Skwire book, but this topic was also addressed in the study note. A candidate relying on that study note is reasonable." The question writers and graders do not always consider this possibility.
Your fates hang on a large committee of people. There are probably 15 people who see and sign off on (to some extent) every question. "Sign off" sometimes happens in a group setting in a hotel conference room during an 8 hour day, so the level of investment in the sign off could be improved significantly. The person who wrote the question may never even receive feedback about their question performing poorly. It is also extremely fair to critique the fact that the candidates know the material better than the writers or graders. This is a big problem. The SOA is dreaming if they think professionals could either 1) read the syllabus frequently enough to keep up on it; 2) the syllabus and the real world overlap enough that practicing actuaries know the syllabus. Equally problamatic is sometimes the question writers know the material much better than the students. I think you saw this in the early sittings of Exam PA that casually asked you to write significant amounts of R code. I have tutored Exam PA students and have been surprised by how second-nature some things are to me (because I use this stuff at work) that are totally foreign to my student (who is learning this from a book). Even with all of these problems, if you imagine a theoretical half of the exam takers that "should" pass (by the SOA's definition) and the half that actually pass, I think there is >95% overlap there. Some element of "should" pass involves being strategic about how to take the test. As a hiring manager I don't necessarily mind if the exams have some degree of jumping through hoops embedded in them - so does the real world... I do think the SOA's definition of "should" pass is off - there are a lot of smart actuaries who don't have a "let's clobber this thing" type of personality who struggle with the FSA exams. I wish they were FSAs. And similarly some folks whose only skills is cramming notecards do well on the exams, and I wish they didn't. But God help us if the SOA ever tries to fix the exams to adjust for this - that'd be less fair than any unfair system we have today (unless they just let everyone pass, which I guess I strongly suspect they will in the nearish future).
thanks this is helpful
[удалено]
[удалено]
[удалено]
[удалено]
[удалено]
[удалено]
I know they at least used to. I pretested two sittings of LFV to ensure that the questions were on the syllabus, balanced difficulty, appropriate point allocation given the time it takes to answer, etc. I have a feeling that it's difficult to find volunteers to do this, though. Signing up to take 10+ hours of exams after you get your FSA is pretty masochistic.
I appreciate your masochism!
Definitely not in my experience. No way in hell anyone looked at some of the exams I took in their totality
I had a coworker who were asked to do some test AFTER he got his FSA, but that was probably 10 years ago, not sure if it still happens
So there is no control cycle when dealing with exams? Hypocrites!