The Greatest Myth about CASPer Test Busted – Our frank opinion for applicants, their parents, pre-health advisors, admissions deans and directors on how to easily prepare for this highly coachable test.
“To avoid criticism say nothing, do nothing, be nothing.” – Aristotle
We strongly encourage all applicants, their parents, and admissions professionals to proceed with caution when deciding to use CASPer or when reviewing claims made by its creators and its for-profit administrator, as none of their claims appear to have been independently verified. We have seen claims by them that may be hurtful to applicants' chances of acceptance, their CASPer scores, and worse studies suggest that situational judgment tests may cause bias.
Note that we do not endorse CASPer and its use is not recommended because, in our opinion, it adds unnecessary cost to applicants, the general literature suggests that situational judgement tests cause bias and its efficacy has not been independently validated and one can argue that there is a reason why most universities do not use CASPer. In fact, If you are too worried about the test, you may want to consider applying to schools that don't require it. If you have no choice, we are here to help you ace your test and we provide many free resources.
The greatest myth of all about CASPer test is that "you can't prepare for CASPer" or that it is "immune" to coaching or you can't really "study" for this test. Our students do not require convincing because while we cannot comment about other CASPer prep programs, we know our CASPer prep programs work and the result they produce is the reason we have become the leader in CASPer preparation. In fact, most students also joke about this myth and comment how illogical it sounds. Furthermore, we even have scientific proof to bust this myth but before we share our research results with you, let's think about this myth for a moment.
Nobody is born with personal and professional characteristics
What does CASPer claim to test? It claims to test personal and professional characteristics such as empathy, communications skills and ethics. All these traits are learned behaviors. Nobody is born with any of these. You either learn these behaviors as part of your upbringing or through deliberate training. Normally, but not always, people from higher socioeconomic status learn these skills naturally while growing up because of their social environment. The rest of us must learn these skills actively on our own. When you understand that simple concept, you'll understand how ridiculous it is for someone to claim that it's not possible to prepare for such a test and now you are going to be armed with an intelligent response whenever someone says something as absurd, including the administrators or some random online forum member who may actually be an agent of a test administrator or one of your fellow applicants trying to misguide others in hopes of gaining a competitive advantage. Myth busted.
Study indicates that on average applicants improve their practice scores by 23% with appropriate preparation
If that doesn't convince you, consider this. We recently conducted a double-blind study of our CASPer prep program to examine its effectiveness. Double-blind means that neither our students nor our consultants were aware we were conducting the study until after the study completion. This is important because we wanted to avoid confounding factors that could interfere with our study.
The study included 24 applicants with an upcoming CASPer test who were selected randomly to participate in the BeMo study. The applicants' baseline performance was determined by conducting a realistic mock CASPer test (CASPer SIM™) using BeMo's CASPer Prep program protocols followed by expert feedback and numeric scoring identical to the scoring system used by the CASPer test administrators. We coached the applicants by identifying areas of improvement and re-tested them using additional independent mock CASPer practice tests. Each applicant received a total of 3 practice CASPer tests followed by expert feedback.
The results speak for themselves. Our students significantly improved their CASPer SIM score by 23% compared to their baseline performance. To put things in perspective notice that 23% is huge when you are competing with thousands and may be even tens of thousands of applicants. Even 1-2% could give you an edge in a fierce competition. 23% is the difference between a rejection letter and an acceptance letter. 23% is the difference between disappointment and thousands of dollars in waste and enjoying a fulfilling career for the rest of your life. Myth busted again.
Figure 1. Applicants' CASPer scores improved by 23% on average after only 3 CASPer preparation sessions and expert feedback with BeMo.
The above two points are sufficient to convince the few applicants who are on the fence or mistakenly guided by the test administrators' comments. Nevertheless, the test administrators make a few other conjunctures that are illogical and false, in our opinion, that merit commenting here.
Writing a test does not equate with practicing for that test.
They say that they have done some “research” on their “own internal data” and concluded that there's no evidence of practice effects.
It is critical to note that the administrators did not examine any specific coaching program. For instance our coaching programs are restricted to actual students who are writing the test with no affiliation with the test administrators. The administrators and their agents are not permitted to participate in our coaching programs or even take our practice tests based on our terms of service. Therefore, it is not possible for such a company to make any claims about preparation given the absence of such data and such research. Rather, in their “research” they simply looked at average scores of applicants across scenarios for a given test. Then claiming that applicants’ average scores was unchanged throughout the test. For example, if applicants on average received a score of 6 out of 9 on the first scenario in the test, they also received a 6 out of 9 on subsequent scenarios, on average. Well, you see the flaw in this rationale again?
That’s not practice. You don’t get practice while you are writing a test. Practice happens way before you actually write the test. Furthermore, as we have always said, practice does not make perfect. Practice makes permanent. It is only perfect practice that makes perfect. You have probably heard the famous quote: "The definition of insanity is doing the same thing over and over again and expecting a different result." That means you only get better on CASPer if you practice using realistic simulations followed by expert feedback so you can learn from your mistakes. How else are you supposed to get better? In other words, it is not possible to magically learn a new skill by mindlessly engaging in the activity. Rather, you need a coach to tell you what you are doing well, and more importantly what you are doing poorly and how to do better. You keep doing this until you get better based on the judgement of an expert. Myth busted yet again.
Fact: BeMo does keep track of your score before and after preparation.
The administrators go on to make yet another false claim. They say some companies make the claim that their preparation program is proven to increase your scores but they don’t know how you scored before and after your preparation. False again; we just showed you that in fact that’s exactly what we did in our double-blind study. We compared each of our students’ baseline practice scores with their practice scores following our preparation program and on average their CASPer SIM scores significantly increased by 23%, in just three sessions. This shows that CASPer is actually a lot easier to prepare for than other forms of admissions tests. Notably, as we discussed, the administrators don't know your before and after scores either, so how can they make such claims?
Fact: Wealth does not correlate with probability of use of admissions preparation programs.
But there’s more. The administrators baselessly state that the performance of students taking preparation programs is likely confounded by the fact that students who use preparation programs are more affluent. Incorrect again. There is clear evidence that test preparation programs do not contribute to the well known socioeconomic bias present in professional schools. Instead studies have suggested that it is admissions screening practices that may result in bias against applicants from lower income levels.
Two independent studies by SortSmart in the United States and Canada demonstrated that wealth does not correlate with the probability to use admissions preparation services. This makes perfect sense. The cost of our services are a fraction of what it costs to obtain a professional degree or attend an in-person multiple mini interview (in-person format of CASPer) for example, which requires applicants to travel thousands of miles and pay thousands of dollars for flight and accommodation, never mind the hefty application costs. Furthermore, professional school students on average have to pay up to $50,000 dollars per year just for tuition or risk going into debt and later forced to pay all back with interest. Therefore, it’s easy to see how naïve and false such a conjecture appears, even without the facts that show admissions prep does not correlate with wealth.
The two studies which included a random and representative sample of medical students and residents had a margin of error of +/-5% and suggest that: admissions appears to favor the wealthy (Fig. 2); the wealthy are the most represented group regardless of admissions screening tool used to select them (Fig. 3); 94% of accepted applicants would support a new and improve admissions screening tool (Fig. 4); wealth does not correlate with the probability to use admissions preparation services (Fig. 5).
Figure 2. Admissions Favors the Wealthy. While the median household income in the United States is $60K/year, 69% of medical school students and residents reported household income of over $60k/year at the time of application.
Figure 3. The wealthy are most represented group regardless of admissions tool used to select them.
Figure 4. 94% of a students and residents would support a new and improved admissions screening tool.
Figure 5. Wealth does not correlate with the probability of use of admissions preparation services.
The findings are similar to a report by the New York Medical College (NYMC) that showed underrepresented minority applicants scored lower on CASPer, compared to other applicants, and that males scored lower than females creating a gender bias. This is corroborated by independent published studies about the Multiple Mini Interview (MMI), which is the offline version of CASPer from the same university. MMI has been found to be biased against male applicants and underrepresented minority groups score lower on these types of interviews.
Administrators cannot tell whether or not you have prepared for the test.
The administrators go even further when you thought it couldn’t get any worse, claiming that they have "heard" students recommending to others that preparation did not help them and that they are telling you all this so you don't "waste your money". Here’s another false opinion spread as facts. This type of claim is vague, broad, unverifiable, and unconvincing. How many students from using how many different prep programs did they sample? How do they define “help”? Did it not increase their score, increase it not as much as they expected? Or did it not help with the butterflies they felt in their bellies the day of the test? Therefore, such generalizations spread as facts should be viewed with extreme caution. Second, online forum users with random names are probably not your most trustworthy options and could simply be representatives of admissions offices and administrators as discussed earlier. Third, our success rates and study results speak for themselves. The only waste of money here would be the cost to write this test plus the cost of reapplication process and loss of time if you don’t prepare well in advance.
The administrators say you must "be yourself".
When you thought it couldn't get any worse. The administrators say "don't prepare... we want you to be yourself".
This is contradictory to everything any future professional stands for because “being yourself” implies that you should not even be interested in self-improvement, which is in fact contrary to the one common skill every good professional has: a sense of life-long learning and self-improvement. You must always strive to become your better self.
Why do the administrators make such claims? Are they knowingly lying to applicants?
We don’t know for sure. This is something you must ask them in a public forum. We even asked McMaster to provide us with data about the test in a Freedom of Information request so we can examine the claims however our request was refused. Here are some plausible explanations based on our opinion. Perhaps they feel insecure about their test since it has not be really validated to correlate with actual on-the-job behavior. Rather, the test barely correlates with performance on other future tests. Maybe this is a method of customer acquisition for them because promising to new schools that the test is immune to preparation might entice skeptical admissions deans. Remember, each time they sign up a school to use CASPer, they make a lot of money from applicants because all applicants are forced to take the test. Maybe they just don’t know better and they don't understand that the hallmark of any future professional is their desire to self improve. In our experience, students that seek coaching are actually the best type of applicants who are eager and open to learn more. Or perhaps, they simply want to defame BeMo because they don’t like the outspoken underdog and advocate of students that has been standing up to them and demanding change. We don’t know and we don’t care.
The important question to ask is this: is this really the company that is being trusted to judge others on professionalism?
What we do know is that the for-profit test administrators do not offer any refunds if you get rejected. They do not provide any feedback. They do not provide you with your test scores. They do not have any form of satisfaction guarantee.
On the other had, BeMo does offer a bold satisfaction guarantee because we hold ourselves to a higher standard and we wouldn't create a program that we wouldn't recommend to our own friends and family, at full price. Furthermore, the reason we exist is to make sure no one is treated unfairly as a result of the current admissions practices until a fairer and scientific admissions screening process is created and adopted by most universities. In fact, a significant portion of our revenue is used for research and development of a new admissions screening tool.
You now have 3 choices:
1. You can simply choose to only apply to schools that do not require CASPer so you don't waste your money.
2. You can do absolutely nothing to prepare in advance like most students who get rejected and pretend what they say is true because it's the easier route or
3. You can reject this utterly nonsensical conjecture about preparation, stand up for yourself and get ready for your CASPer test before it's too late.
Ultimately, the choice is yours. Choose wisely.
To your success,
Your friends at BeMo
