What 3 US medical programs are observing since using Casper

November 2, 2020

With the vast majority of disciplinary action by medical boards being attributed to issues of professionalism, it’s critical that medical programs consider applicants’ non-academic qualities, like empathy and compassion, when making their selections. However, traditional tools like personal statements and reference letters are unreliable, and interviews are too time-consuming to be used for a large number of applicants. All of this contributes to the need for a revamped admissions process that uses more evidence-based screening tools to assess non-cognitive competencies and widen pathways to medical programs.

In ongoing research projects with three MD programs in the United States, we’re looking to see how Casper can help evolve their admissions processes so they make decisions with confidence while saving time. These studies specifically seek to answer two questions:

Does Casper measure the non-academic skills that are most important to the program?
Are Casper scores associated with in-program performance?

Here is what the preliminary results show so far:

Casper’s impact on measuring key non-academic skills in the admissions process

In a joint study with a mid-sized medical program in the South-West, we evaluated the association and overlap between competencies assessed in the admissions process and Casper constructs and scores. We found that two key competencies the program measured – maturity and critical thinking – had a significant association with Casper scores. Based on these results, we’re building a process that will help the program use Casper earlier in the process to select the most promising applicants to invite to the MMI.

Interested in finding out who these programs are and getting the final results of the studies?

Subscribe to our newsletter to receive research updates and more content about how our programs are using Casper

Casper and in-program performance

One large program in the South-East US worked with us to conduct a quality assurance assessment focused on the link between Casper and clerkship grades. Initial results show that at a given GPA and MCAT score, applicants with the highest casper scores have a 5 percent higher probability of receiving the highest grade in clerkship. We are now working with the program to build a composite program-specific professionalism metric that incorporates clerkships, course grades, and administrative remarks. We’re also looking to create an admissions model to predict applicant performance in interviews based on screening measures like Casper.

We also worked with a mid-sized program in the same part of the US to evaluate the association between admissions metrics and course grades. The results showed that higher Casper scores were significantly associated with non-academic course grades, including clinical skills and doctoring, after accounting for MCAT scores and demographic characteristics (sex, age, and ethnic group). No significant associations were observed between Casper and more academically-oriented course grades, such as anatomy and biochemistry. Based on these initial results, the program will now work with Altus to add more cohorts of data and expand the study to build predictive models that identifies students who are likely to have issues, fail, or require multiple course attempts.

What does all of this tell us?

Although preliminary, the results show that Casper can reliably measure key non-academic skills that programs assess during admissions and in-program, making it a useful pre-screen tool to prioritize applicants for limited interview spots. All of this helps to save admissions committees valuable time in the admissions process. Based on the experiences of two of these programs, demographic differences in Casper test performance appear smaller than traditional cognitive assessments, although our own research has identified opportunities to further minimize those differences. By balancing academic and non-academic competencies in the admissions process, and using more effective and less-resource intensive screening tools to do so, programs can bring in more well-rounded students.