Beyond Grades: 5 Lessons from the Largest Meta-Analysis of Situational Judgment Tests

September 29, 2026

How can we look beyond grades to evaluate future healthcare professionals who will not only excel academically but also display the social intelligence and professionalism needed to become patient-centred clinicians?

Medical Education, the leading international journal for research on health professions education, recently published the largest synthesis to date on open-response situational judgment test (SJT) validity evidence. 

The evidence shows that, just as standardized academic assessments like the MCAT predict academic performance, Casper and other open-response SJTs predict an applicant’s future interpersonal and professional performance.

On the heels of this landmark meta-analysis, we’re unpacking five essential lessons to help your institution succeed in the ever-evolving landscape of health professions admissions.

1. A Comprehensive Admissions Process? It’s No Longer Optional

Institutions need to attract, retain, and graduate learners who will become the best future healthcare professionals; making a comprehensive admissions process that assesses a broad range of skills essential. To build stronger admissions processes, institutions need to combine academic performance assessments (e.g., grade point average, standardized academic assessments like the MCAT, GMAT, or GRE) with tools, such as SJTs, designed to gauge interpersonal and professional skills.

SJTs are powerful because they assess how an applicant will behave when the right answer isn’t in the textbook. They assess areas such as:

• Communication
• Empathy
• Collaboration
• Judgment
• Ethics
• Professionalism

These tools are most effective when intentionally embedded in the admissions process and aligned with program goals.

2. Not All SJTs Are Built Equally

It’s critical to understand how open-response (or constructed response) SJTs, such as Casper, differ from traditional multiple-choice formats. Each question on a multiple-choice (or selected-response) SJT has a fixed number of responses, and the test-taker has a decent shot at choosing correctly, even if they’re not 100% sure of the answer. 

An open-response SJT, on the other hand, gives the test-taker a blank slate to respond to real-world scenarios with the critical thinking and nuance they deserve. Test takers are able to articulate their reasoning, allowing trained raters to assess not just what they would choose to do but the rationale behind those choices. 

3. Research Shows How to Build Better Cohorts

The recent meta-analysis published in Medical Education shows that open-response SJTs effectively predict learners’ professional performance during their healthcare training.

Specifically, this synthesis of 100 findings across 27 studies shows an average correlation of 0.22 between SJT scores (99 of which were Casper) and professional outcomes, climbing to 0.32 when the in-program outcomes assess the same competencies as Casper. In practice, this means programs that leverage Casper could see an additional 10-15% of their students perform above average on critical interpersonal and professional milestones, compared to randomly selecting from their pool of program applicants without Casper.

Don’t forget: real people are behind these numbers. That’s 10-15% more people you can choose for your program who are likely to be empathetic, well-rounded healthcare professionals, better equipped to serve their communities.

4. Meeting the Moment in Admissions

Admissions teams recognize that AI is changing the face of admissions, particularly regarding applicant authenticity and the utility of traditional admissions tools, such as personal statements. 

Applicants can now use AI to generate written materials to support their applications within moments, making these traditional tools a less effective yardstick for gauging potential success than in the past. SJTs can help programs respond to these evolving challenges. By asking your applicants how they would respond to real-world dilemmas in a proctored, timed environment, they provide your team with evidence-based insights that support your institutional objectives. 

But adding an SJT to your admissions toolkit isn’t just a one-and-done scenario. You must regularly evaluate how your entire admissions process works for both your institution and its healthcare programs, and especially for your learners.

5. One Piece of the Puzzle

An SJT is just one piece (albeit an important one) in your overall admissions puzzle. To put it another way, an SJT shouldn’t stand alone as the sole metric when determining who is the best fit for your institution and program. Instead, it should work alongside and complement other admissions tools, such as interviews, grades, and other test scores, to predict a wide range of future student and professional outcomes. See how the University of Manchester incorporates Casper into their admissions process.

Think of an SJT as adding pixels to the picture. It allows a more fully developed portrait of your applicants to emerge, offering a glimpse of the future healthcare professional they will become. It’s designed to enrich your understanding of these individuals while complementing, not replacing, those traditional and still very relevant evaluation methods.

These are just a few of the insights we explored in our webinar, Beyond Grades: What a Landmark Meta-Analysis Reveals About Assessing Future Healthcare Professionals. This expert panel discussion and live Q&A was led by Dr. Kelly Dore, Acuity Insights Founder and VP, Science and Innovation, and Adjunct Professor at McMaster University. She was joined by:

• Ivy Erickson, P.A.-C, Clinical Assistant Professor at the University of Wisconsin-La Crosse
• Dr. Brian Steele, Ed.D., Associate Dean for Admissions and Assistant Professor of Medical Education at the University of Georgia School of Medicine
• Dr. Michelle Schmude, EdD, MBA, Vice Provost for Enrollment Management and Professor of Medical Education at Geisinger College of Health Sciences

Missed the webinar? Watch the webinar on demand, and gain more takeaways from the meta-analysis, alongside admissions strategies you can put into action at your institution.