What Open-Response SJTs Can Tell Us About Future Professional Performance

June 22, 2026

Exceptional physicians, engineers, teachers, and health professionals are defined by more than their academic knowledge. Their ability to communicate effectively, think critically, exercise sound judgment, and empathize with patients, clients, and students is often what sets the good apart from the great.

Yet for years, higher education has grappled with a difficult question: How do you identify and evaluate these qualities before students enter their chosen professions? Traditional measures such as grades and test scores can provide insight into academic performance, but they often reveal little about the professional and interpersonal skills that are critical to long-term success.

We’re talking about important non-technical competencies, such as communication, empathy, and teamwork. As noted in our recently published report, Beyond Grades: Why Situational Judgment Tests Matter in Higher Education, institutions “are no longer judged only by what they teach, but by who their graduates become and what they are capable of as they enter into the profession.”

New meta-analytic evidence offers a clearer answer. It confirms that open-response situational judgment tests (SJTs), which ask students to respond to realistic scenarios, can provide meaningful insight into how they apply judgment, professionalism, and other critical non-technical skills in context. This is especially true when institutions are intentional about what they measure and how they measure it across the entire learner journey — from admissions to education and learner development.

Enhancing the Conversation About SJTS

Medical Education, the leading international journal for research on health professional education, just published the largest synthesis to date on constructed-response SJT validity evidence. It combines the results of 27 independent studies with 100 effect sizes, and confirms what Acuity Insights has been championing wholeheartedly for years: stronger performance on open-response SJTs translates to stronger downstream professional and interpersonal outcomes.

The synthesis adds a strong, authoritative voice to the growing conversation about measuring these critical skills.

CTA: Find more information on the meta-analysis and its findings.

Skills Institutions Care About Are Often the Hardest to Measure

The skills institutions prioritize — collaboration, ethics, and problem-solving, just to name a few — are often the most difficult to measure. Even when institutions do take steps to measure them, they’re not always done so consistently across the entire learner journey. But, as the meta-analysis indicates, when these skills are both properly and consistently measured during admissions via open response SJTs, they have a tremendous impact downstream on:

• Selection evaluations: An applicant performs well during the rest of the admissions process, including interviews and other selection tools.
• Curricular assessments: Learners perform better in their academic program, such as on exams and in peer evaluations.
• Workplace evaluations: Trainees working in real clinical environments are rated higher by their supervisors.
• Behavioural outcomes: Learners exhibit better real-world behaviours, such as leadership and indicate fewer professionalism issues and remediation efforts.
  

Alignment Drives Stronger Signals

The meta-analysis also shines a spotlight on the importance of construct congruence. Construct congruence means that outcomes are more predictive when institutions intentionally evaluate the same skills throughout the learner journey.

When outcome measures aligned closely with the skills open-response SJTs were designed to assess, predictive validity — the ability to predict future behaviour, performance, outcome, etc. — nearly doubled. In other words, the more conceptually aligned your predictor and outcomes measures are, the stronger your predictions will be.

This is especially critical now, when application volumes have surged and institutions need to look beyond GPA to choose more well-rounded applicants. Also, artificial intelligence (AI) is here to stay and has changed the game. In an Acuity Insights survey, 35% of applicants admitted to using AI tools during the application process. So, it’s no wonder that institutions are being forced toreassess the yardsticks they traditionally used to gauge applicants, such as statements of purpose and references, as signals in admissions.

The implication is significant: Many traditional evaluations may not actually measure the professional competencies that institutions value, making it harder to see a relationship between predictors and outcomes. This creates an opportunity for educators to rethink assessment practices and build more intentional, skills-aligned evaluation systems so that they can better predict who will succeed at the activities that matter most, and better support the development of these skills.

Casper: A More Well-Rounded View

The evidence is clear: when institutions evaluate human skills more intentionally and with better evidence, they give themselves, and their students, a competitive edge. Casper, the most widely used open-response situational judgment test in higher education, is designed to assess social intelligence and professionalism in a structured, scalable way.

The bottom line is that better decisions depend not only on measuring more, but on measuring the right things — and with greater consistency — across a learner’s entire journey. 

Learn more about how it predicts success, optimizes admissions, and widens pathways by lowering academic thresholds for applicants, and book your demo today.