Widening access to medical education using situational judgement tests

August 28, 2019

In the United States and Canada, research shows that disparities in the quality of healthcare continue among racial and ethnic minority groups, as well as those from lower socioeconomic status. American minority populations experience a higher likelihood of chronic and preventable diseases, have double the infant mortality rates, and are more likely to pass away from conditions like breast cancer than their counterparts. This may be in part due to the level of comfort these groups feel with their doctors and the quality of the health care they receive. A review paper revealed that patients prefer having doctors of their own race/ethnicity, and individuals with strong racial/ethnic preferences or mistrust would more likely seek out treatment if their doctor shared their ethnicity.

There is evidence to suggest that increasing the diversity of students in medical school classes could be part of the solution. Studying alongside a diverse group of peers could better prepare medical students for interactions with patients from different backgrounds, which could increase the quality of the medical treatment they provide. Research shows that when groups have members with not only diverse cultural or ethnic backgrounds, but also with diverse perspectives, they’re more likely to have better problem-solving skills than homogenous groups.

Increasing diversity in medical schools is a moral imperative and key to improving the quality of healthcare and the field of medicine as a whole. Even with the strong push in the past decade to increase the representation of underrepresented minorities in higher education, the situation actually seems to be getting worse. Up until now, accepting more diverse applicants with the goal of widening access has proven challenging for schools as programs are still hindered by the weaknesses of traditional admissions tools.

Almost every university uses cognitive measures like GPA and standardized tests (MCAT, UKCAT, BMAT, GAMSAT, OAT, DAT, UMAT, etc.) in their selection process. These tools have demonstrated lower scores amongst disadvantaged groups: underrepresented minority (URM) applicants and lower socioeconomic status (SES) applicants (defined as applicants who had paid employment before the age of 18). Non-academic measures show little differences in race and SES effects, however not all non-academic measures are up to the task. Personal statements and reference letters have demonstrated low reliability and poor validity all around (see our blog posts on why both statements and letters are mostly useless); therefore, it would be difficult to clearly differentiate between applicant groups to see how well, if at all, these two assessment tools address the diversity problem.

Schools need an admissions tool that doesn’t put applicants from underrepresented minorities and/or lower socioeconomic brackets at a disadvantage. To that end, many universities are adopting situational judgment tests (SJTs) to help evaluate an applicant’s personal and professional characteristics, alongside academic performance measures.

Although slight differences still exist between populations from different ethnic and socioeconomic backgrounds when it comes to SJT scores, research has demonstrated that this discrepancy tends to be much less than that observed in cognitive scores. SJTs and other alternative predictor measures can reduce these differences amongst various groups of applicants because they measure non-academic abilities, of which variations for diverse groups are smaller. One study concluded that using alternative predictor measures, like SJTs, alongside the more traditional cognitive assessments would greatly reduce the differences amongst diverse groups. There’s just one problem: administering SJTs can be costly.

Casper is an online SJT for assessing personal and professional characteristics. It is low cost to applicants, conveniently administered through web-based delivery, and easy to use for academic programs at no charge to the institution. The use of Casper in medical education admissions screening has demonstrated positive effects on the diversity of applicants invited to interview allowing admissions teams and committees to have a more diverse pool of applicants to select from.

By: Patrick Antonacci, M.A.Sc., Data Scientist

Photo by Chinnapong on Adobe Stock