TouroCOM-NY finds that Casper scores correlate with humanistic outcomes
October 28, 2021
“Our data on Casper’s predictive validity clearly shows that there are benefits to using Casper in the admissions process…we think it could potentially help us determine the best candidates to invite to interviews after secondary applications.”
Dr. Stacey L. Fanning, Assistant Professor and Course Director of Immunology, Department of Basic Biomedical Sciences, Touro College of Osteopathic Medicine
About the program
Since opening its doors in 2007, the Touro College of Osteopathic Medicine in New York (TouroCOM-NY) has become a highly respected college in the U.S. for osteopathic medicine. It attracts about half of all DO applicants every year, thanks to its cutting-edge curriculum — students learn the material at home and professors track the students’ grasp of the material in class — as well as its commitment to underserved communities. Approximately 80 percent of its hospital affiliates target underserved communities in both rural and urban areas, and their students are active advocates from Harlem and the Hudson Valley to Haiti and beyond.
No. of applicants per year
~10,000
No. of matriculants per year
270
Highlights
Challenge
- With a commitment to producing graduates who will go on to treat patients in underserved communities, TouroCOM-NY sought a more reliable way to assess applicants’ personal and professional characteristics.
Solution
- The program adopted Casper as part of a multi-year research study to explore its predictive capacity with regards to the Class of 2023’s performance in non-cognitive and humanistic assessments in the program.
Results
- Casper positively correlated with performance in the non-cognitive portion of the admissions interview, and it was the strongest predictor of performance in the second-year Objective Structured Clinical Exam (OSCE) and community service hours.
Deep Dive
Of the 20,000+ aspiring Doctors of Osteopathic Medicine (DOs) that apply to American colleges each year, 10,000 of them — about half of all applicants — knock on TouroCOM’s door.
Why the demand? TouroCOM-NY has a few distinct qualities that continue to attract a wide variety of applicants. TouroCOM has both urban and rural campus options in the New York City region, offers a cutting-edge curriculum, and is committed to training osteopathic physicians with an interest in practicing medicine in underserved communities, an ever-growing need in North America. The school is also dedicated to creating a fair and effective admissions process, one that ensures that underrepresented minorities in medicine are given an equal chance at admission. Although TouroCOM does have minimum academic standards during the initial application review process, a fully fleshed-out holistic review process has been used since the very early days of TouroCOM-NY.
All tools — including personal statements, reference letters, the holistic review, and interviews — are meant to complement each other in the admissions process. For decades, academic institutions have used any combination of these tools to admit students to a variety of academic programs. In addition to selecting applicants that have the academic potential to succeed in their medical training, TouroCOM-NY seeks to find applicants who will deliver compassionate care with the highest level of ethics and professionalism.
Students with deficiencies in the areas of personal characteristics and professionalism have negative consequences for academic programs, the hospitals where the students have their clinical rotations, and the students in question. Programs incur the burden of dealing with these students, who can become problematic and unprofessional. Typical admissions tools, grades, MCAT scores, and carefully edited personal statements, are often unreliable in assessing positive personal characteristics and professionalism. As a result, many medical schools and professional organizations strive to seek the best way to consistently and accurately evaluate these attributes in candidates. This approach, to ensure high ethics and professionalism in medical school graduates, is a core value of the teaching philosophy at TouroCOM-NY.
That is why TouroCOM-NY decided to adopt Casper, an online open-response situational judgment test, as part of a three-year research study to see if it could predict humanistic outcomes in their selected students. In other words, TouroCOM-NY wanted to know if adding Casper could help identify applicants who would go on to develop strong doctor-patient relationships and provide compassionate care.
Applicants who meet the program’s academic requirements are invited to submit a secondary application, which includes completing the Casper test. Select candidates are then invited to interview with two faculty members who use a rubric to assess candidates’ cognitive and non-cognitive abilities. Once those interviews are completed and rated, the Admissions Committee performs a holistic review and makes recommendations on offers to the Dean for final approval. As Casper was initially adopted as part of a research study, candidates’ scores were not reviewed or considered for admissions purposes.
So what did the program find out?
At the end of the 2018/19 cycle, TouroCOM-NY discovered that Casper scores positively correlated with the MCAT Critical Analysis and Reasoning subsection. A positive correlation was also observed between Casper scores and interview performance, but just slightly due to the fact that interviews assessed cognitive abilities in addition to personal and professional characteristics.
Dr. Stacey Fanning, and Dr. Tamie Proscia-Lieto, Co-Chairs of the TouroCOM Admissions Committee sought to determine Casper’s predictive capacity in relation to performance in the non-cognitive portion of the admissions interview and performance in-program, specifically second-year OSCE scores and community service. Focusing this second part of the research study on the Class of 2023, the program saw a statistically significant positive correlation between Casper scores and performance in the non-cognitive portion of the interview. This correlation was most notable when applicants were asked about their motivation to pursue a career in primary care. For in-program performance, Casper scores predicted performance in OSCE and community service hours more than the admissions interview, which has a focus on both the values and interpersonal skills of the applicant.
“Our data on Casper’s predictive validity clearly shows that there are benefits to using Casper in the admissions process,” shares Dr. Stacey L. Fanning. “While we’re still determining where it can be best used, we think it could potentially help us determine the best candidates to invite to interviews after secondary applications.”
“Further, the use of Casper scores shows promise as an additional tool for prediction of students’ future performance on non-cognitive, in-program humanistic assessments,” adds Dr. Tamie Proscia-Lieto.
What’s next?
While TouroCOM-NY determines how they can best incorporate Casper scores into their admissions process, the program will also explore additional tools and metrics this cycle that can help paint a more complete picture of applicants’ personal and professional characteristics. These tools include Duet, a value-alignment assessment that compares what applicants value in a program to what the program has to offer.