February 3, 2023

At the end of November 2022, OpenAI released a public update to one of their prototypes; a large-scale language model trained on a massive amount of colloquial conversation and text from the internet. At its core, it is a chatbot, however it has the power, data, and training to sound almost genuinely human. The release of ChatGPT has generated myriad discussions and debates on media, social media, boardrooms, offices, and living rooms. But nowhere more so than in higher education. It has exploded into the higher education news cycle and within institutions, thanks to the possibilities around its use in assignments, essays, personal statements, and more.

The value of personal statements or essays in the admissions process has been in question for some time. These documents are occasionally dubbed “village statements,” underscoring the socio-economic advantages that certain applicants have over others. This advantage may stem from their ability to afford editorial assistance and coaching or their access to mentors and other forms of support. Ultimately, personal essays often fail to offer a genuine assessment of an applicant and can serve as obstacles to educational access for certain individuals.

Earlier this year our co-founder and VP, Science & Innovation, Dr. Kelly Dore, penned an op-ed for eCampus News about the risks and benefits of generative AI. “Using ChatGPT to plagiarize essays and personal statements is a concern, but this tool needs an active and human user who can sift through inaccuracies and provide an empathetic and authentic voice. If a student does over-rely on ChatGPT, they risk setting themselves up for future failure as a professional who lacks the key skills to succeed,” said Dr. Dore.

While universities may decide that personal essays are no longer an effective assessment tool, it’s critical that they find an evidence-based approach to review non-academic skills.

This is where Casper plays a crucial role. Backed by nearly two decades of research, Casper is an evidence-based situational judgment test (SJT) designed to assess an applicant’s soft skills. It empowers institutions to admit individuals with a comprehensive skill set, positioning them for success in their selected academic program and, ultimately, in their future professional endeavors.

Is Casper safeguarded against the use of ChatGPT?

Is Casper protected against the possibility of test takers utilizing ChatGPT to respond to questions? The short answer is ‘yes’. Here’s the long answer:

• Casper has both a written and video response section. For the video-response section, applicants share their answers orally in this section, and it tends to be obvious when someone is reading out loud. Therefore, the use of ChatGPT would be obvious to raters.
• Casper includes scenarios that are short videos with intricate details. Describing the situation to ChatGPT, as well as the questions, would take up a significant amount of the response time, if not all, and affect the quality of the response.
• While evidence shows that ChatGPT is able to pass some multiple choice tests, Casper is not a multiple choice test – it’s an open-response format that requires the formulation of a response within a set timeframe. Because there is no one right answer to each question, an important part of Casper is providing the unique rationale behind the response. Knowing the “why” behind the “what” is vital.
• We have methods within proctoring (audio and visual) and rating to spot cheating or use of unapproved tools during the test. For example, monitoring the pace at which an applicant completes the test – proctors can detected if the applicant is using prepared answers and consequently speeding through the test.
• Existing measures, such as disabling the ability to copy and paste and restricting the time to answer, also make it very difficult, if not impossible, to use ChatGPT during the test.
• When an applicant agrees to undertake the Casper test, they are also agreeing to our policy that any evidence of cheating or obtaining an unfair advantage would result in a breach of our Terms of Use.

Risks of using ChatGPT

Aside from this, there are inherent risks for test takers who may want to use ChatGPT during their Casper test:

• Although ChatGPT can write plausible-sounding answers, it can also produce incorrect or nonsensical answers. Using ChatGPT during a Casper test, if they’re able to, could disadvantage the applicant.
• Answers provided by ChatGPT are more generalized and generate average responses. Using such aids, if they’re able to, could impact the applicant’s evaluation negatively, resulting in a low score from our human raters.
• Good responses to Casper questions draw upon lived experiences and personal values of our test takers. Generalized answers would mean raters don’t see those features in a response

All of these factors contribute to the fact that applicants need to put in genuine effort and provide authentic answers on their Casper test. These responses should truly reflect who they are – this is a unique opportunity to showcase their lived experience and versatility.

There’s no doubt that ChatGPT has transported us into a new reality within higher education, and many are grappling with its impact. However, ChatGPT also opens up a host of new opportunities.

One solution that can help admissions teams in this new reality is evaluating applicants more holistically in higher education admissions through timed, open-response SJTs like Casper.

[cta cta_id=”21453″]