Casper Implementation Guide: A Comprehensive Overview

September 14, 2023

In the competitive world of university and college admissions, the traditional benchmarks of GPA, test scores, and recommendation letters have long been the norm. However, with institutions increasingly seeking well-rounded individuals who will excel academically, professionally, and socially, Acuity Insight’s Casper assessment has been embraced as a part of many institutions’ holistic admissions process.

What is the Casper assessment?

Casper is an online, open-response situational judgment test (SJT) for measuring soft skills tied to social intelligence and professionalism. Casper was developed in a university, for universities with a deep commitment to the success of their programs and learners.

Backed by nearly 20 years of extensive research, Casper consistently proves its ability to predict future student performance by evaluating an applicant’s soft skills. Compared to knowledge-based assessments, Casper consistently shows lower demographic differences, ensuring a fair evaluation of all applicants – regardless of their background. This approach strengthens its position as a more equitable assessment, empowering institutions to identify the most promising candidates by looking beyond their grades.

What makes Casper different from other assessments?

Casper goes beyond book smarts and takes admissions evaluations to a new level, emphasizing the significance of a candidate’s character and interpersonal skills. It poses real-world and professional dilemmas and prompts applicants to respond to these dilemmas while showcasing one or more of the ten soft skills that it assesses for, including:

Communication
Collaboration
Empathy
Resilience
Teamwork

Open Response Format

Casper is the most widely used open-response situational judgment test (SJT) in higher education. The assessment includes both video and word-based scenarios, each with open-ended questions that require timed typed and video responses. By allowing for open-ended responses, the assessment provides authentic answers from applicants that draws on their lived experiences and values – in turn, minimizing the opportunity for applicants to “game the system”.

How is Casper Scored?

We provide programs with an applicant’s percentile and a Z-score – a simple score that can be easily applied to the admissions process.

What are Z-scores, exactly?

A Z-score is a standardized type of score that offers insight into how an individual did on an assessment relative to all other test takers. In Casper, the score shows how the applicant did compared to other applicants in the exact same test instance – or, their peers.

A z-score tells us how close a score is to the average. If it’s positive, the value is above average. If it’s negative, it’s below average. The larger the z-score, the further away it is from the average.

Now, let’s talk about percentiles

Percentiles show us the percentage of people that fall below a certain value in the distribution. For example, a student who scored in the 75th percentile performed better than 75% of the class.

How do programs use Casper scores in their admissions process?

Casper was originally designed as a pre-screening measure for interview selection. As it has grown, programs have found value even when they do not have interviews. We recommend using Casper early in the admissions process – this is where it will have the greatest value for programs.

In general, programs tend to implement Casper scores using one or more of the following methods:

Formulaic
Rubric
Qualitative

Below we dive into each of these approaches, and uncover the benefits and risks of each.

Regardless of the implementation approach programs take, they are encouraged to follow a systematic framework for designing, implementing, and evaluating selection processes (e.g., see Cleland and colleagues, 2023, AMEE guide).

Formulaic Approach

This approach combines Casper and other admissions metrics – giving different weightages to each metric dependent on the program’s goals – to create an overall score for each applicant, which can then be used to rank applicants.

This is an excellent way to apply Casper scores to your admissions process because it’s highly defensible, data-driven, trackable, and easy to implement. However, the formulaic approach is also highly dependent on formula weights – meaning that the formula each program uses and the weight the Casper score is given will impact the results, which is why it’s important that the formula used by each program is reflective of their priorities.

River Fork University: A Fictional Case Study

River Fork University faces a unique challenge as they sift through 3000 eager applicants with only 300 program seats. The admissions team is determined to enhance their process by using a holistic review that includes both academic test scores and personal attributes.

They decide to use a formulaic approach, to ensure they are combining a variety of data points to review every applicant. Their formula considers GPA, GRE scores, and Casper test Z-scores.

In the first round of the admissions process – which leads to 700 interview spots – their formula includes:

1/3 GPA
1/3 GRE
1/3 Casper

Applicants are ranked based on this formula and the top 600 are fast-tracked to receive an offer to be interviewed. The admissions committee completes a thorough file review for applicants ranked 601 to 800 to select who to offer that last 100 interview spots.

Once those 700 interviewees are selected, the next step is to select for the 300 program seats using the following formula:

½ MMI/interview score
½ Pre-interview Formula score

In this scenario, River Fork University uses a formulaic approach. The Casper test scores are applied to a formula early on in the admissions process to help select who will go on to the interview.

Rubric Approach

This approach includes an applicant’s Casper score alongside other metrics in a scoring guide, with each level clearly described. Rubrics also provide interpretable groupings of performance, and are highly defensible and trackable.

The rubric approach is highly dependent on rubric points and definitions that are set out by the program.

Silver Valley University: A Fictional Case Study

Silver Valley University also faces the challenge of selecting 300 students from a pool of 3000 applicants for its renowned program. The institute aims to refine its admission process by incorporating personal and professional attributes into a comprehensive rubric.

The rubric, serving as the foundation for selecting applicants to interview, allocates the same point system to the four components (that is, each measure has the same number of maximum points). The four components considered include GPA, GRE scores, Casper test, and reference letters. An example of the rubric Silver Valley uses to get the score for each applicant is below.

Rubric points	Casper z-score or %ile	GPA	GRE (quant) (rough numbers)	Reference Letter
0	< -1.0 or<16th %ile	< 3.0	< 147<16th %ile	0
1	– 1.0 to 0 or 16th – 50th %ile	3.01 to 3.5	147 – 154.916th – 50th %ile	1
2	0 – +1 pr 51st to 85th %ile	3.51 – 3.9	155 – 165.951st to 85th %ile	2
3	1.01+ >85th %ile	3.91 – 4.0	166+>85th %ile	3

Silver Valley’s rubric has a maximum score of 12 (3 points for each of the 4 measures). To select the 700 interviewees from the pool of 3000 applicants, everyone with a total rubric score of 10 or higher are fast-tracked and offered an interview. Then to fill the remaining spots, the admissions committee completes a full file review for applicants with a total rubric score between 6 and 10. This lets Silver Valley dig deeper into individual histories and experiences for applicants who may be successful in the program.

Once applicants go on to the interview stage, the admissions team reviews MMI/interview evaluations and pre-interview rubric scores. This methodology allows the institute to holistically gauge all candidates’ potential.

Silver Valley University uses a rubric approach to handpick a dynamic cohort, ensuring that their program thrives with a diverse array of well-rounded individuals.

Qualitative Approach

The qualitative approach is a hands-on option where your reviewers use Casper and all other application materials to make a progress decision, without any numerical evaluation. This is a highly flexible approach that is open to your reviewers’ interpretation.

There are risks associated with this approach, namely that there’s a greater dependance on the program’s applicant reviewers and risk of individual biases. It is also extremely time consuming. In order for this to be a defensible process it requires clear documentation.

Paradise Grove College: A Fictional Case Study

Paradise Grove College decides not to pursue a rubric or formulaic approach, instead choosing the qualitative method.

As the committee combs through applications, reviewers see the whole file for each applicant including GPA, Casper, their transcripts, personal statements, and CV. After reading through the whole file, they decide if the applicant should be invited to interview or not before moving onto the next file.

Paradise Grove College has instituted comprehensive bias training and regularly discusses guidelines in an effort to ensure equity within the selection process.

A note on using Casper scores

You should plan to adjust the way that you use Casper – and the rest of your selection measures – as you gather data and analyze the results. This is fundamental to continuous quality assurance and quality improvement of your selection process (Cleland and colleagues, 2023). If you’re new to using Casper, you may choose to start with a moderate weight and gradually develop a specific implementation strategy as you gain confidence in what works best for your process. Take it step by step to find the most beneficial approach for you.

The problem with the screen in/screen out method

Best practices sometimes change, which is why we have updated our recommendations on the screen in/screen out method, otherwise known as cut scores.

Research shows that creating cut scores leads to a strong divide that inhibits flexibility, and can also lead to less diversity in your program cohort. Incorporating mid-range academic scores typically relates to more diverse classes, while keeping high success rates – and cutting students based on a small and insignificant difference in score can cut quality, diverse students who would excel in your program.

Imagine that the key to getting into a program is the difference between a 3.49 and a 3.5 GPA, even though there may be no practical evidence to say that the person with a 3.5 will be successful while the 3.49 won’t. With a cut score you won’t even get a chance to make that call.

While you may have good evidence for a cut score on a particular measure, we encourage you to experiment with new approaches that may deliver better results, like a formula or rubric.

Casper is a research-backed, efficient, and flexible tool that supports programs’ holistic admissions process. Casper identifies applicants with a diverse skill set who might be overlooked when programs focus too heavily on academic scores for admission decisions; it also supports their in-program success. However, each program is unique, and the way that they implement Casper and use Casper scores may vary depending on what’s needed for program success