Disruption in admissions: why letters of intent and personal statements don’t give enough insight

April 4, 2019

This month, we’re looking at disruption in admissions. As more and more institutions are moving to a holistic approach, many new factors are playing a key role in how admissions professionals assess applicants. But, how are institutions approaching holistic admissions processes, and where is there still work to be done? This series takes a look at letters of intent and personal statements, demonstrated interest, and new laws surrounding admissions that focus on holistic practices to ensure integrity.

Kicking off our disruption series, we look at two of the major pieces admissions professionals use to develop a holistic understanding of applicants: personal statements and letters of intent. While they have always been useful tools for admissions, a new focus on how a holistic process can produce a varied student body that is rich in potential has significantly increased their importance.

But, is it really disruption if there is simply a bigger focus on something that already exists? Well, this new focus on holistic admissions has also shown that letters of intent and personal statements may not be reliable or add enough value to the holistic process. We take a look at what letters of intent and personal statements are trying to add to the admissions process, where they fall short, and how disruption can fill in the gaps.

Planned persona

In a holistic process, letters of intent and personal statements try to give admissions teams insights on an applicant’s personality. As a result of the careful planning applicants put into writing their letters and statements, research shows that these insights are often unreliable.

Any applicant will start their letter of intent process by researching the institution they are applying to. They read about the values, mission statements, recent news, professors, and much more, and then craft a letter of intent that speaks to these core values presented by the institution.

Applicants are crafting their personalities to fit a mold they feel the institution has created. While the reason for applying to that specific institution may be genuine, the personality the applicant  presents is not. Admissions officers can not understand how well an applicant will fit into their institution, and later a profession, because the applicant has forced him or herself into a mold specifically designed to appeal to that institution.

Admissions officers are getting a perfected version of an applicant’s goals and self-assessed people skills. This gives little valuable information into how that applicant will perform in the program, and whether they will succeed? Professionally.

So how can disruption fill the gaps?

Research that compares situational judgement tests (SJTs) against personal statements specifically, shows that SJTs are more able to predict meaningful performance outcomes and reliable scores.

So, institutions that have incorporated SJTs into their process as a replacement or complement to personal statements and letters of intent, will have a better understanding of an applicant’s authentic personality, and more importantly, reliable data to predict how that personality fits the institutions program.

Who wrote this?

The added emphasis on the letter of intent and personal statement in a holistic process also created an entire new business: professionally written letters of intent and personal statements.

There are many tutors and master’s students who are paid to craft, or edit, letters of intent for applicants. These writers explain how applicants should write the letter, give key focus points, and in some cases, just write a letter on behalf of the applicant.

In fact, there is evidence to support that letters of intent and personal statements do not even show an applicant’s writing ability since they have been so heavily edited or written by others.

Admissions are now receiving these “expertly written” letters that give no information about the applicant, since the applicant had very little to do with the writing process. And while hiring a writer is not illegal in anyway, the process is as detrimental to a holistic admissions process as hiring a test taker to boost scores.

How can disruption fill the gaps?

Back to SJTs and even multiple-mini interviews , both insure that the applicant is actually the person responding to the questions. In both cases, the applicant goes through a verification process to ensure test administrators and admissions professionals are engaging with the actual candidate. Again, this may seem unnecessary, but given the recent scandal where applicants paid others to take tests, this is a valid step.

Even when administered online, a trustworthy SJT administrator takes multiple steps to ensure the applicant is not receiving help.

So, the disruption SJTs and multiple mini-interviews bring to admissions are helping to ensure that personality and performance indicators are actually coming from the applicant and not a paid professional.

Often leaves out key factors

While test scores and academic achievements give insights on how a student will perform in-class, there is still a need to know how applicants will interact with classmates, and future colleagues. Letters of intent and personal statements try to give insights into these areas, but miss one key factor: an applicant may purposely, or even unintentionally, leave out pieces of their own personality or behaviour because they feel these traits are negative.

For example, an applicant may say in their letter of intent that they thrive in group work situations. They may give examples of past experiences where they completed a large-scale project as part of a team and succeeded in their goals. What this anecdote does not show is their method of achieving results may have been based on doing all the work, ordering the other team members to do things instead of collaborating, or simply took a back seat and did not actively contribute to the group’s ideas.

All of this information is left out of the letter of intent, because the applicant believes it reflects negatively on them. But, maybe this shows that the applicant is willing to work harder to achieve a result. Or, they have a talent for leading through difficult situations. Or, maybe, they succeed when given clear direction and goals.

So how can disruption fill the gaps?

Research into constructed-response, or open-response questions, in a timed situation with proper administration, enable trained evaluators to fully understand an applicant’s personal and professional characteristics.  

Since the response needs to be fully written out, and is in a timed environment, the applicant is more likely to honestly answer how they would act in a given situation. So, responses will be more authentic.

Trained evaluators are then able to take these responses and produce an accurate assessment of an applicant’s personal traits.

No structured review process

There is a key question when it comes to letters of intent, or even personal statements: “How do admissions teams evaluate them?”. The answer is incredibly complicated because there is no set standard.

Unfortunately, every institution evaluates these letters differently, and there is no set metric to determine the usefulness of this step in the process.

While some institutions focus on past experience to measure skills, others may look closely at goals to understand future aspirations and success. Every different admissions team can read the letter differently and will have a different focus point and take away.

On top of this, there is rarely a metric or study done to evaluate the impact of using this information. Once the students are in the program, it’s assumed they will do well. But very few institutions look at each batch of applicants and determine if their approach to using letters of intent made any discernible difference.

So how can disruption fill in the gaps?

Institutions are beginning to understand the value of comparing applicants on entrance and graduation. For example, researchers from Belgium found that closed-response format SJTs administered to applicants were able to predict medical internship ratings and practitioner performance even 9 years after taking the test.

What this means is that data taken from SJTs is reliable and can be used to gather insights on past applicants. By analyzing this data, admissions teams can monitor how their selection strategies affect the student population. They can then adjust these strategies using reliable data and insights, giving institutions a metric they can use to evaluate the impact of their holistic strategy.

What’s disrupting?

The letter of intent is still beneficial for other reasons. It does give applicants a reason to deeply understand an institution’s core values and decide if that institution really is the best fit for them.

However, from an admissions standpoint, the letter of intent is not giving the insights institutions need to make a holistic decision and strategy.

There is a loss of authenticity because of the time spent researching and crafting the perfect letter. Key insights on personality are missing because an applicant may feel some of their qualities are negative. And in the end, there is no way to tell if the applicant actually wrote their own letter, which is essentially the same as looking at self-submitted academic achievements with no verification.

The disruption comes from the data and insights offered by SJTs and other new and innovative admissions tools. Constructed-question SJTs are more able to provide valuable insights on applicant’s’ personal and professional qualities. This data is reliable enough that long term strategies can be implemented, analyzed, and changes can be made to improve the student population and increase the holistic method’s effectiveness.

Thinking to spearhead a change in your organization? Read how other admissions professionals have changed their processes for the better, with data-driven and measurable people skills assessment, here.

Photo by Glen Carstens-Peters on Unsplash