January 18, 2024

When looking at the question of sharing data with learners it becomes a bit of a Goldilocks problem. How much feedback is too much, leading to overload and increased anxiety? How much is too little, where the learner doesn’t have a clear idea of where they stand or how they can approve? How much is just right?

This week on the Holistic Success Show we welcome Sylvia Mioduszewska, Senior Product Manager at Acuity Insights, to talk about the benefits and drawbacks when sharing individual performance data with students.

Read on to discover the advantages and disadvantages of sharing that data, as well as how programs can adjust and move forward to create better informed and motivated learners.

What data do health science programs currently share with learners to help them understand their progress through their program?

Like many things, the type and amount of information currently shared to learners from their programs varies greatly across schools. While some consistency exists, each institution is unique in their approach. 

This variety is exemplified by the numerous assessment methods available in every health science program, not just within a given institution. These take the form of:

• Graded examinations (often in the form of quizzes)
• Formative assessments
• Summative assessments
• Peer evaluations
• Self-assessments
• Self-reflections
• Portfolio reviews
• Labs and evaluations
• Research projects

This doesn’t take into account the clinical side of a health science program, which have their own types of assessments in the forms of clinical evaluations and feedback from simulated or standardized patients.  

How do programs manage this large and varied data set?

Different programs will have their unique approaches to managing the large and varied data set of their students, but the problems of that management is what unites them. As previously discussed at greater length in Episode 7 of The Holistic Success Show with Senior Product Manager of Analytics at Acuity Insights, Adam Tarnowsky, the scale of the data that is shared across multiple departments creates difficult obstacles. 

In developing Analytics, early interviews with medical schools in 2017 revealed that in some cases 30 different data sources were being looked at to understand how learners, faculty, and programs were performing. Some of these methods of gathering data were still using paper, a notoriously difficult medium to integrate back into a unified system. An excel spreadsheet in one department may not attach to another in a different department, making it difficult for cross-department discussions to have access to the same material.

What trends are emerging with data sharing with learners?

It might seem counterintuitive at first, but as the data collected by programs increases, the trend in sharing that data has been to reduce the amount accessible by the learner. There’s this idea of trying to take the stress and burden off learners from watching their assessments and scores and worrying and focusing more on confidence and mastery of skills and knowledge and promoting a growth mindset. So there’s been a kind of pulling back of sharing some of those assessments.

What are the benefits of sharing data with learners?

Here are some of the benefits to sharing data with learners:

• Clearer understanding: Through direct feedback, learners are able to have a clearer understanding of their strengths and weaknesses with the subject matter. In health professions, where continuous learning and self-improvement are essential to their lifelong career, having greater clarity about where they stand in relation to the material is invaluable.
• Depth of understanding: Sharing performance data promotes a reflective practice with students, leading to better integration of the knowledge and skills that are so important.
• Motivated learning: Constructive criticism, when done correctly, motivates learners to improve and builds trust between learners, their faculty and their programs. When learners have access to some of their performance data, it creates a heightened sense of responsibility and accountability to address areas of concern.

What are the drawbacks to sharing data with learners?

The drawbacks of sharing data with learners creates:

• Heightened stress and anxiety: Readier access to more detailed and frequent reporting often leads to greater feelings of stress and anxiety in the learner. Learners in EPA-based curriculums often report feelings of constantly being watched and assessed. In their struggle to meet the required number of observations for progression, they end up focusing on the reporting itself and not the skills and knowledge they are building.
• Feedback overload: Similar to planning a route, a learner needs to know where they’re going, how to get there, and how to navigate obstacles. When too much feedback is given, it creates a paralyzing overload, pulling focus away from the material.
• Comparison between learners: Learners are human beings, and comparison amongst peers is natural. With the focus on the results of learning (grades, assessment, etc) instead of the learning itself, it creates an atmosphere that promotes risk aversion to new challenges and learning opportunities that could be valuable, as well as hindering collaboration and teamwork. 

Recommendations

Just like Goldilocks, programs are looking for that just right balance between sharing too much and too little individual performance data with students, and the truth is that there is never going to be one just right option. What works for some will not work for others. Just right is a moving target. So what can programs do?

It begins with communication. Having the data isn’t enough, it’s about having that data packaged in a way that helps communicate to the learner where they are and how to get where they’re going. This requires an active discussion with the learner and the program. If the learner is unsure of where their strengths and weaknesses are, the data they are provided is not doing the job. They need to be able to communicate their struggles with the end goal of greater understanding of how to achieve their goals.

To do that, regular reviews of assessment methods and strategies is vital. This ensures an alignment between the program objectives and the learners’ needs. Additionally, moving away from traditional graded systems to prioritize competence and mastery can promote deeper learning and understanding in the learner.

However the learner and program get there, the end goal is to have both participants to be on the same page. 

At the conclusion of the rotation, regardless of whether a final evaluation or assessment is provided to the learner, they should have a clear understanding of their strengths and weaknesses, as well as their performance. Ambiguity should be eliminated, and feedback should be continuous, preferably delivered verbally, and whenever feasible, conducted in-person or through some form of face-to-face interaction.

Watch the full episode to learn more about this delicate but integral balance in sharing individual performance data with learners!