Why use learning analytics?

There are a number of potential reasons for Universities to invest in learning analytics (1): curriculum design, testing marketing effectiveness, insights into service provision etc, but for most institutions interest appears to be primarily about some form of student success. This may be about raising students’ awareness of their own learning, providing early warnings of at risk students or rethinking support for individuals or groups.

At my University (Nottingham Trent University) we have identified four success criteria:

• Improving student success
• Improving relationships with staff (particularly with personal tutors)
• Improving students’ awareness about their own learning
• Improving data for institutional systems

What do we mean by student success?

Success is a personal lived experience. We work in a central department for a university with over 28,000 students. Much as we’d like to, we can’t create a personalised individual measure of success for each student. We have a conflict, the tool is meant to support the way that we personalise support and the learning experience for student. However we need to work with some form of shorthand. Therefore, we tend to use two benchmarks:

• Progression from the first to the second year
• Achieving a ‘good’ degree classification (60% +)

We tend to describe learning analytics in these two frameworks, for example:

“In 2015-16, 95% of students with high average engagement progressed from the first to the second year.”

“In 2015-16, the grade point average for first year students with low average engagement was 52%, for students with high average engagement, 63%”(2)

These are reasonably good benchmarks (and after all, we need something to frame our work).

• They relate to outcomes that most students would aim for (3)
• They are reasonably comprehensible
• They are relatively easy to remember
• They are reasonably comparable against other institutions (and ultimately that’s why we chose these measures)

But they can only describe the student experience in very broad terms and mask the individual details.

• What if a student has struggled against all types of adversity, completes their degree, but has a final grade below 60%?
• What if a student decides that they are on the wrong course and moves to a different course/ institution entirely?

The metrics are useful, offer us insights and provide a focus. We do however need to be careful not to ascribe additional meaning to these metrics, or give them an objectivity that they don’t deserve.

We may also want to remember Goodhart’s Law

“When a measure becomes a target, it ceases to be a good measure.” (Strathern (1997) (4)

We need to be careful to avoid ignoring an individual’s successes just because it doesn’t meet a broad brush metric.

 

---

(1) The Wikipedia page on learning analytics is brilliant. It not only offers a useful overview of the state of the art, but the ‘definition’ section opens with an academic debate about definitions. 

(2) Amongst students who progressed to the second year.

(3) At time students appear particularly obsessed with the ‘good degree’ classification. 

(4) http://conferences.asucollegeoflaw.com/sciencepublicsphere/files/2014/02/Strathern1997-2.pdf