In a recent discussion about how we help academics to use learning analytics, a very wise colleague made the point that we need to constantly remind them of the relationship between average engagement and success. This arose during a conversation about the accuracy of each individual alert raised. They can never be 100% accurate and still be useful; there’s always going to be a degree of uncertainty because we’re asking staff to intervene before something has gone really wrong. (See this post on WonkHE for further discussion of some of the issues.)

I think this is right, but the fact remains that our core message is about the average of thousands of students’ engagement with their courses. Staff have to remember that learning analytics has to be based on averages from within very large data sets. There’s a very human tendency to say ‘ah yes but not in this case’. Everyone I know in HE knows at least one person who did really well at university without appearing to try, but the brutal truth is that we remember these exceptional people precisely because they are exceptional.

The truth is that across large sets (like the first year entrants 2017-18 below) there’s a very strong association between time on task and academic progression/ success. Students who are more engaged are simply more likely to succeed.

Nonetheless I think that one of the tensions is that there’s a huge disconnect between the individual student sat in front of the academic or study adviser and these statistics. There’s the individual and then there’s this amorphous/anonymous blob of data and staff don’t always appear able to connect the student and the very apparent risk.

Chaos theory was a big thing in popular science about ten years ago. I don’t think it’s in the popular consciousness quite as much these days. I do wonder if that’s because we think with big data we can actually understand the immense complexity of systems such as the weather. One of the most famous analogies was, of course, the butterfly effect: a butterfly flapping its wings in Brazil would (when combined with billions of other chance events) create a tornado in a particular location of the US. The problem is that it’s difficult to demonstrate to that individual butterfly (let’s assume we can overcome the language and cognition issues) the association between its behaviour and the massive effect caused.

I think the analogy is true with learning analytics. The data from the individual student contributes to the big picture of all student success, but it can feel that there’s a huge conceptual disconnect in the mind of tutors, advisers and students between the data set and the individual and behaviours that make it.