I’ve recently published a paper with a colleague Dr Rebecca Siddle for a special edition of Assessment and Evaluation in Higher Education, The paper explored the impact of ‘no-engagement’ alerts for identifying students in need of further support and contrasted it with using background characteristics. In this paper, we used widening participation status as a proxy for poorer socio-economic background. The essence of the paper is that:

• Learning analytics is an effective way to target learners in need of further support. The ‘no-engagement’ alerts were effective at providing early warnings. As students generated more alerts they were both less likely to progress to the next year and less likely to achieve higher grades.
• We don’t use socio-economic factors in our core algorithm, nonetheless, students from disadvantaged backgrounds were still more likely to be identified because they engaged less with academic activities.
• The ‘no-engagement’ alerts are more efficient at identifying students at risk than background characteristics. If we base alerts on WP status, we will generate lots of false positive alerts (about three for every student we successfully identify as being at risk). However, if we use ‘no-engagement’ alerts we generate less than one false positive alert for every accurate alert raised.

We took a strong view when we first started using learning analytics that demographics shouldn’t be in the algorithm for deciding risk. I still feel that’s the right approach, the role of background in algorithms is rightly a cause for concern. We didn’t include the following graph in the paper (for lack of space), but I think it helps make an important point. On average, Widening Participation students had a lower average engagement rating than their peers. But if you look at the two bars, WP students engagement is bifurcated, pushed to the extremes, they are both more likely to be highly-engaged and lowly-engaged than their peers (*1). Targeting students using background would target both sections of the WP cohort.

There’s room for considering strategies for targeting within cohorts of historically disadvantaged groups, but we remain concerned about adding background into algorithms.

Ed Foster & Rebecca Siddle (2019) The effectiveness of learning analytics for identifying at-risk students in higher education, Assessment & Evaluation in Higher Education, DOI: 10.1080/02602938.2019.1682118

(*1) We see the same pattern in most groups with poorer attainment. The one exception is male students who are just less engaged on average than their female peers.