The last post was about some of the psychological hurdles that we need to overcome in order for students to realise that they need to change and the one before about the importance for tutors of relationship building and setting short term goals. This short piece is about how the use of learning analytics data might break into students’ consciousness and help overcome innate resistance to change. I’m using staff surveys conducted in 2016/17 as the basis of this post written up as part of the ABLE Erasmus+ project.

Strength Vs Credibility: pricking students’ consciousness

Evans (2012) suggests that when looking at evidence, we pay attention to two slightly different factors. Firstly the strength of the evidence, secondly its credibility. In learning analytics:

• Strength of evidence = metrics such as risk of failing, choice of language (“high risk”), how red/ angry/ exclaim-y the various graphics might be.
• Credibility – transparency between the metric and the outcome being warned about, consistency/ repeatability of the evidence and at the point where a staff member speaks to a student their personal credibility in the eyes of a student.

Evidence from NTU Personal Tutors

In June 2017, 130 tutors answered questions about personal tutoring: barriers, challenges and successes. Forty-three tutors (33%) felt that using the Dashboard in tutorials had helped them to change student engagement. it appeared that the most important function was breaking into student’s consciousness. This may be particularly important for helping first year students to understand how expectations were different to their prior education. There were other aspects, for example early warning, but I’m going to concentrate only on those points that relate to how learning analytics data was used by tutors in tutorial conversations.

For example: “I have shown them the evidence of their lack of engagement on Dashboard so that they can no longer argue / believe that “everything is fine” and it has forced them to re-evaluate their actions resulting in improvements in engagement” and “I think that in some cases students who had engaged rather poorly with their course had not fully appreciated how poor their engagement had been in recent months. Showing them the actual data helped them to recognise the extent to which they had disengaged, which in turn created a stimulus for a helpful conversation about how they might address this issue.”

One tutor described how they used the engagement data as part of their mid-term review “… I conducted 121 tutorials with students and showed them evidence of their engagement, and these students could be seen to engage more from this point”.

It is important to note that the engagement data is not presented in isolation, but contextualised against peers on the course. One tutor noted how showing this comparison appeared to make a difference “This is speculative, but I did show students how their performance compared to the rest of the class and I think this, at the very least, encouraged them to think about how they were engaging”.

One problem, particularly for students on large courses, is that they can potentially fade anonymously into the background. Given that one of the most recent blog posts was about the importance of relationships as a tool for motivating students, this feels a real problem. One tutor noted that the Dashboard was useful to make it clear to the student that a staff member was aware and interested in them “I agree, it is a good tool so benchmark students against others. It also shocked some of the students to know that I was able to see that information.”

I feel that there’s a real tension in these quotes. Staff are undeniably describing using data to ‘shock’ students out of an unhelpful pattern of engagement. There is of course a libertarian argument that it’s students right as individuals and fee-paying customers to be left alone. I understand that argument, but we have a duty and a responsibility to make students aware that their behaviour may lead to undesirable outcomes. It remains their choice to act on that behaviour, but I do sincerely believe that we ought to be making students aware of the potential risks of their learning strategies.

References

Evans, D., (2012), Risk Intelligence: How to live with uncertainty, Croydon, Atlantic Books