I’m most of the way through Nate Silver’s “The Signal and the Noise: the art and science of prediction” I’m really struck by the fact that as I’m reading, I’m learning new tools and reference points to critique our learning analytics work. It’s quite possible that if I’d a background in social sciences, mathematics or computer science, this would be second nature to me. But it’s not.

Should I be worried?

I’ll be frank, I’m still not 100% certain that I understand what Bayesian reasoning is and I’m often looking for best fit patterns, where a bit more patience is probably more appropriate.

I’m not going to over-worry this. I’m not employed as a data scientist (and work with a really good one). Our Dashboard is there to help us support student success and after many years working in retention, study skills and transition into HE, I know how the Dashboard needs to work. I know that I’m in role to represent the needs of the end users: students and staff at the University.

CP Snow’s “The Two Cultures and the Scientific Revolution” (1959), critiques the UK education system for being too dominated by the humanities “the majority of the cleverest people in the western world have about as much insight into [physics] as their neolithic ancestors would have had.” Sixty years on, I find it interesting that with the (extremely unfortunate) exception of the British ruling class, the very reverse seems true. We appear to be living in an age where the new gods are the scientists making the future: smart phones, electric cars, large hadron colliders, space rockets etc. If anything, the humanities are viewed as this quaint thing that we used to do, but it doesn’t really contribute to the economy or society at large. Like the social sciences, humanities bring meaning, ethics, humanity even, to our world. In a field as important as data, we need alternate voices and views.

Does being a humanities graduate bring any unique benefits to working in big data?

My undergraduate degree was a joint honours in History & English. I scraped a 2:1. Ironically all my best grades were in English, yet I always thought of myself as more of a historian.

How has studying English helped my work?

I hope I write reasonably well (please feel free to disagree). Moreover, I spend my life looking for metaphors and stories to explain and illuminate complex ideas. As an undergraduate, I loved William Blake and the Romantic poets; I want to make a clever point about how as s student of literature, my vision is somehow ‘truer’ than that of a scientist, but I don’t quite buy Blake’s lines:

“Now I a fourfold vision see
And a fourfold vision is given to me
‘Tis fourfold in my supreme delight
And threefold in soft Beulah’s night
And twofold Always. May God us keep
From Single vision & Newton’s sleep”

William Blake, Letter to Thomas Butt, 22 November 1802

Blake is referring to the power of imagination (in his view divinely-given) being inherently better than Newton’s observations and measurements. I don’t agree that one is inherently better than the other, we need both. We need creative thinkers working in big data alongside detail-oriented individuals, if reading Blake helps, that’s not a bad thing.

How has studying History helped my work?

I think there’s probably a much stronger case here that studying history still benefits my work. I don’t think I saw it at the time, but history is all about evaluating data. My undergraduate degree was a lovely eclectic mix of the history of Japan, the USA, the decline of rural England and other topics. My dissertation was a study of the attitude of the 17th Century English attitude towards the Irish and Native Americans (surprisingly colourblind it turns out, but on the whole rather unpleasant to both groups).

It wasn’t until I did some undergraduate teaching years later that I actually discovered the word ‘historiography’(1). I had the chance to teach on a first-year module that focussed on evidence by studying hoaxes and conspiracy theories. My lecture was on Holocaust Denial, particularly David Irving versus Penguin books & Deborah Lipstadt. The point of the lecture was that whilst Irving used historical techniques, he shouldn’t be considered a historian due to the highly partisan way that he selected his sources. This matters a lot, it’s conceptually the same as medical researchers faking evidence. Whilst it might not be the same as releasing an unsafe medicine, deliberately-faked history is just as a dangerous as deliberately-faked news.

Overall, I think that being a humanities graduate taught me about complexity, nuance, and the difficulty of defining truth.

One of the interesting aspects of the Signal and the Noise is that each chapter explores progressively more-complex systems. Early chapters are about weather forecasting, baseball analysis and earthquake forecasting. Later chapters investigate the stock market, climate change denial and terrorism. What makes the later chapters more complex is people. People don’t have perfect understanding of any given situation, they act on a range of factors, some rational, some not. Putting people into models is massively more complex than natural phenomenon such as the weather. Moreover, people are often flawed in the way that they interpret data, particularly messy, noisy data. In these situations, we should be very cautious about the search for simple answers, particularly in contested areas. It’s far too easy to cope with complexity by accepting conspiracy theories, which Silver argues “… might be thought of as the laziest form of signal analysis. As the Harvard professor HL Gates says “Conspiracy theories are an irresistible labor-saving device in the face of complexity”” (Silver, 2012, pg. 417)

Studying the humanities prepared me to work in big data.

Bitchy (and probably unfair) addendum

Oh. And then there’s copy editing. What century is the 21th?

(1) Essentially the study of how we do history, conceptually a bit like stats 101 for historians, and yes it should definitely have been part of my degree.