In the UK, 2018 might come to be seen as an important year in our appreciation of just how significantly data is playing a role in our shaping our lives in ways that are surprising, horrifying and certainly without meaningful consent. We can be grateful of the actions of journalists in the Guardian and Channel Four news in breaking the Cambridge Analytica scandal that uncovered just how cynically voters were manipulated in the run up to the UK Brexit referendum. Given that the raison d’être of the company was to weaponise social media for those political parties with the deepest pockets, this should be no surprise. What was truly unsettling was how effectively it worked in the UK, a country with a supposedly-sophisticated electorate with 350 years of some form of democracy at work. The complete inability of regulators, civic society, politicians and journalists to expose, challenge and ultimately prevent such manipulation is both depressing and terrifying.

Surely, somebody ought to have been able to prevent this. Surely?

“Everybody Lies” by Seth Stephens-Davidowitz is similarly unsettling.

Stephens-Davidowitz starts with the other electoral shockwave of 2016: Donald Trump’s victory over Hillary Clinton to become the 45th President of the USA. And it’s not pleasant.

Stephens-Davidowitz’s central thesis is that people are more honest in their relationship with Google than they are in activities such as academic surveys. In effect, if you want to know the truth of someone, don’t ask them; check their Google history. He starts with an amusing diversion about the problems with relying on people to tell the truth about their sex lives, but quickly turns to something really dark. In 2016, the pollster Nate Silver searched for factors that correlated with a tendency for an area to vote Trump. The strongest correlation turned out to be with a map that Stephens-Davidowitz built in 2012.  Areas that supported Trump most strongly in 2016 were those that had made the most searches in Google for the term “n*****” in 2012.

This is an enlightening (and at times unsettling) read. Perhaps what’s most appealing about the book is that it’s a very accessible introduction to data science and the way that, without us noticing, it is changing our world. Stephens-Davidowitz is at times like a kid in the proverbial candy store. He is in the first generation of data scientists with access to an almost infinite quantity of rich data revealed through our digital footprints. He excitedly points out all manner of lessons and challenges and it’s never dull. The book’s three sections: ‘Data, Big & Small’; ‘The Powers of Big Data’; and ‘Big Data, Handle with Care’ offer a solid introduction to the approaches used, potential and risks of big data.

If you work in data, or are a citizen, this book is important. We have a long series of battles ahead as we struggle with the tension between free services we receive and the price of giving our data away to the likes of Google or Facebook. In the future, we will look back on this time and be slightly baffled about what we gave away unthinkingly in just the same way as we can’t understand now why children’s toy makers used to put lead in paint. When that change takes place will depend on how many people actually understand how data is being used. Read the book,