Favourite big data proving nothing story #1 (yes we’re doing shaggy dog stories now)

A few years ago a colleague attended an event with a company of data specialists. The company was experimenting with personalised learning for professionals. As I understand it they had lots of interesting ideas about personalising online learning. For example, if you got an answer wrong about marketing, you’d be routed to an easier set of questions. All well and good.

But they company was really buzzing about a side project they’d done. The company had asked a cohort of business executives to record what they had for breakfast before taking tests over a period of some months. By the end of the course they claimed that they were able to predict the whether or not someone was going to pass based on what they ate for breakfast.

I’ve always thought this story was fishy, but had the opportunity to think about it recently. Now it’s possible that they did find the miracle effect of muesli, but they probably didn’t.

  • It’s quite plausible that there’s an effect associated with missing breakfast, but surely this just proves that humans are creatures of habit? If superstar-student Gloria eats the same breakfast every morning and gets the best grades each time then surely they’re just showing that superstar-student Gloria eats the same breakfast each day.
  • The bit missing from the story always felt to be the reveal where they said ‘… and when the business executives switched to eating kippers they all passed’.
  • I’d love to be proven wrong and will happily switch to porridge/tiramisu on toast/whatever, but this feels particularly bad data science.
  • I did start to look up others examples of dubious data, but stopped when I found a graph showing the correlation between people drowning by falling into a pool and films Nicholas Cage appeared in. I’m not going to top that.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.