The reaction time of a typical human being is apparently 250 miliseconds. In other words, it takes us about quarter of a second to get out of the way of a speeding snowball, or catch the dropped vase.

That stuff’s easy – 1. Danger – 2. React – 3. Try not to look embarrassed when our response was all wrong and we tripped over our shoes, walked into the lamp post etc.

It’s much harder to react to something that hasn’t happened yet.

At the time of writing, England had undergone three lockdowns in response to the coronavirus pandemic. Each of these lockdowns was a difficult decision made by a politician to disrupt the economy and interfere with the family and personal lives of millions of people. Your view on whether each lockdown was too soon or too late is as likely to be shaped by your political views, your comfort with risk and your personal circumstances as much as it is by your statistical acumen. At each decision point the Conservative Government stated that it was following the scientific evidence, but at the same time appeared to be trying to balance the economic impact of lockdowns and, it appeared, the views of backbench fringe lockdown deniers/ libertarian hardliners.

These were no doubt difficult decisions, seemingly made more difficult by the political ideologies of the ministers involved: this must have felt a lot like a right wing government implementing the nanny state. I’ve previously written about the role heuristics play in decision-making. One of the most important is confirmation bias. We essentially use data to confirm our decisions rather than approach a situation objectively or use the data neutrally. Deciding to lockdown must have flown in the face of the Government’s instincts and perceptions of what had worked previously and the role of the individual versus the state.

Consequently, one criticisim is that the Government has followed scientific advice, just a fortnight too late.

When to lock down

But the decision is made more complicated as it’s reliant on interpreting data about something that hasn’t happened yet. I suspect that humans are particularly bad at making predictions and acting on them (as a species we still bet on horses, the stock market etc.). What makes this set of decisions particularly difficult though is the death toll of delaying. The following graph uses data from the UK government’s official statistics. I’ve mapped the dates that the three lockdowns started (green line) and the peak fatalities (red line) following each lockdown (effectively the time lag between the intervention and any positive impact). I’ve used deaths as the measure as I think comparing incidences is more complicated as testing has got better as time progressed. I’m aware that there are conspiracy theories about COVID including, depressingly, about deaths, but this felt a more reliable measure (if significantly sadder).

Daily deaths from Covid-19 showing the dates of lockdown (green line) and peak deaths (red line) before impact of lockdown had time to work

First lockdown, 23rd March 2020

On 23rd March, 186 people died within 28 days of a positive COVID 19 test. The first peak of COVID-related deaths came 16 days later when 1,073 people died (477% increase). As I recall there were at least several weeks where the Government appeared be reluctant to initiate a lockdown. To some extent why should they? The 1918 flu pandemic was effectively out of living memory and the SARS and MERS outbreaks scarcely reached the UK. Outside a few Hollywood movies, a pandemic was practically unimaginable. The scientists and politicians appeared stuck in a mindset that COVID had a flu-like severity, not something 10 times more lethal.

Second lockdown, 5th November 2020

On 5th November, 299 people died within 28 days of a positive COVID 19 test. When that wave peaked 13 days later 396 people died, a relatively small rise of 32%.

Third lockdown, 5th January 2021

Finally, the third lockdown started on 5th January 2021, on that day 818 people died of COVID-19, 14 days later the third wave peaked with 1,241 deaths (52% increase).

The decisions around the second and third lockdowns were profoundly shaped by the Government not appearing to want to be described at the Government that cancelled Christmas.

Surprising headline – lockdowns really work

I’ve not really looked at the correlation between the start of lockdown and the peaks of fatalities. In all three cases, that peak is about a fortnight after lockdown (16 days, 13 days, 14 days). Of course the country can’t lock down forever, but you’d have to be particularly obtuse to argue that lockdowns don’t work in reducing deaths from Covid.

How to make predictions seem more tenable to decision-makers

1. Don’t start from here. Decision-makers need to see evidence of previous predictions. Our Dashboard is good at spotting students at risk. It’s never perfect though, and probably more importantly I don’t invest enough time in producing annual reports for decision makers (who knew this post would turn to therapy?). Relationships with decision-makers need making and trust needs establishing.
2. Speak the language of your audience. I’m in a conversation with a valued colleague at the moment that involves the use of pseudo R squared. If you are familiar with pseudo R squared, the sorting hat just put you into one house, if you’re not, welcome to the other. This is a big deal, people are paralysed by complicated data. At the point that they need to make a decision they’re probably still on the back foot because they don’t fully understand the data, or the method of acquiring, but they REALLY don’t want to look stupid. In these instances they’re unlikely to take difficult decisions.
3. Make your analysis transparent. There is a reason I feel academics like atttendance monitoring when it’s only one possible way to test if students are engaged. It’s really easy to understand what 100% attendance looks like. If your analytics uses a lot of complex maths, you should be explaining it. A lot.
4. Make decision making easy. Probably the main reason that there’s so much interest in nudge theory is that most of it is easy to do. If you’re already sending a million electronic messages, colouring half green and half pink is easy. Even if it’s a difficult decision, remember to spell out what the decision is rather than just standing back in front of your most marvellous chart and expect them to work it out for themselves.
5. Speak the language of your audience (part 2). I’ve listened to lots of representatives from the Government’s SAGE committee interviewed by the media about lockdown. For the past few weeks, journalists really like the question “When would you lift the lockdown?”. The scientists being interviewed twist themselves in knots to avoid answering. I understand why, it oversteps their role, they don’t want to later be shown to be wrong thereby undermining any advice they do give etc., etc.,  but it also gives the impression that they’re not partners in this most difficult decision-making process. I wonder if the SAGE committee use that approach in meetings or more forcefully state a position. I started this post being critical of politicians, but I wonder if there’s more that could be done by scientists to (transparently) push a particular approach rather than leaving politicians to work it out for themselves? (*)

(*) in the unlikely event that you’re actually a SAGE scientist, I’d love to know how you strike that balance.