Measuring decision making

Almost a century ago, early marketing theorists began to design surveys to try to understand what people wanted before selling it to them. On realising there were inconsistencies between what people said they wanted and how they actually behaved, psychologists and market researchers developed more advanced techniques to unpick motivations.

But even in-depth discussions are limited in providing accurate insights into decision making and behaviour. One of my favourite examples is a qualitative study into individual tax debtors, to understand why they failed to pay self-assessed tax. The focus group based study confidently suggested that a sample of the general population would be more likely to pay when presented with information about how the taxes would be spent. However, the Behavioural Insights Team (BIT) then tested this hypothesis in the field using a randomised controlled trial (RCT) and found them no more likely to. The group who had expenditure explained to them did not show more willingness to pay than the group who did not get the explanation.

The group who had expenditure explained to them, did not show more willingness to pay than the group who did not get the explanation.

For the last few years, BIT have been encouraging governments, police forces and other agencies to routinely experiment as a means of measuring and improving the effectiveness of their operations. For this approach to be most efficient, it requires three principal elements. First, we should measure impact by observing actual behaviour, as opposed to self-reported behaviour or attitudinal measures. Where possible, routinely collected data (e.g. software updates) should be used as a measure of actual behaviour. Second, evaluations should create a counterfactual or ‘business as usual’ group – the experimentation element – against which a comparison can be made to determine impact. Third, there should be a willingness to add small adaptations or ‘nudges’ to key touchpoints, from letters to text messages to websites to face-to-face communication, with the aim of incrementally encouraging the desired decisions.

There are important limitations to this approach, especially relating to crime and security. RCTs should be paired with other evaluation methods to ensure some outcomes are not missed. But in many other policy domains, such use of rapid, low cost, empirical field trials has significantly advanced our understanding of decision making, allowing us to develop more efficient services and more effective policies.

We’ve seen this approach lead to hundreds of millions of pounds of tax debt advanced to the Treasury; a reduction in the over-prescription of antibiotics; increases in education attainment; reductions in racial disparities in Police recruitment, and many more, all at practically zero marginal cost.

Think small

The most important recent finding from this kind of public policy research has been the confirmation of Daniel Kahneman’s hypothesis, that ‘the environmental effects on behaviour are a lot stronger than most people expect’. That means that the small adaptations referred to above (if chosen correctly) can have disproportionately large impacts on human behaviour.

Since its inception, BIT has conducted in excess of 500 large-scale field trials that demonstrate this. But one of the most powerful examples relating to security comes from Carnegie Mellon. Experimenters found that participants in a study were more likely to divulge sensitive, personal information via an online survey when questions were phrased indirectly rather than directly. In a separate experiment, participants were more likely to divulge sensitive, personal information on a website that looked unprofessional, which the same participants rated as significantly less secure than a professional-looking website, which was rated as seeming more secure.

Environmental adaptations can have large impacts, but also unexpected ones. BIT recently partnered with a medium-sized police force to measure the impact, using an RCT, of body-worn video cameras. In addition to positive justice-related outcomes, we observed a reduction in the number of sick days taken by officers, yet no reduction in spells of sickness, suggesting the cameras had some effect on speeding up recovery times.

Humanising law enforcement and security regimes

Understanding how these environmental details affect decision making can (if harnessed effectively) be a powerful tool for policymakers and service deliverers to improve security outcomes. But we should primarily see built-in nudges as a way to optimise the systems and security regimes that serve our users and the population, and collect data that demonstrate how behaviour is being guided.

This approach to security is in its infancy. But one early observation is that security professionals tend to adopt an impersonal tone when dealing with the public, whether automated or face-to-face. To encourage decision making that is likely to support security concerns, we might consider a more human manner.

We recently partnered with West Midlands Police to increase compliance in one of the highest causes of harm in the UK: dangerous driving. We adapted the ‘Notice of Intended Prosecution’ – a letter sent to drivers caught speeding – to make it a bit more…well… ‘human’. Rather than using legalistic language that only talked of sanctions, we explained how speeding limits were set and why drivers should comply with them.

We tested the impact of the revised sanction with an RCT, over a period of 6 months observing more than 15,000 drivers. The results were startling: a 13.7% increase in payment of the fine; a 41.3% reduction in those eligible for prosecution; and, most impressive of all, after another six months we observed a 21% reduction in future speeding offences in the West Midlands alone.

Further evidence to support the humanising of security-relevant, compliance-based interactions comes from procedural justice, a concept that promotes openness and fairness in processes over which users or the public have little agency. In Queensland, Australia, a procedural justice prompt for police officers conducting random breath tests increased compliance with police directives, improved levels of satisfaction with the police, perceptions of police fairness and confidence in the police.

In many aspects of security, multiple touchpoints exist with end-users, the public, offenders, suspects, witnesses, victims or indeed actors within the system. Many of these touchpoints provide opportunities to test how small adaptations to business as usual can affect human responses, which cumulatively could have a significant effect on security. It’s time we exploited them.