April 5, 2019   4 mins

Efficiency is good, right? Who doesn’t want to do more with less, reuse and recycle, end food waste and food miles, minimise their impact on the earth and make the most of their time on it?

What helps us do these things is efficiency: the one thing of which we can never have too much. But what if our constant striving to streamline is actually having a negative effect?

That’s the argument advanced by Edward Tenner in his new book, The Efficiency Paradox. Efficiency is “something self-evidently desirable,” he says, “until it isn’t”.

His beef is with big data (in which large collections of information are analysed computationally to reveal patterns, trends and associations), which is great at helping us to do more with less, or even to do less with much less. It’s not so good at helping us do something new and hitherto unimagined –those things that extend our capabilities.

Things like flight. It wasn’t big data that invented the aeroplane, but risk-taking humans who took a chance on heavier-than-air flight being possible, in many cases throwing themselves from high places in pursuit of their dream. Once you have aeroplanes, efficiency – through the use of big data – can do wonderful things, such as minimise unwanted environmental effects, reduce prices to put flight within reach of ordinary people, and let us choose the seat we want online.

This is what Tenner, quoting Clayton Christensen, Derek van Bever and Bryan Mezue, calls “efficiency innovation”, typical of the platform-based disrupters such as Uber and Amazon, that “makes existing goods and services available to more people at lower prices”. But “market-creating innovation” that generates the new products or services, such as photocopiers or aeroplanes, takes longer to bear fruit.

Take the labour market. Optimists about the future of jobs in an increasingly automated age point to previous waves of mechanisation, when new industries arose to employ redundant workers. Cars created new jobs for people who previously worked with horses, for example, and then aviation created an entirely new industry that today employs about 10 million people worldwide, directly and indirectly.

To some extent, I share that optimism; but for that pattern to persist in future, we need market-creating innovation to keep creating new industries. Those bigger gambles that won’t pay off for years, if ever, are less appealing to investors than the quick hits of market-disrupting efficiency innovation.

The most valuable companies in the US in 1917 made their money from steel, oil and the telegraph or telephone business. Today, the top five worldwide by market value are all in data, including Facebook and Google, whose business model rests on delivering advertising with ever-greater efficiency.

You could argue that those platforms are providing radically new products and services, albeit as a side product. Certainly, their existence has transformed the way we communicate with each other and find information. We learn more every week about how much data they collect about us – their users – and yet we continue to use the platforms we know are digitally surveilling and profiling us.

This means that our experience of news and of our friends’ opinions, of videos created by users like us, or of political campaigns, is significantly mediated by algorithms that filter the endless tide of digital output. When those algorithms are gamed to feed us untruths, or horrific live videos of a suicide or murder, we ask why they can’t be made more efficient in preventing that kind of abuse.

The problem is, as Tenner puts it, that these “algorithms are written neither to advance a political outlook nor to establish the truth, but to maximise traffic and increase advertising revenue for the platform company”. The efficiency that the algorithms are designed to optimise is not for the benefit either of society or of the individual service user.

But “efficient at what, and for whose benefit?” is only the first question we should be asking. Tenner’s argument goes much deeper, challenging whether efficiency in itself is achievable or even desirable.

Let’s go back to the “market-creating innovation” that produces new products and services. How can we make that innovation process itself more efficient? “The paradox of efficiency,” says Tenner, “is that progress towards greater efficiency is wasteful.” To make leaps of insight and creative imagination, human beings need to be inefficient. We need boredom, serendipity and “desirable difficulty”. We understand concepts better when we have to work harder to get them.

Experiments with reading on paper versus onscreen, with handwritten versus typed note-taking, and even with more and less legible typefaces, point to this “desirable difficulty” as a help in grasping underlying ideas. If you want to win a pub quiz, browse the internet on your smartphone. But to grasp big ideas, read a paper book in an archaic font, or listen to a lecture while making notes by hand.

Tenner has many other examples of how pursuing efficiency through technology and data can lead to its opposite. Using wearable tech to monitor physical activity and nudge us towards healthier lifestyles turns out, in experiments, to reduce activity, weight loss, and even happiness and well-being. Time and energy saved by home automation through the Internet of Things will probably be expended in the battle to secure networks against hackers.

None of this means we should turn our backs on the algorithms – only that we should not rush to defer to their mechanical judgment. “Balancing algorithm and intuition to achieve inspired inefficiency,” as Tenner puts it, can free us from repetitive and time-consuming tasks to be creative and inventive. Combining contextual knowledge of a subject with the processing power of search engines or data analysis can deliver breakthroughs in discovery or invention.

The danger comes when we outsource our judgment, underestimating our human minds, shaped through the messy process of life in human society, and overestimating what algorithms can do. We may be fallible but, unlike computers, we understand what it means to live in this world, and we can imagine other worlds in which we might live in the future.

If we never go beyond doing the same things with increasing efficiency, we are missing opportunities that are no less important for being hard to quantify. Inefficiency is an unlikely rallying cry for the digital age, but it opens doors we might otherwise not even see.


Timandra Harkness presents the BBC Radio 4 series, FutureProofing and How To Disagree. Her book, Big Data: Does Size Matter? is published by Bloomsbury Sigma.

TimandraHarknes