No, computers haven’t learned how to cheat

Today’s UnPacked features a story that should have a got a lot more attention than it did.

Covered late last year by Devin Coldewey of the TechCrunch website, it concerns a type of AI system called a generative adversarial network or GAN.

A GAN consists of two halves: a ‘discriminator’ and a ‘generator’. By having labelled items of data fed to it, the discriminator learns to identify the patterns that correspond to the labels – so that it can go on to identify similar patterns in unlabelled data. An example would be a system set up to identify particular faces in CCTV footage.

The generator works the other way round – it generates or modifies data to create simulated patterns with the objective of convincing an observer that they correspond to a particular label. A rather sinister example might be the simulation of CCTV footage containing recognisable faces. Feedback from the observer enables the system to learn to produce evermore convincing fakes.

By pitching its generator and discriminator components against one another, a GAN computerises the feedback process, thus greatly speeding up the rate of learning.

The TechCrunch story concerns an experiment by Stanford and Google researchers, in which a GAN was given the task of converting aerial photography into the sort of image you might see on a sat nav app:

“The intention of the researchers was, as you might guess, to accelerate and improve the process of turning satellite imagery into Google’s famously accurate maps. To that end the team was working with what’s called a CycleGAN — a neural network that learns to transform images of type X and Y into one another, as efficiently yet accurately as possible, through a great deal of experimentation.”

The system did a pretty good job – turning the fine detail of the photograph into the simplified and stylised components of a map.

Then the researchers tried the process in reverse – getting the system to convert the map into a convincing simulation of an aerial photograph. That’s when things started getting creepy:

“In some early results, the [AI] was doing well — suspiciously well. What tipped the team off was that, when the agent reconstructed aerial photographs from its street maps, there were lots of details that didn’t seem to be on the latter at all. For instance, skylights on a roof that were eliminated in the process of creating the street map would magically reappear when they asked the agent to do the reverse process…”

To be clear, in undertaking the reverse task the computer only had the map to work from, not the original photograph. So how on Earth did it recreate details that were present in the original photograph, but not in the map that it did have access to?

Upon investigation, the answer was that the details were present in the map, but hidden as subtle image variations that were imperceptible to the human eye. The computer then used that concealed information to recreate the suspiciously accurate version of the original photograph. It was a manoeuvre akin to smuggling a text book into an exam hall – a solution of a kind, but subversive of the purpose of the test.

Is this evidence of artificial creativity? Indeed, of AI cunning and deception?

The headline of the TechCrunch piece is “This clever AI hid data from its creator to cheat at its appointed task”. When the Daily Mail covered the story a few days a latter it went with “The AI that learnt to hide data from its creators to CHEAT at tasks they gave it”.

One can hardly blame the headline writers for seizing upon the juiciest interpretation of a remarkable story, but as Devin Coldewey makes clear, the computer wasn’t cheating:

“One could easily take this as a step in the ‘the machines are getting smarter’ narrative, but the truth is it’s almost the opposite. The machine, not smart enough to do the actual difficult job of converting these sophisticated image types to each other, found a way to cheat that humans are bad at detecting. This could be avoided with more stringent evaluation of the agent’s results, and no doubt the researchers went on to do that.

“As always, computers do exactly what they are asked, so you have to be very specific in what you ask them.”

What really happened was a highly sophisticated, but essentially mindless, process of trial and error. In generating a vast number of potential solutions to the problem it was tasked with – and discarding those that didn’t work – the system just happened upon a work-around. There was no sneakiness involved, no conscious calculation of any kind – just a brute process of ‘artificial natural selection’, greatly accelerated by computer.

The real danger posed by artificial intelligence isn’t one of rebellion. Far from breaking the rules, our machines will follow them mechanically – finding solutions that may not be envisaged by the rule makers (i.e. us), but which are fully in keeping with, and implied by, our instructions.

If the end result turns out to be a shortcut to nowhere, the exploitation of a loophole or perhaps a disastrous unintended consequence, then the fault will be entirely ours.

---

Peter Franklin is Associate Editor of UnHerd. He was previously a policy advisor and speechwriter on environmental and social issues.

peterfranklin_