Creeping Towards Mordor by Daniel Witt

New Developments in Mind-Reading Technology and the Problem of Power

If you were worried about some benevolent world government lacking full access to our inmost thoughts, I have good news! In a new study, experimenters claim to have made a leap in mind-reading technology.

The study “Semantic reconstruction of continuous language from non-invasive brain recordings” demonstrates some ability to translate a human subject’s thoughts into words and sentences using fMRI (functional MRI) scans decoded by the GPT-1 language model and other software.

The technique was clever. Experiment subjects listened to hours of podcasts while undergoing brain scans. The AI model had access to both the brain scan data and the podcast data, and it was directed to find the correlations between the words said in the podcast and the activities of the recipient brain.

After hours of training, the AI model was then cut off from the podcast data and forced to infer sentences based on the brain data alone while the subjects listened to new podcast stories.

It worked. The AI was able to construct fairly accurate sentences based on the brain scans. For example, the podcast input:

i didn’t know whether to scream cry or run away instead i said leave me alone i don’t need your help

resulted in brain activity that was decoded as:

started to scream and cry and then she just said i told you to leave me alone you can’t hurt me anymore i’m sorry.

Next, they tested the ability of the AI to interpret imagined speech, absent any external stimulus. Again, it was successful, albeit with less accuracy than when it was analyzing perceived speech from podcasts.

Finally, they tested the ability of the AI to decode non-language-based thoughts by having the subject watch silent videos. Sure enough, the program was able to describe some of the events of the videos based only on the brain activity of the viewer.

Question: Is This Mind Reading?

Answer: Sort of. MIT Technology Review quotes Romain Brette, a theoretical neuroscientist:

The way the algorithm works is basically that an AI model makes up sentences from vague information about the semantic field of the sentences inferred from the brain scan. There might be some interesting use cases, like inferring what you have dreamed about, on a general level. But I’m a bit skeptical that we’re really approaching thought-reading level.

What that boils down to seems to be that the machine can be used to read thoughts, but only at the vaguest level. The sentences constructed by the language model are wild extrapolations, more indicative of the generative and predictive capacity of the AI than of its depth or accuracy at interpreting brain scans. Think of autofill, but with brain activity as the only prompt.

Of course, it could go farther in the future. One might be tempted to say that it never will, because the mind cannot be reduced to the activity of the brain. True – but it doesn’t need to be. All that is needed is for thoughts to have consistent corollaries in brain activity at the word and sentence level. The experiment indicates that they sometimes do. If it works in principle, there’s no telling where it could lead. The technology could even now be much farther along than the study indicates – the paper was submitted for publication in 2022 and the experiment used only GPT-1, not the current GPT-4 which has impressed so many people with its ability to interpret language.

Another Piece in the Panopticon

This innovation comes at a time when governments around the world are becoming increasingly focused on mind-control. It has been reported that China is developing weapons that can disrupt brain activity at long range to incapacitate human targets. This is part of a broader arms-race between the U.S. and NATO and China to dominate the field of “cognitive warfare.” That why the U.S. would never in a million years do what Elon Musk and others have suggested and place a 6-month moratorium on AI development; if the U.S. started behaving that way, China or some other country would just shrug and step into its place as the dominant world power.

But while the impetus behind the arms race may be the desire to defeat foreign rivals, no one can doubt that many in the U.S. government would have no scruples about using the technology on their own citizens. (Of course, there is little question about the scruples of the Chinese government.) Ten years after Edward Snowden’s revelations about U.S. government spying, many analysts think that the invasive efforts of U.S. intelligence agencies have gotten worse, not better.

Good News! For Now

You’ve probably already thought of several ways this mind-reading technology could be misused by our rulers in the future. The authors of the study are also concerned, and have addressed some questions surrounding the potential dangers.

First – they say that the technology does not work unless it has been specifically trained on the brain of the target. That means an agency couldn’t train the AI generically and then use it on anyone. That’s good – for now.

Second – the experimenters tested whether the technology could be resisted by intentionally thinking about something unrelated to the content of the podcast. They found that this method of resistance worked. Again, good news – until someone finds a way around it.

“Left naked to the Lidless Eye”

It’s comforting that this technology hasn’t quite reached the level of irresistible mind-reading. But the fact that these questions even need to be asked should cause any reasonable person concern. The authors of the paper concluded that “it is critical to raise awareness of the risks of brain decoding technology and enact policies to protect each person’s mental privacy.” While we can certainly try, it’s difficult to be optimistic. Freedom of speech is out of fashion; it’s hardly unthinkable that future Gen Z lawmakers would be happy to punish thought-crimes, if such a thing became possible. And nobody can doubt that many in the U.S. government, and other world governments, are already perfectly willing to use whatever tools of surveillance they find at their disposal.

It’s not merely that power corrupts. Power attracts the power-hungry. This truth is obvious to the point of being tautological. If a system exists that can grant power, those who desire power will flock to it. Even if a new system of power is used wisely by one generation, the first generation will inevitably age out and be replaced by a group of up-and-comers that is likely more comprised of power-seekers. If the ability to read minds is out there, it will eventually be used by the sort of people who will misuse it. We can try to pass laws to limit the use of technology, but people who hunger for power will always be looking for ways to circumvent those laws – and, in sad fact, many of those same people start their quests for power by choosing to become lawmakers.

The ideal of “progress” we keep hearing about mainly refers to the accumulation of power in the hands of humanity, and the inevitable end of this kind of progress is dystopia. That’s because humanity is fallen and cannot handle power. That’s why Tolkien, in his insightful masterpiece The Lord of the Rings, exposed the noble-sounding idea of using the Ring of Power for good as naïve at best and dangerously egotistical at worse. The only way to preserve the life and dignity of the free peoples of Middle Earth was to destroy the Power forever.

But the sad fact is, in the real world there is no practical way to do this. There is no Mount Doom into which we can simply drop “technological progress.” Frodo’s path is not available to us. The question we need to ask ourselves, then, is simply how we can go on living good and noble lives within a developing dystopia.