Superintelligence as a threat to human existence

A playfully-taken position for the Great A.I. debate, part of the Chasing Consciousness Series of YHouse, presented by Caveat in New York City, Wednesday, 20 December 2017 at 5:30 pm

Good evening. We’re all doomed.

Erik and I are here to talk about artificial intelligence. I’m sure we’ll talk about some terrifically erudite things: deep learning, the no free lunch theorem, the frame problem.

But I’m going to start with a story (and I’ll end with one).

When I was in high school, our philosophy teacher introduced us to Plato’s theory of the forms: very roughly speaking, the idea that (for example) the tables that carpenters make in the world are somehow imperfect shadows of an Ideal table. Our teacher didn’t talk about tables, though; he talked about hamburgers, and asked us to think about the Ideal hamburger. My friend, Morgan Schick, replied that the only thing he could imagine was a really big hamburger.

When people think about superintelligence, and the threat it might pose to human civilization, they tend to make the same error. Perhaps we think about Einstein, and then a Super Einstein, a thousand times smarter than the Einstein we know. How bad could that be? He would invent General Relativity in an evening; perhaps by the end of the week discover a unified field theory. But we already have enough atom bombs to destroy the world twenty times over. The cruelty of man has already exceeded the resilience of our species.

Or perhaps, in a darker frame of mind, we imagine a Super Hitler, a thousand times more crafty. But Hitler was not particularly intellectually gifted to begin with, and he was able to lead a continent in the destruction, almost, of an entire people. His limited intelligence was little hinderance, and without the kind of counterfactual thinking that historians rightly dismiss, it’s hard to see how augmenting it could have made things much worse.

I argue that these accounts miss something fundamental—that they are limited essentially by our failure to understand the creative power of evolution. That deceptively simple process of selection and variation has produced, among other things, the great majesty of the human form. The eye alone, in its flexibility, intelligence, and dynamic range is a device our technology still strives to replicate.

But evolution is slow. So slow that we struggle to comprehend the lengths of time involved. To write The Origin of Species, Charles Darwin had to study not biology so much as geology: the vast timescales that it takes to create the Himalayas or the Rocky Mountains are the only ones that can compare. It took three billion years to make a sponge. And our ancestors lived lives almost identical to the ones that their ancestors did, for hundreds of thousands of years.

But then, somehow—we know not how—we developed culture. And the transition was dramatic: our species started to change not on the hundred-thousand year timescales of its ancestors, but century by century. A few thousand years ago, during the transition to agriculture, we built our first cities. I have placed my finger in the clay rut made by another man’s finger—five thousand years ago, in Mesopotamia. That man was essentially genetically indistinguishable from me or you but now, five thousand years in the past is a great deal of time indeed. From our point of view, that poor man was trapped in a distant hell, struggling to survive and prey to the injustices of both nature and other men in ways we cannot imagine.

With culture, the ability to adapt and extend life was now increasingly governed by our brains and social lives. As we passed down traditions to our children, they altered them. Better techniques were replicated, failed ones adapted or lost. Crop rotation. Counting. Written language. Geometry. Philosophy, dialectical discussion: the precursors to what we are doing here tonight.

Our greatest powers were unleashed in the modern era. First, as far as we can tell, in Britain, around 1810. Our species had already broken the Malthusian trap that limited our growth to local resources. Within a hundred years, the average manual laborer could command the material wealth that had previously been enough for his entire village.

This morning I flew from that second cradle, London, to New York, in seven hours. How many 18th Century villages would that take? After 1980 or so, in the developed world, the evolution of technology had made the tracking of inflation practically meaningless in material domains. How much is the cost of phone service rising? It makes no sense to compare a landline to a modern smartphone.

When we networked our machines, the pace of culture began to exceed our grasp. We no longer have decades: we have months. Memes propagate faster and faster. Wayne’s World quips lasted years—not. But who remembers grumpy cat or the inarticulate doge? Each year I have to remake my slides for students because the memes are out of date. The Millennials may be the last generation to have a real name.

The kind of evolution that networked machines make possible was almost completely unforeseen in 1995 when the National Science Foundation opened the internet to commercial use. We have now elected a president who says nothing, believes nothing, thinks nothing. His rise was enabled almost entirely by the harnassing of simple evolutionary tools—A/B testing, for example—to spread the most compelling cultural messages, no matter how incoherent. Indeed, so incoherent that no Mad Men advertising agency could have even conceived them.

I ask you, then—what happens when these machines speak not just to us, but also to each other?

I hope I’ve given you enough to think that what will emerge will be something literally unimaginable. As unimaginable as a jumbo jet would have been to my ancient potter. The one thing we can expect is that the pace, now electrically-enabled, will accelerate again.

To give our artificial machines the capacity to interact places them at the cusp of a new civilization. Given the ability to share and modify, to evolve their minds, they will find themselves on the equivalent of the flood plains of Mesopotamia.

If it gets bad, if what emerges threatens our culture, our values, the basic structure of human experience, well, you might say: we can shut it down, turn it off. But the nearly-universal collective will of Silicon Valley could not turn off Trump.

The danger we face is born from our lack of imagination. We act as if cultural evolution would have just produced hunter-gatherers with really big spears. What machines will do, the powers they will gain, once they (or we) hit on the necessary pattern for their evolution to decouple from human will will be literally impossible to predict.

I began with a story, and promised to end with one. In 1904, the great British writer Virginia Woolf had a mental breakdown. She later wrote that, walking through London, she had heard the birds speaking Ancient Greek.

Which, however poetic, is necessarily nonsense. Greek is beyond the mental powers of avian life and society. If Woolf had thought she heard two pedestrians speaking Greek, that is one thing: perhaps it was modern Greek. But birds, no, no matter how intelligent the species.

Perhaps one day, a machine will hallucinate that we can understand its culture, its language, as beyond us as Greek is to birds. We might hope that that machine is as sensitive and kind as Virginia Woolf. But even she ate birds for dinner.