Superintelligence as a threat to human existence

A playfully-taken position for the Great A.I. debate, part of the Chasing Consciousness Series of YHouse, presented by Caveat in New York City, Wednesday, 20 December 2017 at 5:30 pm

Good evening. We’re all doomed.

Erik and I are here to talk about artificial intelligence. I’m sure we’ll talk about some terrifically erudite things: deep learning, the no free lunch theorem, the frame problem.

But I’m going to start with a story (and I’ll end with one).

When I was in high school, our philosophy teacher introduced us to Plato’s theory of the forms: very roughly speaking, the idea that (for example) the tables that carpenters make in the world are somehow imperfect shadows of an Ideal table. Our teacher didn’t talk about tables, though; he talked about hamburgers, and asked us to think about the Ideal hamburger. My friend, Morgan Schick, replied that the only thing he could imagine was a really big hamburger.

When people think about superintelligence, and the threat it might pose to human civilization, they tend to make the same error. Perhaps we think about Einstein, and then a Super Einstein, a thousand times smarter than the Einstein we know. How bad could that be? He would invent General Relativity in an evening; perhaps by the end of the week discover a unified field theory. But we already have enough atom bombs to destroy the world twenty times over. The cruelty of man has already exceeded the resilience of our species.

Or perhaps, in a darker frame of mind, we imagine a Super Hitler, a thousand times more crafty. But Hitler was not particularly intellectually gifted to begin with, and he was able to lead a continent in the destruction, almost, of an entire people. His limited intelligence was little hinderance, and without the kind of counterfactual thinking that historians rightly dismiss, it’s hard to see how augmenting it could have made things much worse.

I argue that these accounts miss something fundamental—that they are limited essentially by our failure to understand the creative power of evolution. That deceptively simple process of selection and variation has produced, among other things, the great majesty of the human form. The eye alone, in its flexibility, intelligence, and dynamic range is a device our technology still strives to replicate.

But evolution is slow. So slow that we struggle to comprehend the lengths of time involved. To write The Origin of Species, Charles Darwin had to study not biology so much as geology: the vast timescales that it takes to create the Himalayas or the Rocky Mountains are the only ones that can compare. It took three billion years to make a sponge. And our ancestors lived lives almost identical to the ones that their ancestors did, for hundreds of thousands of years.

But then, somehow—we know not how—we developed culture. And the transition was dramatic: our species started to change not on the hundred-thousand year timescales of its ancestors, but century by century. A few thousand years ago, during the transition to agriculture, we built our first cities. I have placed my finger in the clay rut made by another man’s finger—five thousand years ago, in Mesopotamia. That man was essentially genetically indistinguishable from me or you but now, five thousand years in the past is a great deal of time indeed. From our point of view, that poor man was trapped in a distant hell, struggling to survive and prey to the injustices of both nature and other men in ways we cannot imagine.

With culture, the ability to adapt and extend life was now increasingly governed by our brains and social lives. As we passed down traditions to our children, they altered them. Better techniques were replicated, failed ones adapted or lost. Crop rotation. Counting. Written language. Geometry. Philosophy, dialectical discussion: the precursors to what we are doing here tonight.

Our greatest powers were unleashed in the modern era. First, as far as we can tell, in Britain, around 1810. Our species had already broken the Malthusian trap that limited our growth to local resources. Within a hundred years, the average manual laborer could command the material wealth that had previously been enough for his entire village.

This morning I flew from that second cradle, London, to New York, in seven hours. How many 18th Century villages would that take? After 1980 or so, in the developed world, the evolution of technology had made the tracking of inflation practically meaningless in material domains. How much is the cost of phone service rising? It makes no sense to compare a landline to a modern smartphone.

When we networked our machines, the pace of culture began to exceed our grasp. We no longer have decades: we have months. Memes propagate faster and faster. Wayne’s World quips lasted years—not. But who remembers grumpy cat or the inarticulate doge? Each year I have to remake my slides for students because the memes are out of date. The Millennials may be the last generation to have a real name.

The kind of evolution that networked machines make possible was almost completely unforeseen in 1995 when the National Science Foundation opened the internet to commercial use. We have now elected a president who says nothing, believes nothing, thinks nothing. His rise was enabled almost entirely by the harnassing of simple evolutionary tools—A/B testing, for example—to spread the most compelling cultural messages, no matter how incoherent. Indeed, so incoherent that no Mad Men advertising agency could have even conceived them.

I ask you, then—what happens when these machines speak not just to us, but also to each other?

I hope I’ve given you enough to think that what will emerge will be something literally unimaginable. As unimaginable as a jumbo jet would have been to my ancient potter. The one thing we can expect is that the pace, now electrically-enabled, will accelerate again.

To give our artificial machines the capacity to interact places them at the cusp of a new civilization. Given the ability to share and modify, to evolve their minds, they will find themselves on the equivalent of the flood plains of Mesopotamia.

If it gets bad, if what emerges threatens our culture, our values, the basic structure of human experience, well, you might say: we can shut it down, turn it off. But the nearly-universal collective will of Silicon Valley could not turn off Trump.

The danger we face is born from our lack of imagination. We act as if cultural evolution would have just produced hunter-gatherers with really big spears. What machines will do, the powers they will gain, once they (or we) hit on the necessary pattern for their evolution to decouple from human will will be literally impossible to predict.

I began with a story, and promised to end with one. In 1904, the great British writer Virginia Woolf had a mental breakdown. She later wrote that, walking through London, she had heard the birds speaking Ancient Greek.

Which, however poetic, is necessarily nonsense. Greek is beyond the mental powers of avian life and society. If Woolf had thought she heard two pedestrians speaking Greek, that is one thing: perhaps it was modern Greek. But birds, no, no matter how intelligent the species.

Perhaps one day, a machine will hallucinate that we can understand its culture, its language, as beyond us as Greek is to birds. We might hope that that machine is as sensitive and kind as Virginia Woolf. But even she ate birds for dinner.

SapphoBot, Data Science, Lovers and Beloveds

Thou shalt not sit
With statisticians nor commit
A social science.
W. H. Auden, Under Which Lyre

There is always the lover, and always the beloved. As Michel Foucault suggests, the only remaining question is how to allocate them: who is allowed to sleep with whom, and under what circumstances. Consider the dilemma of the (extremely charming) young Phaedrus, in the dialogue with Socrates that bears his name: what kind of lover should a person, seeking to be loved, take? Socrates’ answer, of course, is that he should cleave to one inspired by a particular kind of divine madness: “the fourth and last kind of madness, which is imputed to him who, when he sees the beauty of earth, is transported with the recollection of the true beauty; he would like to fly away, but he cannot; he is like a bird fluttering and looking upward and careless of the world below; and he is therefore thought to be mad.” One need not be a paid-up subscriber to Dorothy Parker’s cynical view that one of you is lying to think the oppositions of the lover/beloved relationship tells us something true about this madness. If the symmetry be broken spontaneously, of a moment, even rehealed and rebroken, it is still for a time, a broken symmetry, maddening to those under its spell.

Despite the great inconveniences it can pose to well-ordered state, this madness is recorded down to our own day. Today, indeed, we blow this process up onto the largest possible scales: as bots retweet Russian propaganda and mad leaders, we task them also with reminding us of the torments of the visions granted by love, and soothing us, perhaps, as we undergo them. That is thanks to SapphoBot, a little program that shares the works of the great Lesbian poet, who did for love what Aeschylus did for tragedy, and Socrates for philosophical dialogue.

Who reads poetry? We do, now, at the rate of one fragment every two hours. Drawing randomly, SapphoBot breaks off a Sapphic text from the classicist/poet Anne Carson’s translations in If Not, Winter—what little we have, torn off in its turn from an Egyptian mummy’s wrappings or an exemplar sentence in a textbook grammar, and shares it instantly to the 17,000 (or so) of her followers spread around the world.

Let us (as the social scientists say) operationalize that crucial dyadic granted to us from the Greek estate. Those subscribing to SapphoBot’s feed have a choice: to touch the heart, indicating a personal response, or to re-tweet, sharing her work under their own name, adjacent to, and interspersed with, the things they write themselves. When a subscriber re-tweets, she speaks in the voice of the lover; when she touches her heart, she plays the beloved. We place ourselves along each axis, sometimes the lover, sometimes the beloved, and signal accordingly; each fragment, now, records both the number of speakers, and the number of (responding) beloveds.

One way to view this strange and automated window on an infinitely distant, infinitely close, past is at the top of this article: a simple scatterplot. Each point on this figure corresponds to a Sappho fragment; the horizontal location of the point shows the lover’s retweets, while its vertical position shows the beloved’s heart-like responses.

Some simple things at first. There are more beloved-responses than there are lover’s declarations. This might have puzzled Phaedrus and Socrates, who would have understood the yielding of the beloved to the lover to be — at least potentially — a shameful matter. But the internet makes the beloved-responses (at least partially) hidden from public view: to <heart> a text is a private matter, while a declaration, conversely, is shared with all the lover’s followers (here, considering twitter, it is hard not to imagine the Greek agora, one where philosophy and love coexist with pride, public shaming, and hidden vice…)

While the beloved responses outnumber the lover’s declarations, it is also the case that the response is sub-linear. In practical terms, what this means is that the declarations that are most common are less popular with the beloveds than you might expect. If you double the popularity of a declaration among the lovers, you only increase the responses of the beloveds by about 68%, a relationship mathematically expressed by saying that the beloveds scale as the three-quarters power of the lovers. (I had hoped to find a three-halves scaling, which would allow for an analogy between lovers and beloveds, on the one hand, and Kepler’s third law of planetary motion, relating the axis and period of an orbit; regardless, this empirical law now demands an equivalent Newton of the heart, to explain its emergence from first principles.)

For those quibbling scientists, it’s worth noting that the three-quarters power-law of lovers and beloveds contrasts with the behavior on the Finnegans Wake Bot, a similar concept. In this case, I’ve described retweets as “writer”, and likes as “reader” responses. In contrast to the differences we find between lovers and beloveds in Sappho, readers and writers, in Finnegans Wake, are essentially equivalent roles: any passage will have, on average, a similar number of readers and writers. And as a passage becomes more and more popular with writers, it becomes similarly popular with readers.

There are many lessons hidden here for the lover seeking her beloved. Implicit in the sub-linear scaling—that pseudo-Keplerian three-quarters law—is that beloveds have a wider range of tastes than lovers. The speeches of lovers are more unequal than the more pluralistic desires their beloveds demand. The songbirds sing in a restricted range; the beloveds, by contrast, are more likely to respond to the unexpected than one might expect.

Lovers, in their madness, misjudge in other ways as well: they fail to realize that what it pleases them to say may not please their beloveds equally well. Consider the red band, which highlights a population of passages that lovers, at least, seem to treat equally. The scatter up and down that red band, however, shows how beloveds are a different matter. Among these passages that their lovers treat equally, they prefer some much more than others. At the two extremes within that red band, we find these two (where the “]” in the beloved-scorned text indicates a fragmentary feature)—

where are you gone leaving me behind?
no longer will I come to you
no longer will I come
(~18 retweets; ~88 likes)

]no pain
(~19 retweets; ~40 likes)

The message is simple. Lovers: declare not your pain, tempting though it is! Your beloveds really mourn what you have done to them, and have little pity for the pains you receive in return.