Shundo David Haye – Zen Priest

There was an article about AI recently in the New Yorker which did not make me feel any better about the subject. If I were to sum up my feelings, I would be leaning on the side of the “doomers,” and wondering why humanity – or at least a small, well-funded part of it – is ploughing ahead with this when some heedfulness would not amiss. Perhaps, like climate change, we cannot make ourselves care enough to do things differently. Here were three quotes that give a flavour of the article:

‘Clara Collier, the editor of Asterisk, a handsomely designed print magazine that has become the house journal of the A.I.-safety scene, told me, of the e/accs, “Their main take about us seems to be that we’re pedantic nerds who are making it harder for them to give no fucks and enjoy an uninterrupted path to profit. Which, like, fair, on all counts. But also not necessarily an argument proving us wrong?” Like all online shitposters, the e/accs can be coy about what they actually believe, but they sometimes seem unfazed by the end of humanity as we know it. Verdon recently wrote, “In order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates.” Grace told me, “For a long time, we’ve been saying that we’re worried that A.I. might cause all humans to die. It never occurred to us that we would have to add a coda—‘And, also, we think that’s a bad thing.’ ”’

‘In his talk, Nielsen told a story about a house party where he’d met “a senior person at a well-known A.I. startup” whose p(doom) was fifty per cent. If you truly believe that A.I. has a coin-toss probability of killing you and everyone you love, Nielsen asked, then how can you continue to build it? The person’s response was “In the meantime, I get to have a nice house and car.” Not everyone says this part out loud, but many people—and not only in Silicon Valley—have an inchoate sense that the luxuries they enjoy in the present may come at great cost to future generations. The fact that they make this trade could be a matter of simple greed, or subtle denialism. Or it could be ambition—prudently refraining from building something, after all, is no way to get into the history books.’

‘Grace uses online prediction markets—another rationalist attempt to turn the haphazard details of daily life into a quantitative data set—to place bets on everything from “Will AI be a major topic during the 2024 Presidential debates?” to “Will there be a riot in America in the next month?” to her own dating prospects. “Empirically, I find I’m good at predicting everything but my own behavior,” she told me.’

This post first appeared on my Patreon page.