AI and existential risk: Live podcast recording with Nir Eisikovits and James Hughes

Eisikovits and Hughes opened by reversing their usual roles. On their podcast, Prosthetic Gods, Nir Eisikovits plays the self-described Luddite and James "Jay" Hughes the techno-humanist optimist — but for this live evening it was Hughes who built the case that artificial intelligence might end the human race, and Eisikovits who set out to take it apart. Both direct the Applied Ethics Center at UMass Boston, and both warned the audience early that the question everyone wanted to discuss — extinction-level "existential risk" — might be the least interesting one on the table.

Hughes broke "artificial general intelligence" into its parts: software already capable of replacing human labor, systems that have begun to recursively improve themselves, and the still-speculative moment of "waking up" into self-awareness and wants of its own. As a sociologist, he argued that AIs are best understood as sociotechnical systems — reflections of the corporations, users, and groups whose interests they encode — and that whether or not a model ever wakes up, a tool with the power of a nuclear weapon is dangerous in whatever hands hold it.

Eisikovits countered that technical genius is no warrant for prophecy, reaching back to Socrates on the craftsman who, having mastered his trade, presumes to speak on everything else: that Geoffrey Hinton won a Nobel for neural networks tells us nothing about his ability to predict what the machines will do. Doom, he suggested, is partly self-aggrandizement and partly good marketing, and it rests on two oversights — an inflated faith in raw intelligence as the engine of worldly success, and a forgetting that catastrophe still needs "boots on the ground," with many fragile steps between a capable model and a ruined world. He would spend ninety-nine of a hundred worried minutes on nuclear weapons, he said, and the last one eating peanuts.

Where the two converged was on existential risk with a small "e" — not the end of the species but the slow erosion of how we live. They traded worries about AI companions and therapists, a generation of students for whom writing is quietly disappearing, and the human tendency to anthropomorphize anything that talks back. Eisikovits reframed 2001: A Space Odyssey to make the point: the danger is not HAL refusing to open the pod bay doors, but Dave preferring HAL's company — and HAL's tailored movies and inexhaustible patience — to that of his fellow astronauts. As Eliot put it, the world ends not with a bang but a whimper.

From there the conversation widened to the genuine promise of AI in medicine and self-driving cars, the politics of inevitable inshittification and tech monopolies, and a closing argument over post-scarcity and whether human nature itself can — or should — be edited. Pressed by host Gary Oberbrunner on what happens "if AIs become better at being human than humans are," the pair disagreed productively to the end: a Burkean skeptic of fixed human nature beside a technoprogressive who believes we can, and probably will, change it. Fitting questions, in other words, for a room trying to think a thousand years ahead.