In his talk "Superintelligence: The Idea That Eats Smart People," Maciej Cegłowski takes apart the fear that a self-improving AI will bootstrap itself into a god and wipe out humanity — the Nick Bostrom, paperclip-maximizer scenario beloved in Silicon Valley.
He finds the whole thing "somewhat silly, and full of unwarranted assumptions." Intelligence, he argues, isn't a single dial you can crank; Stephen Hawking, for all his brilliance, still couldn't talk a cat into a carrier. He calls AI doomerism "Religion 2.0," a "nerd Apocalypse" that promises immortality and a God you build yourself. "AI risk is string theory for computer programmers," he writes — "fun to think about, interesting, and completely inaccessible to experiment."
His real worry is about the people it attracts: those "who can reach preposterous conclusions from a long chain of abstract reasoning, and feel confident in their truth, are the wrong people to be running a culture."
Previously:
