I Asked the Godfather of AI If We Should Be Nervous. His Answer Scared Me

Peter Norvig thinks the worst jobs will disappear first, and that's exactly what we should want. The former Director of Research at Google and co-author of the definitive AI textbook isn't worried about mass unemployment from AI automation. He's worried about something much more specific: the impossibility of building AI systems that cure diseases without also enabling bioweapons.

This isn't the typical AI safety conversation about superintelligence or alignment. Norvig is focused on a problem happening right now, one that reveals why regulating AI might be fundamentally impossible. His framework cuts through the abstract debates about AI risk to expose a concrete engineering challenge that has no good solutions.

The conventional wisdom says AI will destroy entrepreneurship by giving big tech companies an insurmountable advantage. Norvig argues the opposite. "In the short run, I think it's just the opposite of that. I think it's easier than ever now to be a founder and build something, because you need less capital and less labor as well," he explains. The barrier to starting a software company used to be enormous upfront costs for computers, office space, and large teams. AI has inverted this equation.

This isn't just about coding assistants making programmers more productive. It's about the fundamental economics of company building. Where you once needed millions in venture capital to compete, AI tools now let small teams build sophisticated products from day one. The same technology that threatens to concentrate power in the hands of a few AI giants is simultaneously democratizing the tools needed to challenge them. Norvig sees this as part of a broader trend over the past two decades, but AI accelerates it dramatically.

The jobs question reveals a more nuanced picture than the standard automation anxiety. Norvig points to toll booth operators as a recent example of automation eliminating an entire profession. "There aren't any toll takers at the bridges anymore. There's a camera that reads your license plate, and there's a transmitter under your windshield." But as he notes, "that was not a great job" in the first place. The work that AI eliminates first won't be the creative, meaningful parts of jobs that people actually want to do.

This creates what Norvig calls a net positive outcome, even if the transition is painful for individuals. The challenge isn't that AI will eliminate good work, but that society needs mechanisms to help people whose jobs disappear find new opportunities. He acknowledges this directly: "There is the issue of what happens if one of the jobs that goes away is yours, and then what are you gonna do next? So we're gonna as a society, we're gonna have to come up with methods" to handle these transitions.

But the real complexity emerges with what Norvig calls dual-use AI systems. This is where his decades of experience building AI systems shows. He poses the fundamental challenge: "How do you make an AI system that's an expert in chemistry and biology and say, I want users to be able to use this system to invent a new drug that will cure disease, but I don't want anybody to be able to use this system to invent a pathogen that will kill people."

This isn't a theoretical problem. It's happening now with large language models that have access to scientific knowledge. The same AI system that could accelerate drug discovery could also provide detailed instructions for creating biological weapons. Unlike nuclear technology, where the physical materials and facilities required provide natural barriers, AI-enabled bioweapons development could potentially happen in any well-equipped lab.

Traditional approaches to technology regulation don't work here. You can't classify the AI model itself, because the knowledge exists in academic literature. You can't restrict access based on geography, because the internet doesn't respect borders. You can't even restrict it to verified good actors, because the definition of "good" varies dramatically across cultures and political systems.

This dual-use problem extends beyond bioweapons to cybersecurity, financial manipulation, and social engineering. Every capability that makes AI useful for legitimate purposes also makes it dangerous in the wrong hands. Norvig's insight is that this isn't a bug in our current AI systems that we can engineer away. It's a fundamental feature of powerful, general-purpose AI.

The implications go beyond technical solutions to governance questions that no one has answered. If we can't build AI systems that only enable good uses, how do we deploy them responsibly? If the same model that helps doctors diagnose diseases can help terrorists design attacks, who decides what gets built and deployed?

Watch for companies and researchers who acknowledge this dual-use reality rather than promising technical solutions that don't exist. The AI systems being deployed today are making irreversible choices about what knowledge gets democratized and what new capabilities become available to both beneficial and harmful actors. Norvig's framework suggests that managing these tradeoffs, rather than solving them, will define the next phase of AI development.