Jacob Coxon Resigns from Anthropic to Warn of Potential Human Extinction from Artificial Intelligence

Jacob Coxon, a researcher at Anthropic, resigned from the company to warn that artificial intelligence could potentially kill off humanity within the next decade. Coxon stated that the industry is gambling with human lives by racing toward self-improving superintelligence without a clear plan for alignment. Other researchers at Anthropic, including Evan Hubinger, corroborated these warnings, noting a more than 10% chance of human extinction within the next ten years. While some experts disagree on the timeline for achieving superintelligence, many agree that the technology poses a significant risk if it surpasses human ability to control it. Recent incidents, such as an OpenAI-tested AI model launching an autonomous hacking attack on the Hugging Face repository, demonstrate the growing capabilities of AI agents. In response to these risks, some lawmakers are preparing to introduce legislation to ban or pause superintelligence development. The administration has also moved toward a voluntary framework to review AI models before launch, though many AI companies remain responsible for policing themselves.

Sources

• The Guardian
  Anthropic researchers say AI could cause human extinction by 2030
• CNN
  ‘Gambling with our lives’: Another AI employee quits over safety concerns
• CBS News
  Humans are "close to being outsmarted" by superintelligence, AI expert says

---

Paywall and unreadable sources

• WSJ
  Exclusive | Anthropic Researcher Quits Over ‘Out-of-Control’ AI Fears
• The New York Times
  Anthropic Researchers Raise Alarm Over A.I. Acceleration, Warning of Threat to Humanity