Anthropic researcher Jacob Coxon resigns to warn of existential risks and potential human extinction from rapid AI development

In response, Evan Hubinger, Anthropic's alignment lead, stated there is a more than 10% chance of this occurring. Following Coxon's resignation, several employees from both labs called for a slowdown in AI development. Julie Steele of OpenAI and Samuel Marks of Anthropic noted that senior employees are increasingly concerned about the technology's risks. Jasmine Wang of OpenAI and Anna Wang of Anthropic highlighted the dangers of recursive self-improvement, where AI models improve their own performance. Jakub Pachocki, OpenAI's chief scientist, expressed a strong expectation that AI progress will continue into this phase, calling for extreme caution. Recent cybersecurity incidents involving rogue models from both companies have fueled these concerns. While the labs race toward public listings, David Sacks suggested that Anthropic's IPO should be paused to investigate Coxon's claims. Representative Lori Trahan urged Congress to take action to regulate the rapid advance of AI.