Silicon Valley executives and investors have been drawn into a growing debate over the risks posed by artificial intelligence after the abrupt resignation of Anthropic researcher Jacob Coxon sparked renewed warnings about the possibility of advanced AI causing catastrophic harm.
Coxon, 27, who previously worked at OpenAI before joining rival AI company Anthropic, announced his resignation on Tuesday. He said people developing increasingly powerful AI systems were taking enormous risks, warning that future systems could become capable of carrying out sophisticated cyberattacks.
“They are gambling with our lives,” Coxon said, adding that “these will soon be superhuman systems that can hack anything”.
Coxon is among several current and former employees of Anthropic and OpenAI who have raised concerns about AI safety. Some current Anthropic staff members also responded to his comments.
Evan Hubinger, a team lead at Anthropic, wrote on X that the company genuinely believed AI could potentially kill all humans. He estimated the risk at more than 10% within the next decade.
The warnings have prompted scepticism among some investors and technology executives, particularly as Anthropic and OpenAI are reportedly preparing for possible initial public offerings. Critics have questioned whether dire predictions about AI could also help generate investor interest by emphasising the potential scale of the technology.
Anthropic chief executive Dario Amodei has previously warned that AI could eliminate a large share of entry-level white-collar jobs and said the technology would “test who we are as a species”.
George Arison, chief executive of dating app Grindr, told the BBC that the latest comments reflected what he described as an “anti-civilisational worldview” at Anthropic. He said he had instructed some engineers at the company to stop using Anthropic’s technology.
Nvidia chief executive Jensen Huang also discussed Coxon’s comments at the Goldman Sachs conference in San Francisco, according to people who attended. Huang reportedly rejected the warnings, having previously described the idea that AI could end humanity as “complete nonsense”.
Other technology investors have criticised the warnings. Brad Gerstner, head of Altimeter Capital, called Coxon’s comments “ridiculous hyperbole”. Hugging Face chief executive Clement Delangue also questioned the focus on extinction risks.
The debate shifted after Amodei published an essay on Saturday calling for a slowdown in AI model development and global regulation. He described the risks associated with advanced AI as serious.
Concerns have intensified following reports that Anthropic’s AI systems have demonstrated advanced capabilities in hacking and cybersecurity. The company has also said it recently stopped actors who attempted to use its technology in activities including bioweapons development and cyber-espionage.
Some investors believe the warnings could encourage governments to impose stricter rules. US Senator Bernie Sanders has backed legislation proposing a temporary pause on advanced AI development, citing concerns that humans could lose control of increasingly powerful systems.
The Trump administration, however, has largely supported rapid AI development, arguing that the United States must maintain its lead over China. President Donald Trump said this week that he had no concerns about AI causing human extinction, but worried that the US could face serious consequences if it failed to remain ahead in AI.
Despite the debate, the financial ambitions of the industry remain strong. Anthropic was recently valued at $965 billion, while OpenAI was valued at $852 billion. OpenAI chief executive Sam Altman has suggested that a stock market listing is more likely in 2027 than this year.
For investors, the immediate question may be less about AI’s distant risks and more about whether the companies developing it can become profitable.
