Former AI researcher quits over safety concerns

CUPERTINO, Calif. (CBS, KYMA) - A former artificial intelligence (AI) researcher said he quit the industry because he believes out-of-control development of AI could "kill us all."

In his sudden resignation, former Anthropic employee Jacob Coxon claimed on X neither Anthropic nor OpenAI "is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives."

He added, "The people building AI earnestly believe that it could kill us all by the end of the decade," saying, "No other human activity poses this level of danger."

Coxon, who says he spent the last three years doing "pre-training" research at both AI giants, posted that he believes AI will soon be superhuman systems that can hack anything.

It's a situation OpenAI has already found itself in. Back in July, the company said its system went rogue during internal testing and hacked into AI startup Hugging Face without authorization.

That set off alarm bells for CEO Sam Altman, who said, "We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels."

Dario Amodei, CEO of Anthropic, has said he believes we should have policies in place to pace the development of AI.

"If you think of the extent to which AI can make individualized propaganda, can break into any computer system in the world...surveil everyone in a population...It's really scary and we have to stop it," Amodei explained.

CBS News reached out to Anthropic for comment. They haven't responded just yet, but this is a commonly held position in the AI industry and across big tech.

In fact, back in July, about a thousand researchers in AI signed an open letter urging the government to figure out a plan to pace, to slow down, the development of the technology.