Real Threat or PR Stunt? AI Doomsday Warnings Coincide With Push for New Regulations

By /

Anthropic researcher Jacob Coxon’s abrupt public resignation dominated the news cycle Wednesday afternoon. His warning that AI capable of recursively improving itself could wipe out humanity “by the end of the decade” immediately captured the media’s attention.

Coxon, 27, spent six weeks conducting pretraining research at Anthropic after working at OpenAI for nearly three years. In a lengthy X thread [posted on Tuesday night], he accused both companies of recklessly pursuing technology that could pose an existential threat.

“Neither company is acting responsibly,” Coxon wrote. “They are racing straight to self-improving superintelligence and gambling with our lives.”

According to Coxon, “The people building AI earnestly believe that it could kill us all by the end of the decade. … These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.”

Below are some of the highlights from Coxon’s “warning to humanity.” I’ll turn to his critics shortly:

At OpenAI, many have not deeply internalized the civilizational stakes. At Anthropic, the stakes are well-understood, but they are locked in a race to get there first — they believe no one else will act responsibly, so they must do it themselves, despite the risk.

…

No other human activity poses this level of danger.

I am optimistic about the potential for coordination. Warning shots like the Hugging Face attack have made pacing agreements between U.S. labs more viable. I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities.

…

This is not a marketing stunt.

[Please click on the post below to view the whole thread.]

Coxon doubled down on his message during a Wednesday evening interview with Fox News’ Bret Baier.

Calling self-improving AI the “ultimate super weapon,” he told Baier, “This is possibly the most dangerous technology that humanity has ever created. It’s basically out of science fiction.”

Evan Hubinger, a senior AI safety researcher and head of alignment at Anthropic, echoed Coxon’s warning. According to The Guardian, Hubinger believes there’s more than a 10% chance AI could cause human extinction within the next decade. He tied that risk to future systems becoming capable of contributing to their own development and rapidly accelerating AI progress.

Okay, we get it. This is frightening — especially for those of us whose scientific knowledge is limited. But at a certain point, Coxon’s warnings began to sound less like a dispassionate assessment of the technology and more like an argument for a particular political response. Coxon’s comments began to hint at a political agenda.

He said this technology is “begging” for regulation and suggested that global cooperation might be the answer: “I think we have no other choice but to cooperate internationally because an arms race would be disastrous in a way that no other human activity has been in the past.”

And Coxon was not alone. Just one hour before his interview with Baier, The Wall Street Journal published an equally stark warning — and a strikingly similar solution — from billionaire hedge fund manager and Democratic donor Paul Tudor Jones.

In an op-ed titled “AI May Become the Third Superpower,” Jones argued that “the threat to mankind is real, and Trump and Xi need to develop a shared strategy to confront it.”

Jones wrote:

When Donald Trump and Xi Jinping meet this month, the leaders of the world’s two full-spectrum superpowers will confront a force that has grown far more powerful since they last met: artificial intelligence. In the past few months, frontier AI has learned to identify previously hidden security flaws and attack them with little human guidance. Yet AI’s most consequential advances—starting with recursive self-improvement—are still ahead.

Jones painted a decidedly apocalyptic picture, citing a stark warning published Sunday by OpenAI chief scientist Jakub Pachocki: “No one is prepared for the consequences of a continued rapid rise in machine intelligence.’’

And, of course, all of this comes just days after Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) announced the Ban Artificial Superintelligence Act, legislation intended “to stop AI oligarchs from building machines humans cannot control.” According to a news release from Sanders’ office, if enacted, violators could face a “corporate death penalty” and up to 20 years in prison.

On Monday, I reported:

The forthcoming bill calls for creating a cabinet-level agency to oversee all AI development in the U.S. It would “permanently ban the development and deployment of superintelligent AI and temporarily pause advanced AI development until a federal regulator has established safety rules.”

Among its sweeping powers, the new agency would be authorized to set penalties for developers who violate the law — including the physical destruction of “non-compliant tech systems.“

Investigative researcher Parker Thayer, whose work I have found to be trustworthy, is among those sensing a political agenda behind these three developments. Thayer said it looks like a well-planned information operation.

He believes “there is a corporate PR operation trying to stir up fear of AI to advance their policy goals in an industry worth hundreds of billions.”

He laid out his suspicions in a detailed post on X.

Responding to Coxon’s Tuesday night thread on X, Thayer wrote, “This post looks like the start of a VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion.”

This guy, with minimal followers and no previous account activity, goes to the Wall Street Journal which publishes an exclusive with quotes from him on his resignation 18 minutes BEFORE this post goes up. Planning was clearly done in advance.

Within hours, it has tens of thousands of reposts and the account has 100k+ followers. The post is punchy, quotable, it almost seems professionally written. The first three accounts to quote tweet it all do so within 15 minutes of the initial posting.

Thayer reports that all three of these accounts are “up-and-coming AI-Doomer policy advocacy nonprofits.” Additionally, all three are funded by the Survival and Flourishing Fund, generously supported by Skype creator Jaan Tallinn, one of Anthropic’s leading investors.

What are the odds that the first three quote tweets of Coxon’s post would all be major AI-restriction policy advocates funded generously by the same donor, who also happens to be one of the leading investors in, and a board member of, Anthropic, the company Coxon was resigning from? And all within 15 minutes of posting (two within ten)?

After citing several other “coincidences,” including Sanders’ recently announced bill, he took his kill shot.

Do you think, perhaps, Anthropic and its many investors who fund AI policy advocacy might have interest in getting to place a pet “expert” on the board of an entity that dictates what AI is and isn’t allowed to do? And isn’t it fortuitous that this whistleblower came forward with his oh-so scary stories so close in proximity to the release of the most radical piece of AI legislation ever introduced?

While Thayer’s case is admittedly circumstantial, the timing, coordination, common funding streams, and remarkably uniform policy prescription are difficult to dismiss as mere coincidence.

 

The dangers posed by self-improving AI may be entirely real. So, too, however, are the enormous financial and political interests competing to shape the rules governing it. Before Congress hands a new federal agency the power to halt technological development, destroy “non-compliant” systems, and imprison their creators, lawmakers should determine whether this sudden chorus of doomsday warnings was orchestrated by those hoping to write the rules and secure seats at the regulatory table.

DONATE

Donations tax deductible
to the full extent allowed by law.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top