Trump may be forced to reveal secret rules feds use for AI safety testing

Unspoken rules

Trump’s secret reviews of frontier AI models may hide corruption, lawsuit says.

Four federal agencies have been sued amid calls to release information about the secret framework that the Trump administration uses to conduct safety reviews of frontier AI models prior to release.

In a Wednesday press release announcing the lawsuit, a nonpartisan nonprofit called Protect Democracy alleged that “almost no details” have been released to the public or Congress. To everyone except a few vague “trusted partners,” it remains unclear what the government’s review process looks like, which companies are involved in constructing the framework, or what legal authority Trump officials have to conduct the reviews.

“Neither the identities of those entities nor the criteria by which they were selected have been made public,” Protect Democracy said.

That’s a problem, Protect Democracy alleged in a statement to Ars, since decisions about “which AI models are approved and released may be the most important policy question of this White House.” And it appears that at least one AI firm, OpenAI, has “negotiated a private agreement with the Federal Government to limit distribution of its cutting-edge AI models to government-vetted partners.”

The group’s lawsuit is seeking an order requiring officials to produce all information sought by September 30. That includes “unclassified procedural and contractual architecture: the framework’s text, the terms of participation, the identity of participants, and the process and criteria by which access to frontier models is granted or withheld,” the complaint said. They also want an injunction restraining officials from improperly withholding records that aren’t classified.

“The executive branch is now choosing which companies can release their products and which customers get access to a technology that could shape the future of not only American industry and national security, but economies and governments abroad as well,” Protect Democracy’s release said. “All with no oversight from Congress, the public, or tech experts outside of the executive branch.”

Secret rules allegedly hide corruption

According to Protect Democracy, the framework could potentially be “corrupt,” with officials from the Office of the National Cyber Director, the Office of Science and Technology Policy, the Treasury Department, and the Commerce Department potentially favoring rapid deployments for AI firms that Trump likes. And on the flip side, the public has already seen Trump retaliate against an AI firm for being too woke, Protect Democracy noted, pointing to a judge’s recent ruling that it was illegal for Trump to blacklist Anthropic.

As AI technology advances and cybersecurity risks escalate— OpenAI’s model hacking Hugging Face is the most obvious recent example—the public can’t afford to blindly trust Trump to act in good faith when choosing which frontier models should be rigorously tested, Protect Democracy said. And perhaps even worse, the public cannot know if new vulnerabilities discovered in the real world are due to agencies failing to complete a proper review, running outdated or ineffective tests, missing important checks, or missing steps that align with the latest mitigation strategies advanced by leading AI safety experts.

“We deserve to know what’s in the framework,” the group said, joining other calls for the agencies to be more transparent about AI model deployments.

In a post on X last month, US Representative Greg Casar (D-Tx.) accused Trump of “completely failing to keep us safe from the dangers of AI” while taking “millions from AI billionaires.”

“Now, in the wake of extremely dangerous AI cybersecurity problems, he says he’s set up ‘voluntary’ review that no one has seen,” Casar said. “Asleep at the wheel. Too busy cashing in to protect our jobs or national security.”

More recently, another Democrat, California state Senator Josh Becker, urged the court to grant Protect Democracy’s request. In a declaration supporting their complaint, Becker noted that California is mulling a bill, SB 813, that would establish a process where independent organizations would set baselines for AI safety standards. If passed—unlike Trump’s approach—California’s plan would make the latest benchmarks, standards, and methodologies used to evaluate risks posed by AI systems transparent to the public, Becker said. Already, public input has shaped the bill, he noted, with one provision withdrawn following public backlash.

“In contrast to the Administration’s approach, every step of SB 813’s development has been public,” Becker wrote, emphasizing that “we are accountable for the framework we have set.”

No one knows Trump’s “trusted partners”

The Trump administration has been rushing to set up the voluntary safety review process ever since the government flagged Anthropic’s Mythos 5 model as too dangerous to release earlier this summer. The plan was to rapidly expand industry collaborations so that government teams at the Center for AI Standards and Innovation could review prerelease models with reduced or removed safeguards to “thoroughly evaluate national security-related capabilities and risks.” But that would be impossible, Trump realized, without industry experts being fully transparent and explaining what exactly new models could do.

In July, the White House launched a clearinghouse, GOLD EAGLE, that relies on industry partners to help agencies flag cybersecurity vulnerabilities across many sectors and industries. Again, Protect Democracy noted that the “White House did not identify any company participating in GOLD EAGLE, the terms on which they are participating, or any legal authority for the program.”

Then, on August 3, the White House announced that it had completed the voluntary framework for reviewing AI models before their public release. Both the framework and GOLD EAGLE are actively being used to review frontier models as the technology advances, the White House has confirmed.

Some parts of the government’s review process are classified. The executive order explained that Trump planned to develop “a classified benchmarking process” that would assess advanced capabilities of AI models and determine when models should be designated as frontier AI.

However, Protect Democracy’s complaint said the framework itself is notably not designated as classified. Despite that, the White House has refused to share details with the public. Instead, a spokesperson told reporters that “just because things are unclassified, that doesn’t mean we are going to broadcast them to everyone.”

Additionally concerning, the term “covered frontier model” is not defined, Protect Democracy said. If the definition is too narrow, some dangerous AI models could slip through the cracks, and if it’s too broad, agencies recently gutted by DOGE cuts may be spread too thin to assess too many models at once. Without more clarity, the public “can only hope this powerful technology is being governed safely and in the public interest,” Protect Democracy alleged.

So far, the framework has only been shared with “a select set of AI companies,” which excludes meaningful stakeholders, such as AI safety groups currently warning of emerging societal risks, as well as smaller or emerging AI firms whose input or model reviews may be just as critical.