The U.S. government is accelerating the development of a regulatory framework as it expands cooperation with major tech firms to screen AI models for national-security risks before public release.
On May 5 (local time), the Commerce Department’s AI Standards and Innovation Center (CAISI) announced agreements with Google DeepMind, Microsoft and xAI to establish a system that evaluates AI models’ performance and risks prior to public deployment.
Under the agreements, companies must submit their latest models to CAISI before launch. The agency will use those submissions to conduct pre-deployment evaluations and targeted studies. Officials said they will analyze models with some safeguards relaxed or removed to focus on risks that could affect national security, including cyber, biological and chemical threats.
“An independent, rigorous evaluation framework is essential to understanding advanced AI’s capabilities and national-security implications,” CAISI said, calling the cooperation an important step for the public interest. The agency has already completed more than 40 evaluations, including work on non-public models.
The move builds on cooperation begun in 2024 with OpenAI and Anthropic. At that time, the Biden administration established the AI Safety Institute Consortium (AISIC) to accept models from leading firms for safety checks; the organization was later renamed under the Trump administration and the government has now broadened the scope of collaboration.
The White House is also weighing additional steps to tighten AI oversight. Officials are discussing a new public-private working group to formalize pre-release verification of AI models and are considering pursuing the effort through an executive order.
The push reflects growing security concerns after advanced AI systems evolved faster than expected. In particular, Anthropic’s publicly disclosed model “Claude Miso” drew government attention for a notably improved ability to detect and potentially exploit software vulnerabilities. Because of the cyber risks, Anthropic has distributed the model in a controlled way, providing access only to selected firms.
Anthropic CEO Dario Amodei met with senior White House officials to discuss the model’s risks and potential uses.
As powerful AI models raise the prospect of cyberattacks and other societal risks, experts say that pre-release verification could emerge as a new global standard.
By Chan Park | cpark@aitimes.com
Most Commented