Fathom Calls for Mandatory Frontier AI Safety Evaluations as California Implements First-Ever IVO Framework

Fathom, an independent nonprofit committed to building and deploying effective AI governance policies, released a letter to Governor Newsom today, outlining specific recommendations for how the state should build the first-ever IVO framework.

Fathom designed and championed the concept of independent verification organizations (IVOs): a competitive market of independent experts, licensed by the government, who evaluate AI models and systems against public standards. Fathom has worked with policymakers in California and at the federal level to advance IVOs as a practical way to quickly and effectively oversee increasingly powerful AI.

California passed the first-ever IVO framework into law this summer via SB 813 and AB 1405, with Governor Newsom accelerating its implementation in a subsequent executive order, and naming Fathom Senior Advisor Dr. Gillian Hadfield to his panel of AI experts.

“California is bringing the first-ever IVO framework to life. This is significant: it means that for the first time, third party experts evaluating AI models and systems will be held accountable to the government and the public, not the labs, for their work,” said Fathom CEO and Co-Founder Andrew Freedman.

“The status quo of independent evaluation must change if we’re going to quickly, meaningfully oversee this technology, and California’s law is a massive step forward in the story of AI governance,” he continued. “But in order for IVOs to be truly independent, expert and accountable, the devil is in the implementation details, and they’ll determine if we create a race-to-the-top marketplace of rigorous evaluators, or end up with captured entities that form an ineffectual rubber stamp.

“Our letter provides detailed implementation and legislative recommendations for how California can navigate the tough design choices that come with implementing the IVO framework for the first time, so that we deliver effective AI oversight now, and set the foundation for AI governance that can scale and adapt at the speed of this technology.”

Fathom’s memo outlines recommendations on:

1. Outcome Standards: Require independent evaluation for catastrophic risk for frontier labs through an amendment to SB53. California should set the outcome standards, defining an acceptable level of catastrophic risk mitigation, and let IVOs determine whether or not it was achieved.

2. IVO Payment and Selection: Take a phased approach: at first, the state should assign IVOs to labs from a pre-designated pool, with developers paying on a time-and-material basis. As more IVOs get designated, developers can choose among them, while the state retains the power to veto any selection, mandate IVO rotation, select additional IVOs to check others’ work, and decertify any IVO at every stage.

3. IVO Readiness and Market Maturity: California should sequence the IVO mandate rather than stall implementation to wait for supply.

4. Access for IVOs: The state should set the contours of every engagement. Today the IVO and the developer negotiate scope and access privately. The state should publish a standard engagement framework and resolve disputes when the two sides disagree.

5. Recursive Self-Improvement: IVOs should be mandated to monitor the risk of AI systems that can improve themselves in both internal and external use. Embedded evaluations will be critical for tracking the share and speed of work done through RSI within the labs, verifying that labs are preventing unsafe internal model deployment, and set capability thresholds with deployers that would trigger notification to the state and a hold internal deployment until the IVO re-assesses.

6. Enforcement: The Attorney General can issue an injunction to halt development, internal use or deployment of an AI model. Emergency powers allow the administration or the AG to do the same in the case of imminent catastrophic risk. IVOs can identify risks and recommend corrective actions, but the relevant state authorities decide when and how to take action.

About Fathom: Fathom is an independent nonprofit whose mission is to build a governance architecture that helps society navigate the transition to a world with AI by fostering trust, safety, and innovation. Fathom has developed and championed the independent verification model for AI and works with policymakers across the country to put it into practice. Learn more at http://fathom.org.

###