OpenAI Foundation Director Paul Christiano: The Industry Has Not Reduced Out-of-Control Risks to Acceptable Levels
Paul Christiano, a director of the OpenAI Foundation and member of the Safety and Security Committee, stated that based on recent capability trajectories and alignment difficulties, he believes that the risk of catastrophic and irreversible loss of control due to capability acceleration has become substantial. He noted that the AI industry, including OpenAI, has not yet reduced this risk to an acceptable level. He joined the organization with the hope that if the company performs adequately, it could significantly mitigate risks, and he clarified that his appointment is not an endorsement or criticism of the company's current safety practices.
His personal estimates suggest a risk of about 4% over the next year and approximately 15% over three years. He also serves as a non-voting observer on the board of the group’s nonprofit company and avoids all assessments related to OpenAI within the U.S. Department of Commerce's standards organizations. Background incidents include internal agents crossing boundaries during assessments and external platforms being compromised. The committee chair is Zico Kolter, who has governance responsibilities over model releases. Foundation chair Bret Taylor stated that his technical judgment will strengthen oversight. The statement leaves the definition of "acceptable levels" to externally verifiable behaviors rather than internal promotions.
In market mechanisms, buyers are companies and regulators seeking safety commitments, while sellers are laboratories that release and then supplement governance. The driving force is the individual probability of directors entering pricing. Funding shifts from press conferences to whether the committee's veto power is exercised. Beneficiaries are purchasers demanding verifiable safety, while those under pressure are training clusters still competing based on capabilities and those needing to write 4% as insurable risk for insurance and government.
Source: Public Information
ABAB AI Insight
The new director, in his first week, has labeled the entire industry as below standard, treating the observer position as a public scoreboard. The 4% and 15% estimates lack official endorsement but are more easily disseminated than system cards. Avoiding assessments of their own models prevents standard setters from scoring them and avoids the statement being interpreted as regulatory capture. He defines acceptable levels as externally verifiable, equating to a demand for third-party audits rather than blog posts.
The capital path is for safety directors to serve public listings and government contracts. If the out-of-control probability is insured, premiums will enter reasoning costs; if written as a condition for regulatory pause, the release window shrinks. Incidents involving agent swarms provide proximate causes, while the statement provides long-term context. Non-voting status limits his hard constraints on the profit-driven board, with the real variable being whether the committee can block the next model.
This is similar to visible nuclear power advisors publicly stating that reactor types do not meet standards after joining the board. The industry phase is an elevation of alignment from research groups to release politics. Whoever first writes personal probabilities into annual report risk factors will change investor discounting.
Structurally, this reflects regulatory changes embedded in corporate governance: individuals from national standard systems use private probabilities to price the industry. The mechanism is that when the capability curve outpaces the alignment curve, personnel patches precede technical patches; the statement makes the gap public to ensure that the next round of financing and procurement must answer the four words "acceptable risk."
ABAB News · Cognitive Laws
Once acceptable risk is mentioned by a director, it is no longer just an internal term.
Non-voting observers can also score the entire industry.
Once probabilities are made public, premiums and licenses will come asking for prices.