I am a person who reads forty reviews before buying a kettle. Coffee goes cold while I compare descriptions of how the lid feels. So I come at the question of regulating AI with a certain amount of sympathy for the people who have to decide with far less information than I want for a kettle, and far more riding on the answer.
The question is simple to state. Capabilities keep changing, the evidence about risk arrives slowly and unevenly, and someone still has to decide whether to write a rule today. This post is about how to make that decision without pretending the uncertainty is gone. I will lay out the problem, look at two real bills that went different ways, borrow a couple of ideas that seem to hold up, and then sketch the decision tool I would build.
The problem has a name
David Collingridge described it in 1980 in The Social Control of Technology. Early in a technology's life, when it is cheap to steer, you cannot tell what its effects will be. By the time the effects are visible, the technology is entrenched and changing it is expensive and slow. People call this the Collingridge dilemma, and it has an information half and a power half.
The International AI Safety Report published in January 2025, chaired by Yoshua Bengio, describes the same trap in the terms policymakers face now and calls it an evidence dilemma. Acting early on limited evidence might turn out to be ineffective or unnecessary. Waiting for strong evidence could leave society unprepared, or make mitigation impossible if capabilities jump suddenly. The report's suggestion is modest: early-warning systems and risk-management frameworks may ease the dilemma. It does not claim to dissolve it.
I like that restraint, because the usual way this debate gets framed is as a choice between caution and speed, as if one of them were free. Neither is.
Two bills, two different bets
California gives a useful pair of examples. SB 1047 would have applied to models trained with more than 10^26 operations of compute that also cost over $100 million to train. Governor Newsom vetoed it in September 2024. As reported at the time, his objections were that it regulated on the basis of the cost and the amount of computation used to develop a model, that it applied stringent standards to even basic functions when a large system used them, that smaller specialized models could be as dangerous or more so, and that it could give the public a false sense of security. He said he preferred risk-based legislation that looks at deployment in high-risk settings, critical decision-making and sensitive data.
A year later, in September 2025, he signed SB 53, the Transparency in Frontier Artificial Intelligence Act. It still uses a 10^26 compute line to define a frontier model, and adds a revenue test of $500 million for what it calls large frontier developers. But what it asks for is different. Large developers must publish a framework describing how they assess and mitigate catastrophic risk and review it at least annually. Frontier developers must publish transparency reports when they release a new or substantially modified model. Critical safety incidents must be reported to the state's emergency services office within 15 days, or within 24 hours if there is an imminent risk of death or serious injury. Whistleblowers are protected, and penalties reach $1 million per violation.
My reading, and it is only my reading, is that the second bill is a different bet about uncertainty. It does not try to decide in advance which models are dangerous. It makes the developers show their reasoning and report what goes wrong, so that the next round of rules rests on evidence. I would not call it a solution. The compute line is still doing work it may not be able to do well, which I will get to in a later post, and a transparency regime depends on what gets disclosed. But it is a more honest response to not knowing.
The cost of getting it wrong in each direction
It helps to be specific about what each error costs, because "precaution" tends to be treated as the safe default when it is not.
Regulating too early risks locking in the wrong proxy. If a rule is written around something measurable today, like training compute, and the link between that measure and actual risk turns out to be loose, the rule either misses what matters or burdens what does not. Newsom's point about smaller specialized models is the version of this that actually happened. Sara Hooker made the argument in 2024 that compute thresholds are poorly grounded because the relationship between compute and risk is uncertain and changing quickly. A rule can also create false comfort, which Newsom named too.
Regulating too late runs into the power half of Collingridge. Once a capability is widely deployed, by the time anyone can demonstrate harm, the tools to contain it may no longer exist. The International AI Safety Report makes this point about open-weight models, where access cannot be rescinded once weights are public.
The trouble is that these two errors are not symmetrical in how they show up. Over-regulation gets noticed quickly, by the firms affected. Under-regulation tends to surface later, after something has happened. That asymmetry pushes political systems toward waiting, and it is worth naming so that it can be corrected for deliberately.
Ideas that seem to hold up
Pre-committed triggers. Holden Karnofsky's 2024 piece for the Carnegie Endowment proposes if-then commitments: if a model has a particular dangerous capability, specific safeguards must be in place, and if they are not ready, development or deployment pauses. Each commitment has a tripwire, an evaluation-based condition for detecting it, and a required response. The appeal is that it does not require anyone to agree on whether the risk is near. It only requires agreeing on what you would do if it were. He also recommends setting evaluation triggers before the real danger line, to leave a margin for error, and he is plain about the limits: evaluations may miss capabilities, it is probably never possible to fully rule out a tripwire capability, and voluntary commitments alone are unlikely to be enough.
Rules that expect to be wrong. The EU AI Act presumes a general-purpose model has high-impact capabilities above 10^25 floating-point operations, but Article 51 also lets the Commission amend that threshold by delegated act to account for developments such as algorithmic improvements or more efficient hardware. I think this is underrated. The drafters wrote the expectation of their own proxy aging into the text.
Institutions built to learn. Dietz, Ostrom and Stern argued in Science in 2003, writing about shared natural resources, that fixed rules are likely to fail under uncertainty because they lean too heavily on current knowledge. They asked for governance that supplies trustworthy information, handles conflict, secures compliance, provides supporting infrastructure, and keeps the capacity to change. It is a general argument, not an AI-specific one, and I would not claim it transfers neatly. But the list reads like a checklist for what a monitoring body would need.
A monitoring posture with a way out. The NTIA report from July 2024 recommended monitoring a portfolio of risks from open-weight models rather than restricting weights immediately, with steps to collect evidence, evaluate it, and act if risks warrant it. That is a decision to wait that comes with a plan for when to stop waiting.
I also see a link to David Teece's dynamic capabilities, which I use elsewhere in my research. His three capacities are sensing, seizing and transforming. A regulator that cannot sense (no incident data, no evaluation access) cannot seize in time, and one that cannot transform (no amendment path) will be stuck with last year's proxy.
The tool I would build
Here is what I would put on a single page for any rule made under uncertainty. It would be a short decision record with these fields:
- What we believe, and how confident we are.
- What we do not know, stated as specific questions.
- The indicator we will watch for each unknown, and who produces it.
- The threshold at which we act, and what the action is.
- Who owns the decision when the threshold is crossed.
- A review date, and what would make us bring it forward.
- What would make us conclude we were wrong.
The last field is the one I care about most, because it is what turns uncertainty into something a public can hold officials to. A policymaker who says "we are not certain" has not yet said anything checkable. One who says "if evaluation results show X by this date, we will do Y, and here is who decides" has made a commitment that can be audited.
To stress-test the format I would run it backwards on past decisions. Take the SB 1047 veto: fill in the record the governor's office could have published, and ask whether the unknowns it named (do smaller models matter, does compute track risk) had indicators and owners. Then do the same for SB 53's reporting requirement. The exercise would show whether the format forces useful specificity, or whether people will fill it with vague text. I have not run it and I am not going to claim a result.
Communicating uncertainty without losing accountability
There is a real worry that publishing uncertainty gives officials cover. "We said we weren't sure" can become an all-purpose alibi. The decision record answers that by tying every stated uncertainty to a trigger and an owner. Uncertainty is allowed, but it has to come with a plan and a name.
It also helps to say which kind of uncertainty is in play. Some of it is about facts that further research could settle, such as how much uplift a model gives a novice. Some is about the future, such as how fast capabilities will grow, which research can only narrow. And some is about values, such as how much risk is tolerable. Mixing these together is how a disagreement about values gets disguised as a disagreement about evidence.
Where I could be wrong
Triggers can be gamed, and a lab that controls its own evaluations has an incentive to stay under the line. Karnofsky's piece is explicit that evaluations may miss capabilities. Precommitment can also make decisions rigid at the moment flexibility is needed, which is the Dietz, Ostrom and Stern concern in reverse. And I am writing as someone who has never had to defend a rule in front of a legislature. The process constraints, the lobbying, the calendar, all of it affects what is feasible, and I may be underweighting it.
What I am fairly confident about is narrower. "We do not know yet" is a fine thing for a policymaker to say, provided that the next sentence says what they will watch, what will make them act, and when they will look again.
If you work on policy under uncertainty, or have used if-then commitments or decision records in practice, I'd like to hear what I've missed. Get in touch →
References
Bengio, Y., et al. (2025). International AI Safety Report. arXiv:2501.17805.
Clifford Chance. (2025, October). SB 53: California sets standards for AI transparency.
Collingridge, D. (1980). The Social Control of Technology. Frances Pinter.
Dietz, T., Ostrom, E., & Stern, P. C. (2003). The struggle to govern the commons. Science, 302(5652), 1907-1912.
European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 51. Official Journal of the European Union.
Hooker, S. (2024). On the limitations of compute thresholds as a governance strategy. arXiv:2407.05694.
Inside Global Tech. (2024, September 30). California governor vetoes AI safety bill.
Karnofsky, H. (2024, September 13). If-then commitments for AI risk reduction. Carnegie Endowment for International Peace.
National Telecommunications and Information Administration. (2024, July 30). Dual-use foundation models with widely available model weights. U.S. Department of Commerce.
Teece, D. J. (2007). Explicating dynamic capabilities: The nature and microfoundations of (sustainable) enterprise performance. Strategic Management Journal, 28(13), 1319-1350.