Skip to content
WorkAboutInsightsNewsletterGet Your Free Brand Audit →
← Insights
9 min read

Compute Thresholds: What They Catch and What They Miss

· Amina Zakim

A speed limit is a proxy. Nobody cares about speed as such. We care about crashes and deaths, and we use speed because it is easy to measure, hard to argue about, and tied closely enough to harm that the rule works. The proxy is good because the link is stable. Faster cars hit harder, on roughly the same physics, year after year.

Several AI laws now run on a proxy of their own: the amount of computing power used to train a model. I want to look at whether that link is stable enough to carry the weight, and at what a better-designed threshold might look like. The arithmetic in this post is mine, built on published estimates, and I will show it so you can check it.

What compute governance is, and why people like it

Sastry and colleagues, a large group that includes Lennart Heim, Markus Anderljung, Gillian Hadfield and Yoshua Bengio, wrote the best overview I know. They argue that compute is an unusually good point of intervention compared with data or algorithms: it can be detected, excluded and quantified, and the supply chain is highly concentrated. They describe three kinds of governance capacity. Visibility means using compute to give regulators insight into who is building what. Allocation means steering compute toward beneficial uses. Enforcement means restricting irresponsible or malicious development. They also list the risks of doing it badly: privacy harms, economic effects and concentration of power.

Notice that visibility is the least controversial of the three, and the one a threshold serves best. A threshold says "if you cross this line, we get to know." That is a modest ask, and it can be enforced because large training runs are hard to hide.

Where the lines are drawn

The EU AI Act's Article 51 presumes a general-purpose model has high-impact capabilities, and so may carry systemic risk, when the cumulative compute used for training exceeds 10^25 floating-point operations. The same article lets the Commission amend the threshold by delegated act to account for developments such as algorithmic improvements or more efficient hardware, with the stated aim that thresholds keep pace with the state of the art.

California's SB 1047 used 10^26 operations plus a $100 million cost test, and was vetoed in 2024, with the governor objecting that it regulated by cost and computation and that smaller specialized models could be as dangerous. SB 53, signed in 2025, defines a frontier model by the 10^26 line and adds a revenue test of $500 million to identify large developers. Sara Hooker's 2024 paper notes that thresholds of this kind also appear in the US Executive Order on AI safety, and argues that, as currently implemented, they are unlikely to mitigate risk, because the connection between compute and risk is uncertain and changing quickly and because predicting which abilities emerge at which scale is harder than the policy assumes.

So the proxy is widely used and its main critic has a serious case. The question is how serious.

Why a fixed line ages

Ho and colleagues at Epoch AI estimated how fast the compute needed to reach a given level of language model performance has been falling because of better algorithms. Their central estimate is that it halves roughly every 8 months, with a 95% confidence interval of 5 to 14 months. The caveats matter: the study covers pretraining only and excludes post-training improvements, algorithmic gains and compute scaling happened together so their contributions are hard to separate, the benefit of an innovation depends on scale, and it is unclear whether the rate will continue.

Taking the estimate at face value, here is what it implies. If the compute needed for a given capability halves every T months, then a fixed line gets effectively looser over time by a factor of 2 to the power of (months divided by T).

Halving time After 12 months After 24 months After 36 months
14 months (slow end) 1.8x 3.3x 5.9x
8 months (central) 2.8x 8x 23x
5 months (fast end) 5.3x 28x 147x

In plain terms: a capability that takes 10^25 operations to reach today would need about 1.25 x 10^24 two years from now at the central estimate, and about 4.4 x 10^23 after three years. A model trained well below the line could match what the line was meant to catch. I would not read the table as a forecast. It is a way of seeing how sensitive the answer is to one assumption, and the sensitivity is large: the uncertainty range spans from 3x to 28x in two years.

This is the strongest version of the aging argument, and it tells us about pretraining only. It says nothing about post-training gains, which the study excludes, so it may understate how fast the gap opens.

What thresholds are good for

I do not think the conclusion is to abandon them. A fixed compute line is good at three things.

It is objective. Training compute can be measured and audited without a debate about what a model "can do," which matters for enforcement and for fairness between firms.

It is cheap to trigger. If the consequence of crossing is a notification, not a ban, the cost of being wrong in either direction is small. A model that crosses by accident files a report. A model that dodges the line gets caught by the next layer, if there is one.

It is hard to fake. Training at scale leaves traces in energy use, chip purchases and cloud contracts, which is the point of Sastry and colleagues' visibility argument.

What it is poor at is telling you whether a particular model is dangerous. Hooker's critique is right about that, and Newsom's veto message pointed at the same weakness when it said smaller models might matter as much.

The design I would propose

The way I would resolve this is to stop asking the threshold to do the job of a risk assessment, and layer three mechanisms.

Layer one: compute for notification. Crossing a compute line triggers a duty to notify the regulator and to disclose basic information about the training run. Nothing else. This uses the proxy for what it is good at.

Layer two: capability evaluation for obligations. Safety duties attach to what a model can do, tested by evaluations that follow the kind of rubric I described in my post on eval scores. This is where Hooker's point lands: capability, not size, should decide who bears the heavier duties.

Layer three: deployment context for use rules. Where a system is used, on whom, with what consequences, carries the rules that respond to real-world harm. That is the risk-based direction Newsom described.

On top of that I would add a mechanism for the aging problem. The EU already has a power to amend the threshold, but a discretionary amendment tends to happen late, because it takes political will. I would add a scheduled review, say every 12 months, with a published efficiency estimate and a default adjustment rule. If the halving time is 8 months, a 12-month review should lower the line by a factor of 2^(12/8), about 2.8. If the true figure is at the slow end, 14 months, the factor is 1.8, and at the fast end, 5 months, it is 5.3. I would let the regulator choose within that range based on evidence, but require them to publish their choice and reasoning. That way a missed review is itself visible.

Where this design is weak

There are real problems, and I would rather list them than hide them.

Automatic tightening widens scope. Lowering the line every year pulls in smaller developers and research groups, including open-source projects, and the compliance burden for them is not the same as for a large lab. A notification duty is light, but it is not free. Sastry and colleagues explicitly warn that badly designed compute governance can concentrate power, and a threshold that keeps dropping could end up doing exactly that.

The proxy can be gamed. A developer can split a training effort, or use techniques that shift capability into post-training, where the cumulative-training-compute count may not capture it. I do not know how large that loophole is in practice. The study I cited excludes post-training, which is the very area I am worried about, so I am working from absence of evidence there.

Efficiency estimates are themselves uncertain. I used a published central estimate with a wide interval. A regulator who adopts a number will be tempted to treat it as settled, and the entire argument of this post is that it is not.

Finally, I am assuming a regulator with the capacity to run reviews, publish reasoning and take comment. Many jurisdictions do not have that capacity yet, which is a separate problem.

What I take from it

A threshold is a tripwire for paperwork, not a verdict on risk. If you treat it as a tripwire, an annual review is a fair price for keeping it useful. If you treat it as a verdict, you end up with a rule that looks precise and slowly stops matching the world it was written for.


If you work on compute governance and have better efficiency estimates, or can see where my review mechanism breaks, I'd like to hear from you. Get in touch →


References

European Union. (2024). Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 51. Official Journal of the European Union.

Clifford Chance. (2025, October). SB 53: California sets standards for AI transparency.

Ho, A., Besiroglu, T., Erdil, E., et al. (2024). Algorithmic progress in language models. Advances in Neural Information Processing Systems 37 (NeurIPS 2024). arXiv:2403.05812.

Hooker, S. (2024). On the limitations of compute thresholds as a governance strategy. arXiv:2407.05694.

Inside Global Tech. (2024, September 30). California governor vetoes AI safety bill.

Sastry, G., Heim, L., Belfield, H., Anderljung, M., Brundage, M., Hazell, J., et al. (2024). Computing power and the governance of artificial intelligence. arXiv:2402.08797.

Want me to look at your brand?

Get Your Free Brand Audit →