Lists of important problems are easy to write and not very useful. Everything on them is important, which is the trouble. What I actually want to know when I look at a field is narrower: given that I have a finite number of hours, a laptop, no lab, and no compute grant, where would an hour of work do the most good?
This post is my attempt to answer that for AI safety, using the seven earlier pieces in this series as raw material. It is a gap map in the plain sense: where the field seems thin, what a small, well-scoped project in each gap would look like, and how I would choose among them. I am writing as someone outside the labs, which limits what I can see and gives me a slightly different view of what is missing. Both of those are reasons to hold the conclusions loosely.
How crowded is the field?
Start with some numbers. In April 2024 the Emerging Technology Observatory published a count of AI safety research. Looking at about 30,000 safety-related articles between 2017 and 2022, they found that safety made up about 2% of all AI research, a share they described as a drop in the bucket. Forty per cent of those articles had American authors, 12% Chinese authors and 19% European authors, and among the top-cited 10% the American share rose to 58%. The clusters they named included data poisoning, algorithmic fairness, explainable machine learning, gender bias and out-of-distribution detection.
I need to flag what that count can and cannot tell me. The definition of "safety" is broad, so the 2% includes work on fairness and explainability, not only the catastrophic-risk work people usually mean by the term. It covers English-language articles. And the dates stop in 2022, before the generation of models that made the field so visible. So the number says safety work is a small slice of AI research. It does not say which parts of the slice are neglected.
A second source gives a different angle. The 2025 Stanford AI Index reports that organisations often recognise responsible AI risks without acting on them. In a McKinsey survey it cites, fewer organisations took active steps to mitigate than considered a risk relevant, in every category. The widest gap was for intellectual property infringement, where 57% called it relevant and 38% were actively mitigating, and the reputational risk gap was 45% against 29%. The same chapter notes that major model developers report results on the same general capability benchmarks, but no comparable standard exists for safety evaluations, and that most evaluations are internal and nonstandardised.
Put the two together and you can see where my hypothesis comes from.
The hypothesis
My working claim is that the deployment layer is under-studied relative to the model layer. By deployment layer I mean the organisations that take a model someone else built and put it in front of customers, patients or citizens, with their own prompts, data, vendors and review processes. Much of the published safety work concerns what the model can do. Much less concerns what happens inside the organisation that adopts it, which is where the AI Index data shows a gap between awareness and action.
I want to be honest that this is a hypothesis built from suggestive evidence. The ETO count does not separate model-level work from deployer-level work. The AI Index figures come from a survey of organisations, not a map of the research literature. So I would not publish the claim as it stands.
How I would test it
The test is a count. Draw a random sample of 300 papers from a defined AI safety corpus. Write a coding guide that sorts each paper by the level it studies: the model, the system built around it, the deploying organisation, or society at large. Have two people code the sample independently, report their agreement, and calculate the share at each level with a 95% confidence interval. Before looking at any data, I would write down the result that would make me drop the hypothesis. A reasonable line would be: if papers about the deploying organisation make up 10% or more of the sample, I stop describing it as neglected.
I have not run this count and I am not presenting its result. It is the work the claim owes before it earns a place in anything I publish.
Five gaps and a project for each
Each of the earlier posts in this series ended up pointing at a specific gap. Here they are with the smallest project that would produce something real.
1. Release decisions for open weights have little independent replication. From my post on open weights: the worst-case fine-tuning study for gpt-oss was run by the developer. A project would be a replication protocol that an outside group could follow, with a stated risk threshold and a plan for handling dangerous findings. It needs compute, access to capable models and a careful approach to information hazards.
2. Evaluation claims are rarely checked for quality. From my post on passing eval scores: Bean and colleagues supplied a construct validity checklist, but I am not aware of anyone applying that kind of rubric systematically to the safety claims in published model documentation. The project is a desk audit of, say, 20 model cards using my seven-question rubric, scored by two coders, with the profile of "stated and adequate," "stated but weak" and "not stated" published for each. It needs no compute.
3. Compute thresholds age without a review habit. From my post on thresholds: the arithmetic shows how fast a fixed line loosens under published efficiency estimates. The project is a short policy memo with a proposed review mechanism and a sensitivity analysis. It is desk work, but it needs an expert reviewer to catch my mistakes.
4. There is no near-miss system for deployers. From my post on near-misses: the project is the nine-field taxonomy tested on at least ten public incidents, coded by two people, with a report on which fields cannot be filled from public information.
5. Safety testing is thin outside English. From my post on language: a pre-registered pilot comparing English, Modern Standard Arabic and Darija, with native-speaker prompts and two annotators. It needs a small API budget and paid annotators.
I would add a sixth that cuts across: oversight is claimed more than it is measured. From my post on oversight: the project is a sampling-and-catch-rate protocol, with seeded failures to estimate a reviewer's detection rate, tested with a handful of volunteer reviewers on a simple task. It needs no compute and very little money.
How I would choose
I would score each project on five questions.
- Can an independent person do it? Does it need compute, special access or institutional backing?
- Does it carry an information hazard? Could the output help someone cause harm?
- Is the result useful whichever way it comes out? A project whose null result is also informative is much safer to start.
- Can it be checked by others? Is the method simple enough that someone else can reproduce or challenge it?
- Is the gap real? Does the hypothesis test above, or a similar check, support it?
On those criteria, my order would be the oversight protocol, then the evaluation-claim audit, then the near-miss taxonomy test, then the language pilot, with the open-weight replication last, and probably not alone. The reasons are practical. The first three need no compute, carry little hazard, and give useful answers either way. The language pilot needs annotators and money. The replication needs resources and care that I would only want to take on with an institution behind me.
An eight-week plan for the first project
For the oversight protocol, the plan would be this.
Weeks 1 and 2. Define a simple review task with a known ground truth, such as spotting factual errors in short AI-generated summaries. Write the sampling plan and the scoring rules. Register the hypotheses and the sample-size reasoning, including the table from my oversight post.
Weeks 3 and 4. Build the item set: a few hundred outputs with a controlled number of seeded errors at known rates. Pilot with two or three people to find problems in the instructions.
Weeks 5 and 6. Run the study with a small group of reviewers under two conditions, a short time budget and a longer one, recording catch rates for each.
Week 7. Analyse. Report catch rates with intervals, and compare them with what the sampling table predicts.
Week 8. Write up, including everything that went wrong.
Stop conditions. If reviewers' catch rates cannot be separated from guessing in the pilot, the task is badly designed and I fix it before going on. If I cannot recruit enough participants to estimate a catch rate with a usable interval, I report that as the finding and stop.
What would change my mind
I would revise this map if the paper count showed deployer-level work is already well covered, if someone showed me the evaluation-claim audit had already been done, or if a person who works in a lab told me the real bottleneck is something I cannot see from outside. That last one is the most likely. Gap maps made from the outside tend to over-weight what is visible in public documents and under-weight the practical constraints of building and testing these systems.
Why publish the plan at all
Writing the map down in this form is partly for me. Each project has a deliverable, a method someone can criticise, and a stop condition, which stops it from turning into a vague interest. It is also an invitation. If you work in this area and you can see why one of these is already solved, or already abandoned for a good reason, I would like to know.
If you work in this area and can see why one of these gaps is already filled, I'd like to know. Get in touch →
References
Bean, A. M., Kearns, R. O., Romanou, A., Hafner, F. S., et al. (2025). Measuring what matters: Construct validity in large language model benchmarks. NeurIPS 2025, Datasets and Benchmarks Track. arXiv:2511.04703.
Emerging Technology Observatory. (2024, April 3). The state of global AI safety research. Center for Security and Emerging Technology, Georgetown University.
Greenblatt, R., Shlegeris, B., Sachan, K., & Roger, F. (2024). AI control: Improving safety despite intentional subversion. arXiv:2312.06942.
McGregor, S. (2021). Preventing repeated real world AI failures by cataloging incidents: The AI Incident Database. Proceedings of the AAAI Conference on Artificial Intelligence, 35(17). arXiv:2011.08512.
Stanford Institute for Human-Centered AI. (2025). The AI Index 2025 Annual Report, Chapter 3: Responsible AI.
Wallace, E., Watkins, O., Wang, M., Chen, K., & Koch, C. (2025). Estimating worst-case frontier risks of open-weight LLMs. arXiv:2508.03153.
Yong, Z.-X., Menghini, C., & Bach, S. H. (2023). Low-resource languages jailbreak GPT-4. arXiv:2310.02446.