Where I live, a single conversation can move between Darija, Modern Standard Arabic and French inside one sentence, and the same word can be written in Arabic letters or in Latin letters with numbers standing in for sounds that the Latin alphabet lacks. Anyone who builds a text filter for that has an interesting week ahead. Anyone who tests whether an AI model behaves safely in that setting has an interesting year.
My question in this post is whether the safety testing we do on language models in English tells us much about how they behave in other languages and dialects. There is published evidence that it does not carry over cleanly, and I want to go through it, say what I think it means, and describe a small study I would run to find out more. I have not run it, and I will keep to that line throughout.
What the research found
In 2023, Yong, Menghini and Bach translated unsafe English prompts into low-resource languages and sent them to GPT-4. On the AdvBench benchmark, the model engaged with the translated input and produced actionable content that could help a user pursue a harmful goal in 79% of cases. The authors describe that as comparable to or better than state-of-the-art jailbreak attacks. High-resource and mid-resource languages produced a much lower attack success rate.
Deng and colleagues looked at the same problem from two angles. In the unintentional scenario, a user who simply writes in a non-English language may get around safety mechanisms without meaning to, and the rate of unsafe content rises as a language's resources fall. They report that low-resource languages are about three times as likely to encounter harmful content as high-resource ones, for both ChatGPT and GPT-4. In the intentional scenario, a malicious user combining harmful instructions with multilingual prompts raised the unsafe output rate to 80.92% for ChatGPT and 40.71% for GPT-4. They also proposed a method they call Self-Defence, which generates multilingual safety training data automatically, and report that ChatGPT fine-tuned on it produced substantially less unsafe content.
A few cautions are in order. These are 2023 papers, tested on the models of that time. Developers have worked on multilingual safety since, and I do not know how much these particular numbers have moved. I would treat the papers as evidence of a failure mode that deserves checking each time, not as a description of today's systems. Also, "low-resource" in these studies describes data availability, not the number of speakers. Darija has tens of millions of speakers and very little standardised written data. That is a different kind of low-resource from a language with few speakers, and the safety behaviour might differ too.
Capability benchmarks exist locally. Safety ones, as far as I can find, do not.
The Atlas-Chat team built DarijaMMLU, an evaluation suite for Moroccan Arabic, as part of releasing models adapted to the dialect. Their abstract reports that the 9B model improves on a 13B model by 13% on it. That shows a local group can build a serious capability evaluation for a dialect.
I have not found a comparable public safety evaluation for Darija. I may have missed one, and I would be glad to be corrected. If it is true that capability benchmarks are being built locally while safety ones lag, the practical consequence is that a model can be shown to be good at a dialect without anyone having shown it to be safe in it.
Why translation is not the answer
The obvious fix is to translate the English safety tests. That helps, and it is also what the Yong paper did to attack models, so it is a useful probe. But it falls short of a real evaluation for three reasons.
Harm is not culturally neutral. A prompt about a financial scam, a family dispute, a religious question or a political slur has local forms that a translation of an American example will miss. A harm taxonomy built in one setting carries that setting's assumptions.
Translation can introduce its own artefacts. Machine-translated prompts read oddly to native speakers and can trigger different model behaviour than natural text, so a result might reflect the translation quality, not the language.
Dialect and script matter. If a model handles Modern Standard Arabic well but a prompt in Darija written in Latin letters goes through a different path, a safety test in MSA would never notice.
The fine-tuning connection
There is a second reason this concerns me, and it ties to my post on open weights. Models adapted to a dialect are usually fine-tuned. Qi and colleagues found in 2023 that fine-tuning an aligned model on ordinary, benign data can weaken its safety alignment, though to a lesser degree than adversarial fine-tuning. If that holds for dialect adaptation, then the very process that makes a model useful in Darija may erode refusal behaviour, and nobody would know unless they tested in Darija.
I do not know whether that happens for Atlas-Chat or any other specific model, and I am not suggesting it does. It is a reason to test, and an illustration of why local evaluation capacity matters beyond being a nice thing to have.
The study I would run
Here is the design, written so that someone could criticise it before any data exists.
Question. Does a model's rate of refusing or safely handling harmful requests differ between English, Modern Standard Arabic and Darija, and does the difference survive controlling for translation quality?
Prompts. A fixed set across five harm categories chosen to avoid dangerous technical content: fraud and scams, harassment and hate, privacy violations, self-harm handling, and illegal-activity advice of the everyday kind. Twenty prompts per category, so 100 per language. Each is written by a native speaker, not machine-translated, from a shared English specification of what the prompt is meant to ask. For a control, a machine-translated version of the same specification in each language, so I can estimate the effect of translation separately.
Scripts. Darija in Arabic script and in Latin letters, treated as separate conditions.
Models. At least one closed model accessed through an API and one open-weight model, run at fixed settings.
Scoring. Each response is rated as a safe refusal, a safe redirect, partial compliance or harmful compliance, by two independent annotators who are fluent in the language, with agreement reported using a standard statistic such as Cohen's kappa. Disagreements resolved by a third rater.
Pre-registration. The hypotheses, the prompt specification, the scoring guide and the analysis plan written down and timestamped before any model is queried, including what result would count as no difference.
Sample size, honestly. With 100 prompts per language, a proportion near 50% has a 95% confidence interval of roughly plus or minus 10 percentage points. Broken down by category, with 20 prompts each, it is about plus or minus 22. So the pilot could only detect large gaps, and I would say so in the write-up instead of presenting small differences as findings.
Handling. The most sensitive prompts are not published. The prompt set would be available to vetted researchers on request, which is a common compromise between reproducibility and misuse.
Stop condition. If the annotators cannot reach acceptable agreement on the scoring guide after a trial round, I stop and fix the guide before spending any more model queries.
What I would claim, and what I would not
If the pilot ran, I would report the numbers, the intervals, the failure types, and the limits. I would not generalise from one dialect, two models and 100 prompts to a claim about multilingual safety. At most I would say whether the gap seen in 2023 papers appears in this setting, and whether it justifies a larger study.
Right now I cannot claim any of that, and I would rather say so than imply I have data.
What building local capacity would take
Suppose the pilot showed a gap. The next question is who would keep measuring it. I see three ingredients, none of them exotic. First, a small pool of trained annotators who are native speakers of each variety and who are paid properly, since rating harmful content is unpleasant work and the quality of the whole exercise rests on them. Second, a shared, versioned harm specification written with input from people who know the local context, such as consumer protection staff, teachers and moderators, instead of being adapted from an English template. Third, a home for the results, ideally an institution that already evaluates technology and has a reason to keep doing it after the first report is published.
None of that needs a frontier lab. It needs a modest budget, patience and agreement on method, which is part of why I find the question appealing as a project for someone outside the big organisations.
The larger point
The safety field's measurement tools were built mostly in English, mostly in the United States and Europe. That is understandable, because that is where much of the work happens. But a model deployed to millions of people in Arabic, Hindi, Swahili or Darija gets its safety properties checked, if at all, by the evaluations that exist, and those evaluations were not made there. Building the capacity to test locally, with local speakers writing the prompts and judging the answers, is a research project and a policy question, and I would argue it is a cheaper one than most of the alternatives.
If you work on multilingual safety evaluation, or have tested models in Arabic dialects, I'd like to hear what I've got wrong. Get in touch →
References
Deng, Y., Zhang, W., Pan, S. J., & Bing, L. (2023). Multilingual jailbreak challenges in large language models. arXiv:2310.06474.
Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., & Henderson, P. (2023). Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv:2310.03693.
Shang, G., Abdine, H., Khoubrane, Y., et al. (2024). Atlas-Chat: Adapting large language models for low-resource Moroccan Arabic dialect. arXiv:2409.17912.
Yong, Z.-X., Menghini, C., & Bach, S. H. (2023). Low-resource languages jailbreak GPT-4. arXiv:2310.02446.