RAG in 75+ languages: what breaks first
Building retrieval that works in Urdu, Arabic, and Tagalog as well as it does in English. Spoiler: the embeddings were the least of it.
Working out of Dubai recalibrates your assumptions about language. The workforce on our platform spans dozens of nationalities; a system that only performs in English isn’t “mostly done” — it’s broken for half its users, quietly, in ways your English-speaking team will never notice from the dashboard.
Getting the Context Engine to hold up across 75+ languages was a longer road than any architecture diagram suggested. In rough order of when each thing bit us:
Cross-lingual retrieval fails asymmetrically
The naive hope: multilingual embeddings put “annual leave policy” and its Urdu equivalent near each other, so everything just works. The reality: it works impressively in demos and unevenly in production. Query in English against English documents, great. Query in Urdu against English documents — the dominant real pattern, since corpora skew English — and quality drops off a cliff whose height varies by language pair.
What actually moved the needle: evals sliced per language pair (aggregate scores hid everything), and for weaker pairs, query translation as a retrieval assist — search with both the original query and its English translation, merge, re-rank. Inelegant. Effective.
Tokenization is a hidden tax
The same paragraph costs roughly 2–3x more tokens in Urdu or Arabic script than in English on most tokenizers. That’s not just a bigger bill — chunk-size settings tuned on English content silently truncate meaning elsewhere, and context budgets fit fewer retrieved passages precisely for the users whose retrieval is already weakest. We ended up setting chunk boundaries in characters-per-script rather than one global token count. Ugly, but it stopped the bleeding.
OCR quality is wildly uneven across scripts
Our document pipeline handled English scans beautifully and mangled Urdu ones — right-to-left text, Nastaliq script, and mixed-direction documents (Arabic body, English numerals and dates) confused every off-the-shelf layout model to some degree. Vision-model OCR closed most of the gap for us, but the lesson generalizes: your ingestion quality has a language distribution, and you should know what it looks like before your users find out for you.
“Answer in the user’s language” needs a definition
User asks in Arabic about a document written in English. Answer in what? We tested assumptions against actual user preference and got it wrong on the first guess: people overwhelmingly wanted answers in their query language, but quoted policy passages left in the original, because the original wording is what’s binding. That’s now the rule: respond in query language, quote in source language, label both.
The meta-lesson
Every one of these failures was invisible in our English-language evals and obvious the week we started slicing by language. Multilingual isn’t a feature you add — it’s a dimension you measure along, or don’t. If your user base speaks twelve languages and your eval set speaks one, you don’t have twelve times less coverage. You have zero coverage, eleven times.