Optimizing AI Services for Multilingual User Experiences

Many companies are rolling out AI first or AI augmented services that directly or indirectly interact with users. This presents some potentially thorny issues because LLMs are dependent on the quantity and quality of training data to perform well. An overwhelming amount of that data is in English and a handful of other languages. Localizing your user interface and other customer touchpoints is pretty easy because that can all be done with human augmented AI/MT translation processes. However it doesn’t matter how good your UI and website look in Thai if the underlying service is dysfunctional in that language.

Have Native Speakers Test The User Experience

One of the reasons it is a good idea to hire bilingual staff where possible is they can evaluate the product and look for functional issues while running it in their language. One of the problems with AI is it can generate language that “sounds” right but is factually incorrect. This is something native speakers can spot that may go unnoticed by other testers (and you really don’t want to turn a broken experience loose on your users).

This is also something a transcreation agency like Mother Tongue can help with. These agencies hire copywriters who are native speakers of the target language. They can assess the quality of translations, style and accuracy of responses in their respective languages.

The criteria you are asking your bilingual staff to judge it on is how accurate or correct the responses are. Minor translation errors and stilted tone are okay if the response to the query is accurate. Most users treat AI services as experimental, and will be forgiving of translation glitches if it is mostly right. On the other hand, if the process of translating back and forth results in nonsense responses due to garbled prompts, that is a bigger problem.

Be Cautious With Language Releases (Or At Least Use The BETA Label)

It is generally not a good idea to shotgun release new languages without doing thorough testing first. Even then it is a good idea to label the release as BETA and also to compare user metrics versus English. Users are generally pretty tolerant of minor translation errors in the UI, so those generally won’t hurt usage much, but if they are using your service to do research and getting nonsensical results they may churn out and not come back.

Secondary and Low Resource Languages

Secondary and low resource languages (languages that have sparse training data) pose especially difficult problems. This is for two reasons. One there may be orders of magnitude less training data to work with, so while your product might perform like ChatGPT in English, it could be a lot less reliable in Hindi. To make matters worse, a lot of the content on the open web is machine translated into these languages, which creates a garbage in → garbage out problem which makes the problem even worse if that material is used to train the model.

Use English As A Bridge Language

One approach you can use is to use a translation AI like DeepL or Google Translate LLM to translate to and from English. Some AI providers may already be doing this behind the scenes, so it is worth investigating that before you add your own translation layer, which could just get in the way. There are some risks to doing this as there will typically be some loss of information in each direction. The best models deliver accurate translations 80-90% of the time, which can make the difference between a good prompt and one that will produce garbled results. The challenge is that a small translation error may snowball into a much larger error if the AI model is thrown off by the translated prompt.

To understand the limitations in training data, take a look at the number of Wikipedia entries in different languages. This is a proxy for the availability of training data in other domains. The pattern is clear. The amount of training data in well resourced languages is large, but it drops off a cliff for secondary languages.

If you do this, it’s probably a good idea to make this clear to users, so if they speak a better resourced language, they can use that instead.

Secondary and low resource languages are often underserved or not served at all by translation engines. Sometimes you can find specialist translation platforms that target specific languages. What you’ll typically see is that translation accuracy is not as good for secondary languages, which will cause information loss in both directions. The good news is many people understand one or more of the other top international languages where AI platforms perform well. For example, French is widely spoken in parts of Africa, so users there might find the platforms work best in French versus local languages and dialects.

Languages As Information Silos

Another issue to be aware of is that information can be siloed by language. Let’s say you have questions about Japanese baseball players. You are probably going to have better luck asking questions in Japanese because information about players there may not be available in English.

The general heuristic here is as follows:

Think about where the information you are looking for is likely to live and what language(s) is it in.
Present your query in that language, or direct the AI tool to translate your query to that language and use that as a prompt.
Repeat these steps as needed
Translate the responses back to the original query language
Optionally ask the AI to aggregate and summarize the responses.

The decision to make here is whether to do this on behalf of the user or instruct the user on how to be successful with cross language queries. My guess is that the high end AI services are doing some form of this to support secondary languages and to deal with this siloing problem. The issue is that most vendors are not transparent about the internal workings of their systems.

Prompt Engineering Considerations For Product Teams

When fielding queries from users in low resource languages, here are a few things to consider.

Writing the system prompt in English, while accepting user input in their preferred language.
Instructing the model explicitly to reason in English, and then respond in the user’s preferred language (use English as a bridge language and translate in and out).
Using few-shot examples in the target language to anchor the model’s behavior.
Identify the query domain (e.g. Japanese baseball players) and assess whether there is likely to be siloed information.

Good News : Native Multilingual LLMs Are On The Way

One encouraging development is that countries are developing their own AI models that are trained in relevant languages to build natively multilingual services. In Switzerland, ETH Zurich and EPFL are developing an open source, multilingual LLM that others can build on. I expect to see more developments like this to create AI models to serve other regions. My personal hope is that open source, domain expert models will win out in the long run, because will enable countries that might otherwise be underserved to build their own models. India is a great example, with the number of people and diversity of languages spoken there.

I’ve never liked the idea of an AI that tries to be an expert in everything. That seems like an expensive thing to build and maintain. But imagine an AI that is trained up to a college student level, and then trains on everything related to Japanese baseball. These sort of expert systems can, like the localization industry, be specialized by language and subject matter, which will enable small and mid size companies to thrive. This is also how our modern economy operates as a network of specialists.

Localizing AI Based Services