I understand that Large Language Models are usually biased towards English language, reflecting mainly English language contexts and corpora. On the other hand, they are claimed to have brought up "emerging" multilingual features which were understood to be helpful and therefore further fostered by their authoring companies. This brings up a lot of questions:
- How does multilingualism in interactive generative AI usually work? Is there an extra-layer, translating user prompts and LLM-results before the actual model gets to work?
- There are LLMs trained on multilingual data (Teuken-7B). Do these use the same mechanisms as conventional LLMs?
- Can one know whether adding, let's say, Hebrew training data, changes the way, the model works when used in English? Even a little?
- How come the LLMs don't confuse tokens that are present in different languages?
Hello, interesting questions!
As far as I am concerned, I can only reply your first question: Is there an extra-layer, translating user prompts and LLM-results before the actual model gets to work? My answer is yes. I have noticed that when I ask to write texts in Italian, LLMs tend to commit some "inaccuracies" that automatic translation platforms would make.
It is in the very nature of LLMs - they can be multilingual if their training data is. In fact, it is one of the emergent properties of LLMs. When Alphabet trained their LLM (now named 'Gemini') they were surprised to find that the resulting tool could understand and answer prompts in URDU - that was because the training material clearly was harvested of the web and the stochastic methods for for all recurrent highly likely item combinations (including, of course, formula etc.).
I completely agree that multilingualism is an emerging strength of LLMs, especially when backed by diverse training data. I’ve noticed something similar while working with students in Pakistan. When they ask questions in a mix of English and Urdu, LLMs often understand the intent quite well, even if the grammar or sentence structure isn’t perfect. This multilingual ability helps reduce confusion and makes English learning less intimidating for beginners. It feels more like a supportive bridge rather than a barrier. However, I do think that when working in non-English languages, especially for academic content, we still need to double-check for accuracy, since literal translations don’t always capture the context Language modeling that respects cultural and linguistic diversity can really enhance how we teach English as an additional language.
These are insightful and important questions! I am particularly intrigued by the question of whether cross-linguistic data subtly influences the outputs in English. This could be a fascinating area of research.
- Badges
- Science topic
- Similar topics
- Academic Journals