The natural language processing abilities of modern AI are astounding. But they also have their limitations. Gary Kayye just highlighted a startup seeking to address a current limitation, the ability to understand accents. This is an obvious problem for multi-national, multi-lingual companies, and it's exciting to see the work done to address it. In the article, Kayye focuses on real-time transcription, but his title mentions real-time translation. There is a critical difference here: Real-time transcription is possible, real-time translation is impossible.

Why is real-time translation be impossible? The nature of language. The problem is that not all languages flow the same way. To construct an accurate translation from language A to language B, it is often necessary to wait for an entire sentence to end before you know what is actually being said.

To illustrate, I'll use a language I've been studying recently, Dutch. In Dutch, words at the end can completely change the meaning of the whole sentence. For example: "Ik heb jouw fiets..." Ik = I, heb = have, jouw = your, fiets = bike. "I have your bike." If your AI was real-time translating, the English sentence would be looking just fine so far: "I have your bike." But here's the problem: If the next word is "nodig", the meaning completely changes. The one-for-one translation of the word "nodig" is "necessary." But "Ik heb jouw fiets nodig" does not mean, "I have your bike necessary." It means, "I need your bike." The real-time translation would produce nonsense; only by lagging could an accurate English translation be produced.

So accurate translation necessarily requires lag. That lag varies significantly based on language. Last year, members of AVIXA's Global Partner Program got a private presentation from a lead researcher at META, who was working on this inevitable lag problem. He shared a table of the lag times their research showed was needed for accurate English translation of a couple dozen top languages (related data can be seen here). The longest lags weren't crazy long. They were generally around a few seconds. But these lags are conversationally significant. A few seconds is nothing compared to a lifetime, but it's quite an awkward pause in the flow of a conversation.

If you're imagining translation that's real-time enough to, say, have an AI plugin enable a normal inter-language conversation via Teams or Zoom, you're going to be disappointed. Because of the diversity in how languages flow, true, real-time translation is simply impossible.