AI Can Translate. The Know-How Is Checking It.

Start reading

1. Introduction: Fluency Is Not Accuracy

Somewhere in the past three years, "we use AI translation" became a default answer in procurement calls, in client pitches, and in workflow diagrams. What didn't change at the same speed was the question that follows: and how are you checking it?

The research is unambiguous. AI translation tools, including the most advanced large language models, produce consistent, measurable errors across domains. Some of those errors are obviously wrong. Many are not. The most dangerous ones are fluent, plausible, and invisible to anyone who cannot read the source language. And as enterprises and LSPs integrate AI into more of their workflows, the volume of unreviewed output is growing faster than the infrastructure designed to catch what is wrong.

This article compiles findings from 23 peer-reviewed studies, institutional reports, and regulatory guidance documents, organized across three questions that matter most to language professionals and their clients: Is AI translation reliable? Is AI better at checking than generating? And what happens when translation errors go undetected at scale?

We also spoke with two of the people building one answer to these questions: Marco Baglioni, CEO of LanguageCheck.ai, and Anthony Neal Macri, CMO at LanguageCheck.ai.

Part One

2. The AI Translation Accuracy Problem

The conversation around AI translation quality tends to focus on fluency, and on that metric, modern systems have made extraordinary progress. Neural machine translation and large language models produce output that reads naturally. That readability is, paradoxically, the source of much of the risk.

When an output sounds wrong, a human reviewer catches it immediately. When it sounds right but carries an incorrect term, a subtly inverted meaning, or a culturally loaded phrase in the wrong register, it passes undetected. Researchers have a name for this class of error. They call it a fluent mistranslation, and the academic literature has been documenting them at scale.

Key Statistics: AI Translation Error Rates

Legal Translation: Where Errors Have Legal Consequences

A 2023 comparative study found that AI translation tools produced error rates of 15–25% when processing legal documents, including mistranslated terminology, incorrect interpretation of legal concepts, and structural issues that altered the meaning of entire clauses. Professional legal translators achieved above 98% accuracy for the same content. Researchers Moneus and Sahari (2024), writing in the International Journal of Linguistics, Literature and Translation, further concluded that AI-based legal translations often struggle with precision, leading to potential misinterpretations in critical contexts.

The Hallucination Problem in Translation Models

In 2023, researchers publishing in Transactions of the Association for Computational Linguistics (MIT Press) analyzed hallucination behavior across large multilingual translation models, including ChatGPT and GPT-4. Their finding: translation errors produced by LLMs are qualitatively different from those of traditional NMT systems — and crucially, almost all hallucinated outputs could not be self-corrected without external intervention. The model does not know it hallucinated. It cannot flag what it cannot detect in itself.

A 2025 survey published in Frontiers in Artificial Intelligence sharpened this further: high-confidence hallucinations — outputs that appear fluent and coherent but are factually incorrect — are particularly dangerous and difficult to detect automatically. Standard lexical evaluation metrics like BLEU and ROUGE, which many organizations still use to assess MT quality, fail entirely to catch this class of error.

Cultural Nuance: The Metric That Standard Benchmarks Miss

In 2025, researchers published the first multilingual, human-annotated benchmark focused explicitly on cultural nuance in machine translation. The benchmark assessed leading LLMs — including GPT-5, Claude Sonnet 3.7, and Mistral Medium 3.1 — across idioms, puns, holidays, and culturally specific concepts.

The result: mean overall quality across all models was 1.68 out of 3. Idioms scored 1.65/3. Puns scored only 1.45/3. The researchers described this as a "persistent gap between grammatical adequacy and cultural resonance" — precisely the gap that causes real-world localization failures.

Lexical Choice: The Foundational Weakness

And a foundational finding from MT research that has never been overturned: according to research cited by BYU's editing research unit, "the most frequent correction for all systems is the lexical choice… the main weak point of all systems is incorrect lexical choice." Even after retraining, one major MT system only improved lexical accuracy from 26% to 68%, leaving nearly one-third of word-level choices requiring human intervention.

Part Two

3. AI Is Better at Checking Translations Than Generating Them

The argument isn't that AI has no role in translation. It clearly does, and it has changed the economics of the industry irreversibly. The argument is about where AI creates reliable value. The research points consistently in one direction: quality estimation and automated error-checking produce better outcomes than generation alone, especially in high-stakes domains.

This distinction matters enormously for how LSPs position themselves and how enterprises build their localization stacks. AI that generates and AI that verifies are fundamentally different tools. Conflating them is the source of most deployment failures.

Quality Estimation Cuts Post-Editing Time by More Than Half

The IntelliCAT study (ACL 2021) tested a quality estimation-assisted post-editing interface against traditional approaches. Post-editing with QE support achieved a 52.9% reduction in total translation time compared to translating from scratch, while simultaneously improving output quality (-6.01 TER / +6.15 BLEU on WMT 2020 English-German). Professional translators gave the interface a System Usability Score of 88.61 — well above the industry "excellent" threshold of 85. The key mechanism: QE flagged problematic segments before humans ever reviewed them, concentrating effort where it was actually needed.

Practitioners Already Know They Need to Check

A 2025 survey published in the International Journal of Research and Innovation in Social Science found that 74.4% of practitioners always double-check AI translation outputs, while 23.1% do so sometimes. Only 2.6% trusted AI translation without any review. The researchers noted this reflects widespread practitioner awareness that MT outputs should not be accepted at face value, echoing the academic literature's consistent warning about fluent but inaccurate translations.

In other words, the translation industry already operates as if AI translation requires verification. The problem is that most of that verification is manual, inconsistent, and unscalable. Quality estimation tools systematize what trained reviewers do intuitively — and do it at a speed and consistency no human team can match.

Quality Estimation Enables Scalable, Responsible AI Translation

LanguageLine Solutions, in a published case study with TAUS and the EPIC platform, documented a specific operational problem: post-editors working without quality flagging faced machine-translated documents without knowing which sentences required serious work — leading to inefficient review, increased cognitive load, and inconsistent output quality. Automated quality estimation resolved this by directing human attention to flagged segments and clearing high-confidence content for lighter review.

Language Scientific's 2025 analysis states plainly: "Human-in-the-loop is not optional in life science translation — it is the foundation of responsible AI." It identifies quality estimation and automated post-editing as the maturing technologies that make scalable quality possible, and warns that LSPs deploying AI carelessly, allowing errors and hallucinations to reach clients, risk permanent trust erosion.

🤖 Free Course: AI-Powered Localization Quality by Marco Baglioni (LanguageCheck.ai CEO) →

Part Three

4. What Happens When AI Translation Goes Unchecked

The theoretical case for translation quality assurance is well-established. The empirical case — what actually happens when bad AI translations reach end users — is more sobering. The documented consequences span patient safety, legal liability, brand equity, and public health.

In Healthcare, Patient Safety Is Directly at Stake

A peer-reviewed article published in Discover Public Health (Springer Nature, 2025) analyzed AI translation practices in European clinical settings and found persistent and overlapping risks: inaccurate translations, bias, and unclear liability when errors occur. The paper documents cases where miscommunication due to poor AI translation led to incorrect treatment, delayed care, and compromised patient autonomy, and calls for mandatory regulatory guidelines governing AI translation use in clinical environments.

Under Section 1557 of the Affordable Care Act, machine translations must be reviewed by a human professional when accuracy is essential to access. The American Translators Association explicitly confirms this standard. Courts and regulatory bodies generally do not consider unreviewed machine-translated medical or legal documents to be reliable.

In Regulated Industries, the Error Taxonomy Is Already Documented

A 2024 IEEE study found that contextual errors, including omissions, hallucinated additions, and step sequencing errors, accounted for 32% of misinterpretations in AI-translated engineering documentation. A 2023 study published in the Journal of Medical Systems found that misinterpreted instructions were a contributing factor in 21% of global medical device-use errors. In pharmaceutical contexts, the Institute for Safe Medication Practices identifies number and unit translation errors, including wrong dosage instructions, as among the top three causes of medication-related harm globally.

In Mental Health, Fluent Mistranslations Become Invisible Risks

A 2024 research paper published on arXiv examined critical translation errors in Arabic mental health content. The finding is striking: state-of-the-art NMT systems produce fluent mistranslations that are difficult to detect by end users — and in the mental health domain, these errors cause false negatives where urgent signals of depression or suicidal ideation pass undetected. When AI translation is embedded in content moderation pipelines without quality checking, the failure mode is not just inaccuracy. It is invisibility.

In Brand and Commercial Contexts, Trust Erodes Fast

Industry data published in January 2026 found that approximately 30% of localization failures in 2024 were directly caused by over-reliance on unreviewed AI output. A 2025 Nimdzi report confirmed that brands skipping proper localization of slogans and UX copy lose up to 25% in engagement and conversion rates. And a 2024 consumer survey found that when customers encounter poorly translated content, 75% report decreased trust in the brand, and 64% say they are less likely to make a purchase.

The Hidden Cost of Gender Bias

A peer-reviewed study published on arXiv (2024) quantified the economic cost of gender bias in machine translation. Using behavioral data from approximately 90 participants, feminine translation required twice as long and four times the editing operations compared to masculine translation — a cost that automatic evaluation metrics entirely failed to detect. The research concluded this represents "unfair service disparities" — and documented that the economic burden falls disproportionately on the translators performing the work.

For Language Professionals

5. What This Means for Your Practice

The research doesn't argue against AI in translation workflows. It argues against AI without accountability. And it maps a clear path forward: the LSPs and enterprises that will sustain quality and client trust at scale are those that build structured verification into the workflow, not as an afterthought but as infrastructure.

Quality estimation has moved from a research concept to a production tool. The 52.9% time reduction documented in the IntelliCAT ACL study wasn't achieved by replacing human translators. It was achieved by giving them a better signal about where to focus.

For independent language professionals, this means positioning QA tooling as part of your service offering — not just as a personal safeguard, but as a differentiator in pitches to enterprise clients who increasingly want documented evidence of quality control. For LSP operators, it means building QE into your standard workflow before a single one of your clients asks why you didn't.

The research collected in this article represents a snapshot of a rapidly expanding evidence base. What it consistently shows is that the industry's assumption that AI translation is good enough, or that post-editing will catch what matters, is not supported by the data. The errors that reach clients are not the obvious ones. They are the ones that looked fine.

🎓 From Post-Editor to AI Quality Specialist — 6-hour live course →

6. Actionable Checklist

• ✅ Define quality thresholds for each content type and audience
• ✅ Use automated segment-level QA integrated with your CAT environment
• ✅ Flag fewer than 30% of segments — concentrate human effort where it matters
• ✅ Calibrate error categories to your domain (legal, medical, technical, marketing)
• ✅ Produce documented QA reports for enterprise clients
• ✅ Track recurring error types to refine glossaries, style guides, and prompts
• ✅ Place QA before human post-editing, not after
• ✅ Train your team on how to interpret QE scores and flags
• ✅ Know regulatory requirements (e.g., Section 1557) applicable to your clients

7. FAQ

How accurate is AI translation compared to human translation? It depends on the domain. For legal documents, studies show AI error rates of 15–25%, while professional human translators achieve 98%+ accuracy. For general content with low complexity, AI can produce usable output — but it always requires review.

What is a fluent mistranslation? A translation that reads naturally and confidently but contains factual or semantic errors. These are the most dangerous type because they are invisible to end users who cannot check the source language.

Can AI check its own translations for errors? No. Research shows that LLMs cannot self-correct hallucinated outputs without external intervention. The model does not know what it got wrong. This is why quality estimation tools provide an independent verification layer.

How does LanguageCheck.ai differ from a standard QA tool? It is purpose-built for AI-powered translation verification, performing segment-by-segment analysis at up to 1,000 words per minute. It flags fewer than 30% of segments for review and integrates with all major CAT tools via XLIFF. It does not generate translations — it verifies them.

Which CAT tools does LanguageCheck.ai support? Phrase, memoQ, XTM, and Trados via XLIFF.

Is QA mandatory? Under Section 1557 of the Affordable Care Act, machine translations must be reviewed by a human when accuracy is essential. In legal, pharmaceutical, and medical device contexts, unreviewed AI translation is generally not considered reliable by courts and regulatory bodies.

How much time can quality estimation save? The IntelliCAT study (ACL 2021) documented a 52.9% reduction in total translation time when using QE-assisted post-editing.

8. Call to Action