Pro Interpreters vs. AI Challenge: Who Translates Faster and Better? | WIRED

Summary

In a quest to determine the efficacy of AI in translation, Wired pitted the KUDO speech translator against human interpreters in a series of tests. The evaluation focused on conveying emotions, intonation, and nuances in speeches by King Felipe of Spain, Colombian President Gustavo Petro, and El Salvadorian President Nayib Bukele. While KUDO excelled in capturing technical terms and maintaining content completeness, it struggled with conveying emotion, syntax, and grammar. Human interpreters highlighted the AI's limitations in capturing the flow of speech and the figurative language. The conclusion suggests AI translators are suitable for informal situations but not for high-stakes scenarios requiring precise language and deep understanding.

Takeaways

• 🧩 AI Interpreters vs. Humans: The script discusses a comparison between AI interpreters, specifically KUDO, and human interpreters to determine which is more effective in conveying speech nuances.
• 🌐 COVID-19 Speech Test: The first test involves a speech by King Felipe of Spain about the COVID crisis, emphasizing the importance of emotion and connection with the audience.
• 🤖 AI's Content Completeness: In the first test, KUDO performed well in terms of content and completeness, but struggled with word choice and conveying the emotional depth of the speech.
• 🎤 Spontaneity and Flow: The second test with Colombian President Gustavo Petro's speech highlights AI's difficulty with capturing the flow of speech and interpreting pauses correctly.
• 📈 Strength in Technical Terms: AI proved effective in capturing and translating technical terms and titles, as demonstrated in the second test.
• 💬 Challenges with Syntax and Grammar: The script notes that AI often made mistakes with syntax, word choice, and grammar, which can affect the accuracy of the interpretation.
• 🇸🇻 Speed and Clarity: In the third test with El Salvadorian President Nayib Bukele's speech, AI managed to keep up with the speed but had issues with clarity and accuracy.
• 🌐 Global Inflation Example: AI's interpretation of the speech about global inflation demonstrated its ability to convey complex economic concepts, albeit with some ambiguity.
• 🛑 Limitations in High-Stakes Situations: Interpreters suggest that AI should not be trusted in high-stakes situations such as courtrooms, legal systems, or important diplomatic discussions.
• 🔍 Need for Flexibility and Openness: While AI is not a replacement for human interpreters, there is a call for examining and understanding how AI can be constructively used to aid interpreters.
• 🔄 The Future of Interpretation: The script concludes that the field of interpretation should be open to new ideas and flexible in integrating AI tools where appropriate.

Q & A

What is the purpose of testing KUDO, the speech translator, against human interpreters?

The purpose is to determine which method better transmits all necessary elements for people to understand what they're hearing, including emotion, intonation, completeness, spontaneity, and speed.

What were the key aspects of King Felipe of Spain's speech that were important to focus on during the test?

The key aspects were emotion and the ability to connect with the audience, as the speech addressed the citizens of Spain during a difficult time of the COVID crisis.

How did KUDO perform in terms of content and completeness when translating King Felipe's speech?

KUDO performed well in terms of content and completeness, but struggled with word choice in some areas, leading to borderline nonsensical translations.

What specific term did KUDO struggle to translate accurately in the context of King Felipe's speech?

KUDO struggled to accurately translate the term referring to the 'emotional and physical burden' of healthcare workers as 'the great emotional charge and physics on their backs'.

What was the focus of the second test involving former senator and current Colombian president, Gustavo Petro?

The focus was on spontaneity, examining how KUDO would handle a speech with a natural flow and pauses.

How did KUDO's performance differ when translating Gustavo Petro's speech compared to the first test?

KUDO was effective at capturing terms of art but struggled with pauses, creating 'islands' of complete thoughts that disrupted the natural flow of the speech.

What were the challenges KUDO faced when interpreting President Nayib Bukele's speech?

KUDO faced challenges with syntax, word choice, and sometimes grammar in English, and occasionally chose words that were not the most accurate way a native English speaker would phrase something.

What was the main takeaway from the test involving President Nayib Bukele's speech?

The main takeaway was that while KUDO was able to capture all content, it had difficulties with the nuances of language, such as metaphorical meaning and the natural flow of speech.

In what situations would the narrator not trust the use of AI for translation?

The narrator would not trust AI for translation in high-stakes situations such as courtrooms, legal systems, medical interactions, asylum cases, or high-level diplomatic talks.

What is the narrator's view on the role of AI in the future of interpretation?

The narrator believes that AI is not a replacement for human interpreters but a tool that can be used constructively and carefully to aid interpreters, provided it is understood and respected for its capabilities.

What does the narrator suggest as an appropriate use for AI translation like KUDO?

The narrator suggests using AI translation in informal situations where there is not a high level of consequence for mistakes.