Pro Interpreters vs. AI Challenge: Who Translates Faster and Better? | WIRED
Summary
TLDRIn a quest to determine the efficacy of AI in translation, Wired pitted the KUDO speech translator against human interpreters in a series of tests. The evaluation focused on conveying emotions, intonation, and nuances in speeches by King Felipe of Spain, Colombian President Gustavo Petro, and El Salvadorian President Nayib Bukele. While KUDO excelled in capturing technical terms and maintaining content completeness, it struggled with conveying emotion, syntax, and grammar. Human interpreters highlighted the AI's limitations in capturing the flow of speech and the figurative language. The conclusion suggests AI translators are suitable for informal situations but not for high-stakes scenarios requiring precise language and deep understanding.
Takeaways
- 𧩠AI Interpreters vs. Humans: The script discusses a comparison between AI interpreters, specifically KUDO, and human interpreters to determine which is more effective in conveying speech nuances.
- π COVID-19 Speech Test: The first test involves a speech by King Felipe of Spain about the COVID crisis, emphasizing the importance of emotion and connection with the audience.
- π€ AI's Content Completeness: In the first test, KUDO performed well in terms of content and completeness, but struggled with word choice and conveying the emotional depth of the speech.
- π€ Spontaneity and Flow: The second test with Colombian President Gustavo Petro's speech highlights AI's difficulty with capturing the flow of speech and interpreting pauses correctly.
- π Strength in Technical Terms: AI proved effective in capturing and translating technical terms and titles, as demonstrated in the second test.
- π¬ Challenges with Syntax and Grammar: The script notes that AI often made mistakes with syntax, word choice, and grammar, which can affect the accuracy of the interpretation.
- πΈπ» Speed and Clarity: In the third test with El Salvadorian President Nayib Bukele's speech, AI managed to keep up with the speed but had issues with clarity and accuracy.
- π Global Inflation Example: AI's interpretation of the speech about global inflation demonstrated its ability to convey complex economic concepts, albeit with some ambiguity.
- π Limitations in High-Stakes Situations: Interpreters suggest that AI should not be trusted in high-stakes situations such as courtrooms, legal systems, or important diplomatic discussions.
- π Need for Flexibility and Openness: While AI is not a replacement for human interpreters, there is a call for examining and understanding how AI can be constructively used to aid interpreters.
- π The Future of Interpretation: The script concludes that the field of interpretation should be open to new ideas and flexible in integrating AI tools where appropriate.
Q & A
What is the purpose of testing KUDO, the speech translator, against human interpreters?
-The purpose is to determine which method better transmits all necessary elements for people to understand what they're hearing, including emotion, intonation, completeness, spontaneity, and speed.
What were the key aspects of King Felipe of Spain's speech that were important to focus on during the test?
-The key aspects were emotion and the ability to connect with the audience, as the speech addressed the citizens of Spain during a difficult time of the COVID crisis.
How did KUDO perform in terms of content and completeness when translating King Felipe's speech?
-KUDO performed well in terms of content and completeness, but struggled with word choice in some areas, leading to borderline nonsensical translations.
What specific term did KUDO struggle to translate accurately in the context of King Felipe's speech?
-KUDO struggled to accurately translate the term referring to the 'emotional and physical burden' of healthcare workers as 'the great emotional charge and physics on their backs'.
What was the focus of the second test involving former senator and current Colombian president, Gustavo Petro?
-The focus was on spontaneity, examining how KUDO would handle a speech with a natural flow and pauses.
How did KUDO's performance differ when translating Gustavo Petro's speech compared to the first test?
-KUDO was effective at capturing terms of art but struggled with pauses, creating 'islands' of complete thoughts that disrupted the natural flow of the speech.
What were the challenges KUDO faced when interpreting President Nayib Bukele's speech?
-KUDO faced challenges with syntax, word choice, and sometimes grammar in English, and occasionally chose words that were not the most accurate way a native English speaker would phrase something.
What was the main takeaway from the test involving President Nayib Bukele's speech?
-The main takeaway was that while KUDO was able to capture all content, it had difficulties with the nuances of language, such as metaphorical meaning and the natural flow of speech.
In what situations would the narrator not trust the use of AI for translation?
-The narrator would not trust AI for translation in high-stakes situations such as courtrooms, legal systems, medical interactions, asylum cases, or high-level diplomatic talks.
What is the narrator's view on the role of AI in the future of interpretation?
-The narrator believes that AI is not a replacement for human interpreters but a tool that can be used constructively and carefully to aid interpreters, provided it is understood and respected for its capabilities.
What does the narrator suggest as an appropriate use for AI translation like KUDO?
-The narrator suggests using AI translation in informal situations where there is not a high level of consequence for mistakes.
Outlines
π€ Comparing AI and Human Interpreters
The narrator discusses the potential threat of artificial intelligence to jobs, including interpreters. Wired tested a speech translator called KUDO against human interpreters to evaluate which better conveys emotion, intonation, completeness, spontaneity, and speed. The test involved comparing translations of speeches from various public figures, focusing on different aspects like emotion and content accuracy.
π£οΈ Emotional Accuracy in Translation
In the first test, Barry and KUDO were given a 2020 speech from King Felipe of Spain about the COVID-19 crisis. The AI performed well in terms of content and completeness but struggled with word choice and conveying the speech's emotional nuances. While the AI produced mostly accurate translations, it sometimes rendered phrases in ways that were borderline nonsensical, failing to capture the king's emotional intent.
ποΈ Capturing Spontaneity in Speeches
The second test involved a 2021 speech by Colombian President Gustavo Petro. The AI was effective at capturing terms of art and specific phrases, such as 'president-in-office' and 'minister of finance.' However, it struggled with spontaneity and maintaining the flow of the speech, often creating disjointed and incomplete thoughts due to the speaker's pauses and inflections.
π Handling Fast-Paced Prepared Speeches
In the third test, Barry and KUDO interpreted a 2021 speech by El Salvadorian President Nayib Bukele, which was delivered at a high speed. The AI excelled in capturing all the content accurately, even outperforming the human interpreter in terms of completeness. However, it often made mistakes in syntax, word choice, and grammar, choosing words that were not the most accurate or natural in English.
π When to Use AI in Interpretation
The final section discusses the appropriate contexts for using AI solutions. While AI can be useful in informal situations with low consequences for mistakes, it is not reliable for high-stakes environments like courtrooms, medical interactions, or diplomatic negotiations. The importance of understanding and respecting the tool's limitations is emphasized, advocating for its careful and constructive use.
π§ Embracing AI as a Helpful Tool
The conclusion highlights the need to understand the power of AI tools to use them effectively and benefit everyone. Emphasizing flexibility and openness to new ideas, the narrative encourages leveraging AI constructively while recognizing it is not a replacement for human professionals.
Mindmap
Keywords
π‘Artificial Intelligence (AI)
π‘Interpreters
π‘Inaccurate Interpretation
π‘Emotion
π‘Intonation
π‘Completeness
π‘Spontaneity
π‘Syntax
π‘Word Choice
π‘Metaphorical Language
π‘Professional Interpreter
Highlights
Artificial intelligence is threatening jobs, including interpreters, highlighting the need for accurate interpretation in critical contexts.
Inaccurate interpretation can significantly impact high-stakes situations, such as nuclear deals.
Wired tested KUDO, a speech translator, against human interpreters to evaluate effectiveness in transmitting emotion, intonation, completeness, spontaneity, and speed.
The test focused on emotion using a 2020 speech from King Felipe of Spain about the COVID crisis.
Human interpreters emphasize the importance of connecting emotionally with the audience during a crisis.
KUDO performed well in content and completeness but struggled with word choice and capturing the emotional tone accurately.
The AI translation included nonsensical phrases, such as 'great emotional charge and physics on their backs,' instead of recognizing healthcare workers' emotional and physical burdens.
Test on spontaneity using a 2021 speech from Gustavo Petro revealed AI's strength in capturing specific terms but struggled with maintaining flow due to pauses.
AI effectively translated terms like 'president-in-office' and 'special drawing rights' but created disjointed speech segments due to handling pauses poorly.
Human interpreters can analyze speaker's body language and rhythm, which AI currently cannot replicate.
Test on speed using a 2021 speech from Nayib Bukele demonstrated AI's ability to retain content but struggled with syntax, word choice, and grammar.
AI translations often included ambiguous phrases, such as 'new currency issues,' which could confuse listeners.
Professional interpreters process and deliver information in a coherent and culturally appropriate manner, which AI struggles to match.
AI solutions may be suitable for informal situations with low consequences for mistakes but are not reliable for high-stakes contexts like courtrooms or diplomatic negotiations.
Understanding the limitations and potential of AI tools is essential for their constructive use in aiding human tasks.
AI cannot replace human interpreters but can be a valuable tool when used appropriately and with an understanding of its limitations.
Transcripts
- [Narrator] Artificial intelligence
has been threatening everyone's jobs,
including interpreters.
Inaccurate interpretation can be the difference
between a good or bad nuclear deal.
So Wired tested a speech translator called KUDO
versus human interpreters to see which better transmits all
of the necessary elements
for people to understand what they're hearing,
including emotion, intonation, completeness,
spontaneity and speed.
- [KUDO] Liars jobs.
- [Narrator] To test out if AI is really better
than a human.
Let's begin.
Test one.
- Here we go.
- [Narrator] We gave Barry and KUDO a 2020 speech
from King Felipe of Spain.
The speech is about the COVID crisis.
The subjects will focus on emotion.
- Because of the pandemic.
I parachuted into this not knowing anything
about the speech, when it was given.
In this speech, what matters more
is the emotion and the ability to connect
with the audience because he's speaking to the citizens
of Spain at a very difficult time
when many have suffered loss,
are going through a number of challenges.
This reminds us of very deep feelings.
At this time, many of you are fighting against the illness
and what it has done in your homes.
And the words are just the vehicle to get that across.
- [Narrator] Now let's evaluate the speech translator.
- [KUDO] Good evening, I address you on this Christmas Eve
when we are living truly exceptional circumstances
due to the pandemic.
- In terms of content and completeness,
the AI did very well.
- [KUDO] The virus has broken into our lives,
bringing suffering, sadness or fear,
altered our way of life.
- It struggled with word choice in a number of areas.
It referred to an overflow in some hospitals.
- [KUDO] And also overflow in some of our hospitals.
- Now, what that was referring to
is at that time when hospitals were overwhelmed,
when the king was recognizing the healthcare workers
for the work that they had done
and he recognized the tremendous physical
and emotional burden that they were carrying
and the AI translated that
as the great emotional charge and physics on their backs.
If you know Spanish, you can go back and figure out
how it is that the AI determined to use those words.
But obviously, that is borderline nonsensical.
If you look
at what the speech-to-speech translation produced,
it was by and large, quite accurate.
However, it's very difficult to get the sense
of what the king is seeking to portray
through the AI in its current state.
- [Narrator] Test two, spontaneity.
We gave Walter and KUDO a 2021 speech from former senator
and now current Colombian president, Gustavo Petro.
- The most powerful and nonproductive sectors
of this country, 20 billion pesos a year.
So I took the hour beforehand to look him up,
look at some of his videos.
Just listen to the level of language he employs.
Listen to his rhythms.
Listen to, you know, watch his body language.
- [Narrator] Now let's see how the speech translator did.
- [KUDO] You are, Mr. President-in-Office.
[Gustavo speaking in Spanish]
- [KUDO] In the year of 2019, this Congress decided to give
to the most powerful and unproductive sectors
of the country, 20 billion pesos a year.
- The particular strength of artificial intelligence
was in capturing those terms of art.
Artificial intelligence captured president-in-office
or sitting president,
[Walter speaking in Spanish]
minister of finance,
[Walter speaking in Spanish]
special drawing rights.
That's a consecrated term.
It's a term that's enshrined
that you can find in any number of glossaries.
That's the official translation, and I missed that.
Official caucuses as well
that the artificial translation rendered that very clearly
and it pulls those in and did so very well.
That's what I felt it was most effective at.
And where I felt it had the most shortcomings
was because of the pauses.
- [KUDO] When El Pais has a stagnant economy
and an impoverished middle class
what this tax reform brings.
[Gustavo speaking in Spanish]
- [KUDO] It is to increase taxes
on the productive company,
small and medium fundamentally.
[Gustavo speaking in Spanish]
- [KUDO] An absolute contradiction and absurdity.
- The artificial intelligence creates these little islands,
these little bubbles that read like complete thoughts
but are not because there's a flow.
There's a flow that's coming after the pause.
And thanks to an absolutely aggressive social policy,
making a gift to the bankers.
- [KUDO] And thanks to an absolutely
aggressive social policy,
how to issue money to give to bankers.
- So the pauses in this particular speech seem
to throw the artificial intelligence off.
You have the vocal inflection.
Increasing taxes on those who govern Colombia.
That you know allows you to understand
there's something else coming.
I'm not finished yet.
Unproductive capital, those who don't work
but prey on the state, on the people and on nature.
Thank you, sir.
- [Narrator] Test three.
We gave Barry and KUDO a 2021 speech
from El Salvadorian President Nayib Bukele.
The speech is highly crafted, prepared
and read from a teleprompter at a high rate of speed.
- We are going to make use of BANDESAL.
I will tell you that it was a very challenging experience
for me as a professional interpreter
to interpret that speech accurately and completely.
What you have to do is process that information
and extract the ideas to be able
to give your listener something that's usable
for them to be able to follow from that huge amount
or that torrent of words that's coming at you.
- [Narrator] Now let's evaluate the speech translator.
- Salvadoran people, good morning.
As you all know,
the world has suffered an increase in prices.
- When I was able to go back and listen
to the AI interpretation, what I noticed
throughout the entire speech
is that it did not miss any of the content.
- [KUDO] The cost of these goods increases
to match the supply of new dollars,
new euros, new yen, and therefore,
that affects world prices.
[Nayib speaking in Spanish]
- [KUDO] That is why we are experiencing a global inflation
that hits the pockets of everything, of all the people
in the world.
And El Salvador is no exception.
- I was taken aback
by the artificial intelligence's ability
to get all of the content and hold onto it.
So in terms of completeness, it was very good and better
than what I was able to do at that rate of speed.
However, it often had mistakes and challenges
with making the syntax, the word choice,
and sometimes the grammar in English,
and sometimes it would choose words
that weren't really the most accurate way,
the way a native English speaker would say something.
It talked about new currency issues.
- [KUDO] This inflation generated by the incorporation
of new currency issues, usually dollars, euros, yen.
- Which was really ambiguous
because it was talking about new infusions
of cash or currency into the monetary system.
And people could confuse issues
with problems and think, okay,
what's the problem with these new currencies?
What is happening is that the process
that the professional interpreter has been doing
for as long as professional interpreters have been working
is to process that information
and then serve it up to the listener already processed
to know this is what's being talked about.
So rather than saying new currency issues,
I would've said infusions or injections
of more cash into the monetary system.
- [Narrator] So for a profession that started
with the Nuremberg trials and continues to be used
for important meetings and negotiations today,
when is it appropriate to use an AI solution?
- I would say in informal situations, situations
where there is not a high level of consequence for mistakes.
I would not trust this in courtrooms.
I would not trust this in the legal system.
- If there is a decision that has to be made,
a medical interaction, asylum cases.
- I don't know that I would trust it
for situations that require metaphorical language,
that don't catch the figurative meaning.
- Do I want to have a high-level diplomat
or a president of my country speaking
to another president through AI to make important decisions
about the future of literally millions of lives?
No. - We need to look at it.
We need to examine it.
We need to find out how we can use this constructively
and how we can use it carefully to aid us and not be afraid
of it, because that's no use if we're not flexible.
We're eminently in a world where we need to be flexible
and we need to be open
to new ideas and new ways of doing things.
- It is not a replacement, and you have to understand that.
And if we're able to understand the power of the tool,
we can respect the tool and use it to everyone's benefit.
Browse More Related Video
The Role of the Interpreter
Nayib Bukele asume segundo mandato como presidente de El Salvador
How the UN Translates Everything in Real-Time
Yuval Noah Harari: AI is a βsocial weapon of mass destructionβ to humanity | GZERO World
Artificial Intelligence (AI) for People in a Hurry
Natural Language Processing in Artificial Intelligence in Hindi | NLP with Demo and Examples
5.0 / 5 (0 votes)