Kategorie: machine translation

Natural Language Processing – What’s in it for interpreters?

„Natural Language Processing? Hey, that’s what I do for a living!“ That’s what I thought when I heard about the live talk „A Glimpse at the Future of NLP“ (big thank you to Julia Böhm for pointing this out to me). As I am always curious about what happens in AI and language processing, I registered right away. And I was not disappointed.

In this conference, Marco Turchi, Head of the Machine Translation group at Fondazione Bruno Kessler, presented recent developments in automatic speech translation. And just to make this clear: This was not about machine interpretation, but about spoken language translation (SLT): spoken language is translated into written language. This text can then be used, e.g., for subtitling. Theoretically, it could then also be passed through TTS (text to speech) in order to deliver spoken interpretation, although this is not the purpose of SLT.

The classic approach of SLT, which has been used in the past decades, is cascading. It consists of two phases: First, the source speech is converted into written text by means of automatic speech recognition (ASR).  This text is then passed through a machine translation (MT) system. The downside of this approach is that once the spoken language has been converted into written text, the MT system is ignorant of, e.g., the tone of the voice, background sounds (i.e. context information), age or gender of the speaker.

Now another, rather recent approach relies on using a single neural network to directly translate the input audio signal in one language into text in a different language without first transcribing it, i.e. converting it into written text. This end-to-end SLT translates directly from the spoken source text, thus has more contextual information available than what a transcript provides. The source speech is neither „normalised“ while being converted into written text, nor divided into segments that are treated separately from each other. Despite being very new, the quality of end-to-end SLT this year has already reached parity with the 30-year-old cascade approach. But it also has its peculiarities:

As the text is not segmented automatically (or naturally by punctuation, like in written text), the system must learn how to organise the text into meaningful units (similar to, but not necessarily sentences). I was intrigued to hear that efforts are being made to find the right „ear-voice-span“ or décalage, as we human interpreters call it. While a computer does not have this human problem of limited working memory, it still has to decide when to start producing its output – a tradeoff between lagging and performance. This was the point when I decided I wanted to ask some more questions about this whole SLT subject, and had a video chat with Marco Turchi (thank you, Marco!), just to ask him some more questions that maybe only interpreters find interesting:

Question: Could an end-to-end NLP system learn from human interpreters what a good ear-voice-span is? Are there other strategies from conference interpreting that machine interpreting systems are taught to deal with difficult situations, like for example chunking, summarising, explaining/commenting, inferencing, changes of sentence order, or complete reshaping of longer passages? (and guessing, haha)? But then I guess a machine won’t necessarily struggle with the same problems humans have, like excessive speed  …

Marco Turchi: Human interpreting data could indeed be very helpful as a training base. But you need to bear in mind that neural systems can’t be taught rules. You don’t just tell them „wait until there is a meaningful chunk of information you can process before you start speaking“ like you do with students of conference interpreting. Neural networks, similar to human brains, learn by pattern recognition. This means that they need to be fed with human interpreting data so that they can listen to the work of enough human interpreters in order to „intuitively“ figure out what the right ear-voice-span is. These patterns, or strategies, are only implicit and difficult to interpret. So neural networks need to observe a huge amount of examples in order to recognise a pattern, much more than the human brain needs to learn the same thing.

Question: If human training data was used, could you give me an idea of if or how the learning system would deal with all those human imperfections, like omissions, hesitations, and also mistakes?

Marco Turchi: Of course, human training data would include pauses, hesitations, and errors. But researchers are studying ways of weighing these „errors“ in a smart way, so it is a good way forward.

Question: And what happens if the machine is translating a conference on mechanical engineering and someone makes a side remark about yesterday’s football match?

Marco Turchi: Machine translation tends to be literal, not creative. It produces different options and the problem is to select from it.  To a certain extent, machines can be forced to comply with rules: They can be fed preferred terminology or names of persons, or they can be told that a speech is about a certain subject matter, let’s say car engines. Automatic domain adaptation, however, is a topic still being worked on. So it might be a challenge for a computer to recognise an unforeseen change of subject. Although of course, a machine does not forget its knowledge about football just because it is translating a speech about mechanical engineering. However, it lacks the situational awareness of a human interpreter to distinguish between the purposes of different elements of a spoken contribution.

Question: One problem that was mentioned in your online talk: real-live, human training data is simply not available, mainly due to permission and confidentiality issues. How do you go about this problem at the moment?

Marco Turchi: The current approach is to create datasets automatically. For our MuSt-C corpus, we have TED talks transcribed and translated by humans. These translations with their spoken source texts are then fed into our neural network for it to learn from. There are other such initiatives, like Facebook’s CoVoSt or Europarl-ST.

Question: So when will computers outperform humans? What’s the way forward?

Marco Turchi: Bringing machine interpreting to the same level as humans is not a goal that is practically relevant. It is just not realistic. Machine learning has its limitations. There is a steep learning curve at the beginning, which then flattens at a certain level with increasing specificity. Dialects or accents, for example, will always be difficult to learn for a neural network, as it is difficult to feed it with enough of such data for the system to recognise it as something „worth learning“ and not just noise, i.e. irrelevant deviations of normal speech.

The idea of all this research is always to help humans where computers are better. Computers, unlike humans, have no creativity, which is an essential element of interpreting. But they can be better at many other things. The most obvious are recognising numbers and named entities or finding a missing word more quickly. But there will certainly be more tasks computers can fulfill to support interpreters, which we are still to discover while the technology improves.

Thank you very much, Marco!

After all, I think that I prefer being supported by a machine than the other way around. The other day, in the booth, I had to read out pre-translated questions and answers provided by the customer. It was only halfway through the first round of questions that my colleague and I realised that we were reading out machine translations that had not been post-edited. While some parts were definitely not recognisable as machine translations, others were complete nonsense content-wise (although they still sounded good). So what we did was a new kind of simultaneous on-the-fly post-editing … Well, at least we won’t get bored too soon!

Further reading and testing:

beta.matesub.com (generates subtitles)

http://voicedocs.com/transcriber (transcribes audio and video files)

https://elitr.eu/technologies (European live translator – a current project to provide a solution to transcribe audio input for hearing-impaired listeners in multiple languages)

https://towardsdatascience.com/human-learning-vs-machine-learning-dfa8fe421560

http://iwslt.org/doku.php?id=offline_speech_translation

https://www.spektrum.de/news/kuenstliche-intelligenz-der-textgenerator-gpt-3-als-sprachtalent/1756796?utm_source=pocket-newtab-global-de-DE

https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3

About the author

Anja Rütten is a freelance conference interpreter for German (A), Spanish (B), English (C) and French (C) based in Düsseldorf, Germany. She has specialised in knowledge management since the mid-1990s.

DeepL – not too bad, even if it turns marriage into war

After Microsoft Translate and Google Translate, last week I decided to take a closer look at DeepL’s beta desktop application. I had to prepare over 50 Power Point slides filled with text about quite a number of rulings of the European Court of Justice. I was pretty sure these would be read out at high speed in the meeting and I had no time to prepare in their entirety. As DeepL’s neural networks were trained on the basis of Linguee’s databases, I had half hoped that if I had the original text of an ECJ ruling, or part of an EU regulation, DeepL would just magically replace the English text with the official German version and save me the hassle of looking it up in Eurlex or Curia myself. Admittedly, I was also tempted by DeepL’s extremely user-friendly handling: You simply highlight the word or text you need to be translated, Press CTRL+C twice, see if you like the translation and press Enter to replace the original text with the translation. Also, if there is a particular word you don’t like in the translation proposed, you click on it and DeepL offers you alternatives to choose from in a drop-down menu (improving its own system on the basis of the user’s choice). I was then a bit disappointed to see that DeepL didn’t just replace the official English version of an EU text with the official German version, with both of them being readily available on the internet.  No human translator would take the trouble of translating something that has already been translated and/or verified by expert translators. But then DeepL obviously is not pretending to be human …

All in all, I find the quality of the translation quite impressive. A sample translation from English into German and vice versa is included at the bottom of this article. Of course, it goes without saying that machine-translated texts are not there to be read out pretending you are interpreting simultaneously or you pre-translated it yourself. And also that when using your client’s confidential data, you buy DeepL Pro to make sure no such information is saved on DeepL’s servers. Apart from these banalities, these are some points that require special attention:

Consistency: The same term may be translated differently in the same paragraph. I had nominal value translated into Nennwert and Nominalwert.

Context: When a person invests in a certificate issued by a bank, it is clearly a Zertifikat in German and not an Urkunde.

Plausibility: When an investor brings a tort action against a bank, this does not mean Ein Anleger leitet eine unerlaubte Handlung gegen eine Bank ein (i.e. the investor acts unlawfully) – as this means rather the opposite. The official German version talks of erhobene Klage wegen Haftung dieser Bank aus unerlaubter Handlung.

Robustness: Make sure your original text has no typos! There are typos that are not detected (yet) by machines, because the „wrong“ word is actually a real word, too. Such tiny mistakes often go unnoticed by human readers, because we tend to auto-correct them on the basis of the words we expect to read in a certain context. However, such minor mistakes in the original text can sometimes lead to quite disturbing mistranslations. For example, a non-martial (instead of non-marital) partnership was translated by DeepL into Nicht-Kriegsgesellschaft (i.e. non-war partnership).

Appropriate terminology: Some translations just don’t sound right or are not exactly to the point, like e.g. a person’s status which would be referred to as the Personenstand (civil or marital status) in German instead of simply saying status, which could be anything. A bailiff practice would be Gerichtsvollzieherbüro rather than Gerichtsvollzieherpraxis.

In the end, it always boils down to the same rules, which by the way apply to each and every minute of simultaneous interpreting (or looking up a word in any dictionary, even the most reliable one): Always look for the meaning of a text and constantly run plausibility checks.

About the author:
Anja Rütten is a freelance conference interpreter for German (A), Spanish (B), English (C) and French (C) based in Düsseldorf, Germany. She has specialised in knowledge management since the mid-1990s.

DeepL Sample Translations:

Original DE DeepL EN>DE Original EN DeepL DE>EN
20.12.2012    | DE | Amtsblatt der Europäischen Union | L 351/1 20.12.2012 | DE | Amtsblatt der Europäischen Union | L 351/1 20.12.2012    | EN | Official Journal of the European Union | L 351/1 20.12.2012 | EN | Official Journal of the European Union | L 351/1
VERORDNUNG (EU) Nr. 1215/2012 DES EUROPÄISCHEN PARLAMENTS UND DES RATES über die gerichtliche Zuständigkeit und die Anerkennung und Vollstreckung von Entscheidungen in Zivil- und Handelssachen VERORDNUNG (EU) Nr. 1215/2012 DES EUROPÄISCHEN PARLAMENTS UND DES RATES über die Zuständigkeit und die Anerkennung und Vollstreckung von Entscheidungen in Zivil- und Handelssachen REGULATION (EU) No 1215/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL  on jurisdiction and the recognition and enforcement of judgments in civil and commercial matters REGULATION (EU) No 1215/2012 OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL on jurisdiction and the recognition and enforcement of judgments in civil and commercial matters
vom 12. Dezember 2012 vom 12. Dezember 2012 of 12 December 2012 of 12 December 2012
(Neufassung) (Neufassung) (recast) (recast)
Artikel 1 Artikel 1 Article 1 Article 1
(1)   Diese Verordnung ist in Zivil- und Handelssachen anzuwenden, ohne dass es auf die Art der Gerichtsbarkeit ankommt. Sie gilt insbesondere nicht für Steuer- und Zollsachen sowie verwaltungsrechtliche Angelegenheiten oder die Haftung des Staates für Handlungen oder Unterlassungen im Rahmen der Ausübung hoheitlicher Rechte (acta iure imperii). 1.   Diese Verordnung gilt in Zivil- und Handelssachen unabhängig von der Art des Gerichts. Sie erstreckt sich insbesondere nicht auf Steuer-, Zoll- oder Verwaltungsangelegenheiten oder die Haftung des Staates für Handlungen und Unterlassungen in Ausübung staatlicher Gewalt (acta iure imperii). 1.   This Regulation shall apply in civil and commercial matters whatever the nature of the court or tribunal. It shall not extend, in particular, to revenue, customs or administrative matters or to the liability of the State for acts and omissions in the exercise of State authority (acta iure imperii). 1. This Regulation shall apply in civil and commercial matters, whatever the nature of the court or tribunal. In particular, it shall not apply to tax, customs or administrative matters or to the liability of the State for acts or omissions in the exercise of State authority (acta iure imperii).
(2)   Sie ist nicht anzuwenden auf: 2.   Diese Verordnung gilt nicht für: 2.   This Regulation shall not apply to: (2) It shall not apply to:
a) | den Personenstand, die Rechts- und Handlungsfähigkeit sowie die gesetzliche Vertretung von natürlichen Personen, die ehelichen Güterstände oder Güterstände aufgrund von Verhältnissen, die nach dem auf diese Verhältnisse anzuwendenden Recht mit der Ehe vergleichbare Wirkungen entfalten, a) den Status oder die Rechtsfähigkeit natürlicher Personen, Vermögensrechte aus einer ehelichen Beziehung oder aus einer Beziehung, die nach dem auf diese Beziehung anwendbaren Recht vergleichbare Wirkungen wie die Ehe haben; (a) | the status or legal capacity of natural persons, rights in property arising out of a matrimonial relationship or out of a relationship deemed by the law applicable to such relationship to have comparable effects to marriage; a) | the marital status, legal capacity, capacity to act and legal representation of natural persons, matrimonial property regimes or matrimonial property regimes on the basis of relationships which, under the law applicable to such relationships, have comparable effects to marriage,
b) | Konkurse, Vergleiche und ähnliche Verfahren, b) Konkurs, Verfahren im Zusammenhang mit der Liquidation insolventer Unternehmen oder anderer juristischer Personen, gerichtliche Vereinbarungen, Vergleiche und ähnliche Verfahren; (b) | bankruptcy, proceedings relating to the winding-up of insolvent companies or other legal persons, judicial arrangements, compositions and analogous proceedings; (b) bankruptcies, settlements and similar proceedings,
c) | die soziale Sicherheit, (c) | Sozialversicherung; (c) | social security; c) Social security,
d) | die Schiedsgerichtsbarkeit, (d) | Schiedsverfahren; (d) | arbitration; (d) arbitration,
e) | Unterhaltspflichten, die auf einem Familien-, Verwandtschafts- oder eherechtlichen Verhältnis oder auf Schwägerschaft beruhen, (e) Unterhaltspflichten, die sich aus einer familiären Beziehung, Abstammung, Ehe oder Verwandtschaft ergeben; (e) | maintenance obligations arising from a family relationship, parentage, marriage or affinity; e) | Maintenance obligations based on a family, relationship or marriage law relationship or on affinity,
f) | das Gebiet des Testaments- und Erbrechts, einschließlich Unterhaltspflichten, die mit dem Tod entstehen. (f) Testamente und Erbfolge, einschließlich Unterhaltspflichten, die sich aus dem Tod ergeben. (f) | wills and succession, including maintenance obligations arising by reason of death. (f) the field of wills and succession, including maintenance obligations arising from death.
Artikel 2 Artikel 2 Article 2 Article 2
Für die Zwecke dieser Verordnung bezeichnet der Ausdruck Für die Zwecke dieser Verordnung: For the purposes of this Regulation: For the purposes of this Regulation, the following definitions shall apply
a) | „Entscheidung“ jede von einem Gericht eines Mitgliedstaats erlassene Entscheidung ohne Rücksicht auf ihre Bezeichnung wie Urteil, Beschluss, Zahlungsbefehl oder Vollstreckungsbescheid, einschließlich des Kostenfestsetzungsbeschlusses eines Gerichtsbediensteten. | Für die Zwecke von Kapitel III umfasst der Ausdruck „Entscheidung“ auch einstweilige Maßnahmen einschließlich Sicherungsmaßnahmen, die von einem nach dieser Verordnung in der Hauptsache zuständigen Gericht angeordnet wurden. Hierzu gehören keine einstweiligen Maßnahmen einschließlich Sicherungsmaßnahmen, die von einem solchen Gericht angeordnet wurden, ohne dass der Beklagte vorgeladen wurde, es sei denn, die Entscheidung, welche die Maßnahme enthält, wird ihm vor der Vollstreckung zugestellt; a) „Urteil“ ist jede von einem Gericht eines Mitgliedstaats ergangene Entscheidung, unabhängig von der Bezeichnung der Entscheidung, einschließlich eines Dekrets, einer Anordnung, einer Entscheidung oder eines Vollstreckungsbescheides, sowie eine Entscheidung über die Bestimmung der Kosten oder Ausgaben durch einen Beamten des Gerichts. | Für die Zwecke von Kapitel III umfasst das „Urteil“ vorläufige, einschließlich Schutzmaßnahmen, die von einem Gericht angeordnet werden, das nach dieser Verordnung in Bezug auf den Inhalt der Angelegenheit zuständig ist. Sie umfasst keine vorläufige, einschließlich schützende Maßnahme, die von einem solchen Gericht angeordnet wird, ohne dass der Beklagte vorgeladen wird, es sei denn, das die Maßnahme enthaltende Urteil wird dem Beklagten vor der Vollstreckung zugestellt; (a) | ‘judgment’ means any judgment given by a court or tribunal of a Member State, whatever the judgment may be called, including a decree, order, decision or writ of execution, as well as a decision on the determination of costs or expenses by an officer of the court. | For the purposes of Chapter III, ‘judgment’ includes provisional, including protective, measures ordered by a court or tribunal which by virtue of this Regulation has jurisdiction as to the substance of the matter. It does not include a provisional, including protective, measure which is ordered by such a court or tribunal without the defendant being summoned to appear, unless the judgment containing the measure is served on the defendant prior to enforcement; (a) ‚decision‘ means any decision given by a court or tribunal of a Member State, whatever the judgment may be called, such as a judgment, order, order for payment or enforcement order, including the determination of costs and expenses by an officer of the court. | For the purposes of Chapter III, the term „decision“ shall also include provisional, including protective, measures ordered by a court having jurisdiction as to the substance of the matter under this Regulation. Such measures shall not include provisional, including protective, measures ordered by such a court without the defendant having been summoned, unless the decision containing the measure is served on him before enforcement;
b) | „gerichtlicher Vergleich“ einen Vergleich, der von einem Gericht eines Mitgliedstaats gebilligt oder vor einem Gericht eines Mitgliedstaats im Laufe eines Verfahrens geschlossen worden ist; b) „Gerichtsvergleich“ ist ein Vergleich, der von einem Gericht eines Mitgliedstaats genehmigt oder im Laufe des Verfahrens vor einem Gericht eines Mitgliedstaats geschlossen wurde; (b) | ‘court settlement’ means a settlement which has been approved by a court of a Member State or concluded before a court of a Member State in the course of proceedings; (b) „court settlement“ means a settlement approved by a court of a Member State or concluded before a court of a Member State in the course of proceedings;
c) | „öffentliche Urkunde“ ein Schriftstück, das als öffentliche Urkunde im Ursprungsmitgliedstaat förmlich errichtet oder eingetragen worden ist und dessen Beweiskraft | i) | sich auf die Unterschrift und den Inhalt der öffentlichen Urkunde bezieht und | ii) | durch eine Behörde oder eine andere hierzu ermächtigte Stelle festgestellt worden ist; c) „öffentliche Urkunde“ ist ein Dokument, das im Ursprungsmitgliedstaat formell erstellt oder als öffentliche Urkunde eingetragen wurde und dessen Echtheit: (i) | bezieht sich auf die Unterschrift und den Inhalt des Instruments; und | (ii) | (ii) | wurde von einer Behörde oder einer anderen zu diesem Zweck befugten Behörde eingerichtet; (c) | ‘authentic instrument’ means a document which has been formally drawn up or registered as an authentic instrument in the Member State of origin and the authenticity of which: | (i) | relates to the signature and the content of the instrument; and | (ii) | has been established by a public authority or other authority empowered for that purpose; (c) „authentic instrument“ means a document which has been formally drawn up or registered as an authentic instrument in the Member State of origin and the probative value of which relates to the signature and the content of the authentic instrument and which has been established by an authority or other authority empowered to that effect;
d) | „Ursprungsmitgliedstaat“ den Mitgliedstaat, in dem die Entscheidung ergangen, der gerichtliche Vergleich gebilligt oder geschlossen oder die öffentliche Urkunde förmlich errichtet oder eingetragen worden ist; d) „Herkunftsmitgliedstaat“ ist der Mitgliedstaat, in dem gegebenenfalls die Entscheidung ergangen ist, der gerichtliche Vergleich genehmigt oder geschlossen wurde oder die öffentliche Urkunde formell ausgestellt oder eingetragen wurde; (d) | ‘Member State of origin’ means the Member State in which, as the case may be, the judgment has been given, the court settlement has been approved or concluded, or the authentic instrument has been formally drawn up or registered; (d) „Member State of origin“ means the Member State in which the judgment has been given, the court settlement approved or concluded, or the authentic instrument formally drawn up or registered;
e) | „ersuchter Mitgliedstaat“ den Mitgliedstaat, in dem die Anerkennung der Entscheidung geltend gemacht oder die Vollstreckung der Entscheidung, des gerichtlichen Vergleichs oder der öffentlichen Urkunde beantragt wird; e) „ersuchter Mitgliedstaat“ ist der Mitgliedstaat, in dem die Anerkennung der Entscheidung geltend gemacht wird oder in dem die Vollstreckung der Entscheidung, des Gerichtsverfahrens oder der öffentlichen Urkunde angestrebt wird; (e) | ‘Member State addressed’ means the Member State in which the recognition of the judgment is invoked or in which the enforcement of the judgment, the court settlement or the authentic instrument is sought; (e) „requested Member State“ means the Member State in which recognition of the judgment is sought or enforcement of the judgment, the court settlement or the authentic instrument is sought;
f) | „Ursprungsgericht“ das Gericht, das die Entscheidung erlassen hat, deren Anerkennung geltend gemacht oder deren Vollstreckung beantragt wird. f) „Ursprungsgericht“ ist das Gericht, das dem Urteil, dessen Anerkennung geltend gemacht oder dessen Vollstreckung angestrebt wird, zugestimmt hat. (f) | ‘court of origin’ means the court which has given the judgment the recognition of which is invoked or the enforcement of which is sought. (f) „court of origin“ means the court which delivered the judgment, the recognition of which is sought or the enforcement of which is sought.