The Ergonomics for the Artificial Booth Mate (EABM) project conducted at the University of Ghent has been catching my attention for a while, with all these experiments on cognitive load in the booth run by Bart Defrancq. The CAI tools associated with the project provide live-prompting of terms and numbers in the booth along with a complete transcript of the source text.
So once more, the AI workstream of AIIC’s science hub met for a CAI tool test session, this time with Bart Defrancq, who gave us a live demo and patiently answered even the strangest questions.
Here’s what the current version of the tool looks like:
Key facts about your glossaries
The requirements for glossaries to be uploaded into the system are the usual “one word per cell” and “no extra stuff apart from the very word you want the system to recognise” rules, so synonyms, comments etc. need to go into separate lines.
You can upload glossaries and select them for the tool to recognise and display terms.
The glossary cannot be edited in real time while the transcript is running.
Only one glossary can be selected for a session, but the glossaries can be very long (experience has shown that glossaries including over 3000 items wouldn’t slow down the system).
For the sake of efficiency and speed, no resources other than the uploaded glossary (e.g. online dictionaries) can be consulted.
Term recognition
Interestingly, the term recognition module has a nice balance between a too generous fuzzy matching, which could produce a lot of noise and display many irrelevant terms, and a very rigid 100%-matching rule, which would only show terms exactly as they appear in the glossary. In fact, this tool uses a grammar module in order to recognise singular and plural forms as well as conjugated verb forms. In order for this to work, the glossary needs to contain a part-of-speech tag, telling the system if the word in question is a noun or a verb.
Supported languages
The tool currently supports six European languages as an input, i.e. English, French, German, Spanish, Italian, and Dutch. More languages could theoretically be implemented, subject to sufficient funding. The target language can be any: whatever information in whatever language the third column in the glossary contains will be displayed. So even though the input, i.e. source language, is restricted to six languages, you may have support for the language you are working into, even if it is Chinese, Russian, Vietnamese or any other language.
Settings
You need to select an input language from English, French, German, Spanish, Italian, or Dutch before you start working. As explained above, your target or output language may be any, the terms displayed will just be the ones that figure in the third column.
The tool offers many options to adjust the prompting to match your preferences. You can adjust how many seconds each displayed term remains on the screen and the number of elements displayed, disable the running transcript altogether if you find it too distracting, let the system detect whether you are interpreting at all (“voice activity detection”), have the transcript delayed until it has captured a complete sentence, so that you don’t have to read the auto-correction – although this can be tricky because it tends to exceed our usual decalage – and you can adjust the number of words you want to get in one go (chunks) in the running transcript (“chunk buffer size”).
Terms are not repeated if they appear several times in subsequent sentences, i.e. the tool will not show you the same term over and over again for up to three sentences. The term will re-appear on your screen only in the fourth sentence after its first occurrence.
The new version
And if this wasn’t enough, there is already a new version around:
And this new version brings a major improvement for all of us who work from more than one language: it automatically recognises the source language, which can still be Dutch, English, French, German, Italian, or Spanish. The target or output language needs to be selected from those same languages.
Second major improvement in this new version: Glossaries can include more than two languages, the tool will find the correct language automatically.
Of course, it would be nice to see more, especially non-European, languages supported by the tool. But at least it can be used as a monolingual transcription and support tool, as long as the source language is always one of EN, FR, DE, IT, ES, NL and only the target language spoken by the interpreter is a non-supported language. In this case, even a further workaround might be applied: The terms of the non-supported language, e.g. Russian, could be included into the glossary under a different “fake” heading, for example German, as long as this is only used as a target and not as a source language. When selecting German as a target or output language, the Russian terms would then be shown (credits to my colleagues Magda and Michelle from the AI workstream for coming up with this brilliant idea!).
Technicalities
Currently both the old and the new system only work for onsite meetings, not for videoconferences. It can be connected to a console using a double mini jack. You can perfectly run it on your own booth laptop if you don’t mind a crowded screen.
Both versions are currently running on a safe, local university server in Gent, the underlying LLM for speech recognition being WhisperAI. For the tool to be used outside the university, it could be installed on any other server and connected to any speech-to-text machine available on that server. For such an installation, a subscription fee would be charged. Anyone interested, feel free to get in touch with Bart!
————————–
About the author:
Anja Rütten has specialised in tec, information and terminology management since the mid-1990s. She holds a professorship in interpreting studies and Computer-Aided Interpreting at the Cologne University of Applied Sciences.
——————-
Disclaimer:
Views or opinions expressed are solely my own and do not express the views or opinions of my employer.
Leave a Reply