185TRANSLATION AND CONTRASTIVE STUDIES
Silvana Orel Kos
University of Ljubljana, Slovenia
2024, Vol. 21 (1), 185-208(316)
journals.uni-lj.si/elope
https://doi.org/10.4312/elope.21.1.185-208
UDC: 81'322.4:378
Introduction of Machine Translation into 
Audiovisual Translation Teaching
ABSTRACT
Subtitling has long been the primary type of audiovisual translation in Slovenia. A steady 
increase in the production of audiovisual content, including English-speaking movies and 
TV series, along with the development of machine translation, have seen a growing need and 
opportunity for subtitling globally. The Department of Translation Studies (University of 
Ljubljana) has offered courses in subtitling to MA students for over twenty years. In the course 
of this time, teaching methods have been adapted to embrace new technologies. Recently, 
students have shown interest in incorporating machine translation into the subtitling course. 
The paper presents an analysis of a subtitling assignment in which students were asked to 
post-edit DeepL’s Slovene translation of English formatted movie subtitles. The post-edited 
subtitles were compared with the subtitles done by a group of students from scratch. The 
results show considerable differences in the students’ multimodal awareness and the overall 
quality of the subtitles between the two groups.
Keywords: subtitling, AVT teaching, machine translation, post-editing, multimodal 
awareness
Uvajanje strojnega prevajanja v poučevanje 
podnaslovnega prevajanja
IZVLEČEK
Podnaslovno prevajanje je primarna zvrst avdiovizualnega prevajanja v Sloveniji. Zaradi 
razmaha ustvarjanja avdiovizualnih vsebin, ki vključujejo filme in nadaljevanke v angleščini, 
in razvoja strojnega prevajanja se po vsem svetu večajo potrebe in možnosti za podnaslovno 
prevajanje. Oddelek za prevajalstvo Univerze v Ljubljani že več kot dvajset let nudi študentom 
magistrskega študija seminarje iz podnaslavljanja. V tem času so metode poučevanja sledile 
novim tehnološkim možnostim. Študentje v zadnjih letih kažejo zanimanje za vključevanje 
strojnega prevajanja v seminar podnaslavljanja. Članek predstavlja analizo naloge iz 
podnaslavljanja, v kateri so študentje popravili in uredili slovenski prevod angleških filmskih 
podnapisov, ki ga je ustvaril prevajalnik DeepL. Tako urejene podnapise smo primerjali s 
podnapisi, ki jih je druga skupina študentov ustvarila v celoti. Izsledki kažejo precejšnje razlike 
v multimodalnem zavedanju in nasploh kakovosti podnapisov med študenti obeh skupin.
Ključne besede: podnaslavljanje, poučevanje avdiovizualnega prevajanja, strojno prevajanje, 
popravljanje strojnih prevodov, multimodalno zavedanje
186 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
1 Introduction
New technologies have seen an increase in new audiovisual media services over the last 
twenty years. According to the Audiovisual Media Services Directive issued by the European 
Commission in 2010, audiovisual media services are defined as “mass media in their 
function to inform, entertain and educate the general public”, including related non-private 
commercial communication, while excluding subsidiary occurrences of audiovisual elements 
in advertising and gambling activities (European Commission 2010, 3). The Directive 
highlights the educational impact of the audiovisual media on their users. In fact, in the years 
prior to the adoption of the Directive, spanning the two EU enlargements in 2004 and 2007, 
the European Commission published several documents in which it stressed the importance 
of developing multilingualism via language learning methods.
In its 2008 Communication on Multilingualism, the European Commission conveyed its 
belief that “[t]he media, new technologies and human and automatic translation services 
can bring the increasing variety of languages and cultures in the EU closer to citizens and 
provide the means to cross language barriers” (European Commission 2008, 12), stressing 
that languages can be informally learned via the media. These services “can thus also be a great 
source of informal language learning through ‘edutainment’ and subtitled films” (European 
Commission 2008, 12). To promote this objective, the Commission has supported subtitling 
and invited the Member States to encourage it. Two years later, in 2010, the Commission 
launched a 12-month project in 33 European countries aimed at exploiting the true potential 
of subtitling to encourage language learning and strengthen foreign language competence. 
While the final report waters down the potential of subtitling in this context, it concludes 
that subtitling helps to “improve the mastery of foreign languages”, can “raise awareness 
and provide motivation for language learning” in pupils, students and immigrants, with 
multilingualism and higher education being as such indicative of a preference for subtitling 
over dubbing (Media Consulting Group 2011, 26). Despite the Commission’s call for 
linguistic diversity, the study showed the prevalence of North American film production and, 
consequently, the prevalence of English over other languages. The Commission recommended 
the use of subtitling as an educational tool in both the expected educational environments, 
i.e., (secondary) schools, (all) universities and language teaching organizations, as well as in 
actions by broadcasters. One of the Commission’s strategic recommendations was not only 
to use subtitles to support language teaching and learning, but also to encourage “students 
and pupils” to create subtitles (cf. Media Consulting Group 2011, 28). The underlying ease 
of this proposal is surprising since it takes a whole MA course to master the theoretical and 
practical rudiments of subtitling. Such a recommendation may lead to the reinforcement 
of the stereotype that subtitling does not require any special training while, paradoxically, 
viewers will, deservedly or not, violently criticize the quality of TV subtitles. Last but not 
least, the Commission recommended involvement of “media professionals to develop and/
or make available quality European films in subtitled versions”, especially in languages 
of limited scope, as well as to encourage especially young adults in predominantly non-
subtitling countries to watch subtitled programmes (Media Consulting Group 2011, 29). 
The Commission’s recognition of the educational scope of subtitled programmes has been 
seconded by the rise of new audiovisual media services.
187TRANSLATION AND CONTRASTIVE STUDIES
In line with the recommendations above, the European Commission legally regulated the 
nature and provision of the services with the Audiovisual Media Services Directive in 2010. 
Directive 2010/13/EU stipulated the accessibility of audiovisual media services to ensure 
“freedom of information, diversity of opinion and medial pluralism” and that the provisions 
of programmes should also cover accompanying text-based content, “such as subtitling 
services and electronic programme guides” (European Commission 2010, 3). The Directive 
was amended in 2018 to include programmes and user-generated content shared via social 
media services (European Commission 2018, 69–70), summoning the EU Member States 
to make audiovisual content immediately accessible especially to persons with visual and 
hearing impairments, with subtitling being again listed as one of the features to this end.
The European Parliament adopted a report in early 2023 in which it urged the Member 
States to facilitate accessibility through dubbing, subtitling, audio description or other means 
in all languages, both official and regional and minority languages (Kammerevert 2023). The 
onset of neural machine translation and large language models did not go unnoticed, as the 
2023 report brings up the “potential of using artificial intelligence” as the implementation 
tool and “calls on the Commission and the Member States to promote this in a strategic 
and targeted manner” (Kammerevert 2023, 13). More specifically, artificial intelligence (AI) 
should be used in the production and distribution of the AV content to promote freedom 
of expression, the exchange of information and “detect illegal content” (Kammerevert 2023, 
29–30). In fact, the European institutions have been using AI to promote their video content, 
while the EU web portal Europeana has offered high-quality video clips on European heritage 
to both professionals and the general public to proofread their AI-generated subtitles and 
captions, notably through a resource developed by the 2021–2022 Europeana Subtitled 
project (Europeana, n.d.).
2 Audiovisual Machine Translation
While research on machine translation (MT) started in the USA in the 1950’s, it was 
only around the turn of the millennium, about ten years after a dramatic rise in academic 
research on audiovisual translation (AVT), that the technical, rather than semantic, aspects 
of subtitling drew the attention of MT researchers.1 Although software developers would 
present their findings as successful and promising (see Popowich et al. (2000) for the English-
Spanish language combination, Armstrong et al. (2006) between German and English), the 
training material was ill-suited to the context, and the project results regarding semantic 
accuracy were vague. Nevertheless, the US company Global Translation Inc. still provided 
live machine translation of audiovisual (AV) content in eight languages, although Díaz Cintas 
(2005, 20–21) noted the dubious quality of its product, suggesting that the introduction of 
the tool was an economically driven enterprise.
2.1 Statistical Machine Translation in Subtitling
Modern MT has made tremendous leaps in quality since the introduction of the first open-
source statistical MT engines in the mid-2000’s, most notably the Google Translate service 
1 For a detailed survey of MT development of subtitling, see Bywood et al. 2017.
188 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
in  2006. The first batch of 30 languages supported by Google Translate was headed by 
English, which has also functioned as a relay language for most of the GT languages. Slovene 
was added in 2008, becoming one of the first 40 GT-supported languages. Koletnik Korošec 
(2011, 16–17) provides useful insight into the response of Slovene students of translation 
to GT-generated texts, and discusses the didactic potential of MT. Also supported by the 
findings in Koletnik Korošec (2011), the students were aware of MT’s limitations at the 
time. Unless used in predictable textual environments, early GT-generated texts were derided 
as a source of comical linguistic transformations, occasionally making the service a toy rather 
than a tool. Simultaneously, research on subtitling using statistical MT was developed to 
incorporate increasing huge training corpora. Once again, the methodology was debateable, 
as (due to copyright protection) the materials would be collected from freely available subtitle 
files on the internet, as done by Tiedemann (2007). Sadly, the subtitles were mostly produced 
by amateur subtitlers with insufficient linguistic and subtitling competences. 
Volk (2009) built a statistical MT system for subtitling from English between Swedish and 
Danish, and had automatically produced subtitles post-edited by professional translators, 
who showed different tolerances to the quality of the subtitles, and noted that quality varied 
across film genres. His findings showed that the system was able to achieve good results 
with a large high-quality parallel corpus, and even though the automatic translation quality 
was evaluated differently, the customer was convinced of the system’s benefits and started to 
employ the system. 
Volk and Harder (2007, [1]) saw the direct written representations of spoken discourse in 
subtitles as less than ideal, and highlighted the need for textual condensation in subtitling. In 
the beginning, subtitles were perceived as easily trainable material, since they were regarded 
as short, syntactically simple, repetitive textual units embedded in comfortably alignable 
timecodes. Working with corpora has shown that since subtitles feature a wide range of 
genres and elements of literary language (register, wordplay, poetry) and a variety of topics 
and styles, movie subtitling is closer to the translation of fiction, especially plays, rather than 
non-fiction.
The use of statistical MT was explored by the SUMAT (SUbtitling for MAchine Translation) 
project between 2011 and 2014. The aim of the EU project was to develop a commercial 
online post-edited subtitling service which would be used with AV programmes broadcast by 
public TV networks. The system developers pointed out that the expansion of the market was 
hindered by issues “such as cost, time and quality” (Petukhova et al. 2012, 21; Sepesy Maučec 
et al. 2012, 167), and aspired to higher productivity for lower economical remuneration. 
The project involved nine European languages, with translation between English and Dutch, 
French, German, Portuguese, Spanish, and Swedish, while the Serbian-Slovene pair was chosen 
as “a test-case of an under-resourced language pair” (Etchegoyhen et al. 2014, 46). The training 
corpora included aligned parallel subtitles, most of which were collected from the unprofessional 
OpenSubs website. Despite the good results on objective ratings and an average gain of almost 
40% in productivity, the post-editing process was not evaluated positively by professional 
AV translators, whom the authors referred to as “a key aspect for any eventual adoption of 
machine translation technology in professional subtitling” (Etchegoyhen et al. 2014, 46). 
189TRANSLATION AND CONTRASTIVE STUDIES
Even though the service was aimed at professional subtitling, the authors would strive to further 
improve the quality of machine translation to reduce “the cognitive effort in post-editing 
machine translated text in the open subtitling domains” (Etchegoyhen et al. 2014, 46, 52).
Providing another view of the SUMAT project, Bywood, Georgakopoulou and Etchegoyhen 
(2017, 502–3) aim at a more energetic integration of the MT technology into the subtitling 
workflow. They see the need for the creation of a new profile – “subtitle post-editor” – with 
either the “existing subtitlers” being trained to meet the needs of post-editing, or training 
existing post-editors in subtitling skills (Georgakopoulou and Bywood, 2014, 28 in Bywood 
2017, 502), “to seek out people whose skills and experience include high levels of attention to 
detail and possibly a tolerance for repetitive work and train them specifically in subtitle post-
editing” and to make postgraduate AVT courses embrace the need for post-editing.2 This 
alienating corrective rather than inspiring creative focus on translation opens a bleak future for 
the AVT profession.
2.2 Neural Machine Translation in Subtitling
The mid-2010’s marked a quantitative leap in machine translation. Artificial neural networks 
are now capable of predicting the likelihood of syntagmatic and paradigmatic structures in text 
production. The new approach has introduced new terminology which takes the mechanical 
imagery to organic images: machines have been increasingly thought of as forms of (artificial) 
intelligence, the notion of externally induced training has been replaced by deep learning, 
while service development has been rephrased as AI evolution. The human-like interface was 
further enhanced by the introduction of ChatGPT in late 2022, a generative pre-trained 
transformer, developed by OpenAI. ChatGPT makes use of deep learning architectures 
and large learning language models, and is often presented as a chatbox mimicking human 
conversation. 
As of 2017 there have been two open-source neural machine translation services which 
have gained most popularity: Google Translate and DeepL. The latter is a Germany-based 
machine translation provider, which added support for a dozen more European languages in 
early 2021, when Slovene became one of its 26 languages. Despite the Department’s initial 
discouragement with regard to employing AI services in postgraduate translation classes, the 
students’ accidental copying of MT sources into their assignments revealed they were using 
and post-editing the MT texts as best as they could. The recent leap forward in neural machine 
translation coincided with the COVID-19 pandemic, which upset the regular educational 
patterns between early 2020 and mid-2021. Other than the deplorable lack of physical in-
class experience, which is conducive to the process of human learning, the pandemic did 
not interfere with the very technical aspects of AVT teaching. Indeed, it may have been the 
greater reliance on digital technologies during the two years of studying from home that 
sparked a stronger interest in the use of MT among (technology-minded) students, especially 
the promising solutions from DeepL. 
2 See Mezeg (2023) for a discussion on the needs of developing post-editing skills in students of translation. 
190 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
Indeed, DeepL has proven to be semantically more accurate than Google Translate, and 
recent developments in AI technologies have further improved the generation of automatic 
or real-time intralingual and interlingual subtitles (Poibeau 2023). The last few years have 
seen several new platforms that allow automatic (or manual) transcription and translation of 
video audio, such as Descript, Matesub, Rev, Sonix, Whisper. Online subtitle services claim 
they provide “professional” video editing software to modify the content and export files in 
industry standard formats. The services advertise almost a 100% accuracy, even though their 
platforms are integrated with Google Translate, which has proven inferior to DeepL. The 
option of (post-)editing by such platforms, however, does not differ greatly from that when 
using the free software application Subtitle Edit, aimed at subtitle creation according to user-
defined time and space restrictions.
The quality of the original English subtitle files that (Slovene) professional AV translators are 
given as templates has been declining in recent years. The extreme lengths of lines containing 
the entire utterance, hardly manageable reading speeds, occasionally misheard words and 
awkward time spotting may indicate that such files have been automatically generated without 
observing any high-quality subtitling standards. Editing such files to meet the intended 
subtitling standards can be more time consuming than creating subtitles from the first step.
Digital translation resources used to be seen as tools in the hand of a translator, but now they 
appear to have been getting out of (their) hand. AI language models have been superseding 
manual translation, the quality scope however, has been limited to informative, repetitive, 
preferably (well)-written discourse for a limited number of language pairs. Poibeau (2022, 
6018) reassesses machine translation in terms of human intelligence and shows that “human 
parity” and “super human performance” is possible for a limited number of languages, text 
types and “with very literal translations”.
Recent studies on automatic subtitling tools presented in the Findings of the IWSLT 2023 
Evaluation Campaign show that “automatic solutions do not reach the level of quality that 
is necessary in subtitling”, and observe that it is TV series that pose greatest challenges for 
automatic subtitling (Agarwal 2023, 13–14, 15), although such challenges also apply to movies.
Research that shows awareness of professional subtitling standards thus reveals persistent 
deficiencies in MT systems, most notably in being unable to tackle the multimodal nature of 
AV texts. If anywhere, it is in high-quality subtitling that generative transformers may or will 
develop systems to achieve communicatively suitable and readable subtitles sensitive to the 
relation between speech, image and different levels of contextual meaning.
3 Methodology
Ljubljana MA students are invited to use Subtitle Edit (SE),3 a free and open-source subtitle 
editor developed by the Danish IT expert Nikolaj Olsson. The programme has a built-in 
machine translation option with Google Translate V1 API. In addition, Olsson has developed 
his own machine translation programme from Swedish to Danish, i.e., two languages that are 
3 The subtitling software is available at https://www.nikse.dk/subtitleedit.
191TRANSLATION AND CONTRASTIVE STUDIES
linguistically and culturally related. Even so, Olsson stresses the need for manual correction of 
the automatic translation. As the quality of Google Translate for translation between English 
and Slovene does not reach professional standards, the students of subtitling have not been 
encouraged to use the in-built feature.
During the academic year 2022/23, ten first-year MA students enrolled in my 60-hour 
seminar in AVT of movies and series and some assignments in other genres, i.e., animated 
films for children and documentaries. The group consisted of eight native Slovene speakers 
and two students who had lived outside Slovenia, both speakers of another Slavonic language. 
Except for the bilingual student with some video fan-subtitling experience, the students had 
no prior experience in subtitling. Students were first given some assignments through which 
they gradually built their AVT competence from the beginning.
One third into the seminar they were given a subtitling assignment based on a translation 
by DeepL Translator.4 The students were asked to post-edit a four-minute passage from the 
2014 romantic comedy And So It Goes directed by the American director Rob Reiner, starring 
Michael Douglas and Diane Keaton. The film was critically panned for the clichéd script that 
did a disservice to the great performances by the leading actors, while careful commentators 
criticized the film’s rating for foul and inappropriate language, sexuality and substance abuse. 
Despite the poor reviews upon the movie’s release, the dialogues provide worthwhile subtitling 
challenges. Prior to the MT home assignment, the students had been working on the 
film, which would help them to consider appropriate subtitling strategies for the genre of 
romantic comedy. They had become acquainted with the story’s cultural background, the 
psychological profiles of the main characters, the interpersonal relations (professional and 
social relations between the characters spanning three generations), as well as situation-specific 
communication. The students had gathered some experience in observing textual segmentation 
and adaptation as well as acquired translation strategies that covered stylistic features of register, 
informal conversation, technical language, child’s language, emotional language, idiomaticity, 
humour, sarcasm, cultural specificity (localization), political (in)correctness, mild profanity 
and song translation. While these technical, stylistic and pragmatic aspects of subtitling should 
be crucial to any language pair, Slovene AV translators also have to observe the complexity of 
Slovene morphology. They may need to express features that in English remain unmarked, 
such as grammatical indicators of gender, number, and formality. The relevant information 
may be found in the immediate context or visuals, or elsewhere in the movie.
The students were given English time-coded subtitles containing entire speeches, except for 
very few minor omissions and graphic texts. They were asked to form a maximum of two-line 
subtitles with one speaker per line and observe the (currently valid) national TV parameters of 
a maximum of 35 characters per line and 12 characters per second for two-second+ subtitles 
to allow for a comfortable reading speed of large numbers of viewers watching the PG-13 
movie. They were asked to consider the subtitling strategies that they had been acquiring and 
improving in previous assignments and to deploy new strategies as necessary. The students 
were asked to run the text through DeepL Translator, post-edit the translation in accordance 
4 The web-based translation service is available at https://www.deepl.com/translator. 
192 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
with the instructions given above and comment on their experience. The results of the 2023 
“MT group” (with subtitles provided by a maximum of ten students)5 were compared with the 
results of the 2021 “non-MT” group (with subtitles provided by a maximum of 15 students).6 
The “non-MT” students had received the same instruction in AVT and were given the same 
assignment, with the four-minute passage split into two parts. The students of the “non-MT” 
group were asked to translate the passage on their own, i.e., without machine translation.
The four-minute passage from And So It Goes contains four scenes: (1) the “Paintball” 
scene, (2) the “Landlord” scene, (3) the “Compassion” scene, and (4) the “Real Estate” 
scene. Scene 1 features an emphatic young man’s words of consolation to a stray dog. The 
man’s (rhetorical) question is a cue that opens Scene 2, which pivots around the self-centred 
real estate agent and owner of the building, Oren Little. The hum of overlapping multiple 
speakers indulging in small talk becomes clearer as some tenants accuse Oren of rude and 
selfish behaviour. In Scene 3, Leah’s comment on Oren’s lack of compassion develops into 
an embittered widower’s slow-paced monologue. Following some phatic exchanges, Scene 4 
features a verbal confrontation between three real estate agents, all eloquently defending their 
positions. Their factual data are laden with real estate jargon and flanked by cutting retorts 
that combine marked vocabulary and complex figures of speech.
4 Analysis
The analysis studies the differences in translation decision-making between the MT-group 
and the non-MT group for the grammatical (examples 1–7), lexical (example 8) and stylistic 
categories (examples 9–12) which show the students’ mental processing in relation to the 
multimodally derived meaning. As Figure 1 shows, the percentages for all the categories that 
suggest some affinity with DeepL’s translation strategies are mostly considerably higher with 
the MT-group students than those with the MT-group students.
For the sake of illustrative comparison, the analysis includes comments on automatic 
translations by two automatic subtitling tools, Sonix7 and Matesub,8 added during the time 
of writing the paper. 
4.1 English Grammatical Polysemy
Based on or inspired by natural conversation, movie discourse, very much like drama, commonly 
takes the form of dialogue between the participants in the scene. As conversation generally 
turns on personal and interpersonal topics between the speakers, the discourse is marked by 
first-person and second-person references. The English pronominal system is syncretic and 
thus polysemic. Slovene, on the other hand, uses an extensive system of discrete grammatical 
5 One student, a non-native speaker of Slovene, failed to submit some subtitles.
6 The assignment was then divided into two parts. The first non-MT group consisted of twelve students, the second 
non-MT group of five students, of whom one bilingual student failed to submit some subtitles.
7 Sonix’s home page advertises their tool as “the best automated transcription software in 2024” and “the world’s most 
advanced automated transcription, translation, and subtitling platform” (Sonix, n.d.). The platform is available at 
https://sonix.ai.
8 The web-based subtitling tool is available at https://matesub.com. See Karakanta et al. 2022 for user feedback by 
subtitlers on Matesub.
193TRANSLATION AND CONTRASTIVE STUDIES
morphemes, such as pronominal and verbal inflexions that indicate the grammatical person 
for their gender, number and level of formality, while the predicator is further marked for 
tense, aspect and mood. As the Slovene pronominal-verbal system tends to be monosemic, the 
translator’s interpretation relies on contextual linguistic and extra-linguistic indicators. Poibeau 
(2022, 6020) states that most current MT systems operate at the sentence level. This paper 
shows that DeepL Translator does indeed have difficulties interpreting the context beyond the 
sentence limit, even beyond the line limit, which results in wrong interpretations. Although 
pronominal anaphora is a prominent discoursal feature in movie dialogues, interpretational 
problems with MT from English into Slovene may appear at the supra-sentence level in 
other types of discourse, see Mohar, Orthaber and Onič (2020, 134–35) for examples in the 
translation of poetic fiction by Google Translate and Translator.eu.
4.1.1 The Subject’s Gender
Pronominally, English reveals the Subject’s grammatical gender only with third-person 
references. Having informative finite verb inflections, Slovene omits the Subject pronouns in 
 
FIGURE 1. DeepL-like (mis)translation strategy: the ratios of DeepL-like translations for the numbered illustrative 
examples between the MT-group students and non-MT students. 
 
For the sake of illustrative comparison, the analysis includes comments on automatic translations by two automatic 
subtitling tools, Sonix7 and Matesub,8 added during the time of writing the paper.  
4.1 English Grammatical Polysemy 
Based on or inspired by natural conversation, movie discourse, very much like drama, commonly takes the form of 
dialogue between the participants in the scene. As conversation generally turns on personal and interpersonal topics 
between the speakers, the discourse is marked by first-person and second-person references. The English pronominal 
system is syncretic and thus polysemic. Slovene, on the other hand, uses an extensive system of discrete grammatical 
morphemes, such as pronominal and verbal inflexions that indicate th  grammatical pers n for their gender, number 
and level of formality, while the predicator is further marked for tense, aspect and mood. As the Slovene 
 
7 So ix's h me page advertises their tool as “the best automate transcrip ion software in 2024” and “the world’s 
most advanced automated transcription, translation, and subtitling platform” (Sonix, n.d.). The platform is 
available at https://sonix.ai. 
8 The web-based subtitling tool is available at https://matesub.com. See Karakanta et al. 2022 for user feedback 
by subtitlers on Matesub. 
0% 20% 40% 60% 80% 100%
Obscenity [12b]
Obscenity [12a]
Obscenity [11b]
Obscenity [11a]
Obscenity [10]
Slang phrasal verb [11]
Mild oath [10]
Idiom [9b]
Idiom [9a]
Manager [8b]
Manager [8a]
Mood [7]
Address [6a]
Number [5]
Number [4]
Number [3b]
Number [3a]
Number [2]
Gender [1]
MT Group Non-MT Group
Figure 1. DeepL-like (mis)translation strategy: the ratios of DeepL-like translations for the 
numbered illustrative examples betwee  the MT-group students and non-MT students.
194 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
unmarked communicative positions. The gender, however, is automatically revealed when the 
Subject is bound with predicative adjectival or participial grammatical features. Slovene past 
participles form periphrastic tenses, which, dependent on the narrative, are frequent in movies.
Whereas Poibeau (2022, 6020) finds that current MT systems choose one gender randomly, 
and are “thus often wrong”, this study shows that with non-third person singular references 
DeepL-generated translations, done via English, have shown a bias for the masculine 
grammatical gender.9 Although there are three female characters that present some speech 
in the subtitled passage there are only masculine Subject references outside the present-tense.
However, it may sound surprising that DeepL chose to translate some first-person sentences 
in the “Compassion” scene as if spoken by a female I-character. The MT translation may 
have been affected by the emotional content that is stereotypically attributed to women. 
The feminine gender was chosen even with two subsequent sentences in the same line. The 
Slovene translation of Oren Little’s little soliloquy is indicated by gender symbols following 
the English predicators: 
[1]  (Oren Little:)   
My wife died of cancer!
For two years, I bathed10 (M) her, I changed (M) her.
I cooked (M) for her. I fed (F) her.
I even prayed (F) for her.
And I cried (F) all I’m ever gonna cry (F).
I had (F) compassion. 
All the MT students that provided the subtitles for this scene post-edited the feminine 
participles or rephrased DeepL’s translations accordingly. Although this type of post-editing 
is not difficult, Slovene AV translators have to maintain a constant gender-oriented focus, 
while in non-automatic translation the attention can be delegated to other aspects of 
subtitling. Although automatic subtitling tools have been trained to account for the concept 
of Slovene grammatical gender, they may fail to maintain the focus beyond the sentence 
limit. It is interesting to observe that while Sonix’s automatic speech recognition discerns 
different speakers, the versions of Oren Little’s automatically translated monologue by both 
Sonix and Matesub reflect similar gender-specific emotional stereotypes as seen with DeepL. 
The gender-inflexion issue was not relevant to non-MT students, as there were sufficient 
multimodal sex/gender indicators that are easily spotted by human translators.
4.1.2 Grammatical Number
Grammatical number is a linguistic feature morphologically assigned to nouns, pronouns 
in both English and Slovene, as well as adjectives and participles in Slovene. While English 
9 The masculine gender bias may not be limited to the grammatical case: Mohar et al. (2020, 134) observes the gender-
related mistranslation in the Direct Object.
10 All the relevant features in the subtitles are highlighted by Orel Kos.
195TRANSLATION AND CONTRASTIVE STUDIES
is marked for singular and plural numbers, Slovene is the only official language among 
the EU member states to distinguish the dual, a category marking two entities. As movie 
narratives often address pairs of characters performing joint actions, AV translators need to 
observe when an English plural will imply two entities. The interpretation may be indicated 
or supported through the immediate image and sound, or derived from other multimodal 
information elsewhere in the movie. 
The subtitled passage contains three utterances with the pronoun we that neither DeepL, 
Sonix nor Matesub interpreted as containing the dual. The “Real Estate” scene contains an 
utterance expressed by Claire, an aging real estate agent siding with her retiring colleague 
Oren Little against her young grandson when the three of them meet in the office:
[2]  (Grandma:) 
We’ve been here since your grandfather started this business 44 years ago.
In the non-MT group only one (20%)11 (i.e., bilingual) student out of five students used the 
plural in Slovene. In the DeepL group five (56%) out of nine students retained the MT plural 
reference and four adapted it to dual. The ratio of interpretations with the wrong number is 
considerably higher with the DeepL group.
Interpretation of the “intended” grammatical number is made difficult when there is a mere 
pronoun-predicator agreement against the visuals with several entities, which happens in the 
“Landlord” scene. The first we reference comes from an African-American man conversing 
with other neighbours:
[3a]  (Tenant:) 
You know if he parked over just a little bit, we wouldn’t have to hunt for a space 
on the street. 
At this stage in the scene, the number suggested by the pronoun we may be perceived as a 
collective plural or open to speculation. A minute later, a foregrounded dialogue between the 
man and Oren Little offers a clearer interpretation:
[3b]   (Tenant:)   
My wife is going to give birth soon. Yesterday, we had to park a block away 
because you refuse to move your car just a few feet.
The dual concept in the pronoun we is hinted at by the man’s reference to his pregnant wife, 
while the rest of the neighbours sit listening in the background. None of the ten MT students 
post-edited the plural references in (a) or (b) to express the dual. As many as eight (80%) 
students retained DeepL’s plural references. Two students partially adapted the grammatical 
number, and failed to interconnect the two subscenes (a) and (b), applying a plural (a) + dual 
(b) and a less expected dual (a) + plural (b) combination respectively. In contrast, ten out of 
11 The percentages are given for the number of the non-MT students that provide DeepL-like translations and the MT-
students that retain DeepL's solutions.
196 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
twelve non-MT students applied dual to both utterances, while two non-MT students chose 
the partially contextualized switch from plural (a) to dual (b). No non-MT students rendered 
the English plural references as entirely plural in Slovene.
All students, the MT group included, had been instructed to check or pay attention to 
English plural references against the visuals and discourse or elsewhere in the movie. In an 
assignment preceding the MT task discussed in the paper, a scene with the visuals of some 
children included a similar plural/dual to dual combination of dialogues. The “MT” group 
students, who then translated from scratch, achieved more contextualized translations: five 
(50%) students out of ten fully recognized the dual references, while the “non-MT” group 
achieved even better results: eleven (78.5%) out fourteen students interpreted the English 
plural references as dual in Slovene.
4.1.3 Levels of Formality
Modern Slovene has three levels of address: formal address, informal address, and semi-formal 
address. The forms reflect vertical and/or horizontal relationships to a singular addressee. 
The formal v. informal opposition is the default system for both human dominance and 
distance. The grammatical features are marked pronominally and verbally. The formal address 
uses the second person plural and, where syntactically required, masculine, i.e., linguistically 
unmarked generic gender inflections. The semi-formal address is a combination of formal 
and informal grammatical features, it takes the second person plural finite forms and singular 
gender inflections for participles and adjectives. The semi-formal address regulates public 
and social distance, for example in customer services to imbue a formal conversation with 
some emotional proximity. It is used colloquially and is not normatively recognized. Unless 
presented as a character’s overt stylistic feature, this grammatical mixture is not tolerated in 
professional Slovene subtitles. Compliant with the genre and the character’s sociolinguistic 
profile, Slovene professional subtitles may use substandard vocabulary but no substandard 
grammatical features. Grammatically, all Slovene levels of address to singular referents are 
covered by the English pronoun you. The pronoun’s syncretic nature also accounts for Slovene 
dual and plural and their respective gender markers as well as non-Subject functions. 
English has no grammatical markers for the formal-informal opposition. Attitudes of 
deference are readily expressed lexically and allow for greater swings or immediate transitions 
in familiarity. The speakers’ swings in emotional proximity can be reflected in subtitles of 
inexperienced or incautious AV translators into Slovene who will switch between formal and 
informal addresses to accommodate the implied decrease or increase in social distance.
AV dialogues teem with polysemic you references, including those implied by the Imperative 
mood. The Slovene AV translator derives their (own) optimal multimodal interpretation 
from the number of people involved, social situations and the characters’ personal traits. 
The assignment’s automatic translations show that multimodally blind MT tools (DeepL, 
Sonix, Matesub) heavily rely on the lexical information in the clause when to choose between 
the second person singular and the second person plural with (generic) masculine markers. 
Firstly, basic conversation and simple familiar exchanges with you references, e.g., Are you 
OK? Hold up the towel!, are assigned the second person singular, suggesting that utterances are 
197TRANSLATION AND CONTRASTIVE STUDIES
aimed at an individual person. The register sensitive DeepL appears to employ the strategy 
systematically, Sonix shows a low informal-to-formal threshold, while Matesub seems to follow 
a similar lexis-based strategy, but there are substantial fluctuations. Secondly, utterances with 
you references that contain substandard or vulgar vocabulary correlate with a discrete person 
and a lack of human distance. The three automatic tools used the informal second person 
singular in Slovene to render the utterances that contain substandard vocabulary: Now what 
dumbass would shoot you with a paintball gun?, Something to think about next time you get an 
itch to rat me out?, Don’t be a smart-ass. Thirdly, utterances with you references that contain 
professional terminology and vocabulary that hints at a non-informal conversation correlate 
with the Slovene second person plural, which may either be interpreted as the formal address 
or an actual plural reference. The three automatic tools used the second person plural to 
render in Slovene the utterances that contain some possibly business jargon: Complain to the 
manager, I don’t care if you do own the building, How long have you been in the business?, 
You’ll get five-eight, not a penny more.
The three lexically-based strategies appear to have been systematically used by DeepL, 
producing the same translations in late 2022 and late 2023, when I had the text retranslated. 
DeepL would thus refrain from directing substandard vocabulary at several speakers or from 
using the formal address, while utterances with professional terminology are unlikely to be 
attributed to individuals with familiar interpersonal relationships.
The “Landlord” scene shows a conversation between Oren Little, a grumpy tenant who turns 
out to be the landlord, and a family man, who either speaks for himself or represents a cohort 
of dissatisfied tenants. Over a stretch of 40 seconds the following utterances by the landlord 
are directed at the family man:
[4]  (Oren Little:)
You called the manager on me. (Slovene: 2nd sg)12
[5]  (Oren Little:) 
Something to think about next time you get an itch to rat me out? (Slovene: 
2nd sg)
Nine out of twelve non-MT students used the second person plural with both utterances, 
interpreting the you references as either the formal address to refer to the family man or 
the second person plural to refer to the entire body of complaining tenants. Two non-MT 
students used a plural-to-singular switch and only one student (8%) used the informal 
singular address with both utterances suggesting that the old man is presented as a rude 
person. This personality trait can be supported by the man’s use of vulgar language and general 
attitude towards the young tenants, who are indignant at his inconsiderate parking habits. 
As translated by DeepL, both utterances used second person singular references in Slovene. 
Disregarding the fact that the narrative is open to several interpretations, five (50%) out of ten 
MT students followed the informal singular pattern provided by DeepL, three MT students 
used a plural-to-singular switch, and two MT students chose the second person plural with 
12 The brackets contain Slovene forms as suggested by DeepL.
198 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
both utterances. While the optimal interpretation is open to discussion, the results show that 
the interpersonal relationships were presented in radically different ways by the two groups 
of students. The comparable translations by Sonix and Matesub show the plural-to-singular 
switch. The second person plural in the first utterance probably derives from its rather neutral 
lexical features in comparison with the informal ones in the second.
The “Real Estate” scene contains a mini dialogue between the grandmother and her grandson, 
both working in the family real estate business. Although in Slovene extended family relations 
the formal address is (still) observed by younger generations speaking to older generations, the 
opposite direction has never been the case. In DeepL’s Slovene translation the grandmother 
and her grandson maintain a distanced relationship:
[6]
[6a]  (Grandma:) 
– How long have you been in the business? (Slovene: 2nd pl)13
 (Grandson:)    
– Seven years, as you well know. (Slovene: 2nd pl)
 (Grandson:)    
– Every day is better than the last.
[6b]  (Grandma:)   
– Don’t be a smart-ass. (Slovene: 2nd sg)
The non-MT group consisted of five students to subtitle this scene. Four students interpreted 
the grandmother’s statements informally, while one (bilingual) student (20%) used the 
DeepL-like switch from formal to informal register. Five MT students (including the 
bilingual and the non-native speaker) (50%) out of ten students chose the informal address. 
Three students took up DeepL’s formal-to-informal switch or may have forgotten to post-edit 
the MT formal interpretation in the first statement: a subtitle in the continuation shows that 
they were aware of the grandmother’s informal, patronizing attitude towards her grandson. 
Another two students disregarded the interpersonal relationship between the relatives and 
even hypercorrected DeepL’s interpretation to contain the formal address in all statements by 
the grandmother to her grandson.
The “Real Estate” scene results suggest that DeepL’s translation may diminish the AV 
translator’s attention to contextual meaning or multimodal awareness. A comparison of the 
results for both scenes shows that it was the same five students who disregarded the context 
in the “Real Estate” scene that would also follow DeepL’s terms of address in the “Landlord” 
scene and elsewhere in the assignment. This suggests the students’ general tendency to rely on 
DeepL’s automatic grammaticalization of the interpersonal relationships.
Even though the translation was barely coherent in places, Matesub’s automatic translation 
observed the same strategy as DeepL, while Sonix, having a low informal-to-formal threshold, 
used the formal address with all three relevant utterances. 
13 The brackets contain Slovene forms as suggested by DeepL.
199TRANSLATION AND CONTRASTIVE STUDIES
4.2 Lexis and Style
MT systems have shown a tendency towards direct translation. Dependent on contextual 
interpretation, there are very few fully matching AV translation units between English and 
Slovene. This study shows that students using DeepL may be lulled into disregarding both 
lexico-grammatical and macrocontextual acceptability. This is indicated by their insufficient 
post-editing. The lexico-grammatical acceptability of the MT translations is lower than with 
the non-MT students, even though both groups had been receiving the same assignments 
and comparable teaching instructions. The differences may be generational – also in the sense 
that each new generation of students is more inclined to rely on MT. The following example 
will show the two groups’ different degrees of tolerance to certain lexico-grammatical features.
At the beginning of the “Compassion” scene, after a heated conversation between the landlord 
and his tenants is over, Leah, a widowed tenant who has kept quiet so far, rebukes the self-
centred landlord:
[7] (Leah:)    
You need to have a little compassion. Okay? 
DeepL translated the utterance literally: Morate imeti malo sočutja. Dobro? Both groups had 
been previously shown that instructions and commands containing English modal verbs such 
as need to may be paraphrased as Slovene imperative sentences to achieve a wording that 
is shorter and feels pragmatically more natural. Five out of six non-MT students used the 
imperative in their translations, while one student used a pragmatically smoother modal 
structure: Lahko bi imel malo sočutja. (Literally backtranslated to English: You could have a little 
compassion.) In the MT-group, seven (70%) out of ten students retained DeepL’s translation 
with the modal verb morati (English “must, have to, need to”), two students used the milder 
modal structure lahko bi, whereas only one student used an imperative sentence. In addition 
to this lexico-grammatical feature, the original utterance contains the weak lexical collocation 
have compassion, which does have a formal equivalent in Slovene but will feel stylistically and 
pragmatically more adequate if paraphrased. Three out of six non-MT students rephrased the 
English collocation. The noun compassion was replaced by the semantically corresponding 
Slovene predicative adjective, i.e., biti (vsaj) malo sočuten or the English delexical verb have 
was replaced by the semantically stronger verb pokazati (English “show”). A paraphrase with 
the predicative adjective was used by only one out ten MT students.
Further, the assignment shows MT problems with lexical polysemy. Humans examine multiple 
readings by analysing several threads of multimodal clues, which have so far been inaccessible 
to MT generators. The problem of resolving a polysemic lexeme will be illustrated by the 
term manager. When the word manager takes the full lexicographic form, MT can recognize 
it as a technical term. In movie conversation, technical terms may occur sporadically and will 
appear in their short forms to indicate shared knowledge among the speakers, for example 
“building manager” (Slovene “upravnik stavbe”) becomes manager (Slovene “upravnik”) “the 
listing price” (Slovene “oglaševana cena”) is reduced to the listing (Slovene “cenitev, cena”). The 
reference to “the manager” had been used twice in the previous assignment so the students 
should have been familiar with the exact meaning of the term and the person’s gender, i.e., 
200 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
“(female) building manager”. DeepL’s translation of the three occurrences in the “Landlord” 
scene provided two translations: [8a] vodja (English literal backtranslation: “leader”), which 
is used for both the masculine and feminine genders, and the masculine noun [8b] menedžer. 
All twelve non-MT students remembered the previous assignment and chose the appropriate 
short feminine form upravnica or its variant upraviteljica. Even though the MT-group had 
received the same explanation in their previous assignment, only three out of ten students 
used the appropriately gendered term upravnica. One used the masculine form and replaced 
the feminine pronoun in one of the following subtitles by the masculine to adapt the executive 
function to the traditional social role. Another student chose the masculine form menedžer 
with all the references in the scene, but missed that the feminine pronoun referred back to the 
masculine menedžer. One student interpreted the term as lastnica (English “(female) owner”), 
two students (20%) used DeepL’s first suggestion vodja, while two students failed to see that 
the three nouns and one pronoun had the same referent: one student combined two masculine 
and two feminine references, while the second student used the term uprava (English 
“management”) with one of Oren’s two identical instructions: Complain to the manager.
The example shows a huge discrepancy between the two groups. The non-MT group 
considered both their previous translation experience and the immediate context, including 
the gender clue in the following subtitle that ascertained the information that had been 
shared with them during the previous assignment. Probably relying on DeepL’s ready-made 
translation, most students in the MT-group paid little attention not only to the context – 
which spanned a mere 45 seconds – but also to their previous translation experience.
4.2.1 Stylistic Markedness
Automatic translation has been evolving translation strategies to the extent that rather than 
presenting a mere word-for-word translation, it may produce an incorrect or hallucinatory 
translation. In terms of human subtitling, this abortive translation strategy compares to 
Gottlieb’s (1992, 166) AVT notion of resignation, which results in “differing expression, 
distorted content” and is used with so-called “untranslatable” items.14
The “Real Estate” scene contains the English idiom get the gall (to do something) and its 
variation (not) have gall in two subtitles:
[9] 
[9a]  (Grandma:)  
 So, where do you get the gall to second-guess the listing of an Oren Little? 
[9b]  (Grandson:)  
 I don’t have gall, Grandma. I have comps.
The idiom expresses negative evaluation of the addressee’s insolent behaviour (Smith 2023). 
14 Compared to 2022, Google Translate has barely improved its resignation-like translation of idiomatic and stylistically 
marked expressions (see Simič 2022 for a list of some (informal) Slovene figurative phrases compiled by Lothar Orel 
and their hallucinatory English translations by Google Translate in June 2022). DeepL shows more sensitivity to 
figurative language, while ChatGPT 3.5 has been returning impressive results for the examples found on Orel’s list.
201TRANSLATION AND CONTRASTIVE STUDIES
DeepL recognized the institutionalized figurative expression in the first subtitle. Its translation 
was semantically correct, yet non-idiomatic: Od kod vam toliko drznosti, da (English 
functional backtranslation: “where do you get the audacity to”). The variant not have gall in 
the second subtitle was translated literally: Nimam žolča, babica. The Slovene suggests either 
the grandson’s medical condition of having “no bile” or a quaint sounding metaphorical 
phrase indication that the speaker “has no indignation”. The Sonix automatic translation 
rendered both idioms correctly, but non-idiomatically. Matesub transcribed the first “gall” as 
“goal” and translated it accordingly, whereas the second idiom was rendered with the Slovene 
idiom imeti jajca, which corresponds to the English idiom have the balls. The idiom features 
the opposite polarity, praising the doer’s courage. This idiom was used by one of the MT 
students, while another student expressed the idea of the doer’s “courage” non-idiomatically. 
As the two subtitles express a highly informatively relevant and stylistically rich message 
within seven seconds, one of the MT students reduced the idiom to the (rhetorical) question 
“why”. The rest of the students, MT and non-MT, provided Slovene non-idiomatic verbs and 
phrases to express the idea of audacity. While the idiom have (the) gall is used to criticize the 
doer’s action, the idiom (not) have (the) balls showcases the courage needed to perform a risky 
action. It is worth noting that the crudeness of the abusive idiom debases the old lady, who, 
however, rarely keeps quiet.
4.2.2 Profanity
Offensive and taboo language has been used in movies and TV series to underscore characters’ 
real-life spontaneity, their emotional responses, personality traits or social background. The 
greater tolerance for (or perhaps insensitivity to) the use of vulgar language in film discourse 
over the last few decades has seen increased academic research on related translational and 
sociolinguistic aspects, such as categorization, translation strategies, medium restrictions and 
censorship (for example Mattsson 2006, Díaz Cintas and Remael 2007, Trupej 2015, Díaz-
Pérez 2020).
The four-minute passage contains several examples of mild to strong expressions of offensive 
language. Seeing a boy changing his wet bathing suit in the “Landlord” scene, Oren Little 
cries out:
[10]  (Oren Little:)   
 Oh for Pete’s sake, will you cover your dick? I’m trying to eat a sandwich.
Oren uses the euphemistic oath for Pete’s sake in the movie three times, suggesting that this 
old-fashioned expletive is his standard emotional response. The students’ previous assignment 
contained a scene in which Oren yells out the phrase for the first time. The functions of 
expletives and some subtitling strategies had been discussed in the seminar, while neither the 
non-MT nor MT groups had been given a comprehensive presentation of the topic in the 
lecture format by that point.
Sonix and Matesub are not sensitive to vulgar language and struggled with the speech 
recognition of the mild expletive, both heard it as “I’ll repeat six”. DeepL rendered the phrase 
adequately as a rather old-fashioned and mild blasphemous oath za božjo voljo (English: 
202 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
“for the will of God”). Oren uses the explicative frequently enough to regard the phrase 
an interesting stylistic feature. Still, it is deemed less important than the proposition if the 
temporal or spatial constraints do not allow for more information. Four out of the twelve 
non-MT students and two out of the ten MT students omitted the phrase. Eight non-MT 
and six MT students used short mild oaths in Slovene, while two MT-students (20%), who 
had shown an affinity with the DeepL translation elsewhere in the assignment, retained 
DeepL’s translation. None of the non-MT students came up with DeepL’s marked rendering, 
which is probably little known among the younger generation.
As shown earlier by the you references, DeepL appears to derive a grammatical interpretation 
from the lexis in the immediate context. On the other hand, DeepL’s translation in the 
assignment indicates that it may find it difficult to provide a stylistically adequate translation 
for these very lexical items. For example, the American English slang phrasal verb in Oren’s 
remark:
[11]  (Oren Little:) 
 Something to think about next time you get an itch to rat me out.
was rendered with the stylistically neutral and semantically less appropriate verb izdati 
(English: “betray”), also observed in the Sonix in Matesub automatic translations. The verb 
“betray” was used by only one (8%) out of twelve non-MT students, while four (40%) 
out of ten MT students retained DeepL’s translation. One MT student omitted the verb 
altogether, whereas the rest of the MT group, five students, and six non-MT students chose 
the verb (za)tožiti (English: “tell on”). It was only the non-MT students, five out of twelve, 
who thought of the stylistically equivalent slang Slovene verb (za)špecati. This suggests that 
machine translation both imposes its translation on the AV student and restricts their quest 
for a more suitable/creative translation.
DeepL’s seemingly fumbling ways with the taboo word dick will be further manifested as a 
conspicuous intrusion into the student’s decision-making process. The reference in Oren’s 
utterance [10] will you cover your dick? received in DeepL’s Slovene translation the medical 
equivalent penis, whereas in the next set of subtitles DeepL applied new strategies:
[11]
[11a]  (Oren Little:) 
  So I got a dick in my face, right? Somehow I’m to blame?   
  (Sl: kurac, En: “dick/cock”) 
  (Young child:) 
  He said “dick”.   
  (Sl: ”kurac”, En: “dick/cock”)
[11b]  (Boy:)  
  Sorry about my dick.   
  (Sl/En: penis)
203TRANSLATION AND CONTRASTIVE STUDIES
At first sight, DeepL appears to be torn between the medical and vulgar descriptions. On a 
closer reading, a logic emerges from the contextual lexis. The predicates cover one’s dick and be 
sorry about one’s dick suggest prudent, self-effacing behaviour, hence the penis in translation, 
while get a dick in one’s face and the quotation marks correlate with a brazen attitude 
expressed by the bad word. Sonix’s translations were partly influenced by audio mistakes, and 
backtranslated into English the phrases read: recover your penis, I have a penis in the face, he’s a 
jerk, I’m sorry about my dick/cock. Matesub demonstrates no tolerance for vulgar expressions 
and transfers all the references onomastically, as Dick.
In the four utterances in [10] and [11], three out of twelve non-MT students used the most 
offensive Slovene expression for the male organ kurac/kurec, one student chose the offensive 
expression klinec, which neither reads nor functions well in the utterances, while eight out 
of twelve students chose the multi-purpose word tič, which occupies a central position along 
the axis of stylistically marked expressions denoting the male organ in Slovene. In the four 
utterances in [10] and [11], it was only four out of ten MT students who used the word tič, 
three students used the medical term penis, two students used the most offensive kurac. Not 
paying due attention to the intertextual relations or relying on DeepL’s translation, one student 
first [10] used penis, and the most offensive expressive with the next three utterances [11]. 
Although no students’ renderings correspond entirely to DeepL’s translations, the percentages 
of DeepL-like equivalents are 60% for the MT-group to 25% for the non-MT group.
As the next example shows, DeepL analyses lexical clues for slang and euphemistic words 
alike. In the “Real Estate” scene, the grandmother and Oren Little, having had 44 years of 
experience in the business, belittle Ted’s professional work ethic by reducing him to a baby. 
While the three are talking shop, Claire delivers the following spiteful remark about her 
grandson:
[12]
[12a]  (Grandma:) 
When he was a baby, he used to play with his willy like it was a rambunctious 
puppy.
(Backtr. from Slovene: “When he was an infant/baby, he used to play with his 
watchman like it was a rambunctious puppy.”)
[12b]  (Oren Little:) 
And if he can help me, I’ll play with his willy.
(Backtr. from Slovene: “And if he can help me, I’ll play with his dick/cock.”) 
It is the immediate lexical context that DeepL searches in order to find a clue regarding the 
polysemic word willy, which in this context is used as a (chiefly British) euphemism for 
“penis”. The Slovene translation for the second reference in play with (one’s) may result from 
a higher frequency of written data for the Slovene bad word in combination with the verb 
phrase “igrati se s (svojim)”. The Slovene čuvaj (English “watchman”) for willy appears to be 
a product of an AI hallucination, bringing together perhaps “a rambunctious puppy”, “willy 
wagtail” and “watchdog”. For Sonix and Matesub the two utterances are even harder nuts to 
204 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
crack. Matesub again prefers male names and translates the phrases as play with his Willy and 
play with this. Woody [sic]. Sonix uses a male name in Oren’s remark play with this Willy, 
while the grandmother’s punchline, bizarre as her utterance is even without this translation, 
becomes play with his clit (Slovene ščegetavček).
In [12a], all the non-MT students and six out of nine MT students used a child’s word for 
“willy” (Slovene: lulček, lulek, English “peeny”). Two MT students used the word penis, while 
one student used the most offensive term. No student left DeepL’s semantic mistranslation 
unedited.
Oren’s comment in [12b] takes a mere three seconds, so two out of the four non-MT students 
and five out of the nine MT students resorted to omission and condensation of the phrase 
with his willy, losing the appropriate reference. The textually reduced translations suggested 
that Oren was either going to play or that he was going to play with (his own) willy or that 
he would play with “him/it”, while the “him/it” reference is not clear enough to make an 
impression on the Slovene viewer. Two non-MT students and three MT-students used the 
child’s word, while one MT student retained DeepL’s offensive term.
5 Feedback Analysis
I collected the MT students’ feedback in a nonstructured analysis supported by the following 
guidelines: the students were asked to comment on the time spent on the assignment, the 
positive and negative points of using DeepL and to provide their general view of MT in 
subtitling. Nine students provided some feedback. Compared to the time spent on previous 
assignments, the majority found the MT assignment more time-consuming. Four students 
spent between three and four hours working on what ultimately resulted in 51 to 57 subtitles. 
Two students suggested that the assignment took “more time than usual”. One student 
indicated that the task was performed “faster”, and produced 52 subtitles, while one student 
spent two hours for 57 subtitles and another a mere 40 minutes for 64 subtitles. These last 
two students also appreciated that the overall translation was obtained faster. On the plus 
side, three students found the automatically generated translation a useful starting point 
they could work with. Two students thought that “some segments” were well translated. 
There were, however, more negative responses, and as many as seven out of nine students 
observed that they had to correct the many MT-generated mistakes. Six students pointed out 
that substantial textual reduction had to be done. Four students noticed the tool’s inability 
to cope with context-bound translation, and that “humans” are needed to resolve the issues 
of gender, formality, even terminology. Three students commented on the tool’s reliance on 
the word-for-word translation strategy, and the related uselessness with culture-specific items 
and idiomatic expressions. Three students also thought that the automatic translation and 
bad translations made their subtitling more difficult. One student appreciated the timecodes 
that needed “only some adjustment”, and yet this same student was among three to criticize 
the inaccuracies in the time stamps. With regard to the usefulness of MT, three students 
appreciated the very tool and found it useful for the translation of difficult to understand 
phrases, while three students would prefer to produce their own translations. Two students, 
one from the former and one from the latter group of three, found the experiment an 
205TRANSLATION AND CONTRASTIVE STUDIES
interesting experience. With regard to subtitling, two students might use MT as a source 
of inspiration and another two students said it might be suitable for slow-paced movies or 
movies with “easier content”.
The comparison between the feedback and analysis shows that the students who worked 
faster using DeepL’s translation and original formatting retained more of DeepL’s mistakes or 
introduced their own, since they did not pay enough attention to the multimodal structure 
of meaning. 
Even though digital translation tools are meant to speed up the process of translation and 
to provide accurately communicated messages, this was not the case in my teaching of AV 
translation. The time and intellectual concentration devoted to post-editing appears to be 
counter-productive, while the lack of significant post-editing results in poor quality AV 
translation.
6 Conclusion
The analysis of the subtitles done by the non-MT and MT groups shows considerable 
differences in the multimodal awareness, which is the basis of any adequate AV content 
interpretation. The non-MT group students were better at applying multimodal translation 
skills acquired by that point. The subtitles of the MT group show that the students tended to 
lose focus on the progressing relation between the text and the wider context, including the 
image. The MT-students’ lack of this integrative activity is indicative of machine translation 
logic. Poibeau notes that human translators understand the text by establishing intratextual 
cohesion and “discourse external coherence”, i.e., correspondences between textual elements 
and the extralinguistic reality: “While the machine can reasonably be expected to handle 
issues of text cohesion at some point, discursive coherence is out of its reach, since it requires 
knowledge of the world” (Poibeau 2022, 6020).
As DeepL is not a specific AVT system, the paper studies the categories that to the greatest 
extent possible lie outside the scope of textual adaptation, which is required to achieve 
readable subtitles. All the categories presented in the paper were more adequately solved 
by the non-MT group. The subtitles by the MT group show that most interpretational 
deficiencies emerge from DeepL-generated translations. DeepL fails to consider the context 
which lies beyond the sentence level and, as the analysis shows, beyond the line break. 
Further, the analysis suggests that DeepL (and to different extents both Sonix and Matesub) 
rely on intrasentential lexis-based algorithms that may prefer certain grammatical features in a 
polysemic language like Slovene. While DeepL appears to have an overall tendency to opt for 
the masculine (first-person) Subject, certain propositions are stereotypically or sentimentally 
associated with the feminine grammatical gender. Furthermore, intrasentential lexis governs 
the level of formality and the grammatical number of the Slovene finite verb phrase, i.e., 
substandard vocabulary tends to be associated with informal singular verb features, whereas 
technical jargon will be accompanied by the formal address. Moreover, lexis in the immediate 
context may influence paradigmatic stylistic choices, which may lead to either over- or under-
estimation of profanity.
206 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
While there may have been an overall gap in the subtitling competence between the 2021 
non-MT group and the 2022 MT-group, it is important to note that DeepL’s automated 
translation was not successful in bridging any discrepancies between the generations. In this 
context, questions are raised about the ease of certain EU crowdsourcing initiatives which 
appeal to the general public to either produce subtitles or proofread their automatically 
generated subtitles.
As the feedback shows, most students realized that post-editing the subtitles did not take any 
less time than doing all the work themselves would have. On the other hand, spending any 
less time on post-editing would mean poorer quality subtitles. The students thus welcomed 
the possibility of access to MT tools, but were aware of the pitfalls and the time-consuming 
post-editing process. The study shows that for didactic reasons it is counter-productive to 
introduce MT at the early stages of teaching AV translation. Just as subtitling skills need to 
be acquired, post-editing, if required, should be regarded as a skill that can and should be 
developed. At the very least, students need to learn post-editing as a separate skill to be able 
to critically assess the quality of an automatically translated text. 
MT – or more specifically, automatic subtitling – can only be a safe tool in the hands of 
professional AV translators who are capable of producing high-quality translations by 
themselves, while MT software would help them materialize the text at a faster pace. To help 
achieve this goal, Audiovisual Translators Europe (AVTE) published the Machine Translation 
Manifesto, in which they advocate for the concept of the “augmented translator”, which 
places an active human translator at the centre (rather than the end) of text production while 
using AI-tools “to augment their skills” (Deryagin, Pošta, and Landes 2021, 4).
Post-editing machine-translated texts requires a high level of concentration, as some machine-
translated texts may appear to be formally impeccable and readable, while concealing semantic 
blunders. There is also the danger of impoverishing the store of language by reinforcing 
repetitive linguistic patterns that may sound acceptable or have been uncritically incorporated. 
As this study shows, MT can be misleading, post-editing can be time-consuming, and the 
result can be less remarkable and less rewarding than creative (audiovisual) translation.
References 
Agrawal, Milind, Sweta Agrawal, Antonios Anastasopoulos, Luisa Bentivogli, Ondřej Bojar, Claudia Borg, 
Marine Carpuat, et al. 2023. “Findings of the IWSLT 2023 evaluation campaign.” In Proceedings of 
the 20th International Conference on Spoken Language Translation (IWSLT 2023), edited by Elizabeth 
Salesky, Marcello Federico and Marine Carpuat, 1–61. Toronto, Canada: Association for 
Computational Linguistics. https://aclanthology.org/2023.iwslt-1.1.pdf.
Armstrong, Stephen, Andy Way, Colm Caffrey, Marian Flanagan, Dorothy Kenny, and Minako O’Hagan. 
2006. “Improving the quality of automated DVD subtitles via example-based machine translation.” 
In Proceedings of Translating and the Computer 28, 1–13. London: Aslib. https://
aclanthology.org/2006.tc-1.9 .pdf.
Bywood, Lindsay, Panayota Georgakopoulou, and Thierry Etchegoyhen. 2017. “Embracing the threat: 
Machine translation as a solution for subtitling.” Perspectives 25 (3): 492–508. https://doi.
org/10.1080/0907676X .2017.1291695.
207TRANSLATION AND CONTRASTIVE STUDIES
Deryagin, Max, Miroslav Pošta, and Daniel Landes. 2021. AVTE Machine Translation Manifesto. 
Audiovisual Translators Europe. https://avteurope.eu/wp-content/uploads/2021/09/Machine-
Translation-Manifesto _ENG.pdf.
Díaz Cintas, Jorge. 2005. “The ever-changing world of subtitling: Some major developments.” Research on 
Translation for Subtitling in Spain and Italy, edited by John D. Sanderson, 17–26. Universidad de 
Alicante. https://spiral.imperial.ac.uk/bitstream/10044/1/1401/1/Alicante_05.pdf.
Díaz Cintas, Jorge, and Aline Remael. 2007. Audiovisual Translation: Subtitling. Manchester: St. Jerome.
Díaz-Pérez, Francisco Javier. 2020. “Translating swear words from English into Galician in film subtitles. A 
corpus-based study.” Babel 66 (3): 393–419. https://doi.org/10.1075/babel.00162.dia.
Etchegoyhen, Thierry, Lindsay Bywood, Mark Fishel, Panayota Georgakopoulou, Jie Jiang, Gerard van 
Loenhout, Arantza del Pozo, et al. 2014. “Machine translation for subtitling: A large-scale evaluation.” 
In Proceedings of the Ninth International Conference on Language Resources and Evaluation LREC'14, 
May 2014, edited by Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente 
Maegaard, Joseph Mariani, Asuncion Moreno, et al., 46–53. Reykjavik, Iceland. http://www.lrec-
conf.org/proceedings/lrec20 14/pdf/463_Paper.pdf.
European Commission. 2008. “Communication from the Commission to the European Parliament, the 
Council, the European Economic and Social Committee and the Committee of the Regions: 
Multilingualism: An Asset for Europe and a Shared Commitment.” September 18, 2008. https://
eur-lex.euro pa.eu/LexUriServ/LexUriServ.do?uri=COM:2008:0566:FIN:EN:PDF.
—. 2010. “Directive 2010/13/EU of the European Parliament and of the Council.” March 10, 2010. 
https:// eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2010:095:0001:0024:en:PDF.
—. 2018. “Directive (EU) 2018/1808 of the European Parliament and of the Council.” November 14, 
2018. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:32018L1808.
Europeana. “Europeana subtitled AI pipeline and training suite.” Accessed April 20, 2024. https://
pro.europea na.eu/page/europeana-subtitled-ai-pipeline-and-training-suite.
Georgakopoulou, Panayota. 2012. “Challenges for the audiovisual industry in the digital age: The ever-
changing needs of subtitle production.” The Journal of Specialised Translation 17: 78–103.
Gottlieb, Henrik. 1992. “Subtitling – A new university discipline.” In Teaching Translation and Interpreting: 
Training, Talent and Experience. Papers from the First Language International Conference, Elsinore, 
Denmark 31 May-2 June 1991, edited by Cay Dollerup, and Anne Loddegaard, 161–70. Amsterdam: 
John Benjamins Publishing. 
Kammerevert, Petra. 2023. “REPORT on the Implementation of the Revised Audiovisual Media Services 
Directive (2022/2038(INI)).” European Parliament, Committee on Culture and Education. April 12, 
2023. https://www.europarl.europa.eu/doceo/document/A-9-2023-0139_EN.pdf.
Karakanta, Alina, Luisa Bentivogli, Mauro Cettolo, Matteo Negri, and Marco Turchi. 2022. “Post-editing in 
automatic subtitling: A subtitlers’ perspective.” In Proceedings of EAMT-Translators’ Track, 1–3 June 2022, 
edited by Helena Moniz, Lieve Macken, Andrew Rufener, Loïc Barrault, Marta R. Costa-jussà, 
Christophe Declercq, Maarit Koponen, et al., 261–70. Ghent. https://
aclanthology.org/2022.eamt-1.29.pdf.
Koletnik Korošec, Melita. 2011. “Applicability and challenges of using machine translation in translator 
training.” ELOPE: English Language Overseas Perspectives and Enquiries 8 (2): 7–18. https://
doi.org/10.43 12/elope.8.2.7-18.
Mattsson, Jenny. 2006. “Linguistic variation in subtitling: The subtitling of swearwords and discourse 
markers on public television, commercial television and DVD.” In MuTra 2006 – Audiovisual 
Translation Scenarios: Conference Proceedings, edited by Mary Carroll, Heidrun Gerzymisch-Arbogast, 
and Sandra Nauert. 47–56. Copenhagen. http://
www.euroconferences.info/proceedings/2006_Proceedings/2006_Mat tsson_Jenny.pdf.
Media Consulting Group. 2011. “Study on the use of subtitling. The potential of subtitling to encourage 
foreign language learning and improve the mastery of foreign languages. EACEA/2009/01.” European 
Commission, Directorate-General Education and Culture. June, 2011. http://
publications.europa.eu/reso urce/cellar/e4d5cbf4-a839-4a8a-81d0-7b19a22cc5ce.0003.01/DOC_1.
208 Silvana Orel Kos Introduction of Machine Translation into Audiovisual Translation Teaching
Mezeg, Adriana. 2023. “Ali sploh še potrebujemo prevajalce?” Ars et humanitas 17 (1): 139–54. https://
doi.org /10.4312/ars.17.1.139-154.
Mohar, Tjaša, Sara Orthaber, and Tomaž Onič. 2020. “Machine translated Atwood: Utopia or dystopia?” 
ELOPE: English Language Overseas Perspectives and Enquiries 17 (1): 125–41. https://
doi.org/10.4312/elope .17.1.125-141.
Petukhova, Volha, Rodrigo Agerri, Mark Fishel, Sergio Penkale, Arantza del Pozo, Mirjam Sepesy Maučec, 
Andy Way, et al. 2012. “SUMAT: Data collection and parallel corpus compilation for machine 
translation of subtitles.” In Published Scientific Conference Contribution, 
21–28. http://www.lrec-conf.org/proceedings/lre c2012/pdf/154_Paper.pdf.
Poibeau, Thierry. 2022. “On ‘Human Parity’ and ‘Super Human Performance’ in machine translation 
evaluation.” In Proceedings of the Thirteenth Language Resources and Evaluation Conference, edited by 
Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry 
Declerck, Sara Goggi, et al., 6018–23. Marseille, France: European Language Resources Association. 
https://aclanthology.org/2022.lrec-1.647.
—. 2023. “How machine translation changed the translation industry.” The Responsible AI Forum, TUM, 
Sep 2023, Munich (Allemagne), Germany. https://hal.science/hal-04242026.
Popowich, Fred, Paul McFetridge, Davide Turcato, and Janine Toole. 2000. “Machine translation of closed 
captions.” Machine Translation 15: 311–41. https://www.jstor.org/stable/20060451.
Reiner, Rob, dir. 2014. And So It Goes. Škofljica: Blitz Film & Video Distribution. DVD.
Sepesy Maučec, Mirjam, Marko Presker, Danilo Zimšek, Matej Rojc, Damjan Vlaj, Darinka Verdonik, and 
Zdravko Kačič. 2012. “Izdelava slovensko-srbskega vzporednega korpusa podnapisov za razvoj 
strojnega prevajanja v projektu SUMAT.” In Zbornik Osme konference Jezikovne tehnologije, 8. do 12. 
oktober 2012, [Ljubljana, Slovenia] = Proceedings of the Eighth Language Technologies Conference, 
October 8th-12th, 2012, Ljubljana, Slovenia: zbornik 15. mednarodne multikonference Informacijska 
družba - IS 2012, zvezek C; proceedings of the 15th International Multiconference Information Society 
- IS 2012, vol. C, 167–72. Ljubljana. http://nl.ijs.si/isjt12/JezikovneTehnologije2012.pdf.
Simič, Nina. 2022. “Googlov angleško slovenski prevajalnik ni več najboljši.” Moje finance, June 30, 2022. 
Accessed December 27, 2023. https://mojefinance.finance.si/9001658/prevajalniki?src=undefined.
Smith, Chris A. 2023. “Productivity from a metapragmatic perspective: Measuring the diachronic coverage 
of the low level lexico-grammatical construction Have the N  
(Body Part/Attitude) to ←→< Metapragmatic Comment> using the COHA.” Languages 8 (2): 92: 
1–23. https://doi.org/10.3390/languages8020092.
Sonix. “Sonix.ai.” Accessed December 27, 2023. https://sonix.ai.
Tiedemann, Jörg. 2007. “Improved sentence alignment for movie subtitles.” In Recent Advances in Natural 
Language Processing RANLP 2007, edited by Galia Angelova, Kalina Bontchevam Ruslan Mitkov, 
Nicolas Nicolov, and Nikolai Nikolov, 582–88. Borovets: Bulgaria.
Trupej, Janko. 2015. “Strategije za podnaslovno prevajanje profanosti v slovenščino.” Slavistična revija 63 
(1): 17–28. https://www.dlib.si/details/URN:NBN:SI:DOC-0XTSKK8O.
Volk, Martin. 2009. “The Automatic Translation of Film Subtitles. A Machine Translation Success Story?” 
Journal for Language Technology and Computational Linguistics 24 (3):115–28.
Volk, Martin, and Søren Harder. 2007. “Evaluating MT with translations or translators. What is the 
difference?” In Proceedings of Machine Translation Summit XI: Papers, edited by Bente Maegaard, 499–
506. Copenhagen, Denmark. https://aclanthology.org/2007.mtsummit-papers.