YouTube2Text

AI and the future of translation. In search of evidence — Transcript

by Anthony Pym · 5,887 words · 388 segments · language en · Watch on YouTube

Full transcript

  1. 0:01This is a talk for Translators and Interpreters  Australia, who are very concerned with protecting
  2. 0:09the rights and well-being and the privileges  and the pay of translators and interpreters.
  3. 0:15It's about AI, artificial intelligence,  and its possible impact on our professions.
  4. 0:24And my starting point is that there's lots of  talk around, there are lots of instant experts, and
  5. 0:33we're not very sure of how much of that talk and  the predictions, especially the more catastrophic
  6. 0:39ones about the challenge to humanity, to the  environment, to our professions, to our work, to
  7. 0:46communication, to the quality of language, all  those things, we're not sure how much of that is true.
  8. 0:53It seems to me that what we have  to do is get down and look for some evidence.
  9. 1:01Some of you may remember the millennium bug, the fear at the end of the
  10. 1:09second millennium that all computers would  go wrong because nobody had fixed up the date to
  11. 1:16to change the big numbers. There were  a thousand instant experts around and the
  12. 1:23role of an expert, ladies and gentlemen, is usually  to spread fear and be negative. Why are experts negative?
  13. 1:31Well, because if they're wrong nothing  bad happens - everybody's happy so you forget about
  14. 1:37the expert. And if they're right, then you can  say I told you so. So for experts it's
  15. 1:45just basic risk management strategy to be negative  if in doubt. And that apparently is what could be
  16. 1:53happening here. Let me start my investigation  as best I can and I'll start by just
  17. 2:03pointing out what I'm dealing with or what I'm  talking about here.
  18. 2:10The one you've all heard about I presume is  Chat GPT because you can get on that for free at
  19. 2:18the moment and play around with it. It's been with  us since November 2022; GPT 4 has been released
  20. 2:26publicly so you can play around with that too and  I've also been playing around with Google Bard
  21. 2:32which has been released in experimental  mode and is available in Australia.
  22. 2:41Chat GPT happens to have the fastest growing  user-base in the world ever for an app. So people
  23. 2:49are playing with it; people are having fun; people  are exploring it; and I hope you are too. I have
  24. 2:55been now. I must admit that when I first went into  that little world, I thought, right that's a big
  25. 3:04change. That is impressive. It can translate and  it can give us information about a lot of other
  26. 3:11things and it's going to be incredibly useful but  it might be this moment of singularity when the
  27. 3:21processing capacity of artificial intelligence  is greater than that of the human brain.
  28. 3:27That's a moment of singularity. It has been  reached in the game Go and in chess.
  29. 3:36Perhaps in translation? Perhaps interpreting, too?  We don't know. But certainly my first encounter
  30. 3:45was one of shock and surprise. However, as I  continued, things became a little less frightening.
  31. 3:54Now, a commonsense approach out there among  experts of all kinds is this one is from Sasha Luccioni
  32. 4:06who tweets profusely on this topic: it's the iceberg approach
  33. 4:16where we recognize that there are actual  benefits to the new technology but underneath
  34. 4:23the water there are lots of threats. The complete version finds a lot more threats, some of
  35. 4:31them truly worrying, such as the environmental  consequences - we've got tons of carbon emissions -
  36. 4:38and the ones about representation might concern the work of translators
  37. 4:48as indeed do privacy and copyright infringement So there are lots of things there to be
  38. 4:56scared of certainly and a few things up there to rejoice about. Lynne Bowker, who knows as much
  39. 5:05about this as anybody, who's worked  a lot on machine translation literacy
  40. 5:13has adapted that to translation. So neural MT which is
  41. 5:23what we've had with us - neural machine translation -  since 2016 can be analyzed in much the same terms,
  42. 5:32with a few advantages up the top and  many possible disadvantages
  43. 5:35down the bottom. Hallucinations are just crazy results. What's interesting is that since 2016
  44. 5:49if you look for actual evidence  of growth in these negative things
  45. 5:56it's quite hard to find the evidence and  that's going to be my main point of entry.
  46. 6:03I want to talk about  these things that are said about
  47. 6:08firstly neural machine translation  but then AI such as chat GPT and Bard.
  48. 6:15What's the actual evidence for the bad things? I'll be dealing with transparency first,
  49. 6:23which is a big point, job destruction, changing skill sets, anchoring, which
  50. 6:29we'll come to in a minute, which languages  benefit, which will be marginalized, wage dispersion
  51. 6:36in the industry and intellectual property theft.  That's a lot of things to deal with. I'm going to go fast.
  52. 6:43First, let's be very clear what we're talking about..A lack of clarity
  53. 6:52was evidenced around me where I live and work in Spanish and Catalan. We had people
  54. 7:01who work with subtitles complaining that Squid Game, which was a huge success on Netflix,
  55. 7:10was post-edited: Netflix would give a translator a machine translations and ask
  56. 7:17them to correct the machine translations.  There was a lot of public outcry in various forms
  57. 7:24of the press and online and then a particular  talk there that I picked up and recorded,
  58. 7:31by the head of the Audiovisual Translators Association in Catalonia,
  59. 7:38which is where I am at the moment. She talked about  the massive threat of post-editing and I went
  60. 7:48through her talk you can see I actually took an  automatic speech-to-text and then automatic Google
  61. 7:56Translate version and produced the subtitles,  which actually worked. I produced subtitles
  62. 8:02automatically for her. Going through  the talk, I didn't know what she was talking about
  63. 8:09because it was all mixed up. She talked about  post-editing as if it were raw machine translation
  64. 8:18as if all the mistakes that come from raw  machine translation are the mistakes in post-editing.
  65. 8:24But they're two different things. The  question is: it advantageous to translate from
  66. 8:31scratch without any electronic aids, which few  of us actually do, or put a text through
  67. 8:40machine translation and correct it? That's the only question. Is it better to use
  68. 8:46that passage through machine translation and  correct it or do it from scratch?
  69. 8:52When I teach, we just do experiments. I don't  tell students whether one is better than the other.
  70. 8:57I let them find out for themselves.  We measure results in terms of time spent
  71. 9:03and the quality of the translation at the  end, which is assessed through peer reviewing.
  72. 9:09So it's important when you get  into these conversations
  73. 9:15to stress what you're talking about. We're  not talking about raw machine translation.
  74. 9:21In the case of GPT and Bard, what  they can do for translators is translate
  75. 9:31but the quality is actually not much better or on  a par with DeepL and Google Translate
  76. 9:40for example, and there are many other systems  out there. It's not a huge qualitative advance
  77. 9:49when we give the prompt "translate  from this language to that language".
  78. 9:55Try it for yourself. That will help calm  nerves, I think. However, the new technologies,
  79. 10:03the GPT technologies, can do a lot for us and  help us with what translation memory software does or
  80. 10:12language automation in general, if I put it  together, can do all these things in addition to
  81. 10:19translate, well or badly as a machine translation.  It can obviously bring up your
  82. 10:27previous translations. This is translation  memory technology and you can revise it.
  83. 10:32You can use different feeds. You can have different  machine translation feeds and the GPT
  84. 10:38feed which you now get in Trados Studio to  get alternative translations to choose from.
  85. 10:46Lilt gets you to type and then offers  proposals and they reduce as you type and
  86. 10:52that's one interesting thing that I'll come to in  a minute. I might also add that even a
  87. 10:59free online translation memory suite like MateCat  now has a terminology tool linked to GPT.
  88. 11:08If you're in doubt, you can click on that and you  you get an explanation of what the term means in
  89. 11:14the particular context you're dealing with.  That can be tremendously useful for getting
  90. 11:20translation suggestions about what you're doing.  Propagation automatically making a change in the
  91. 11:28whole text when you decide to translate a term  in a different way. GPT can be used
  92. 11:34to pre-edit the text to translate. If it has to be simplified, we can simplify
  93. 11:40it. or f the grammar has to be corrected. The text will then go through machine translation better.
  94. 11:46It could do that for you. If the sentences have to  be shorter, it can do that for you.
  95. 11:52It can prepare the text so that it will work better in  machine translation. There are online tools where
  96. 11:59we can exchange notes with other translators. We can use speech-to-text; that is, we can speak our
  97. 12:05translations, which often produces a more natural-  sounding translation. We can also go from text to
  98. 12:12speech and I always do that to to revise my  translation: I sit back and listen to it being
  99. 12:18read to me. There's no better way to spot the typos  and the grammatical gaps that happen to be there
  100. 12:26You can revise a translation - the spelling  quite obviously and for grammar and for
  101. 12:31readability and so on. In fact, we're discovering  more things that it can do. For example, aligning
  102. 12:39text your old translations so that  you create a translation memory for reuse,
  103. 12:45GPT technology could do that for you. So I think it's important to insist that
  104. 12:54we're not talking about just machine translation;  we're not talking about just its quality; we're
  105. 13:00talking about all the many different ways you  can interact with it and where it can help you.
  106. 13:05And it the main help is not the core task  "translate A to B". It's all these other things that
  107. 13:14can assist us in our task. Let's just continue  the list: revise the punctuation; translate for a specific
  108. 13:21reader and purpose - this is where GPT really comes in -  you can say translate for a particular reader.
  109. 13:27Terminology management and extraction  as well propagation.
  110. 13:33You can set up your preferences and check consistency. Now all of
  111. 13:43those things concern revision all of those  particular things concern opening up the range of
  112. 13:54things that you can choose between and improving  quality. So my point is:
  113. 14:02what's new here is not the key task of translating;  what's new is everything that's around it.
  114. 14:11We can see this in Audiovisual Translators Europe that has a I think a very good paper
  115. 14:17on automation in the industry. We can see  you the linguistic translator here using tools
  116. 14:26for all these things. They're aware that  all these things can help you so they
  117. 14:34make use of the term "augmented translation" - this is not you being replaced,
  118. 14:41it's you being extended in all these different ways. I agree very much with their conclusion
  119. 14:47here that the real problem with the technology  is that a lot of it has not been developed in
  120. 14:54cooperation with the people actually using it,  with translators and interpreters, which means
  121. 15:00that there are some serious limitations in it  and that the technologies can be much better.
  122. 15:06but refusing the advances of automation  on all those fronts, no that is not the way forward,
  123. 15:13they argue and I generally agree  with them - I think that's a fine paper.
  124. 15:20I have however also been looking  at another big paper, a position
  125. 15:25statement produced by the French  literary translators. You can see it there.
  126. 15:34We have translations online  into English and into Spanish
  127. 15:45and their headline is transparency. They say  they're not against AI, not against any of the
  128. 15:52automation, but what they want is transparency:  just make it clear who is using what.
  129. 16:00I agree. Who could possibly be against transparency? It's just basic honesty.
  130. 16:08However, there are catches. For example, translators  have forever been using dictionaries so in my
  131. 16:15published translation should I list and declare  and thank all the dictionaries I've used?
  132. 16:22I ask people around me often about particular words  and usages. Should I list all of them?
  133. 16:28Perhaps I should in the name of transparency.  Would it hurt to list the machine
  134. 16:35translation systems that you've been using and  how you've been using them? No it won't hurt at all.
  135. 16:43In fact, I'm doing this now. I've been  engaged in a collaborative translation and
  136. 16:49I do in the preface explain exactly whom  I've worked with and which machine translation
  137. 16:55systems I've been using. Why not? One of the things that intrigue me with
  138. 17:02GPT is that if you ask it to translate,  it will translate and give you a text
  139. 17:10and if you ask it to adapt to a particular  readership, it will do that too and Bard will
  140. 17:16do it very well and give you an explanation  at the bottom of exactly what it's done.
  141. 17:24But if I ask that system anything about a  legal matter, not a translation matter, as
  142. 17:30you can see here I've done that: Should I file for  bankruptcy in Australia? I wonder why I'm asking that...
  143. 17:36it will tell you it's crucial to  consult a qualified professional. It will
  144. 17:46declare its limitation and point you to the  use of a professional.
  145. 17:53It's protecting itself in legal terms. That is transparency. I think that it should do exactly the
  146. 18:01same for translations. It can translate for you but  it should say if you want a completely trustworthy
  147. 18:07or you want an adaptive adaptation or you want to  be sure that the translation is going to fulfill
  148. 18:13the purpose, go to a professional translator  just as you would go to a professional lawyer
  149. 18:21or accountant or tax expert. So I  agree very much with transparency
  150. 18:28and a lot more could be done.
  151. 18:36Let's see...
  152. 18:42If you are a translator and you believe your  work has been stolen...
  153. 18:46I asked them have there been cases of translators  who've had their work stolen because of
  154. 18:53machine translation databases or translation  memory databases because that's one of the
  155. 18:59big risks that's always put out there. There have been
  156. 19:04cases where translators have sued companies but  none of them have concerned AI, that is automation,
  157. 19:11or databases as far as I can tell. What's interesting is that you could if you're a
  158. 19:20translator and you believe your work has been stolen, consult with an attorney.
  159. 19:25Ask a professional. So that's transparency.  I would like them to do that for translation.
  160. 19:32I have written to the organizations  involved and made that request and they have
  161. 19:40not got back to me. Does automation destroy jobs? This was in Politico:
  162. 19:50Who killed the EU's translators? Automation is  creeping into European Union institutions
  163. 19:56and translators are among its first victims.  This is in May 2023 and we can see
  164. 20:06that the employment  of translators in the European Commission
  165. 20:14which actually has the biggest translation bureau in the world ever
  166. 20:21is decreasing. The problem is what's  the cause of it. If you can see here,
  167. 20:31from 2012 to 2022 where does the  money go? It goes to outsourcing
  168. 20:42so the number of in-house translators has  decreased because it's cheaper to outsource
  169. 20:48so you can adapt to variable demands. It makes sense as well and they've managed
  170. 20:54to reduce their staff and their budget that way.  The cause is not automation it's just outsourcing.
  171. 21:01So the report is just a load of rubbish. Automation might be in it but we've got no
  172. 21:07numbers to actually show that. We can see here  the budget for translators
  173. 21:17in the last year - these are the most recent numbers  I can get - actually increased by 5.4 percent for
  174. 21:25in-house staff and 11 for contract staff so they  have increased their budget for a translation
  175. 21:31despite the numbers that you just saw and  if I go back here, the cost per page, which you can
  176. 21:39get from the published numbers, has actually  increased although if you go back 10 years
  177. 21:45before then it was about 130 euros so it's got better but it's still pretty bloody high as a cost.
  178. 21:54So if they're using automation to get rid of people and cut costs, they're doing a pretty bad job of it.
  179. 22:01We should get some real data before we  start publishing headlines about automation.
  180. 22:09This is a slightly more serious  study about the same thing.
  181. 22:14They seek evidence from Google Translate and  Chat GPT. So or what they did was look at the
  182. 22:23advertisements for translation jobs on something  called a large online labor market - I think that
  183. 22:31must be ProZ but they're not saying - over  the period in which Google went neural:
  184. 22:38from January 2016 to May 2017. They assumed there was a qualitative change and they
  185. 22:44wanted to see if that affected the employment  of translators. They did find that the demand for
  186. 22:51translations declined by 13 to 20 % , that's a  pretty big band but anyway, and interestingly
  187. 22:58found that the demand for transcreations these are  non-routine cognitive and interactive tasks,
  188. 23:05the kinds that neural machine translation was  not good for, did not decline. And over that
  189. 23:13period they found no significant change in the  rate of pay. Then they assume that something
  190. 23:20like that can be seen for Chat GPT but they don't  have a get convincing numbers.
  191. 23:31I find the study naive. If you know about  the translation industry, I don't think you get
  192. 23:37immediate impact from technological advances.  So Google Translate goes neural and all the others
  193. 23:44go and you're all in the same years 2016-2017, but  it takes a long time for that to filter through
  194. 23:52to any consequence that is substantial for the industry. It is quite possible that lots
  195. 24:00of clients went and tried Google Translate  in that period, which might account for a decline,
  196. 24:08and many of them may have come back the year after  when they found that they could not rely on it.
  197. 24:16This is a similar study but a bit more serious. It looks at advertisement for jobs in
  198. 24:25translation over the period 2014 (not 2024!) to 2019.
  199. 24:43They find that the advent of a neural machine  translation in 2016 did not lead to decreases
  200. 24:51in the demand for language professionals, although  it did increase the demand for transversal skills
  201. 24:57and skills in the use of automation:  skills which are most important for "high quality
  202. 25:07translations of text that machine translation  tools cannot deliver with accuracy."
  203. 25:13So there are two increases: one of them in the technical skills to work with technologies,
  204. 25:18and the other to do the jobs that  machine translation was not good at at the time.
  205. 25:27This in fact contradicts  the previous data.
  206. 25:32So is there job destruction? There's no firm  evidence of that happening.
  207. 25:41Globalization increases,  therefore the number of translations in
  208. 25:45the world increases, so even if our slice of the pie as  professionals gets smaller as more and
  209. 25:51more people do it themselves, the work  available is still considerable.
  210. 26:00We're not losing work because the amount of texts  that are translated is increasing enormously.
  211. 26:08Now let me go back to the French literary translators who said they are speaking
  212. 26:13from experience. I went through that long  document looking for the actual experience of AI
  213. 26:21and this is the only piece  of evidence that they present.
  214. 26:26It's by a researcher in  Vienna and they give this example where
  215. 26:40people using machine translation all gave the  same translation and the people not using machine
  216. 26:46translation gave different translations. They  call this "anchoring" and this would be the problem
  217. 26:52of using machine translation: everybody  gets anchored to what the MT suggests and
  218. 27:00therefore you get tremendous homogeneity in the  group and your culture becomes very homogeneous.
  219. 27:07I do think this actually happens so I'm not arguing against the fact.
  220. 27:16I went to the actual text in question: this is the beginning of "A Very Short Story"
  221. 27:21by Hemingway and we're interested in this sentence here "Luz sat on the bed".
  222. 27:29That sentence could mean "she sat down on the bed"
  223. 27:35or it could mean "she was sitting on the bed". It's just ambiguous
  224. 27:40and nothing else in the story depends on how you  resolve that ambiguity.
  225. 27:48If it were important, Hemingway would have said "sat down"  or "was sitting" but he didn't: he said "sat on the bed".
  226. 27:55For me, it's logical  if you have to make a decision and you're
  227. 28:05going to invest effort in making the decision  and somebody has given you a suggestion
  228. 28:11and the suggestion is as good as all the other  alternatives and the MT says "Luz sat on the bed",
  229. 28:17that's the one it's logical for you  to go for because it's less effort
  230. 28:22and you lose nothing. So I'm really not upset by the fact that people can
  231. 28:31solve a problem with the help of automation when  there is no good or bad solution to the problem.
  232. 28:38I'm fine with that.. Lots of terminology problems
  233. 28:44are solved this way and solved very well. Another point that's raised very often
  234. 28:51with respect to machine translation  and GPT appears in recent work
  235. 28:59on using GPT3. The sentence is this  one: "I'm honored to meet you the future leaders
  236. 29:07of Great Britain and this world". This is Michelle Obama speaking to young people.
  237. 29:14The translation into Portuguese for "I'm honored" comes out male, which is wrong, and then we have
  238. 29:22female future leaders which is wrong too,  because there are men and women. We know that
  239. 29:29machine translation screws up when there are  long sentences and when there are pronouns and
  240. 29:36especially when they're gendered pronouns.  MT can be incredibly sexist but here
  241. 29:41is just a question of being wrong. I translated this into Catalan
  242. 29:49and it gave me correct genders when I said "Michelle Obama".
  243. 29:56I specified the context in the prompt. And I said "a group of young women and men".
  244. 30:06I specified in the prompt and it says feminine and masculine future leaders.
  245. 30:13So if you do the prompt correctly,  the machine will solve the problem really well.
  246. 30:19The sexism is in the society that generated  the databases that go into the machines.
  247. 30:26I suggest the lack of a specific  enough prompt was the problem.
  248. 30:34But let me show you a different way of  solving this.
  249. 30:39So this is the offensive translation: sexist here and sexist here. The first one is easy
  250. 30:46to fix up. Just click on there and "es un honor" (it's an honor) and we solve that problem.
  251. 30:55we can also work on the ST. For example,  "future leaders" becomes "who will lead",
  252. 31:05using the verb, and we make it plural over here,  and we change this and...
  253. 31:21I've got a non-sexist translation just working  here in this lovely little DeepL sandpit.
  254. 31:36What I was doing - I hope you could follow -  was just going into DeepL and playing with
  255. 31:43the alternatives that it suggests. So the the  anchoring phenomenon you recall suggested that
  256. 31:51whatever was proposed by the machine translation  would attract everybody to it, but if you can use
  257. 31:57DeepL this way, modifying the translation here  and modifying the start text over here,
  258. 32:03you can work with those drop-down menus and play  with the translation until it solves the problem.
  259. 32:10So I solved the problem. Instead of saying "I am honored", I put "It is an honor" - I've removed the sexist
  260. 32:17problem. In "To meet you who will lead Great Britain  and the world", I've removed the sexism.
  261. 32:25Now the problem for me is that this  sort of interactive translating that I
  262. 32:33can do in DeepL online should be put into the mainstream translation memory software.
  263. 32:41You can do it in CafeTran, you  can have the DeepL interface there at the bottom,
  264. 32:48but I really want machine translation to  put that capacity to have the drop-down menus and
  265. 32:56select and play with it as you're translating  into the software so that we can all use it to
  266. 33:03enhance our creativity rather than be stuck to  the anchoring phenomenon.
  267. 33:12So yes, anchoring is a problem. I think sometimes it does matter but the case
  268. 33:19the French translators gave is a trivial one. The technology can help us
  269. 33:24overcome the problem if we get better technology.  I don't think we can accuse interactive MT of
  270. 33:33devaluing rich human practices, as in the list by Lynne Bowker,
  271. 33:38or of a lack of creativity. I think that what I  was doing there was a highly creative process.
  272. 33:44Let me move on now. We found that GPT works  well for the main languages I've been working with:
  273. 33:57Spanish, French, English, Chinese, German,  all in combination with English - no problem,
  274. 34:03and between themselves no real problem. But we tried it for Armenian: GPT just gives complete
  275. 34:12rubbish and the neural machine translation  is questionable at best. The people in Dublin
  276. 34:21tried it with Irish and they found that GPT  was much worse with Irish.
  277. 34:29Catalan, although a small language, has a pretty  good set of assets and works
  278. 34:36really well and in fact GPT seems to work  directly with English, not going through Spanish
  279. 34:44as is certainly the case in Google Translate.  So it's getting even better with GPT.
  280. 34:51The problem is that these solutions are working well  for the languages with big resources not for the
  281. 34:58languages with small resources. These are the most frequent languages on the
  282. 35:05Internet that's English that's Russian that's  Spanish French and there's a very long tail of
  283. 35:11very small languages with limited resources.  The answer I think is not that we need
  284. 35:20less automation: we need more of it; we need more  electronic resources. I don't think I've really
  285. 35:27got time to go into this but the work was done  by this group of people working with Icelandic
  286. 35:34and Hausa - small languages comparing  it with French and German and we
  287. 35:41can see the problem that the GPT is really  not doing anything for the smaller languages.
  288. 35:48What's the answer? More automation, build up the  databases, and the machine translation people
  289. 35:54are very interested in that and getting machine  translation to generate databases by analogy.
  290. 36:03I'll move down very quickly to what GPT  is very good at, and Bard, so the large
  291. 36:10language models in general, and that is adapting a  text to a particular audience.
  292. 36:18This is the text in the prompt "translate".  We get that in Catalan and that's a good straight
  293. 36:27translation. There's no problem with it.  If we say "translate for a 12-year-old",
  294. 36:33and we actually gave this example because  we previously used it in class and one of the
  295. 36:41students realized that when Whitlam went  to Canberra he was 12 years old, so that if
  296. 36:50you say "translate for a 12-year-old" you're  inviting personalization of the
  297. 36:58text for the particular reader as was done  by GPT: "born in Kew, Melbourne in the
  298. 37:07summer of 1916 - so they've adapted July - they say it was a really hot day...
  299. 37:15Whitlam went to live in Canberra with his father when  he was 12 years old. Brilliant! His father
  300. 37:23"was a lawyer", so that's much better than a "deputy  Crown solicitor", too complicated for the reader.
  301. 37:30Whitlam studied at the school in  in Canberra and then at the University
  302. 37:34of Sydney. It's done brilliantly by  adapting this "quan tenia 12 anys", when he was 12 years old.
  303. 37:45However, as some of you will have noticed,  it has adapted the text as if it were in Europe.
  304. 37:51In Australia, particularly in Melbourne, July is  a miserably cold month, ladies and gentlemen,
  305. 37:58it's not the summer. So beware: the more you ask the  technology to adapt, it will make changes that
  306. 38:08it thinks are cute - it really does  seem to think - that it calculates are going to
  307. 38:14be attractive to the new reader and the more  changes are made, the higher the risk there
  308. 38:20will be an error. As indeed there is there.  It shouldn't be summer, it should be winter.
  309. 38:26You can re-submit the same prompt and you get different answers.
  310. 38:34Another mistake was made: "Whitlam was born in Kew, a city of Melbourne".
  311. 38:43I don't know. I've been I've been  living there for seven years. I think
  312. 38:48it's a suburb. It might have been a city I don't  know back in 1916, but I certainly wouldn't
  313. 38:54use that explicitation. It was quite  happy just putting Kew, Melbourne. That's all you
  314. 39:02have to say. The more you try to explicitate,  the greater the risk of error and confusion
  315. 39:11Just some more numbers that can be found.  Most of this work is 2023.
  316. 39:19Comparing GPT with neural machine translation in our  own work at the University of Melbourne with
  317. 39:25colleagues working with Chinese and with  colleagues in Spain working with Romance
  318. 39:31languages and Armenian, we have found that  GPT is not much better than what we get from NMT.
  319. 39:39Here we're going Chinese into English with Google, DeepL and Tencent and you
  320. 39:48can find here that the highest quality is with  Google in this case, with Tencent in this case,
  321. 39:56and all of these are much higher than  GPT. German into Chinese the winner is
  322. 40:05Tencent and again and Romanian into Chinese...
  323. 40:12Beware! When you look at these numbers,
  324. 40:18they prove pretty convincingly that the existing
  325. 40:24machine translation systems are better than GPT. The new one was there for GPT-4 by the way.
  326. 40:32The research was done by people working  at Tencent, so they happen to pick examples or
  327. 40:39languages where Tencent came out a winner  in at least two of those cases.
  328. 40:44So there's a lot of playing around  in the examples that are being used.
  329. 40:50This is a similar thing, just showing that  as you can see at the top for these things the
  330. 40:59the machine translation is doing better than GPT. So GPT isn't a threat except there are some
  331. 41:05cases where it does do better. It is not a big threat  to translation. In fact, as translation scholars
  332. 41:12and translators in the community, we've had time  since 2016 to come to terms with this. We shouldn't
  333. 41:20be completely outraged. I'm moving to a conclusion  but I want to step back and have a
  334. 41:29call to calm. Automation has been going on you know  since the Industrial Revolution and before that
  335. 41:35You can study it over time. Computer -based automation would be affecting professions,
  336. 41:45occupations, so a sociologist of work just gets a  list of occupations in the United States from 1950
  337. 41:52another one from 1910. We assume that the impact  of computers has had an effect. Which is the only
  338. 42:02occupation that disappeared? Can you guess? Is it translators? No, we still exist.
  339. 42:08Lift operators or elevator operators in the United States. There are
  340. 42:16still some around - I managed to find them that's  that's the only job because there is a logic
  341. 42:22by which automation increases productivity which  increases demand which increases work so there's
  342. 42:29not necessarily job loss through automation.  That's what we see over the big picture.
  343. 42:36But what we also find or what is found in these studies  is that the more automation comes into a sector
  344. 42:44the more there is wage dispersal, inequality, that  the people who can use the technology to their
  345. 42:52advantage get richer and the people who can't  will stay where they are or get slightly worse off.
  346. 42:58That's a real question for our industry. I have  been fascinated by certain numbers.
  347. 43:05This is from a European Competition Index that is now  a bit dated: over a seven-year period we found
  348. 43:14that in the translation sector the growth  was huge for the big companies and was negative
  349. 43:21for the small companies. That is worrying.  I'm looking for more recent information.
  350. 43:29I find that as companies get bigger up to here, their  profitability increases and out here it's hard to
  351. 43:36say because there are so few companies  that it depends on mergers and acquisitions.
  352. 43:41Similarly up to this point out here, the  bigger the company, the more the profitability,
  353. 43:49although there's some doubt going on there.  So there is some evidence that in general
  354. 43:59the bigger the company, the better the  profitability, and we know from other information
  355. 44:04that the bigger the companies, the more they  invest in the technologies so the technologies
  356. 44:11I've been talking about here are... Well Trados  is now owned by the biggest communication and
  357. 44:19translation company in the world. It has  an interest in exploiting the technology.
  358. 44:27Final point: intellectual property theft. This has been a problem since the early 1990s with
  359. 44:34translation memories: Who owns the translation memory? The company or the translator?
  360. 44:41As I pointed out, I have not found any court  cases dealing with intellectual property theft
  361. 44:47via translation technologies. There are plenty  of cases where translators have sued companies
  362. 44:52and other translators for plagiarism but not  via translation technologies so perhaps I'm
  363. 44:59wrong to look for it. In summary, is  there a lack of transparency? Yes and I think
  364. 45:07we need more transparency and I think it would  be a good thing to apply that across the board.
  365. 45:13Tell us which dictionaries you use; tell us who  you are; tell us who helped you. Why not?
  366. 45:18Let's see translation as a collaborative social activity. Evidence of job destruction? Not yet.
  367. 45:25I haven't seen firm numbers and the numbers that are  published are deceptive. Changing skill sets? Yes,
  368. 45:33towards augmented translation, as we saw in  the audiovisual industry report and I
  369. 45:40think I can see that just tracing where our  graduate students go. AI anchoring?
  370. 45:48Yes, that happens but it may not have negative  consequences and the technology should improve
  371. 45:55to move to get rid of the anchoring, to make MT more of an aid to a creative process.
  372. 46:03Evidence of language marginalization? Yes,  automation is helping the big languages and not
  373. 46:09helping the smaller languages enough. The answer is more automation, more language resources
  374. 46:15for the smaller languages. Wage dispersal?  Yes, but indirectly.. It happens because
  375. 46:25people know how to work with the technology  and the wage dispersion seems to correlate
  376. 46:32more with company size than with anything I can  see dealing with numbers about the technology itself.
  377. 46:38So yes, I think that is happening  but the mediating factor is company size.
  378. 46:43Intellectual property theft? Perhaps it is going on and we haven't heard about it.
  379. 46:54so there might be an iceberg. There are certainly  some very good things at the top. I do enjoy
  380. 47:00working with these technologies. I enjoy working  in that DeepL sandpit and I want to see
  381. 47:05that integrated into mainstream translation memory  technologies. And there are some serious threats
  382. 47:12that we don't see under the iceberg but there's a  lot of misinformation out there and a lot of panic
  383. 47:20about that hidden part of the iceberg. I think as professionals we have to sit back
  384. 47:28and see how it really is affecting us rather  than enter into panic mode. Find out about
  385. 47:34the technology, play with it, experiment with it, find out what works for you.
  386. 47:40As an academic, I have to do research and  and find reliable research before making
  387. 47:47these huge statements about AI being  a threat, not just to translators and
  388. 47:53interpreters but to humanity. I don't know.  Not yet. I haven't seen the evidence of that.

About this transcript

This page contains the full transcript of AI and the future of translation. In search of evidence by Anthony Pym, generated from the public captions YouTube serves with the video. The transcript has 5,887 words across 388 segments, with the original timestamps preserved so you can click any line to jump to that moment in the embedded player.

What you can do with it

Use the transcript to take notes, quote the speaker, build a study guide, generate a summary with ChatGPT or Claude via the YouTube Summary tool, or export it as a timed subtitle file with YouTube to SRT. You can also re-open it in the transcriber to choose a caption track available on the video.

Free YouTube transcript tool

YouTube2Text is a free YouTube transcript generator — no signup, no daily limit. Paste any YouTube link and get the full transcript instantly, with timestamps, click-to-jump, caption language selection, AI prompts for ChatGPT, Claude, and Gemini, and exports to TXT, SRT, VTT, or Markdown.