(ORDO NEWS) — Based on the large linguistic model GPT-3, created by OpenAI, a system for translating literary texts has been developed. The system is open source and anyone can experiment with it.
Experts say that the literary texts that the system translated in many cases look no worse in AI translation than in the translation of a professional translator.
Until recently, just a few years ago, even machine translation of simple semi-formal texts looked rather poor. Today everything has changed, and AI competes with humans in the translation of literary works.
Recent advances in machine learning have greatly improved the quality of automatic translation tools. Currently, these tools are mainly used to translate simple sentences, as well as short texts or informal documents. But not for the translation of fiction.
Literary texts, whether novels or short stories, are still translated by experienced human translators. Although several studies have explored the potential of computational models for translation of literary texts, the results in this area are still unsatisfactory.
Researchers at the University of Massachusetts Amherst conducted a study on the quality of translations into English of literary texts made by machines. Scientists have compared them with translations created by people.
“Machine translation (MT) can complement the work of human translators by improving both learning procedures and their overall effectiveness,” wrote lead author Katherine Tai and her colleagues.
“Literary translation is difficult because the translator must balance semantic equivalence, readability and interpretability of the translation. This makes the literary MT more difficult for computer modeling and evaluation.”
Evaluation of human and machine translation
The main goal of the new work by Katherine Tai and her colleagues was to better understand why modern machine translation tools still fail at translating fictional texts compared to human translation.
“We are collecting a data set (PAR3) of non-English language novels, each of which is compared paragraph by paragraph with both human and automatic English translations,” Tai explained.
PAR3 is a dataset compiled by researchers. It contains 121,000 paragraphs from 118 novels written in languages other than English.
For each of these paragraphs, the dataset includes several different human translations as well as a Google Translate translation.
The researchers compared the quality of a paragraph’s human translation to Google’s, using common metrics to rate machine translation tools. At the same time, scientists asked experienced translators which translations they prefer.
“Using PAR3, we found that experienced literary translators showed a preference for human translations of paragraphs 84% of the time, while current automatic machine translation metrics do not correlate with these preferences,” Tai and her colleagues wrote.
“Experts note that the results of machine translation contain not only incorrect translations, but also stylistic inconsistencies.”
In fact, the results obtained by Tai and her colleagues showed that the metrics adopted for evaluating machine translation (eg, BLEU, BLEURT, and BLONDE) were not particularly effective because human translators did not agree with their results.
But the feedback received from human translators allowed the researchers to identify specific problems with translations created using Google Translate.
And then came GPT-3…
Using expert feedback as a guide, the team eventually created an automatic post-editing model based on GPT-3, a large linguistic model developed by OpenAI. The researchers found that experienced human translators preferred literary translations made by this machine model 69% of the time. This is a very high result.
In the future, the results of this study may form the basis for new research on the use of machine translation tools for the translation of literary texts.
In addition, the PAR3 dataset compiled by Tai and colleagues is publicly available on GitHub and can be used by other teams to train or evaluate their literary translation language models.
—
Online:
Contact us: [email protected]
Our Standards, Terms of Use: Standard Terms And Conditions.