Facebook has announced new artificial intelligence that will be able to perform direct paired translations between more than 100 languages. Facebook AI researcher Angela Fan told Engadget about this. This means that the system will be able to translate texts without using English as an intermediate language.
Most translators first translate texts from the source into English and then into another foreign language. Because of this, the process becomes complicated and the accuracy of some elements of the text is lost. According to Fan, the new AI will be able to produce bi-directional translations (for example, from French to Chinese and vice versa) without using English. According to the BLEU (translated text quality assessment algorithm), the translation quality is 10 points higher than using the English-oriented model.
To create a new AI, the Facebook team used the CommonCrawl service to collect text samples from across the web. After that, the developers analyzed all the material using the FastText system, which the company created several years ago. As a result, it was possible to form a database of 7.5 billion sentences, which the system uses for translation.
How many languages the system will be able to handle in the future is not yet clear. According to Fan, everything depends on the resources that will be at the disposal of the system. Major languages such as Indian, Chinese, or German have a lot of material because they publish a lot of texts on the web. There are many more problems with languages like Zulu, which are not so common.
The company will provide the international research community with an open-source dataset and operating models. It is not yet clear when the system will be introduced into the daily work of the social network.