Using AI to match human performance in translating news from Chinese to English

Compare this to one of Google&#x27;s blog post promoting their MT research: <a href="https:&#x2F;&#x2F;research.googleblog.com&#x2F;2016&#x2F;09&#x2F;a-neural-network-for-machine.html" rel="nofollow">https:&#x2F;&#x2F;research.googleblog.com&#x2F;2016&#x2F;09&#x2F;a-neural-network-for...</a><p>It is:<p>1) More accurate, compared to hyperbole like e.g. &quot;Bridging the Gap between Human and Machine Translation&quot; we have right there in the title the domain: news.<p>2) A more impressive result. This result is on an independently set up evaluation framework, compared to Google&#x27;s which used their own framework.<p>Compare further the papers: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;1609.08144.pdf" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;1609.08144.pdf</a> <a href="https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;uploads&#x2F;prod&#x2F;2018&#x2F;03&#x2F;final-achieving-human.pdf" rel="nofollow">https:&#x2F;&#x2F;www.microsoft.com&#x2F;en-us&#x2F;research&#x2F;uploads&#x2F;prod&#x2F;2018&#x2F;0...</a><p>These researcher appear to have been much clearer about what they&#x27;re actually claiming, and also used more standard evaluation tools (Appraise) and methodology rather than something haphazardly hacked together.

As impressive as it may be, these people should refrain from claiming &#x27;human-like&#x27; translation from a system that has no way of &#x27;knowing&#x27; anything about context, other than statistical occurrences.<p>It is certain that, on occasion, the system will make such mistakes as stating the opposite of what is being said in the first place, or attribute one action to the wrong person, and what not. Perhaps on average it&#x27;s as good as a person, but this system will make mistakes that disqualifies it from being used without a bucket of salt.

The most impressive ML translation tool I&#x27;ve seen so far is DeepL[0].<p>Sometimes, it manages to translate whole articles without errors.<p>[0]: <a href="https:&#x2F;&#x2F;www.deepl.com&#x2F;translator" rel="nofollow">https:&#x2F;&#x2F;www.deepl.com&#x2F;translator</a>

Translate &quot;sentences of news&quot; is very different to translating an entire article, which is obviously what&#x27;s interesting.<p>Is anybody in MT or text comprehension&#x2F;generation really working on systems that construct a model&#x2F;&quot;understanding&quot; of the bigger narrative in a longer-running text? Even just to be able to do correct anaphora resolution across sentence and paragraph boundaries, but intuitively also WSD seems easier if you&#x27;ve got some sort of abstract context over more than just a sentence.

Be careful when reading such claims: <a href="https:&#x2F;&#x2F;www.theatlantic.com&#x2F;technology&#x2F;archive&#x2F;2018&#x2F;01&#x2F;the-shallowness-of-google-translate&#x2F;551570&#x2F;?single_page=true" rel="nofollow">https:&#x2F;&#x2F;www.theatlantic.com&#x2F;technology&#x2F;archive&#x2F;2018&#x2F;01&#x2F;the-s...</a>

What I find most interesting is the multiple training methods used to get the network to improve its performance. They name a few in the article:<p>- dual learning - deliberation networks - joint training - agreement regularization<p>I haven&#x27;t read the paper to see how these are combined but it makes intuitive sense that using multiple training methods can lead to better performance. That is to say, to more effectively search the weight space of the network.

I find these types of &quot;match human performance&quot; claims to be ridiculous, especially when it comes to Chinese -&gt; English translations. Translation is both an art and a science, requiring nuanced understanding of the languages, cultures, and context. It also demands quite a bit of creativity. No translation tool I&#x27;ve tried has come even close to matching human performance of a good human translator, including microsoft&#x27;s tools. AI will need to reach the point where its understanding of language, culture, context, and creative ability matches that of humans to truly be capable of &quot;human performance&quot; in translation.

It&#x27;s obvious that there are limits to how well machine translation can work unless the models have sensory grounding. I wonder if the problem is that people haven&#x27;t figured out how to do sensory grounding or that the hardware is still too slow for it to work.

About translators solely reliant on NN. The thing is, while 70% of output can be well passable, some of the rest can be very weird if original input was not learned. Like a string of gibberish turning into 10 full sentences.<p>You have to score the extent of wrongness too.

Would be nice to have improved MTL performance for Wuxia&#x2F;Xanxia webnovels.

I&#x27;d suggest addressing non-English to non-English translations first, which is usually limited in most engines out there compared to translations to&#x2F;from English.

Amazing when can i start reading chinese cs boards