Alibaba Neural Network Defeats Human In Global Reading Test

Alibaba says its deep neural network model has outscored humans in a global reading test, paving the way for the underlying technology to reduce the need for human input.

The Chinese tech giant's research unit, Institute of Data Science of Technologies (IDST), said it had developed a deep-learning model that attained a score of 82.44 in Exact Match on the Stanford Question Answering Dataset (Squad). Humans had clocked a previous score of 82.304, it said.

Squad comprised more than 100,000 question-and-answer sets based on more than 500 Wikipedia articles, in which participants were required to build machine-learning models to respond to the questions. These models would be evaluated by Squad, which then would run the model on the test set.

Various universities, research institutions, and technology vendors were participants including Tencent, Google, IBM, Microsoft, Samsung, Tel-Aviv University, and South Korea's Kangwon National University. A handful had participated multiple times in the past year including Microsoft Research Asia, which previous score of 82.136 was clocked on December 17, 2017, while Alibaba's previous score of 79.199 was recorded on December 28, 2017.

In its statement Monday, the Chinese vendor claimed to be the first to surpass humans in the test, but Squad listed the Chinese vendor as shared leader alongside Microsoft Research Asia, which scored a higher 82.65. According to the Squad list, Microsoft's score was recorded on January 3, 2018, while Alibaba's was registered on January 5, 2018.

Alibaba said its neural network model was based on the Hierarchical Attention Network, which it explained would read "from paragraphs to sentences to words" to identify phases that could hold potential answers. This underlying technology previously was used in its Singles Day shopping festival to respond to customer inquiries.

The company had said its AI-powered customer service chatbot, Dian Xiaomi, was used to support its online merchants and served an average of 3.5 million users daily across its Taobao and Tmall platforms.

Commenting on the Squad score, Alibaba IDST's chief scientist of natural language processing Si Luo, said: "That means objective questions such as 'what causes rain' can now be answered with high accuracy by machines. We believe the technology underneath can be gradually applied to numerous applications such as customer service, museum tutorials, and online responses to medical inquiries from patients, decreasing the need for human input in an unprecedented way."

Si said Alibaba would be "sharing our model-building methodology" with the community and planned to apply the technology to support its customers in the near future.