Loup Ventures published its annual Smart Speaker IQ test results, revealing how the four major personal assistant AIs performed when 800 questions were asked each. Compared to the previous year, three products had the highest gain in IQ points and were able to respond to a substantially higher percentage of questions compared to 2017. However, the Google Assistant still outperformed them at all levels.
The study was quite simple, involving 800 questions presented to AIs with two categories of results: the question was correctly understood and, if the answer was yes, was a correct answer provided? Google ranked first, with Siri a decent second.
The Google Assistant understood 100% of his questions and was able to provide correct answers to 87.9% of them. Apple's Siri came in second, with 99.6% of queries understood and 74.6% of correct answers.
Amazon's popular Alexa ranked third in the study, comprising 99% of the questions and having only correct answers for 72.5% of the questions. Finally, Microsoft's Cortana was last in the "correctly answered" category, with 63.4 percent, but managed to divert Alexa to third place in the "consultation comprised" with a rate of 99.4 percent.
In terms of correctly answered questions, all four products improved in 2018. Alexa increased by 64%, Google Assistant by 81%, Cortana by 56% and Siri by 52%.
Breaking the results this year, Loup Ventures found that almost all misinterpreted questions have a proper name and that it was usually a local restaurant or town. Natural language processing and voice recognition were noted to have improved among all four assistants.