• Sunday, March 01, 2015


The Bot that almost passed as a Human

Hasan Shahriar

When I told the chatter bot Eugene Goostman, the only programme to have “allegedly” passed the Turing test, that I was supposed to write on him, he replied: “No wonder -- I suspected you to be judged to write on me.”
Developed by Vladimir Veselov, Eugene Demchenko, and Sergey Ulasen in 2001, he is supposed to be a 13-year-old Ukrainian kid with a pet guinea pig. The age was intentional as Veselov thought it was “not too old to know everything and not too young to know nothing.”
On June 7, 2014 the University of Reading organised a Turing test competition at the Royal Society to mark the 60th anniversary of Alan Turing, and Eugene “passed” it by fooling 33 percent of the judges into believing he was a human being.
Thirty judges took part in the event and each participated in conversing simultaneously with a human and a bot five times, not knowing which is which. Ten of them failed to distinguish the bot from human, enabling the organisers to label Eugene as the first ever programme to pass the test.
The computer scientist Alan Turing outlined the Turing test in an academic paper published in 1950. It tests a machine's ability to display intelligent behaviour indistinguishable from humans. The test does not assess the skill to give correct answers but to rather give typical human answers.
However, Eugene can maintain this illusion of “giving typical human responses” for a very short time and even then only if the conversation is an extremely simple one. After two minutes or even less, he'll start fumbling around with his replies, and will even start repeating himself.
Gary Marcus, a professor of cognitive science wrote that Eugene is simply “a cleverly-coded piece of software, heir to a programme called ELIZA that was first developed -- as a joke -- in the nineteen-sixties.”

ELIZA too was taken quite seriously when it first came out. Users would send in problems via Teletype (one of the ancestors to text messaging) and ELIZA would spew out answers, posing as a doctor. When the “patient”, that is the user, exceeds its small knowledge base, it'll start replying with generics. For example, if someone asks: “My arms ache,” it will answer: “Why do you say your arms ache?”  After some time, the conversation would begin to sound nonsensical.
Eugene can make the illusion last longer than ELIZA, and has a “character” of its own, that of a precocious adolescent. He lasts longer by changing the subject constantly and asking questions, even resorting to jokes to hide his limitations.
There's controversy on whether Eugene did pass the Turing test or not. Alan Turing declared that “if a machine could be mistaken for a human being more than 30 percent of the time during a series of keyboard conversations with actual humans, then it just might be 'thinking'.”
But Gary Marcus opines: “Goostman, like ELIZA, relies mainly upon pattern recognition, not genuine understanding. It is a refinement of an old idea, not a fundamental change in artificial intelligence.”
Moreover, all the judges were given only 5 minutes to chat, which some of them felt insufficient and too brief. Also, given that Eugene is supposed to be Ukrainian, the judges were more forgiving with the odd grammatical glitches and awkward wording in English than otherwise.
All that aside, it is  safe to say -- whether or not Eugene passed the test with the help of his tricks -- that we are still eons away from having the capability to ape the soft, calm sentient computer HAL 9000 from A.C. Clarke's “2001: A Space Odyssey”.

Published: 12:00 am Thursday, June 19, 2014

Leave your comments | Comment Policy
ICC Cricket World Cup