Technology : Do what I mean, not what I say

Technology 12 July 1996


HUMAN speech can be recognised by computers almost routinely, as long as the
speaker carefully enunciates each word and sticks to simple phrases. Normal
conversation poses a much tougher challenge, however, and scientists in the US
have now tackled it with a program that responds to the context of the
as well as to the words it recognises.

A team led by James Allen, a professor of computer science at the University
of Rochester in New York state, has programmed a Sun workstation to respond to
voice commands telling it to route imaginary trains between various cities in
the northeastern US. They have shown that their Trains Project machine can
its response to the context of the dialogue, and make sensible replies
even when
it mishears what someone says.

The system works in three stages. A standard speech recognition program
turns sounds into words, which it feeds to software that analyses the
grammatical structure of the sentences. The new part comes in the third
stage of
Allen’s system, which is designed to interpret sentences in the context of the
rest of the dialogue. “It’s a very difficult thing to instil into the
Allen says. People, of course, do it naturally.


As the conversation proceeds, the computer stores the ongoing dialogue in a
large buffer and uses it to make a final interpretation. For example, knowing
that the last instruction was “Let’s start in Detroit” would help the computer
to infer that “garble garble Baltimore” means “Go to Baltimore”.

Allen says that current speech recognition systems do very well with tightly
defined tasks, but understand only 40 per cent of the words spoken in a two-way
conversation. The speech recognition software used in the Trains Project
recognised about 75 per cent of the words spoken by the experimenters.

“When we started, we thought that rates like that probably wouldn’t be
to keep a conversation going well,” says Allen. But they found that their
software could fill in missing details and make corrections to words that were
misunderstood by the speech recognition software. For example, when the machine
thought it heard “Let’s see contain from Detroit to Washington,” it corrected
the sentence to “Let’s take a train from Detroit to Washington.”

At a meeting of the Association for Computational Linguistics in California
last month, Allen let people test the system by saying whatever they liked. He
concedes that the system did not work every time, but says it was successful
more often than not.