Yesterday, I attended another lecture, a series held by the KMDI entitled “Understanding Language Understanding”, which was a clumsy way of saying it was a talk by three professors on computational linguistics. Prof. Gerald Penn and Prof. Graeme Hirst are computer scientists, while Prof. Ian Lancashire offered an alternative perspective from the English studies angle.

Lancashire — recently in the spotlight with his Agatha Christie Research — emphasized the importance of an authorial context. He did not cite specific examples but drew on the experience of being misunderstood as to the intent behind his textual analysis research of Christie’s work. What grew out of respect for a writer, became misconstrued as mischief or some malicious intent.

Prof. Hirst was more philosophical in his talk. He succinctly outlined the limits and opportunities that lie within textual analysis, and hinted at the much broader implications in the analysis of ideological texts. One important thing he noted is that for successful system-user interaction, the two “must share the same domain and framework to engender dialogue”. More specifically, current text-based IR systems matches text-strings, but it is not smart enough to match text-meanings. The issue of engaging in dialogue with a system is AI at large, but before we start training robots to bring us a beer when we say “I’d like a beer.”, there are many applications across the fields.

Prof. Penn’s talk was a refreshing example of someone in the CS field advocating for user-based evaluation, and not merely setting the bar based on often arbitrary and meaningless benchmarks like 0% WER (Word Error Rate) in automated transcriptions of text. According to research done by Munteanu et al. in 2006, often a 25%-45% accuracy rate is sufficient for comprehension. We have expectations of machines to do what humans do, only faster than we could ever process with our brains and our bodies. Yet, if we are designing systems to serve people, conducting research from the user’s perspective first seems to make sense.

As a tangent, this is perhaps one of the key philosophies that makes apple so successful. Since Steve Job returned to Apple over ten years ago, that has been his driving philosophy. Put the user’s needs, expectations, and desires first. Don’t give them a machine, and then tell them they need to learn how to love it. It was ground breaking approach then, and it seems that in the academic and business circles, it still is.

Best conversation in this lecture:

Q: “Can you suggest reasons why there were three paradigm shifts in the three major periods in computational linguistics?”
Hirst: “Well, first of all, there were two paradigm shifts, for a total of three periods…”