SIGdial 2012

13th Annual SIGdial Meeting on Discourse and Dialogue

Enhancing Referential Success by Tracking Hearer Gaze

Presented by:
Konstantina Garoufi
Alexander Koller, Konstantina Garoufi, Maria Staudte, Matthew Crocker

The ability to monitor the communicative success of its utterances and, if necessary, provide feedback and repair is useful for a dialog system. We show that in situated communication, eyetracking can be used to reliably and efficiently monitor the hearer's reference resolution process. An interactive system that draws on hearer gaze to provide positive or negative feedback after referring to objects outperforms baseline systems on metrics of referential success and user confusion.