SIGdial 2013

14th Annual SIGdial Meeting on Discourse and Dialogue

Multimodality and Dialogue Act Classification in the RoboHelper Project

Lin Chen, Barbara Di Eugenio

We describe the annotation of a multimodal corpus that includes pointing gestures and haptic actions (force exchanges). Haptic actions are rarely analyzed as fullfledged components of dialogue, but our data shows haptic actions are used to advance the state of the interaction. We report our experiments on recognizing Dialogue Acts in both offline and online modes. Our results show that multimodal features and the dialogue game aid in DA classification.