Human/Robot Interaction

Research in this area focuses on natural human channels of communication, such as natural language and gestures, to provide easy and friendly multi-modal interfaces between human and robots.   We hypothesize that natural interfaces facilitate fast and easy learning of the interface, thereby permitting the human user to concentrate more on the task at hand, rather than on the tools of the interface.   Furthermore, since human-human interaction is facilitated by natural modes of communication, we believe human-robot interfaces can benefit by mirroring this type of interaction.   We have designed a natural language module that employs a commercial off-the-shelf speech recognition system which maps an acoustic signal to a text string.   The latter is robustly parsed by our in-house natural language understanding system, Nautilus, resulting in a semantic representation.   This representation allows us to perform further linguistic analysis on pronominal dereferencing, quantification, and contextual analysis.   Our linguistic analyses of the input is integrated with various other modules on several different mobile and stationary robot platforms to enable spatial referencing and reasoning, as well as the ability to interact with various domain-specific applications.   The figure illustrates the InterBot architecture, showing the various modules of the multimodal human-robot interface.

Finally, research in this area is prompting us to look at how natural modes of communication can further enhance cooperation and collaboration between multiple human and robotic agents in a discourse.

Click for Full publication list

Contact:

Dennis Perzanowski, Co-Investigator

Alan C. Schultz, Co-Investigator