With robotics rapidly advancing, more effective human–robot interaction is increasingly needed to realize the full potential of robots for society. While spoken language must be part of the solution, our ability to provide spoken language interaction capabilities is still very limited. In this article, based on the report of an interdisciplinary workshop convened by the National Science Foundation, we identify key scientific and engineering advances needed to enable effective spoken language interaction with robotics. We make 25 recommendations, involving eight general themes: putting human needs first, better modeling the social and interactive aspects of language, improving robustness, creating new methods for rapid adaptation, better integrating speech and language with other communication modalities, giving speech and language components access to rich representations of the robot’s current knowledge and state, making all components operate in real time, and improving research infrastructure and resources. Research and development that prioritizes these topics will, we believe, provide a solid foundation for the creation of speech-capable robots that are easy and effective for humans to work with.
@article{margeetal21csl,
title={Spoken language interaction with robots: Recommendations for future research},
author={Marge, Matthew and Espy-Wilson, Carol and Ward, Nigel G. and Alwan, Abeer and Artzi, Yoav and Bansal, Mohit and Blankenship, Gil and Chai, Joyce and Daumé, Hal and Dey, Debadeepta and Harper, Mary and Howard, Thomas and Kennington, Casey and Kruijff-Korbayová, Ivana and Manocha, Dinesh and Matuszek, Cynthia and Mead, Ross and Mooney, Raymond and Moore, Roger K. and Ostendorf, Mari and Pon-Barry, Heather and Rudnicky, Alexander I. and Scheutz, Mattias and St. Aman, Robert and Sun,
Tong and Tellex, Stefanie and Traum, David and Yu, Zhou},
year={2021},
journal={Computer Speech & Language},
url={https://hrilab.tufts.edu/publications/margeetal21csl.pdf}
}