Humans expect interlocutors both human and robot to resolve spoken references to visually-perceivable objects incrementally as the referents are verbally described. For this reason, tight integration of visual search with natural language processing, and real-time operation of both are requirements for natural interactions between humans and robots. We present an integrated robotic architecture with novel incremental vision and natural language processing. We demonstrate that incrementally refining attentional focus using linguistic constraints achieves significantly better performance of the vision system compared to non-incremental visual processing.
@inproceedings{krauseetal13aamas,
title={Incrementally Biasing Visual Search Using Natural Language Input},
author={Evan Krause and Rehj Cantrell and Ekaterina Potapova and Michael Zillich and Matthias Scheutz},
year={2013},
booktitle={Proceedings of AAMAS},
pages={31--38}
url={https://hrilab.tufts.edu/publications/krauseetal13aamas.pdf}
}