As development of robots with the ability to self-assess their proficiency for accomplishing tasks continues to grow, metrics are needed to evaluate the characteristics and performance of these robot systems and their interactions with humans. This proficiency-based human-robot interaction (HRI) use case can occur before, during, or after the performance of a task. This article presents a set of metrics for this use case, driven by a four-stage cyclical interaction flow: (1) robot self-assessment of proficiency (RSA), (2) robot communication of proficiency to the human (RCP), (3) human understanding of proficiency (HUP), and (4) robot perception of the human’s intentions, values, and assessments (RPH). This effort leverages work from related fields including explainability, transparency, and introspection, by repurposing metrics under the context of proficiency self-assessment. Considerations for temporal level (a priori, in situ, and post hoc) on the metrics are reviewed, as are the connections between metrics within or across stages in the proficiency-based interaction flow. This article provides a common framework and language for metrics to enhance the development and measurement of HRI in the field of proficiency self-assessment.
@article{nortonetall22thri,
title={Metrics for Robot Proficiency Self-Assessment and Communication of Proficiency in Human-Robot Teams},
author={Norton, Adam and Admoni, Henry and Crandall, Jacob and
Fitzgerald, Tesca and Gautam, Alvika and Goodrich,
Michael and Saretsky, Amy and Scheutz, Matthias and
Simmons, Reid and Steinfeld, Aaron and Yanco, Holly},
year={2022},
journal={ACM Transactions on Human-Robot Interaction},
volume={11},
url={https://hrilab.tufts.edu/publications/nortonetall22thri.pdf}
doi={10.1145/3522579}
}