The rise of military drones and other robots deployed in ethically-sensitive contexts has fueled interest in developing autonomous agents that behave ethically. The ability for autonomous agents to independently reason about situational ethics will inevitably lead to confrontations between robots and human operators regarding the morality of issued commands. Ideally, a robot would be able to successfully convince a human operator to abandon a potentially unethical course of action. To investigate this issue, we conducted an experiment to measure how successfully a humanoid robot could dissuade a person from performing a task using verbal refusals and affective displays that conveyed distress. The results demonstrate a significant behavioral effect on task-completion as well as significant effects on subjective metrics such as how comfortable subjects felt ordering the robot to complete the task. We discuss the potential relationship between the level of perceived agency of the robot and the sensitivity of subjects to robotic confrontation. Additionally, the possible ethical pitfalls of utilizing robotic displays of affect to shape human behavior are also discussed.
@article{briggsscheutz14ijsr,
title={How Robots Can Affect Human Behavior: Investigating the Effects of Robotic Displays of Protest and Distress},
author={Gordon Briggs and Matthias Scheutz},
year={2014},
journal={International Journal of Social Robotics},
volume={6},
issue={6},
pages={1--13}
url={https://hrilab.tufts.edu/publications/briggsscheutz14ijsr.pdf}
doi={/10.1007/s12369-014-0235-1}
}