Muri13

Summary

Moral Competence in Computational Architectures for Robots is a Multidisciplinary University Research Initiative (MURI), awarded in 2014 by the Department of Defense (via the Office of Naval Research) to a team of researchers from Tufts University, Brown University, RPI, Georgetown University, and Yale University. Its purpose is to identify the logical, cognitive, and social underpinnings of human moral competence, model those principles of competence in human-robot interactions, and demonstrate novel computational means by which robots can reason and act ethically in the face of complex, practical challenges.

For full summary see the Summary page...

Summary

I. Foundations of Moral Competence

Designing robots to achieve moral competence naturally relies on the many layers and connections of concepts, rules, feelings, and judgments that constitute human morality. This project will therefore map out crucial concepts and processes that underlie an agent's ability to interact morally with other agents. This will entail detailed examination of how moral norms are represented and operate, what minimal vocabulary is needed for communication on moral matters, and which cognitive processes underlie judgments and decisions about moral action. Accompanying these efforts will be a robust treatment of affective processes that help generate moral perception, action, and communication. The project then will use these grounded insights to inform thorough logical and mathematical elaborations of modes of inference critical for moral assessments, including deductive, analogical, and counterfactual operations. In these ways the project uncovers the real-life conditions of moral competence while identifying the logical operations and principles that represent and communicate norms and values effectively.

II. Moral Reasoning and Decision-Making

The project will produce and rely on an unprecedented hierarchy (H) of formal systems for robot reasoning and decision-making in the realm of ethics, including natural-language understanding (U) and generation(G) technology that not only allows a robot to understand human commands and queries about ethics, but also to generate proofs and arguments that justify its ethical reasoning and decision-making to humans. The first and fastest part of H is designed to simply flag ethically charged situations, and is inspired by UIMA, of Jeopardy!-winning Watson fame. The next level relies on a hybrid form of reasoning that blends deduction and analogy. The remaining levels of H become increasingly deliberative and expressive. These levels will be multi-operator intensional computational logics, into which U can inject information provided by English-speaking humans, and out of which G can draw and present information to these same humans. H, U, and G will in large measure be provided as open-source resources to a world increasingly (and rightly so!) concerned with the ethical status of actions performed by autonomous robots and other autonomous artificial agents.

III. Computational Architecture

This project takes up the challenge of developing a computational architecture that integrates the different perceptual and logical systems a moral agent needs to master real-life, ethically charged situations. For autonomous systems in particular, this requires an architecture that can manage multiple and parallel components of detection, discernment, and abstraction when encountering physical objects in the environment, natural language, subtle gestures of face and hands, and expressed goals, values, and reasons on the part of other agents. The project will use the cognitive robotic DIARC architecture as an architectural framework that has shown itself uniquely positioned to provide scalable, adaptive, and robotically compatible means for cognition, action, and communication.

IV. Human-Robot Interaction Demonstrations

Throughout its work on moral competence, this project will also make theoretical and empirical advances on the specific challenges and conditions that human-robot interactions pose. This means gaining a more confident understanding of how humans react to robots, in what ways and for what reasons they deny or grant moral agency to robots, and where robots may even exceed ordinary moral competence. Research into those challenges and conditions will guide the development of computational architecture as well as the physical appearance and mobility of robots. The project will culminate in rigorous, demanding demonstrations of robots succeeding at reasoning, reacting, and acting ethically amid shifting conditions, conflicting goals, and competing values.

Sample Publications

Bertram Malle, and Matthias Scheutz (2014) Moral competence in social robots proceedings of ieee international symposium on ethics in engineering, science, and technology (ethics)

Bertram Malle, Matthias Scheutz, Thomas Arnold, John Voiklis, and Corey Cusimano (2015) Sacrifice one for the good or many? People apply different moral norms to human and robot agents proceedings of 10th acm/ieee international conference on human-robot interaction

See more project publications on the Publications page

Press

Questions about what actions robots will and should have the leeway, direction, or authority to carry out have elicited fears, fantasies, and a wide range of future projections. To contribute usefully to these larger public discussions, MURI investigators have sought to discuss critical questions of social responsibility while grounded in real-world research, policy, design, and engineering.

For MURI-related articles and other pieces see the Press tab