Building robots, even for performing simple tasks, requires the designer to assess performance using various parameters. However, sometimes the best solution is not the one that performs best on average. Hence, other ways of evaluating performance are necessary. We ran a broad parameter sweep for an agent-based simulation of a robotic area coverage task with very simple agent controllers in four different task environments. Analysis of the results emphasizes the importance of considering the entire distribution across randomized starting conditions, and not just the mean overall performance, when assessing the effectiveness of parameter settings. Our findings indicate the potential for robotic system designers to constrain or specify the qualities of system performance distributions.
@inproceedings{buckinghametal17ecal,
title={Better Than Average: Analyzing Distributions to Understand Robot Behavior in a Multi-agent Area Coverage Scenario},
author={David Buckingham and Giordano B. S. Ferreira and Matthias Scheutz},
year={2017},
booktitle={Proceedings of 14th European Conference on Artificial Life},
pages={467--474}
url={https://hrilab.tufts.edu/publications/buckinghametal17ecal.pdf}
}