Müller, VC orcid.org/0000-0002-4144-4957 (2020) Measuring Progress in Robotics: Benchmarking and the ‘Measure-Target Confusion’. In: Bonsignorio, F, Messina, E, del Pobil, AP and Hallam, J, (eds.) Metrics of Sensory Motor Coordination and Integration in Robots and Animals: How to Measure the Success of Bioinspired Solutions with Respect to their Natural Models, and Against More ‘Artificial’ Solutions? Cognitive Systems Monographs, 36 . Springer , Cham, Switzerland , pp. 169-179. ISBN 978-3-030-14124-0
Abstract
While it is often said that in order to qualify as a true science robotics should aspire to reproducible and measurable results that allow benchmarking, I argue that a focus on benchmarking will be a hindrance for progress. Several academic disciplines that have been led into pursuing only reproducible and measurable ‘scientific’ results—robotics should be careful not to fall into that trap. Results that can be benchmarked must be specific and context-dependent, but robotics targets whole complex systems independently of a specific context—so working towards progress on the technical measure risks missing that target. It would constitute aiming for the measure rather than the target: what I call ‘measure-target confusion’. The role of benchmarking in robotics shows that the more general problem to measure progress towards more intelligent machines will not be solved by technical benchmarks; we need a balanced approach with technical benchmarks, real-life testing and qualitative judgment.
Metadata
Item Type: | Book Section |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © Springer Nature Switzerland AG 2020. This is an author produced version of a chapter published in Metrics of Sensory Motor Coordination and Integration in Robots and Animals: How to Measure the Success of Bioinspired Solutions with Respect to their Natural Models, and Against More ‘Artificial’ Solutions?. Uploaded in accordance with the publisher's self-archiving policy. |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Arts, Humanities and Cultures (Leeds) > Inter-Disciplinary Ethics Applied (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 27 Apr 2020 13:35 |
Last Modified: | 28 Apr 2020 00:05 |
Published Version: | https://www.springer.com/gp/book/9783030141240 |
Status: | Published |
Publisher: | Springer |
Series Name: | Cognitive Systems Monographs |
Identification Number: | 10.1007/978-3-030-14126-4_9 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:159910 |