Hall, G.T., Oliveto, P.S. and Sudholt, D. (2022) On the impact of the performance metric on efficient algorithm configuration. Artificial Intelligence, 303. 103629. ISSN 0004-3702
Abstract
Algorithm configurators are automated methods to optimise the parameters of an algorithm for a class of problems. We analyse the impact of the cutoff time κ (the time spent evaluating a configuration for a problem instance) on the expected number of configuration comparisons required to find the optimal parameter value for the performance metrics (the measure used to judge the performance of a configuration) that compare configurations using either the best-found fitness values or optimisation times. We first prove that the configurators that use optimisation time as performance metric are not able to tune any unary unbiased algorithm for any function with up to an exponential number of optima using κ ≤ (n ln n)/2. Afterwards, we show that for simple algorithm configuration scenarios the required cutoff time for the optimisation time metric may be considerably larger while using the best fitness metric allows the tuners to configure the target algorithm in linear time in the number of parameters.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2021 Elsevier B.V. This is an author produced version of a paper subsequently published in Artificial Intelligence. Uploaded in accordance with the publisher's self-archiving policy. Article available under the terms of the CC-BY-NC-ND licence (https://creativecommons.org/licenses/by-nc-nd/4.0/). |
Keywords: | Algorithm configurators; parameter tuning; runtime analysis; performance metrics; cutoff time |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number Engineering and Physical Sciences Research Council EP/M004252/1 European Cooperation in Science and Technology CA15140 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 01 Dec 2021 11:06 |
Last Modified: | 04 Nov 2022 01:13 |
Status: | Published |
Publisher: | Elsevier BV |
Refereed: | Yes |
Identification Number: | 10.1016/j.artint.2021.103629 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:181085 |