Zhao, J., Yang, R., Qiu, S. et al. (1 more author) (2024) Unleashing the Potential of Acquisition Functions in High-Dimensional Bayesian Optimization. Transactions on Machine Learning Research. ISSN 2835-8856
Abstract
Bayesian optimization (BO) is widely used to optimize expensive-to-evaluate black-box functions. It first builds a surrogate for the objective and quantifies its uncertainty. It then decides where to sample by maximizing an acquisition function (AF) defined by the surrogate model. However, when dealing with high-dimensional problems, finding the global maximum of the AF becomes increasingly challenging. In such cases, the manner in which the AF maximizer is initialized plays a pivotal role. An inappropriate initialization can severely limit the potential of AF.
This paper investigates a largely understudied problem concerning the impact of AF maximizer initialization on exploiting AFs' capability. Our large-scale empirical study shows that the widely used random initialization strategy may fail to harness the potential of an AF. Based on this observation, we propose a better initialization approach by employing multiple heuristic optimizers to leverage the historical data of black-box optimization to generate initial points for an AF maximizer. We evaluate our approach with a variety of heavily studied synthetic test functions and real-world applications. Experimental results show that our techniques, while simple, can significantly enhance the standard BO and outperform state-of-the-art methods by a large margin in most test cases.
Metadata
Authors/Creators: |
|
||||
---|---|---|---|---|---|
Copyright, Publisher and Additional Information: | This item is protected by copyright. This is an open access article under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. | ||||
Dates: |
|
||||
Institution: | The University of Leeds | ||||
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) | ||||
Funding Information: |
|
||||
Depositing User: | Symplectic Publications | ||||
Date Deposited: | 23 Jan 2024 11:13 | ||||
Last Modified: | 23 Jan 2024 11:13 | ||||
Published Version: | https://openreview.net/forum?id=0CM7Hfsy61 | ||||
Status: | Published online | ||||
Publisher: | OpenReview.net | ||||
Related URLs: |