| Title: |
Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound |
| Authors: |
Takeno, Shion, Tamura, Tomoyuki, Shitara, Kazuki, Karasuyama, Masayuki |
| Source: |
Proceedings of the 39th International Conference on Machine Learning, PMLR 162:20960-20986, 2022 |
| Publication Year: |
2021 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Machine Learning |
| More Details: |
Max-value entropy search (MES) is one of the state-of-the-art approaches in Bayesian optimization (BO). In this paper, we propose a novel variant of MES for constrained problems, called Constrained MES via Information lower BOund (CMES-IBO), that is based on a Monte Carlo (MC) estimator of a lower bound of a mutual information (MI). Unlike existing studies, our MI is defined so that uncertainty with respect to feasibility can be incorporated. We derive a lower bound of the MI that guarantees non-negativity, while a constrained counterpart of conventional MES can be negative. We further provide theoretical analysis that assures the low-variability of our estimator which has never been investigated for any existing information-theoretic BO. Moreover, using the conditional MI, we extend CMES-IBO to the parallel setting while maintaining the desirable properties. We demonstrate the effectiveness of CMES-IBO by several benchmark functions and real-world problems.
Comment: 39pages, 8 figures |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2102.09788 |
| Accession Number: |
edsarx.2102.09788 |
| Database: |
arXiv |
