Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound
| Title: | Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound |
|---|---|
| Authors: | Takeno, Shion, Tamura, Tomoyuki, Shitara, Kazuki, Karasuyama, Masayuki |
| Source: | Proceedings of the 39th International Conference on Machine Learning, PMLR 162:20960-20986, 2022 |
| Publication Year: | 2021 |
| Collection: | Computer Science |
| Subject Terms: | Computer Science - Machine Learning |
| More Details: | Max-value entropy search (MES) is one of the state-of-the-art approaches in Bayesian optimization (BO). In this paper, we propose a novel variant of MES for constrained problems, called Constrained MES via Information lower BOund (CMES-IBO), that is based on a Monte Carlo (MC) estimator of a lower bound of a mutual information (MI). Unlike existing studies, our MI is defined so that uncertainty with respect to feasibility can be incorporated. We derive a lower bound of the MI that guarantees non-negativity, while a constrained counterpart of conventional MES can be negative. We further provide theoretical analysis that assures the low-variability of our estimator which has never been investigated for any existing information-theoretic BO. Moreover, using the conditional MI, we extend CMES-IBO to the parallel setting while maintaining the desirable properties. We demonstrate the effectiveness of CMES-IBO by several benchmark functions and real-world problems. Comment: 39pages, 8 figures |
| Document Type: | Working Paper |
| Access URL: | http://arxiv.org/abs/2102.09788 |
| Accession Number: | edsarx.2102.09788 |
| Database: | arXiv |
| FullText | Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2102.09788 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20210219&spage=&pages=&title=Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound&atitle=Sequential-%20and%20Parallel-%20Constrained%20Max-value%20Entropy%20Search%20via%20Information%20Lower%20Bound&aulast=Takeno%2C%20Shion&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edsarx DbLabel: arXiv An: edsarx.2102.09788 RelevancyScore: 1009 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1008.96203613281 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Takeno%2C+Shion%22">Takeno, Shion</searchLink><br /><searchLink fieldCode="AR" term="%22Tamura%2C+Tomoyuki%22">Tamura, Tomoyuki</searchLink><br /><searchLink fieldCode="AR" term="%22Shitara%2C+Kazuki%22">Shitara, Kazuki</searchLink><br /><searchLink fieldCode="AR" term="%22Karasuyama%2C+Masayuki%22">Karasuyama, Masayuki</searchLink> – Name: TitleSource Label: Source Group: Src Data: Proceedings of the 39th International Conference on Machine Learning, PMLR 162:20960-20986, 2022 – Name: DatePubCY Label: Publication Year Group: Date Data: 2021 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Machine+Learning%22">Computer Science - Machine Learning</searchLink> – Name: Abstract Label: Description Group: Ab Data: Max-value entropy search (MES) is one of the state-of-the-art approaches in Bayesian optimization (BO). In this paper, we propose a novel variant of MES for constrained problems, called Constrained MES via Information lower BOund (CMES-IBO), that is based on a Monte Carlo (MC) estimator of a lower bound of a mutual information (MI). Unlike existing studies, our MI is defined so that uncertainty with respect to feasibility can be incorporated. We derive a lower bound of the MI that guarantees non-negativity, while a constrained counterpart of conventional MES can be negative. We further provide theoretical analysis that assures the low-variability of our estimator which has never been investigated for any existing information-theoretic BO. Moreover, using the conditional MI, we extend CMES-IBO to the parallel setting while maintaining the desirable properties. We demonstrate the effectiveness of CMES-IBO by several benchmark functions and real-world problems.<br />Comment: 39pages, 8 figures – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2102.09788" linkWindow="_blank">http://arxiv.org/abs/2102.09788</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2102.09788 |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2102.09788 |
| RecordInfo | BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Machine Learning Type: general Titles: – TitleFull: Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Takeno, Shion – PersonEntity: Name: NameFull: Tamura, Tomoyuki – PersonEntity: Name: NameFull: Shitara, Kazuki – PersonEntity: Name: NameFull: Karasuyama, Masayuki IsPartOfRelationships: – BibEntity: Dates: – D: 19 M: 02 Type: published Y: 2021 |
| ResultId | 1 |