Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization
| Title: | Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization |
|---|---|
| Authors: | Lin, Jingyang, Wang, Yu, Cai, Qi, Pan, Yingwei, Yao, Ting, Chao, Hongyang, Mei, Tao |
| Publication Year: | 2022 |
| Collection: | Computer Science |
| Subject Terms: | Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition |
| More Details: | Outlier detection tasks have been playing a critical role in AI safety. There has been a great challenge to deal with this task. Observations show that deep neural network classifiers usually tend to incorrectly classify out-of-distribution (OOD) inputs into in-distribution classes with high confidence. Existing works attempt to solve the problem by explicitly imposing uncertainty on classifiers when OOD inputs are exposed to the classifier during training. In this paper, we propose an alternative probabilistic paradigm that is both practically useful and theoretically viable for the OOD detection tasks. Particularly, we impose statistical independence between inlier and outlier data during training, in order to ensure that inlier data reveals little information about OOD data to the deep estimator during training. Specifically, we estimate the statistical dependence between inlier and outlier data through the Hilbert-Schmidt Independence Criterion (HSIC), and we penalize such metric during training. We also associate our approach with a novel statistical test during the inference time coupled with our principled motivation. Empirical results show that our method is effective and robust for OOD detection on various benchmarks. In comparison to SOTA models, our approach achieves significant improvement regarding FPR95, AUROC, and AUPR metrics. Code is available: \url{https://github.com/jylins/hood}. Comment: Source code is available at \url{https://github.com/jylins/hood} |
| Document Type: | Working Paper |
| Access URL: | http://arxiv.org/abs/2209.12807 |
| Accession Number: | edsarx.2209.12807 |
| Database: | arXiv |
| FullText | Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2209.12807 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20220926&spage=&pages=&title=Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization&atitle=Out-of-Distribution%20Detection%20with%20Hilbert-Schmidt%20Independence%20Optimization&aulast=Lin%2C%20Jingyang&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edsarx DbLabel: arXiv An: edsarx.2209.12807 RelevancyScore: 1037 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1037.15771484375 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Lin%2C+Jingyang%22">Lin, Jingyang</searchLink><br /><searchLink fieldCode="AR" term="%22Wang%2C+Yu%22">Wang, Yu</searchLink><br /><searchLink fieldCode="AR" term="%22Cai%2C+Qi%22">Cai, Qi</searchLink><br /><searchLink fieldCode="AR" term="%22Pan%2C+Yingwei%22">Pan, Yingwei</searchLink><br /><searchLink fieldCode="AR" term="%22Yao%2C+Ting%22">Yao, Ting</searchLink><br /><searchLink fieldCode="AR" term="%22Chao%2C+Hongyang%22">Chao, Hongyang</searchLink><br /><searchLink fieldCode="AR" term="%22Mei%2C+Tao%22">Mei, Tao</searchLink> – Name: DatePubCY Label: Publication Year Group: Date Data: 2022 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Machine+Learning%22">Computer Science - Machine Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+-+Computer+Vision+and+Pattern+Recognition%22">Computer Science - Computer Vision and Pattern Recognition</searchLink> – Name: Abstract Label: Description Group: Ab Data: Outlier detection tasks have been playing a critical role in AI safety. There has been a great challenge to deal with this task. Observations show that deep neural network classifiers usually tend to incorrectly classify out-of-distribution (OOD) inputs into in-distribution classes with high confidence. Existing works attempt to solve the problem by explicitly imposing uncertainty on classifiers when OOD inputs are exposed to the classifier during training. In this paper, we propose an alternative probabilistic paradigm that is both practically useful and theoretically viable for the OOD detection tasks. Particularly, we impose statistical independence between inlier and outlier data during training, in order to ensure that inlier data reveals little information about OOD data to the deep estimator during training. Specifically, we estimate the statistical dependence between inlier and outlier data through the Hilbert-Schmidt Independence Criterion (HSIC), and we penalize such metric during training. We also associate our approach with a novel statistical test during the inference time coupled with our principled motivation. Empirical results show that our method is effective and robust for OOD detection on various benchmarks. In comparison to SOTA models, our approach achieves significant improvement regarding FPR95, AUROC, and AUPR metrics. Code is available: \url{https://github.com/jylins/hood}.<br />Comment: Source code is available at \url{https://github.com/jylins/hood} – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2209.12807" linkWindow="_blank">http://arxiv.org/abs/2209.12807</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2209.12807 |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2209.12807 |
| RecordInfo | BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Machine Learning Type: general – SubjectFull: Computer Science - Computer Vision and Pattern Recognition Type: general Titles: – TitleFull: Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Lin, Jingyang – PersonEntity: Name: NameFull: Wang, Yu – PersonEntity: Name: NameFull: Cai, Qi – PersonEntity: Name: NameFull: Pan, Yingwei – PersonEntity: Name: NameFull: Yao, Ting – PersonEntity: Name: NameFull: Chao, Hongyang – PersonEntity: Name: NameFull: Mei, Tao IsPartOfRelationships: – BibEntity: Dates: – D: 26 M: 09 Type: published Y: 2022 |
| ResultId | 1 |