Decoupled Training for Long-Tailed Classification With Stochastic Representations
| Title: | Decoupled Training for Long-Tailed Classification With Stochastic Representations |
|---|---|
| Authors: | Nam, Giung, Jang, Sunguk, Lee, Juho |
| Publication Year: | 2023 |
| Collection: | Computer Science |
| Subject Terms: | Computer Science - Machine Learning, Computer Science - Computer Vision and Pattern Recognition |
| More Details: | Decoupling representation learning and classifier learning has been shown to be effective in classification with long-tailed data. There are two main ingredients in constructing a decoupled learning scheme; 1) how to train the feature extractor for representation learning so that it provides generalizable representations and 2) how to re-train the classifier that constructs proper decision boundaries by handling class imbalances in long-tailed data. In this work, we first apply Stochastic Weight Averaging (SWA), an optimization technique for improving the generalization of deep neural networks, to obtain better generalizing feature extractors for long-tailed classification. We then propose a novel classifier re-training algorithm based on stochastic representation obtained from the SWA-Gaussian, a Gaussian perturbed SWA, and a self-distillation strategy that can harness the diverse stochastic representations based on uncertainty estimates to build more robust classifiers. Extensive experiments on CIFAR10/100-LT, ImageNet-LT, and iNaturalist-2018 benchmarks show that our proposed method improves upon previous methods both in terms of prediction accuracy and uncertainty estimation. Comment: ICLR 2023 |
| Document Type: | Working Paper |
| Access URL: | http://arxiv.org/abs/2304.09426 |
| Accession Number: | edsarx.2304.09426 |
| Database: | arXiv |
| FullText | Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2304.09426 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20230419&spage=&pages=&title=Decoupled Training for Long-Tailed Classification With Stochastic Representations&atitle=Decoupled%20Training%20for%20Long-Tailed%20Classification%20With%20Stochastic%20Representations&aulast=Nam%2C%20Giung&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edsarx DbLabel: arXiv An: edsarx.2304.09426 RelevancyScore: 1057 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1057.31787109375 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Decoupled Training for Long-Tailed Classification With Stochastic Representations – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Nam%2C+Giung%22">Nam, Giung</searchLink><br /><searchLink fieldCode="AR" term="%22Jang%2C+Sunguk%22">Jang, Sunguk</searchLink><br /><searchLink fieldCode="AR" term="%22Lee%2C+Juho%22">Lee, Juho</searchLink> – Name: DatePubCY Label: Publication Year Group: Date Data: 2023 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Machine+Learning%22">Computer Science - Machine Learning</searchLink><br /><searchLink fieldCode="DE" term="%22Computer+Science+-+Computer+Vision+and+Pattern+Recognition%22">Computer Science - Computer Vision and Pattern Recognition</searchLink> – Name: Abstract Label: Description Group: Ab Data: Decoupling representation learning and classifier learning has been shown to be effective in classification with long-tailed data. There are two main ingredients in constructing a decoupled learning scheme; 1) how to train the feature extractor for representation learning so that it provides generalizable representations and 2) how to re-train the classifier that constructs proper decision boundaries by handling class imbalances in long-tailed data. In this work, we first apply Stochastic Weight Averaging (SWA), an optimization technique for improving the generalization of deep neural networks, to obtain better generalizing feature extractors for long-tailed classification. We then propose a novel classifier re-training algorithm based on stochastic representation obtained from the SWA-Gaussian, a Gaussian perturbed SWA, and a self-distillation strategy that can harness the diverse stochastic representations based on uncertainty estimates to build more robust classifiers. Extensive experiments on CIFAR10/100-LT, ImageNet-LT, and iNaturalist-2018 benchmarks show that our proposed method improves upon previous methods both in terms of prediction accuracy and uncertainty estimation.<br />Comment: ICLR 2023 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2304.09426" linkWindow="_blank">http://arxiv.org/abs/2304.09426</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2304.09426 |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2304.09426 |
| RecordInfo | BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Machine Learning Type: general – SubjectFull: Computer Science - Computer Vision and Pattern Recognition Type: general Titles: – TitleFull: Decoupled Training for Long-Tailed Classification With Stochastic Representations Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Nam, Giung – PersonEntity: Name: NameFull: Jang, Sunguk – PersonEntity: Name: NameFull: Lee, Juho IsPartOfRelationships: – BibEntity: Dates: – D: 19 M: 04 Type: published Y: 2023 |
| ResultId | 1 |