A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks
| Title: | A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks |
|---|---|
| Authors: | Tang, Yun, Pino, Juan, Wang, Changhan, Ma, Xutai, Genzel, Dmitriy |
| Publication Year: | 2020 |
| Collection: | Computer Science |
| Subject Terms: | Computer Science - Computation and Language |
| More Details: | Attention-based sequence-to-sequence modeling provides a powerful and elegant solution for applications that need to map one sequence to a different sequence. Its success heavily relies on the availability of large amounts of training data. This presents a challenge for speech applications where labelled speech data is very expensive to obtain, such as automatic speech recognition (ASR) and speech translation (ST). In this study, we propose a general multi-task learning framework to leverage text data for ASR and ST tasks. Two auxiliary tasks, a denoising autoencoder task and machine translation task, are proposed to be co-trained with ASR and ST tasks respectively. We demonstrate that representing text input as phoneme sequences can reduce the difference between speech and text inputs, and enhance the knowledge transfer from text corpora to the speech to text tasks. Our experiments show that the proposed method achieves a relative 10~15% word error rate reduction on the English Librispeech task compared with our baseline, and improves the speech translation quality on the MuST-C tasks by 3.6~9.2 BLEU. Comment: Accepted for ICASSP2021 |
| Document Type: | Working Paper |
| Access URL: | http://arxiv.org/abs/2010.11338 |
| Accession Number: | edsarx.2010.11338 |
| Database: | arXiv |
| FullText | Text: Availability: 0 CustomLinks: – Url: http://arxiv.org/abs/2010.11338 Name: EDS - Arxiv Category: fullText Text: View this record from Arxiv MouseOverText: View this record from Arxiv – Url: https://resolver.ebsco.com/c/xy5jbn/result?sid=EBSCO:edsarx&genre=article&issn=&ISBN=&volume=&issue=&date=20201021&spage=&pages=&title=A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks&atitle=A%20General%20Multi-Task%20Learning%20Framework%20to%20Leverage%20Text%20Data%20for%20Speech%20to%20Text%20Tasks&aulast=Tang%2C%20Yun&id=DOI: Name: Full Text Finder (for New FTF UI) (s8985755) Category: fullText Text: Find It @ SCU Libraries MouseOverText: Find It @ SCU Libraries |
|---|---|
| Header | DbId: edsarx DbLabel: arXiv An: edsarx.2010.11338 RelevancyScore: 1008 AccessLevel: 3 PubType: Report PubTypeId: report PreciseRelevancyScore: 1007.73870849609 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Tang%2C+Yun%22">Tang, Yun</searchLink><br /><searchLink fieldCode="AR" term="%22Pino%2C+Juan%22">Pino, Juan</searchLink><br /><searchLink fieldCode="AR" term="%22Wang%2C+Changhan%22">Wang, Changhan</searchLink><br /><searchLink fieldCode="AR" term="%22Ma%2C+Xutai%22">Ma, Xutai</searchLink><br /><searchLink fieldCode="AR" term="%22Genzel%2C+Dmitriy%22">Genzel, Dmitriy</searchLink> – Name: DatePubCY Label: Publication Year Group: Date Data: 2020 – Name: Subset Label: Collection Group: HoldingsInfo Data: Computer Science – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Computer+Science+-+Computation+and+Language%22">Computer Science - Computation and Language</searchLink> – Name: Abstract Label: Description Group: Ab Data: Attention-based sequence-to-sequence modeling provides a powerful and elegant solution for applications that need to map one sequence to a different sequence. Its success heavily relies on the availability of large amounts of training data. This presents a challenge for speech applications where labelled speech data is very expensive to obtain, such as automatic speech recognition (ASR) and speech translation (ST). In this study, we propose a general multi-task learning framework to leverage text data for ASR and ST tasks. Two auxiliary tasks, a denoising autoencoder task and machine translation task, are proposed to be co-trained with ASR and ST tasks respectively. We demonstrate that representing text input as phoneme sequences can reduce the difference between speech and text inputs, and enhance the knowledge transfer from text corpora to the speech to text tasks. Our experiments show that the proposed method achieves a relative 10~15% word error rate reduction on the English Librispeech task compared with our baseline, and improves the speech translation quality on the MuST-C tasks by 3.6~9.2 BLEU.<br />Comment: Accepted for ICASSP2021 – Name: TypeDocument Label: Document Type Group: TypDoc Data: Working Paper – Name: URL Label: Access URL Group: URL Data: <link linkTarget="URL" linkTerm="http://arxiv.org/abs/2010.11338" linkWindow="_blank">http://arxiv.org/abs/2010.11338</link> – Name: AN Label: Accession Number Group: ID Data: edsarx.2010.11338 |
| PLink | https://login.libproxy.scu.edu/login?url=https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&scope=site&db=edsarx&AN=edsarx.2010.11338 |
| RecordInfo | BibRecord: BibEntity: Subjects: – SubjectFull: Computer Science - Computation and Language Type: general Titles: – TitleFull: A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Tang, Yun – PersonEntity: Name: NameFull: Pino, Juan – PersonEntity: Name: NameFull: Wang, Changhan – PersonEntity: Name: NameFull: Ma, Xutai – PersonEntity: Name: NameFull: Genzel, Dmitriy IsPartOfRelationships: – BibEntity: Dates: – D: 21 M: 10 Type: published Y: 2020 |
| ResultId | 1 |