Findings of the IWSLT 2024 Evaluation Campaign

Authors: Ahmad, Ibrahim Said, Anastasopoulos, Antonios, Bojar, Ondřej, Borg, Claudia, Carpuat, Marine, Cattoni, Roldano, Cettolo, Mauro, Chen, William, Dong, Qianqian, Federico, Marcello, Haddow, Barry, Javorský, Dávid, Krubiński, Mateusz, Lam, Tsz Kin, Ma, Xutai, Mathur, Prashant, Matusov, Evgeny, Maurya, Chandresh, McCrae, John, Murray, Kenton, Nakamura, Satoshi, Negri, Matteo, Niehues, Jan, Niu, Xing, Ojha, Atul Kr., Ortega, John, Papi, Sara, Polák, Peter, Pospíšil, Adam, Pecina, Pavel, Salesky, Elizabeth, Sethiya, Nivedita, Sarkar, Balaram, Shi, Jiatong, Sikasote, Claytone, Sperber, Matthias, Stüker, Sebastian, Sudoh, Katsuhito, Thompson, Brian, Turchi, Marco, Waibel, Alex, Watanabe, Shinji, Wilken, Patrick, Zemánek, Petr, Zevallos, Rodolfo

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2411.05088

SSR: Alignment-Aware Modality Connector for Speech Language Models

Authors: Tan, Weiting, Inaguma, Hirofumi, Dong, Ning, Tomasello, Paden, Ma, Xutai

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2410.00168

MMM: Multi-Layer Multi-Residual Multi-Stream Discrete Speech Representation from Self-supervised Learning Model

Authors: Shi, Jiatong, Ma, Xutai, Inaguma, Hirofumi, Sun, Anna, Watanabe, Shinji

Subject Terms: Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2406.09869

Evaluating the IWSLT2023 Speech Translation Tasks: Human Annotations, Automatic Metrics, and Segmentation

Authors: Sperber, Matthias, Bojar, Ondřej, Haddow, Barry, Javorský, Dávid, Ma, Xutai, Negri, Matteo, Niehues, Jan, Polák, Peter, Salesky, Elizabeth, Sudoh, Katsuhito, Turchi, Marco

Source: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2406.03881

Seamless: Multilingual Expressive and Streaming Speech Translation

Authors: Communication, Seamless, Barrault, Loïc, Chung, Yu-An, Meglioli, Mariano Coria, Dale, David, Dong, Ning, Duppenthaler, Mark, Duquenne, Paul-Ambroise, Ellis, Brian, Elsahar, Hady, Haaheim, Justin, Hoffman, John, Hwang, Min-Jae, Inaguma, Hirofumi, Klaiber, Christopher, Kulikov, Ilia, Li, Pengwei, Licht, Daniel, Maillard, Jean, Mavlyutov, Ruslan, Rakotoarison, Alice, Sadagopan, Kaushik Ram, Ramakrishnan, Abinesh, Tran, Tuan, Wenzek, Guillaume, Yang, Yilin, Ye, Ethan, Evtimov, Ivan, Fernandez, Pierre, Gao, Cynthia, Hansanti, Prangthip, Kalbassi, Elahe, Kallet, Amanda, Kozhevnikov, Artyom, Gonzalez, Gabriel Mejia, Roman, Robin San, Touret, Christophe, Wong, Corinne, Wood, Carleigh, Yu, Bokai, Andrews, Pierre, Balioglu, Can, Chen, Peng-Jen, Costa-jussà, Marta R., Elbayad, Maha, Gong, Hongyu, Guzmán, Francisco, Heffernan, Kevin, Jain, Somya, Kao, Justine, Lee, Ann, Ma, Xutai, Mourachko, Alex, Peloquin, Benjamin, Pino, Juan, Popuri, Sravya, Ropers, Christophe, Saleem, Safiyyah, Schwenk, Holger, Sun, Anna, Tomasello, Paden, Wang, Changhan, Wang, Jeff, Wang, Skyler, Williamson, Mary

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2312.05187

Efficient Monotonic Multihead Attention

Authors: Ma, Xutai, Sun, Anna, Ouyang, Siqi, Inaguma, Hirofumi, Tomasello, Paden

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2312.04515

Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Prediction

Authors: Shi, Jiatong, Inaguma, Hirofumi, Ma, Xutai, Kulikov, Ilia, Sun, Anna

Subject Terms: Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2310.02720

SeamlessM4T: Massively Multilingual & Multimodal Machine Translation

Authors: Communication, Seamless, Barrault, Loïc, Chung, Yu-An, Meglioli, Mariano Cora, Dale, David, Dong, Ning, Duquenne, Paul-Ambroise, Elsahar, Hady, Gong, Hongyu, Heffernan, Kevin, Hoffman, John, Klaiber, Christopher, Li, Pengwei, Licht, Daniel, Maillard, Jean, Rakotoarison, Alice, Sadagopan, Kaushik Ram, Wenzek, Guillaume, Ye, Ethan, Akula, Bapi, Chen, Peng-Jen, Hachem, Naji El, Ellis, Brian, Gonzalez, Gabriel Mejia, Haaheim, Justin, Hansanti, Prangthip, Howes, Russ, Huang, Bernie, Hwang, Min-Jae, Inaguma, Hirofumi, Jain, Somya, Kalbassi, Elahe, Kallet, Amanda, Kulikov, Ilia, Lam, Janice, Li, Daniel, Ma, Xutai, Mavlyutov, Ruslan, Peloquin, Benjamin, Ramadan, Mohamed, Ramakrishnan, Abinesh, Sun, Anna, Tran, Kevin, Tran, Tuan, Tufanov, Igor, Vogeti, Vish, Wood, Carleigh, Yang, Yilin, Yu, Bokai, Andrews, Pierre, Balioglu, Can, Costa-jussà, Marta R., Celebi, Onur, Elbayad, Maha, Gao, Cynthia, Guzmán, Francisco, Kao, Justine, Lee, Ann, Mourachko, Alexandre, Pino, Juan, Popuri, Sravya, Ropers, Christophe, Saleem, Safiyyah, Schwenk, Holger, Tomasello, Paden, Wang, Changhan, Wang, Jeff, Wang, Skyler

Subject Terms: Computer Science - Computation and Language, I.2.7

Access URL: http://arxiv.org/abs/2308.11596

Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks

Authors: Tang, Yun, Sun, Anna Y., Inaguma, Hirofumi, Chen, Xinyue, Dong, Ning, Ma, Xutai, Tomasello, Paden D., Pino, Juan

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2305.03101

Streaming Simultaneous Speech Translation with Augmented Memory Transformer

Authors: Ma, Xutai, Wang, Yongqiang, Dousti, Mohammad Javad, Koehn, Philipp, Pino, Juan

Source: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2021 - 2021 IEEE International Conference on. :7523-7527 Jun, 2021

Relation: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks

Authors: Tang, Yun, Pino, Juan, Wang, Changhan, Ma, Xutai, Genzel, Dmitriy

Source: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) Acoustics, Speech and Signal Processing (ICASSP), ICASSP 2021 - 2021 IEEE International Conference on. :6209-6213 Jun, 2021

Relation: ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention

Authors: Ma, Xutai, Gong, Hongyu, Liu, Danni, Lee, Ann, Tang, Yun, Chen, Peng-Jen, Hsu, Wei-Ning, Koehn, Phillip, Pino, Juan

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2110.08250

From Start to Finish: Latency Reduction Strategies for Incremental Speech Synthesis in Simultaneous Speech-to-Speech Translation

Authors: Liu, Danni, Wang, Changhan, Gong, Hongyu, Ma, Xutai, Tang, Yun, Pino, Juan

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2110.08214

Direct speech-to-speech translation with discrete units

Authors: Lee, Ann, Chen, Peng-Jen, Wang, Changhan, Gu, Jiatao, Popuri, Sravya, Ma, Xutai, Polyak, Adam, Adi, Yossi, He, Qing, Tang, Yun, Pino, Juan, Hsu, Wei-Ning

Subject Terms: Computer Science - Computation and Language, Computer Science - Machine Learning, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2107.05604

SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation

Authors: Ma, Xutai, Pino, Juan, Koehn, Philipp

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2011.02048

Streaming Simultaneous Speech Translation with Augmented Memory Transformer

Authors: Ma, Xutai, Wang, Yongqiang, Dousti, Mohammad Javad, Koehn, Philipp, Pino, Juan

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2011.00033

A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks

Authors: Tang, Yun, Pino, Juan, Wang, Changhan, Ma, Xutai, Genzel, Dmitriy

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2010.11338

fairseq S2T: Fast Speech-to-Text Modeling with fairseq

Authors: Wang, Changhan, Tang, Yun, Ma, Xutai, Wu, Anne, Popuri, Sravya, Okhonko, Dmytro, Pino, Juan

Subject Terms: Computer Science - Computation and Language, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2010.05171

SimulEval: An Evaluation Toolkit for Simultaneous Translation

Authors: Ma, Xutai, Dousti, Mohammad Javad, Wang, Changhan, Gu, Jiatao, Pino, Juan

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2007.16193

Self-Training for End-to-End Speech Translation

Authors: Pino, Juan, Xu, Qiantong, Ma, Xutai, Dousti, Mohammad Javad, Tang, Yun

Subject Terms: Computer Science - Computation and Language, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2006.02490