AudioX: Diffusion Transformer for Anything-to-Audio Generation

Authors: Tian, Zeyue, Jin, Yizhu, Liu, Zhaoyang, Yuan, Ruibin, Tan, Xu, Chen, Qifeng, Xue, Wei, Guo, Yike

Subject Terms: Computer Science - Multimedia, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2503.10522

YuE: Scaling Open Foundation Models for Long-Form Music Generation

Authors: Yuan, Ruibin, Lin, Hanfeng, Guo, Shuyue, Zhang, Ge, Pan, Jiahao, Zang, Yongyi, Liu, Haohe, Liang, Yiming, Ma, Wenye, Du, Xingjian, Du, Xinrun, Ye, Zhen, Zheng, Tianyu, Ma, Yinghao, Liu, Minghao, Tian, Zeyue, Zhou, Ziya, Xue, Liumeng, Qu, Xingwei, Li, Yizhi, Wu, Shangda, Shen, Tianhao, Ma, Ziyang, Zhan, Jun, Wang, Chunhui, Wang, Yatian, Chi, Xiaowei, Zhang, Xinyue, Yang, Zhenzhu, Wang, Xiangzhou, Liu, Shansong, Mei, Lingrui, Li, Peng, Wang, Junjie, Yu, Jianwei, Pang, Guojian, Li, Xu, Wang, Zihao, Zhou, Xiaohuan, Yu, Lijun, Benetos, Emmanouil, Chen, Yong, Lin, Chenghua, Chen, Xie, Xia, Gus, Zhang, Zhaoxiang, Zhang, Chao, Chen, Wenhu, Zhou, Xinyu, Qiu, Xipeng, Dannenberg, Roger, Liu, Jiaheng, Yang, Jian, Huang, Wenhao, Xue, Wei, Tan, Xu, Guo, Yike

Subject Terms: Electrical Engineering and Systems Science - Audio and Speech Processing, Computer Science - Artificial Intelligence, Computer Science - Multimedia, Computer Science - Sound

Access URL: http://arxiv.org/abs/2503.08638

Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens

Authors: Wang, Xinsheng, Jiang, Mingqi, Ma, Ziyang, Zhang, Ziyu, Liu, Songxiang, Li, Linqin, Liang, Zheng, Zheng, Qixi, Wang, Rui, Feng, Xiaoqin, Bian, Weizhen, Ye, Zhen, Cheng, Sitong, Yuan, Ruibin, Zhao, Zhixian, Zhu, Xinfa, Pan, Jiahao, Xue, Liumeng, Zhu, Pengcheng, Chen, Yunlin, Li, Zhifei, Chen, Xie, Xie, Lei, Guo, Yike, Xue, Wei

Subject Terms: Computer Science - Sound, Computer Science - Artificial Intelligence, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2503.01710

Audio-FLAN: A Preliminary Release

Authors: Xue, Liumeng, Zhou, Ziya, Pan, Jiahao, Li, Zixuan, Fan, Shuai, Ma, Yinghao, Cheng, Sitong, Yang, Dongchao, Guo, Haohan, Xiao, Yujia, Wang, Xinsheng, Shen, Zixuan, Zhu, Chuanbo, Zhang, Xinshen, Liu, Tianchi, Yuan, Ruibin, Tian, Zeyue, Liu, Haohe, Benetos, Emmanouil, Zhang, Ge, Guo, Yike, Xue, Wei

Subject Terms: Computer Science - Sound, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Multimedia, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2502.16584

SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines

Authors: Team, M-A-P, Du, Xinrun, Yao, Yifan, Ma, Kaijing, Wang, Bingli, Zheng, Tianyu, Zhu, Kang, Liu, Minghao, Liang, Yiming, Jin, Xiaolong, Wei, Zhenlin, Zheng, Chujie, Deng, Kaixin, Jia, Shian, Jiang, Sichao, Liao, Yiyan, Li, Rui, Li, Qinrui, Li, Sirun, Li, Yizhi, Li, Yunwen, Ma, Dehua, Ni, Yuansheng, Que, Haoran, Wang, Qiyao, Wen, Zhoufutu, Wu, Siwei, Xing, Tianshun, Xu, Ming, Yang, Zhenzhu, Wang, Zekun Moore, Zhou, Junting, Bai, Yuelin, Bu, Xingyuan, Cai, Chenglin, Chen, Liang, Chen, Yifan, Cheng, Chengtuo, Cheng, Tianhao, Ding, Keyi, Huang, Siming, Huang, Yun, Li, Yaoru, Li, Yizhe, Li, Zhaoqun, Liang, Tianhao, Lin, Chengdong, Lin, Hongquan, Ma, Yinghao, Pang, Tianyang, Peng, Zhongyuan, Peng, Zifan, Qi, Qige, Qiu, Shi, Qu, Xingwei, Quan, Shanghaoran, Tan, Yizhou, Wang, Zili, Wang, Chenqing, Wang, Hao, Wang, Yiya, Wang, Yubo, Xu, Jiajun, Yang, Kexin, Yuan, Ruibin, Yue, Yuanhao, Zhan, Tianyang, Zhang, Chun, Zhang, Jinyang, Zhang, Xiyue, Zhang, Xingjian, Zhang, Yue, Zhao, Yongchi, Zheng, Xiangyu, Zhong, Chenghua, Gao, Yang, Li, Zhoujun, Liu, Dayiheng, Liu, Qian, Liu, Tianyu, Ni, Shiwen, Peng, Junran, Qin, Yujia, Su, Wenbo, Wang, Guoyin, Wang, Shi, Yang, Jian, Yang, Min, Cao, Meng, Yue, Xiang, Zhang, Zhaoxiang, Zhou, Wangchunshu, Liu, Jiaheng, Lin, Qunshu, Huang, Wenhao, Zhang, Ge

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2502.14739

CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages

Authors: Wu, Shangda, Guo, Zhancheng, Yuan, Ruibin, Jiang, Junyan, Doh, Seungheon, Xia, Gus, Nam, Juhan, Li, Xiaobing, Yu, Feng, Sun, Maosong

Subject Terms: Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2502.10362

Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-Speech Gesture Generation

Authors: Qi, Xingqun, Pan, Jiahao, Li, Peng, Yuan, Ruibin, Chi, Xiaowei, Li, Mengfei, Luo, Wenhan, Xue, Wei, Zhang, Shanghang, Liu, Qifeng, Guo, Yike

Source: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2024 IEEE/CVF Conference on. :10424-10434 Jun, 2024

Relation: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Authors: Yue, Xiang, Ni, Yuansheng, Zheng, Tianyu, Zhang, Kai, Liu, Ruoqi, Zhang, Ge, Stevens, Samuel, Jiang, Dongfu, Ren, Weiming, Sun, Yuxuan, Wei, Cong, Yu, Botao, Yuan, Ruibin, Sun, Renliang, Yin, Ming, Zheng, Boyuan, Yang, Zhenzhu, Liu, Yibo, Huang, Wenhao, Sun, Huan, Su, Yu, Chen, Wenhu

Source: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2024 IEEE/CVF Conference on. :9556-9567 Jun, 2024

Relation: 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models

Authors: Wu, Shangda, Wang, Yashan, Yuan, Ruibin, Guo, Zhancheng, Tan, Xu, Zhang, Ge, Zhou, Monan, Chen, Jing, Mu, Xuefeng, Gao, Yuejie, Dong, Yuanliang, Liu, Jiafeng, Li, Xiaobing, Yu, Feng, Sun, Maosong

Subject Terms: Computer Science - Sound, Computer Science - Computation and Language, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2410.13267

Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer

Authors: Hou, Siyuan, Liu, Shansong, Yuan, Ruibin, Xue, Wei, Shan, Ying, Zhao, Mangsuo, Zhang, Chao

Subject Terms: Electrical Engineering and Systems Science - Audio and Speech Processing, Computer Science - Sound

Access URL: http://arxiv.org/abs/2410.05151

You Know What I'm Saying: Jailbreak Attack via Implicit Reference

Authors: Wu, Tianyu, Mei, Lingrui, Yuan, Ruibin, Li, Lujun, Xue, Wei, Guo, Yike

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2410.03857

HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Router

Authors: Mei, Lingrui, Liu, Shenghua, Wang, Yiwei, Bi, Baolong, Yuan, Ruibin, Cheng, Xueqi

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2410.02684

OmniBench: Towards The Future of Universal Omni-Language Models

Authors: Li, Yizhi, Zhang, Ge, Ma, Yinghao, Yuan, Ruibin, Zhu, Kang, Guo, Hangyu, Liang, Yiming, Liu, Jiaheng, Wang, Zekun, Yang, Jian, Wu, Siwei, Qu, Xingwei, Shi, Jinjie, Zhang, Xinyue, Yang, Zhenzhu, Wang, Xiangzhou, Zhang, Zhaoxiang, Liu, Zachary, Benetos, Emmanouil, Huang, Wenhao, Lin, Chenghua

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence, Computer Science - Computer Vision and Pattern Recognition

Access URL: http://arxiv.org/abs/2409.15272

SongTrans: An unified song transcription and alignment method for lyrics and notes

Authors: Wu, Siwei, He, Jinzheng, Yuan, Ruibin, Wei, Haojie, Wei, Xipin, Lin, Chenghua, Xu, Jin, Lin, Junyang

Subject Terms: Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2409.14619

Foundation Models for Music: A Survey

Authors: Ma, Yinghao, Øland, Anders, Ragni, Anton, Del Sette, Bleiz MacSen, Saitis, Charalampos, Donahue, Chris, Lin, Chenghua, Plachouras, Christos, Benetos, Emmanouil, Shatri, Elona, Morreale, Fabio, Zhang, Ge, Fazekas, György, Xia, Gus, Zhang, Huan, Manco, Ilaria, Huang, Jiawen, Guinot, Julien, Lin, Liwei, Marinelli, Luca, Lam, Max W. Y., Sharma, Megha, Kong, Qiuqiang, Dannenberg, Roger B., Yuan, Ruibin, Wu, Shangda, Wu, Shih-Lun, Dai, Shuqi, Lei, Shun, Kang, Shiyin, Dixon, Simon, Chen, Wenhu, Huang, Wenhao, Du, Xingjian, Qu, Xingwei, Tan, Xu, Li, Yizhi, Tian, Zeyue, Wu, Zhiyong, Wu, Zhizheng, Ma, Ziyang, Wang, Ziyu

Subject Terms: Computer Science - Sound, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2408.14340

Can LLMs 'Reason' in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation

Authors: Zhou, Ziya, Wu, Yuhang, Wu, Zhiyue, Zhang, Xinyue, Yuan, Ruibin, Ma, Yinghao, Wang, Lu, Benetos, Emmanouil, Xue, Wei, Guo, Yike

Subject Terms: Computer Science - Sound, Computer Science - Computation and Language, Computer Science - Multimedia, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2407.21531

MMTrail: A Multimodal Trailer Video Dataset with Language and Music Descriptions

Authors: Chi, Xiaowei, Wang, Yatian, Cheng, Aosong, Fang, Pengjun, Tian, Zeyue, He, Yingqing, Liu, Zhaoyang, Qi, Xingqun, Pan, Jiahao, Zhang, Rongyu, Li, Mengfei, Yuan, Ruibin, Jiang, Yanbing, Xue, Wei, Luo, Wenhan, Chen, Qifeng, Zhang, Shanghang, Liu, Qifeng, Guo, Yike

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Multimedia, Computer Science - Sound, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2407.20962

VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling

Authors: Tian, Zeyue, Liu, Zhaoyang, Yuan, Ruibin, Pan, Jiahao, Liu, Qifeng, Tan, Xu, Chen, Qifeng, Xue, Wei, Guo, Yike

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Machine Learning, Computer Science - Multimedia, Computer Science - Sound

Access URL: http://arxiv.org/abs/2406.04321

LLMs Meet Multimodal Generation and Editing: A Survey

Authors: He, Yingqing, Liu, Zhaoyang, Chen, Jingye, Tian, Zeyue, Liu, Hongyu, Chi, Xiaowei, Liu, Runtao, Yuan, Ruibin, Xing, Yazhou, Wang, Wenhai, Dai, Jifeng, Zhang, Yong, Xue, Wei, Liu, Qifeng, Guo, Yike, Chen, Qifeng

Subject Terms: Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Computer Vision and Pattern Recognition, Computer Science - Multimedia, Computer Science - Sound

Access URL: http://arxiv.org/abs/2405.19334

ComposerX: Multi-Agent Symbolic Music Composition with LLMs

Authors: Deng, Qixin, Yang, Qikai, Yuan, Ruibin, Huang, Yipeng, Wang, Yi, Liu, Xubo, Tian, Zeyue, Pan, Jiahao, Zhang, Ge, Lin, Hanfeng, Li, Yizhi, Ma, Yinghao, Fu, Jie, Lin, Chenghua, Benetos, Emmanouil, Wang, Wenwu, Xia, Guangyu, Xue, Wei, Guo, Yike

Subject Terms: Computer Science - Sound, Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Machine Learning, Computer Science - Multimedia, Electrical Engineering and Systems Science - Audio and Speech Processing

Access URL: http://arxiv.org/abs/2404.18081