WLB-LLM: Workload-Balanced 4D Parallelism for Large Language Model Training

Authors: Wang, Zheng, Cai, Anna, Xie, Xinfeng, Pan, Zaifeng, Guan, Yue, Chu, Weiwei, Wang, Jie, Li, Shikai, Huang, Jianyu, Cai, Chris, Hao, Yuchen, Ding, Yufei

Subject Terms: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Artificial Intelligence, Computer Science - Machine Learning, I.2.11

Access URL: http://arxiv.org/abs/2503.17924

Context Parallelism for Scalable Million-Token Inference

Authors: Yang, Amy, Yang, Jingyi, Ibrahim, Aya, Xie, Xinfeng, Tang, Bangsheng, Sizov, Grigory, Reizenstein, Jeremy, Park, Jongsoo, Huang, Jianyu

Subject Terms: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2411.01783

The Llama 3 Herd of Models

Authors: Grattafiori, Aaron, Dubey, Abhimanyu, Jauhri, Abhinav, Pandey, Abhinav, Kadian, Abhishek, Al-Dahle, Ahmad, Letman, Aiesha, Mathur, Akhil, Schelten, Alan, Vaughan, Alex, Yang, Amy, Fan, Angela, Goyal, Anirudh, Hartshorn, Anthony, Yang, Aobo, Mitra, Archi, Sravankumar, Archie, Korenev, Artem, Hinsvark, Arthur, Rao, Arun, Zhang, Aston, Rodriguez, Aurelien, Gregerson, Austen, Spataru, Ava, Roziere, Baptiste, Biron, Bethany, Tang, Binh, Chern, Bobbie, Caucheteux, Charlotte, Nayak, Chaya, Bi, Chloe, Marra, Chris, McConnell, Chris, Keller, Christian, Touret, Christophe, Wu, Chunyang, Wong, Corinne, Ferrer, Cristian Canton, Nikolaidis, Cyrus, Allonsius, Damien, Song, Daniel, Pintz, Danielle, Livshits, Danny, Wyatt, Danny, Esiobu, David, Choudhary, Dhruv, Mahajan, Dhruv, Garcia-Olano, Diego, Perino, Diego, Hupkes, Dieuwke, Lakomkin, Egor, AlBadawy, Ehab, Lobanova, Elina, Dinan, Emily, Smith, Eric Michael, Radenovic, Filip, Guzmán, Francisco, Zhang, Frank, Synnaeve, Gabriel, Lee, Gabrielle, Anderson, Georgia Lewis, Thattai, Govind, Nail, Graeme, Mialon, Gregoire, Pang, Guan, Cucurell, Guillem, Nguyen, Hailey, Korevaar, Hannah, Xu, Hu, Touvron, Hugo, Zarov, Iliyan, Ibarra, Imanol Arrieta, Kloumann, Isabel, Misra, Ishan, Evtimov, Ivan, Zhang, Jack, Copet, Jade, Lee, Jaewon, Geffert, Jan, Vranes, Jana, Park, Jason, Mahadeokar, Jay, Shah, Jeet, van der Linde, Jelmer, Billock, Jennifer, Hong, Jenny, Lee, Jenya, Fu, Jeremy, Chi, Jianfeng, Huang, Jianyu, Liu, Jiawen, Wang, Jie, Yu, Jiecao, Bitton, Joanna, Spisak, Joe, Park, Jongsoo, Rocca, Joseph, Johnstun, Joshua, Saxe, Joshua, Jia, Junteng, Alwala, Kalyan Vasuden, Prasad, Karthik, Upasani, Kartikeya, Plawiak, Kate, Li, Ke, Heafield, Kenneth, Stone, Kevin, El-Arini, Khalid, Iyer, Krithika, Malik, Kshitiz, Chiu, Kuenley, Bhalla, Kunal, Lakhotia, Kushal, Rantala-Yeary, Lauren, van der Maaten, Laurens, Chen, Lawrence, Tan, Liang, Jenkins, Liz, Martin, Louis, Madaan, Lovish, Malo, Lubo, Blecher, Lukas, Landzaat, Lukas, de Oliveira, Luke, Muzzi, Madeline, Pasupuleti, Mahesh, Singh, Mannat, Paluri, Manohar, Kardas, Marcin, Tsimpoukelli, Maria, Oldham, Mathew, Rita, Mathieu, Pavlova, Maya, Kambadur, Melanie, Lewis, Mike, Si, Min, Singh, Mitesh Kumar, Hassan, Mona, Goyal, Naman, Torabi, Narjes, Bashlykov, Nikolay, Bogoychev, Nikolay, Chatterji, Niladri, Zhang, Ning, Duchenne, Olivier, Çelebi, Onur, Alrassy, Patrick, Zhang, Pengchuan, Li, Pengwei, Vasic, Petar, Weng, Peter, Bhargava, Prajjwal, Dubal, Pratik, Krishnan, Praveen, Koura, Punit Singh, Xu, Puxin, He, Qing, Dong, Qingxiao, Srinivasan, Ragavan, Ganapathy, Raj, Calderer, Ramon, Cabral, Ricardo Silveira, Stojnic, Robert, Raileanu, Roberta, Maheswari, Rohan, Girdhar, Rohit, Patel, Rohit, Sauvestre, Romain, Polidoro, Ronnie, Sumbaly, Roshan, Taylor, Ross, Silva, Ruan, Hou, Rui, Wang, Rui, Hosseini, Saghar, Chennabasappa, Sahana, Singh, Sanjay, Bell, Sean, Kim, Seohyun Sonia, Edunov, Sergey, Nie, Shaoliang, Narang, Sharan, Raparthy, Sharath, Shen, Sheng, Wan, Shengye, Bhosale, Shruti, Zhang, Shun, Vandenhende, Simon, Batra, Soumya, Whitman, Spencer, Sootla, Sten, Collot, Stephane, Gururangan, Suchin, Borodinsky, Sydney, Herman, Tamar, Fowler, Tara, Sheasha, Tarek, Georgiou, Thomas, Scialom, Thomas, Speckbacher, Tobias, Mihaylov, Todor, Xiao, Tong, Karn, Ujjwal, Goswami, Vedanuj, Gupta, Vibhor, Ramanathan, Vignesh, Kerkez, Viktor, Gonguet, Vincent, Do, Virginie, Vogeti, Vish, Albiero, Vítor, Petrovic, Vladan, Chu, Weiwei, Xiong, Wenhan, Fu, Wenyin, Meers, Whitney, Martinet, Xavier, Wang, Xiaodong, Wang, Xiaofang, Tan, Xiaoqing Ellen, Xia, Xide, Xie, Xinfeng, Jia, Xuchao, Wang, Xuewei, Goldschlag, Yaelle, Gaur, Yashesh, Babaei, Yasmine, Wen, Yi, Song, Yiwen, Zhang, Yuchen, Li, Yue, Mao, Yuning, Coudert, Zacharie Delpierre, Yan, Zheng, Chen, Zhengxing, Papakipos, Zoe, Singh, Aaditya, Srivastava, Aayushi, Jain, Abha, Kelsey, Adam, Shajnfeld, Adam, Gangidi, Adithya, Victoria, Adolfo, Goldstand, Ahuva, Menon, Ajay, Sharma, Ajay, Boesenberg, Alex, Baevski, Alexei, Feinstein, Allie, Kallet, Amanda, Sangani, Amit, Teo, Amos, Yunus, Anam, Lupu, Andrei, Alvarado, Andres, Caples, Andrew, Gu, Andrew, Ho, Andrew, Poulton, Andrew, Ryan, Andrew, Ramchandani, Ankit, Dong, Annie, Franco, Annie, Goyal, Anuj, Saraf, Aparajita, Chowdhury, Arkabandhu, Gabriel, Ashley, Bharambe, Ashwin, Eisenman, Assaf, Yazdan, Azadeh, James, Beau, Maurer, Ben, Leonhardi, Benjamin, Huang, Bernie, Loyd, Beth, De Paola, Beto, Paranjape, Bhargavi, Liu, Bing, Wu, Bo, Ni, Boyu, Hancock, Braden, Wasti, Bram, Spence, Brandon, Stojkovic, Brani, Gamido, Brian, Montalvo, Britt, Parker, Carl, Burton, Carly, Mejia, Catalina, Liu, Ce, Wang, Changhan, Kim, Changkyu, Zhou, Chao, Hu, Chester, Chu, Ching-Hsiang, Cai, Chris, Tindal, Chris, Feichtenhofer, Christoph, Gao, Cynthia, Civin, Damon, Beaty, Dana, Kreymer, Daniel, Li, Daniel, Adkins, David, Xu, David, Testuggine, Davide, David, Delia, Parikh, Devi, Liskovich, Diana, Foss, Didem, Wang, Dingkang, Le, Duc, Holland, Dustin, Dowling, Edward, Jamil, Eissa, Montgomery, Elaine, Presani, Eleonora, Hahn, Emily, Wood, Emily, Le, Eric-Tuan, Brinkman, Erik, Arcaute, Esteban, Dunbar, Evan, Smothers, Evan, Sun, Fei, Kreuk, Felix, Tian, Feng, Kokkinos, Filippos, Ozgenel, Firat, Caggioni, Francesco, Kanayet, Frank, Seide, Frank, Florez, Gabriela Medina, Schwarz, Gabriella, Badeer, Gada, Swee, Georgia, Halpern, Gil, Herman, Grant, Sizov, Grigory, Guangyi, Zhang, Lakshminarayanan, Guna, Inan, Hakan, Shojanazeri, Hamid, Zou, Han, Wang, Hannah, Zha, Hanwen, Habeeb, Haroun, Rudolph, Harrison, Suk, Helen, Aspegren, Henry, Goldman, Hunter, Zhan, Hongyuan, Damlaj, Ibrahim, Molybog, Igor, Tufanov, Igor, Leontiadis, Ilias, Veliche, Irina-Elena, Gat, Itai, Weissman, Jake, Geboski, James, Kohli, James, Lam, Janice, Asher, Japhet, Gaya, Jean-Baptiste, Marcus, Jeff, Tang, Jeff, Chan, Jennifer, Zhen, Jenny, Reizenstein, Jeremy, Teboul, Jeremy, Zhong, Jessica, Jin, Jian, Yang, Jingyi, Cummings, Joe, Carvill, Jon, Shepard, Jon, McPhie, Jonathan, Torres, Jonathan, Ginsburg, Josh, Wang, Junjie, Wu, Kai, U, Kam Hou, Saxena, Karan, Khandelwal, Kartikay, Zand, Katayoun, Matosich, Kathy, Veeraraghavan, Kaushik, Michelena, Kelly, Li, Keqian, Jagadeesh, Kiran, Huang, Kun, Chawla, Kunal, Huang, Kyle, Chen, Lailin, Garg, Lakshya, A, Lavender, Silva, Leandro, Bell, Lee, Zhang, Lei, Guo, Liangpeng, Yu, Licheng, Moshkovich, Liron, Wehrstedt, Luca, Khabsa, Madian, Avalani, Manav, Bhatt, Manish, Mankus, Martynas, Hasson, Matan, Lennie, Matthew, Reso, Matthias, Groshev, Maxim, Naumov, Maxim, Lathi, Maya, Keneally, Meghan, Liu, Miao, Seltzer, Michael L., Valko, Michal, Restrepo, Michelle, Patel, Mihir, Vyatskov, Mik, Samvelyan, Mikayel, Clark, Mike, Macey, Mike, Wang, Mike, Hermoso, Miquel Jubert, Metanat, Mo, Rastegari, Mohammad, Bansal, Munish, Santhanam, Nandhini, Parks, Natascha, White, Natasha, Bawa, Navyata, Singhal, Nayan, Egebo, Nick, Usunier, Nicolas, Mehta, Nikhil, Laptev, Nikolay Pavlovich, Dong, Ning, Cheng, Norman, Chernoguz, Oleg, Hart, Olivia, Salpekar, Omkar, Kalinli, Ozlem, Kent, Parkin, Parekh, Parth, Saab, Paul, Balaji, Pavan, Rittner, Pedro, Bontrager, Philip, Roux, Pierre, Dollar, Piotr, Zvyagina, Polina, Ratanchandani, Prashant, Yuvraj, Pritish, Liang, Qian, Alao, Rachad, Rodriguez, Rachel, Ayub, Rafi, Murthy, Raghotham, Nayani, Raghu, Mitra, Rahul, Parthasarathy, Rangaprabhu, Li, Raymond, Hogan, Rebekkah, Battey, Robin, Wang, Rocky, Howes, Russ, Rinott, Ruty, Mehta, Sachin, Siby, Sachin, Bondu, Sai Jayesh, Datta, Samyak, Chugh, Sara, Hunt, Sara, Dhillon, Sargun, Sidorov, Sasha, Pan, Satadru, Mahajan, Saurabh, Verma, Saurabh, Yamamoto, Seiji, Ramaswamy, Sharadh, Lindsay, Shaun, Feng, Sheng, Lin, Shenghao, Zha, Shengxin Cindy, Patil, Shishir, Shankar, Shiva, Zhang, Shuqiang, Wang, Sinong, Agarwal, Sneha, Sajuyigbe, Soji, Chintala, Soumith, Max, Stephanie, Chen, Stephen, Kehoe, Steve, Satterfield, Steve, Govindaprasad, Sudarshan, Gupta, Sumit, Deng, Summer, Cho, Sungmin, Virk, Sunny, Subramanian, Suraj, Choudhury, Sy, Goldman, Sydney, Remez, Tal, Glaser, Tamar, Best, Tamara, Koehler, Thilo, Robinson, Thomas, Li, Tianhe, Zhang, Tianjun, Matthews, Tim, Chou, Timothy, Shaked, Tzook, Vontimitta, Varun, Ajayi, Victoria, Montanez, Victoria, Mohan, Vijai, Kumar, Vinay Satish, Mangla, Vishal, Ionescu, Vlad, Poenaru, Vlad, Mihailescu, Vlad Tiberiu, Ivanov, Vladimir, Li, Wei, Wang, Wenchen, Jiang, Wenwen, Bouaziz, Wes, Constable, Will, Tang, Xiaocheng, Wu, Xiaojian, Wang, Xiaolan, Wu, Xilun, Gao, Xinbo, Kleinman, Yaniv, Chen, Yanjun, Hu, Ye, Jia, Ye, Qi, Ye, Li, Yenda, Zhang, Yilin, Zhang, Ying, Adi, Yossi, Nam, Youngjin, Yu, Wang, Zhao, Yu, Hao, Yuchen, Qian, Yundi, Li, Yunlu, He, Yuzi, Rait, Zach, DeVito, Zachary, Rosnbrick, Zef, Wen, Zhaoduo, Yang, Zhenyu, Zhao, Zhiwei, Ma, Zhiyu

Subject Terms: Computer Science - Artificial Intelligence, Computer Science - Computation and Language, Computer Science - Computer Vision and Pattern Recognition

Access URL: http://arxiv.org/abs/2407.21783

CoMERA: Computing- and Memory-Efficient Training via Rank-Adaptive Tensor Optimization

Authors: Yang, Zi, Liu, Ziyue, Choudhary, Samridhi, Xie, Xinfeng, Gao, Cao, Kunzmann, Siegfried, Zhang, Zheng

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2405.14377

SEALing Neural Network Models in Encrypted Deep Learning Accelerators

Authors: Zuo, Pengfei, Hua, Yu, Liang, Ling, Xie, Xinfeng, Hu, Xing, Xie, Yuan

Source: 2021 58th ACM/IEEE Design Automation Conference (DAC) Design Automation Conference (DAC), 2021 58th ACM/IEEE. :1255-1260 Dec, 2021

Relation: 2021 58th ACM/IEEE Design Automation Conference (DAC)

SpaceA: Sparse Matrix Vector Multiplication on Processing-in-Memory Accelerator

Authors: Xie, Xinfeng, Liang, Zheng, Gu, Peng, Basak, Abanti, Deng, Lei, Liang, Ling, Hu, Xing, Xie, Yuan

Source: 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) HPCA High-Performance Computer Architecture (HPCA), 2021 IEEE International Symposium on. :570-583 Feb, 2021

Relation: 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA)

SAGA-Bench: Software and Hardware Characterization of Streaming Graph Analytics Workloads

Authors: Basak, Abanti, Lin, Jilan, Lorica, Ryan, Xie, Xinfeng, Chishti, Zeshan, Alameldeen, Alaa, Xie, Yuan

Source: 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) ISPASS Performance Analysis of Systems and Software (ISPASS), 2020 IEEE International Symposium on. :12-23 Aug, 2020

Relation: 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)

Genome wide detection of CNV and their association with body size in Danzhou chickens

Authors: Xie, Xinfeng ^{*, 1}, Shi, Liguang ^{†, 1}, Hou, Guanyu ^†, Zhong, Ziqi ^*, Wang, Ziyi ^*, Pan, Deyou ^*, Na, Wei ^*, Xiao, Qian ^{*, 2}

Source: In Poultry Science December 2024 103(12)

iPIM: Programmable In-Memory Image Processing Accelerator Using Near-Bank Architecture

Authors: Gu, Peng, Xie, Xinfeng, Ding, Yufei, Chen, Guoyang, Zhang, Weifeng, Niu, Dimin, Xie, Yuan

Source: 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA) Computer Architecture (ISCA), 2020 ACM/IEEE 47th Annual International Symposium on. :804-817 May, 2020

Relation: 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA)

Memory-Bound Proof-of-Work Acceleration for Blockchain Applications

Authors: Wu, Kun, Dai, Guohao, Hu, Xing, Li, Shuangchen, Xie, Xinfeng, Wang, Yu, Xie, Yuan

Source: 2019 56th ACM/IEEE Design Automation Conference (DAC) Design Automation Conference (DAC), 2019 56th ACM/IEEE. :1-6 Jun, 2019

Relation: 2019 56th ACM/IEEE Design Automation Conference (DAC)

A Transferable Approach for Partitioning Machine Learning Models on Multi-Chip-Modules

Authors: Xie, Xinfeng, Prabhu, Prakash, Beaugnon, Ulysse, Phothilimthana, Phitchaya Mangpo, Roy, Sudip, Mirhoseini, Azalia, Brevdo, Eugene, Laudon, James, Zhou, Yanqi

Subject Terms: Computer Science - Machine Learning, Computer Science - Hardware Architecture

Access URL: http://arxiv.org/abs/2112.04041

Analysis and Optimization of the Memory Hierarchy for Graph Processing Workloads

Authors: Basak, Abanti, Li, Shuangchen, Hu, Xing, Oh, Sang Min, Xie, Xinfeng, Zhao, Li, Jiang, Xiaowei, Xie, Yuan

Source: 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA) HPCA High Performance Computer Architecture (HPCA), 2019 IEEE International Symposium on. :373-386 Feb, 2019

Relation: 2019 IEEE International Symposium on High Performance Computer Architecture (HPCA)

NNBench-X: A Benchmarking Methodology for Neural Network Accelerator Designs

Authors: Xie, Xinfeng, Hu, Xing, Gu, Peng, Li, Shuangchen, Ji, Yu, Xie, Yuan

Source: 2019 2nd Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2) Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2), 2019 2nd Workshop on. :11-15 Feb, 2019

Relation: 2019 2nd Workshop on Energy Efficient Machine Learning and Cognitive Computing for Embedded Applications (EMC2)

Danzhou chicken: a unique genetic resource revealed by genome-wide resequencing data

Authors: Xie, Xinfeng ^*, Shi, Liguang, Zhong, Ziqi ^*, Wang, Ziyi ^*, Pan, Deyou ^*, Hou, Guanyu, Xiao, Qian ^*

Source: In Poultry Science August 2024 103(8)

MPU: Towards Bandwidth-abundant SIMT Processor via Near-bank Computing

Authors: Xie, Xinfeng, Gu, Peng, Ding, Yufei, Niu, Dimin, Zheng, Hongzhong, Xie, Yuan

Subject Terms: Computer Science - Hardware Architecture

Access URL: http://arxiv.org/abs/2103.06653

Rubik: A Hierarchical Architecture for Efficient Graph Learning

Authors: Chen, Xiaobing, Wang, Yuke, Xie, Xinfeng, Hu, Xing, Basak, Abanti, Liang, Ling, Yan, Mingyu, Deng, Lei, Ding, Yufei, Du, Zidong, Chen, Yunji, Xie, Yuan

Subject Terms: Computer Science - Hardware Architecture

Access URL: http://arxiv.org/abs/2009.12495

SEALing Neural Network Models in Secure Deep Learning Accelerators

Authors: Zuo, Pengfei, Hua, Yu, Liang, Ling, Xie, Xinfeng, Hu, Xing, Xie, Yuan

Subject Terms: Computer Science - Hardware Architecture

Access URL: http://arxiv.org/abs/2008.03752

Face bonding strength of cross laminated northern hardwoods and softwoods lumber

Authors: Musah, Munkaila ^{a, b, 1}, Ma, Yunxiang ^{a, 1}, Wang, Xiping ^{a, c}, Ross, Robert ^{a, c}, Hosseinpourpia, Reza ^{a, d}, Jiang, Xiaolei ^{a, e}, Xie, Xinfeng ^{a, ⁎}

Source: In Construction and Building Materials 29 March 2024 421

Communication Optimization on GPU: A Case Study of Sequence Alignment Algorithms

Authors: Wang, Jie, Xie, Xinfeng, Cong, Jason

Source: 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS) IPDPS Parallel and Distributed Processing Symposium (IPDPS), 2017 IEEE International. :72-81 May, 2017

Relation: 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)

Poly(vinyl alcohol)/nanocellulose film integrated with phenolic waste-based carbon dots for ultraviolet-blocking and flame retardant applications

Authors: Zhang, Longfei, Peng, Limin, Liang, Shanqing, Xie, Xinfeng, Lyu, Shaoyi, Wang, Siqun

Source: In Progress in Organic Coatings November 2023 184