Faster Inference of LLMs using FP8 on the Intel Gaudi

Authors: Lee, Joonhyung, Markovich-Golan, Shmulik, Ohayon, Daniel, Hanani, Yair, Park, Gunho, Kim, Byeongwook, Karnieli, Asaf, Livne, Uri, Shen, Haihao, Huang, Tai, Kwon, Se Jung, Lee, Dongsoo

Subject Terms: Computer Science - Hardware Architecture

Access URL: http://arxiv.org/abs/2503.09975

An Investigation of FP8 Across Accelerators for LLM Inference

Authors: Kim, Jiwoo, Lee, Joonhyung, Park, Gunho, Kim, Byeongwook, Kwon, Se Jung, Lee, Dongsoo, Lee, Youngjoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Performance

Access URL: http://arxiv.org/abs/2502.01070

Debunking the CUDA Myth Towards GPU-based AI Systems

Authors: Lee, Yunjae, Lim, Juntaek, Bang, Jehyeon, Cho, Eunyeong, Jeong, Huijong, Kim, Taesu, Kim, Hyungjun, Lee, Joonhyung, Im, Jinseop, Hwang, Ranggi, Kwon, Se Jung, Lee, Dongsoo, Rhu, Minsoo

Subject Terms: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Artificial Intelligence, Computer Science - Hardware Architecture

Access URL: http://arxiv.org/abs/2501.00210

LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices

Authors: Lee, Jung Hyun, Kim, Jeonghoon, Yang, June Yong, Kwon, Se Jung, Yang, Eunho, Yoo, Kang Min, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2407.11534

To FP8 and Back Again: Quantifying the Effects of Reducing Precision on LLM Training Stability

Authors: Lee, Joonhyung, Bae, Jeongin, Kim, Byeongwook, Kwon, Se Jung, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2405.18710

HyperCLOVA X Technical Report

Authors: Yoo, Kang Min, Han, Jaegeun, In, Sookyo, Jeon, Heewon, Jeong, Jisu, Kang, Jaewook, Kim, Hyunwook, Kim, Kyung-Min, Kim, Munhyong, Kim, Sungju, Kwak, Donghyun, Kwak, Hanock, Kwon, Se Jung, Lee, Bado, Lee, Dongsoo, Lee, Gichang, Lee, Jooho, Park, Baeseong, Shin, Seongjin, Yu, Joonsang, Baek, Seolki, Byeon, Sumin, Cho, Eungsup, Choe, Dooseok, Han, Jeesung, Jin, Youngkyun, Jun, Hyein, Jung, Jaeseung, Kim, Chanwoong, Kim, Jinhong, Kim, Jinuk, Lee, Dokyeong, Park, Dongwook, Sohn, Jeong Min, Han, Sujung, Heo, Jiae, Hong, Sungju, Jeon, Mina, Jung, Hyunhoon, Jung, Jungeun, Jung, Wangkyo, Kim, Chungjoon, Kim, Hyeri, Kim, Jonghyun, Kim, Min Young, Lee, Soeun, Park, Joonhee, Shin, Jieun, Yang, Sojin, Yoon, Jungsoon, Lee, Hwaran, Bae, Sanghwan, Cha, Jeehwan, Gylleus, Karl, Ham, Donghoon, Hong, Mihak, Hong, Youngki, Hong, Yunki, Jang, Dahyun, Jeon, Hyojun, Jeon, Yujin, Jeong, Yeji, Ji, Myunggeun, Jin, Yeguk, Jo, Chansong, Joo, Shinyoung, Jung, Seunghwan, Kim, Adrian Jungmyung, Kim, Byoung Hoon, Kim, Hyomin, Kim, Jungwhan, Kim, Minkyoung, Kim, Minseung, Kim, Sungdong, Kim, Yonghee, Kim, Youngjun, Kim, Youngkwan, Ko, Donghyeon, Lee, Dughyun, Lee, Ha Young, Lee, Jaehong, Lee, Jieun, Lee, Jonghyun, Lee, Jongjin, Lee, Min Young, Lee, Yehbin, Min, Taehong, Min, Yuri, Moon, Kiyoon, Oh, Hyangnam, Park, Jaesun, Park, Kyuyon, Park, Younghun, Seo, Hanbae, Seo, Seunghyun, Sim, Mihyun, Son, Gyubin, Yeo, Matt, Yeom, Kyung Hoon, Yoo, Wonjoon, You, Myungin, Ahn, Doheon, Ahn, Homin, Ahn, Joohee, Ahn, Seongmin, An, Chanwoo, An, Hyeryun, An, Junho, An, Sang-Min, Byun, Boram, Byun, Eunbin, Cha, Jongho, Chang, Minji, Chang, Seunggyu, Cho, Haesong, Cho, Youngdo, Choi, Dalnim, Choi, Daseul, Choi, Hyoseok, Choi, Minseong, Choi, Sangho, Choi, Seongjae, Choi, Wooyong, Chun, Sewhan, Go, Dong Young, Ham, Chiheon, Han, Danbi, Han, Jaemin, Hong, Moonyoung, Hong, Sung Bum, Hwang, Dong-Hyun, Hwang, Seongchan, Im, Jinbae, Jang, Hyuk Jin, Jang, Jaehyung, Jang, Jaeni, Jang, Sihyeon, Jang, Sungwon, Jeon, Joonha, Jeong, Daun, Jeong, Joonhyun, Jeong, Kyeongseok, Jeong, Mini, Jin, Sol, Jo, Hanbyeol, Jo, Hanju, Jo, Minjung, Jung, Chaeyoon, Jung, Hyungsik, Jung, Jaeuk, Jung, Ju Hwan, Jung, Kwangsun, Jung, Seungjae, Ka, Soonwon, Kang, Donghan, Kang, Soyoung, Kil, Taeho, Kim, Areum, Kim, Beomyoung, Kim, Byeongwook, Kim, Daehee, Kim, Dong-Gyun, Kim, Donggook, Kim, Donghyun, Kim, Euna, Kim, Eunchul, Kim, Geewook, Kim, Gyu Ri, Kim, Hanbyul, Kim, Heesu, Kim, Isaac, Kim, Jeonghoon, Kim, Jihye, Kim, Joonghoon, Kim, Minjae, Kim, Minsub, Kim, Pil Hwan, Kim, Sammy, Kim, Seokhun, Kim, Seonghyeon, Kim, Soojin, Kim, Soong, Kim, Soyoon, Kim, Sunyoung, Kim, Taeho, Kim, Wonho, Kim, Yoonsik, Kim, You Jin, Kim, Yuri, Kwon, Beomseok, Kwon, Ohsung, Kwon, Yoo-Hwan, Lee, Anna, Lee, Byungwook, Lee, Changho, Lee, Daun, Lee, Dongjae, Lee, Ha-Ram, Lee, Hodong, Lee, Hwiyeong, Lee, Hyunmi, Lee, Injae, Lee, Jaeung, Lee, Jeongsang, Lee, Jisoo, Lee, Jongsoo, Lee, Joongjae, Lee, Juhan, Lee, Jung Hyun, Lee, Junghoon, Lee, Junwoo, Lee, Se Yun, Lee, Sujin, Lee, Sungjae, Lee, Sungwoo, Lee, Wonjae, Lee, Zoo Hyun, Lim, Jong Kun, Lim, Kun, Lim, Taemin, Na, Nuri, Nam, Jeongyeon, Nam, Kyeong-Min, Noh, Yeonseog, Oh, Biro, Oh, Jung-Sik, Oh, Solgil, Oh, Yeontaek, Park, Boyoun, Park, Cheonbok, Park, Dongju, Park, Hyeonjin, Park, Hyun Tae, Park, Hyunjung, Park, Jihye, Park, Jooseok, Park, Junghwan, Park, Jungsoo, Park, Miru, Park, Sang Hee, Park, Seunghyun, Park, Soyoung, Park, Taerim, Park, Wonkyeong, Ryu, Hyunjoon, Ryu, Jeonghun, Ryu, Nahyeon, Seo, Soonshin, Seo, Suk Min, Shim, Yoonjeong, Shin, Kyuyong, Shin, Wonkwang, Sim, Hyun, Sim, Woongseob, Soh, Hyejin, Son, Bokyong, Son, Hyunjun, Son, Seulah, Song, Chi-Yun, Song, Chiyoung, Song, Ka Yeon, Song, Minchul, Song, Seungmin, Wang, Jisung, Yeo, Yonggoo, Yi, Myeong Yeon, Yim, Moon Bin, Yoo, Taehwan, Yoo, Youngjoon, Yoon, Sungmin, Yoon, Young Jin, Yu, Hangyeol, Yu, Ui Seon, Zuo, Xingdong, Bae, Jeongin, Bae, Joungeun, Cho, Hyunsoo, Cho, Seonghyun, Cho, Yongjin, Choi, Taekyoon, Choi, Yera, Chung, Jiwan, Han, Zhenghui, Heo, Byeongho, Hong, Euisuk, Hwang, Taebaek, Im, Seonyeol, Jegal, Sumin, Jeon, Sumin, Jeong, Yelim, Jeong, Yonghyun, Jiang, Can, Jiang, Juyong, Jin, Jiho, Jo, Ara, Jo, Younghyun, Jung, Hoyoun, Jung, Juyoung, Kang, Seunghyeong, Kim, Dae Hee, Kim, Ginam, Kim, Hangyeol, Kim, Heeseung, Kim, Hyojin, Kim, Hyojun, Kim, Hyun-Ah, Kim, Jeehye, Kim, Jin-Hwa, Kim, Jiseon, Kim, Jonghak, Kim, Jung Yoon, Kim, Rak Yeong, Kim, Seongjin, Kim, Seoyoon, Kim, Sewon, Kim, Sooyoung, Kim, Sukyoung, Kim, Taeyong, Ko, Naeun, Koo, Bonseung, Kwak, Heeyoung, Kwon, Haena, Kwon, Youngjin, Lee, Boram, Lee, Bruce W., Lee, Dagyeong, Lee, Erin, Lee, Euijin, Lee, Ha Gyeong, Lee, Hyojin, Lee, Hyunjeong, Lee, Jeeyoon, Lee, Jeonghyun, Lee, Jongheok, Lee, Joonhyung, Lee, Junhyuk, Lee, Mingu, Lee, Nayeon, Lee, Sangkyu, Lee, Se Young, Lee, Seulgi, Lee, Seung Jin, Lee, Suhyeon, Lee, Yeonjae, Lee, Yesol, Lee, Youngbeom, Lee, Yujin, Li, Shaodong, Liu, Tianyu, Moon, Seong-Eun, Moon, Taehong, Nihlenramstroem, Max-Lasse, Oh, Wonseok, Oh, Yuri, Park, Hongbeen, Park, Hyekyung, Park, Jaeho, Park, Nohil, Park, Sangjin, Ryu, Jiwon, Ryu, Miru, Ryu, Simo, Seo, Ahreum, Seo, Hee, Seo, Kangdeok, Shin, Jamin, Shin, Seungyoun, Sin, Heetae, Wang, Jiangping, Wang, Lei, Xiang, Ning, Xiao, Longxiang, Xu, Jing, Yi, Seonyeong, Yoo, Haanju, Yoo, Haneul, Yoo, Hwanhee, Yu, Liang, Yu, Youngjae, Yuan, Weijie, Zeng, Bo, Zhou, Qian, Cho, Kyunghyun, Ha, Jung-Woo, Park, Joonsuk, Hwang, Jihyun, Kwon, Hyoung Jo, Kwon, Soonyong, Lee, Jungyeon, Lee, Seungho, Lim, Seonghyeon, Noh, Hyunkyung, Choi, Seungho, Lee, Sang-Woo, Lim, Jung Hwa, Sung, Nako

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2404.01954

No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization

Authors: Yang, June Yong, Kim, Byeongwook, Bae, Jeongin, Kwon, Beomseok, Park, Gunho, Yang, Eunho, Kwon, Se Jung, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2402.18096

Label-Noise Robust Diffusion Models

Authors: Na, Byeonghu, Kim, Yeongmin, Bae, HeeSun, Lee, Jung Hyun, Kwon, Se Jung, Kang, Wanmo, Moon, Il-Chul

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2402.17517

DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation

Authors: Woo, Sunghyeon, Park, Baeseong, Kim, Byeongwook, Jo, Minjung, Kwon, Se Jung, Jeon, Dongsuk, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2402.17812

Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models

Authors: Heo, Jung Hwan, Kim, Jeonghoon, Kwon, Beomseok, Kim, Byeongwook, Kwon, Se Jung, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2309.15531

FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization

Authors: Lee, Jung Hyun, Kim, Jeonghoon, Kwon, Se Jung, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2306.00317

Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization

Authors: Kim, Jeonghoon, Lee, Jung Hyun, Kim, Sungdong, Park, Joonsuk, Yoo, Kang Min, Kwon, Se Jung, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2305.14152

Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models

Authors: Kim, Dongjun, Kim, Yeongmin, Kwon, Se Jung, Kang, Wanmo, Moon, Il-Chul

Subject Terms: Computer Science - Computer Vision and Pattern Recognition, Computer Science - Artificial Intelligence, Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2211.17091

AlphaTuning: Quantization-Aware Parameter-Efficient Adaptation of Large-Scale Pre-Trained Language Models

Authors: Kwon, Se Jung, Kim, Jeonghoon, Bae, Jeongin, Yoo, Kang Min, Kim, Jin-Hwa, Park, Baeseong, Kim, Byeongwook, Ha, Jung-Woo, Sung, Nako, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2210.03858

BiQGEMM: Matrix Multiplication with Lookup Table for Binary-Coding-Based Quantized DNNs

Authors: Jeon, Yongkweon, Park, Baeseong, Kwon, Se Jung, Kim, Byeongwook, Yun, Jeongin, Lee, Dongsoo

Source: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis SC High Performance Computing, Networking, Storage and Analysis, SC20: International Conference for. :1-14 Nov, 2020

Relation: SC20: International Conference for High Performance Computing, Networking, Storage and Analysis

Structured Compression by Weight Encryption for Unstructured Pruning and Quantization

Authors: Kwon, Se Jung, Lee, Dongsoo, Kim, Byeongwook, Kapoor, Parichay, Park, Baeseong, Wei, Gu-Yeon

Source: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) CVPR Computer Vision and Pattern Recognition (CVPR), 2020 IEEE/CVF Conference on. :1906-1915 Jun, 2020

Relation: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models

Authors: Park, Gunho, Park, Baeseong, Kim, Minsub, Lee, Sungjae, Kim, Jeonghoon, Kwon, Beomseok, Kwon, Se Jung, Kim, Byeongwook, Lee, Youngjoo, Lee, Dongsoo

Subject Terms: Computer Science - Distributed, Parallel, and Cluster Computing, Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2206.09557

Maximum Likelihood Training of Implicit Nonlinear Diffusion Models

Authors: Kim, Dongjun, Na, Byeonghu, Kwon, Se Jung, Lee, Dongsoo, Kang, Wanmo, Moon, Il-Chul

Source: Advances in Neural Information Processing Systems 2022 (NeurIPS22)

Subject Terms: Computer Science - Machine Learning

Access URL: http://arxiv.org/abs/2205.13699

Modulating Regularization Frequency for Efficient Compression-Aware Model Training

Authors: Lee, Dongsoo, Kwon, Se Jung, Kim, Byeongwook, Yun, Jeongin, Park, Baeseong, Jeon, Yongkweon

Subject Terms: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2105.01875

Encoding Weights of Irregular Sparsity for Fixed-to-Fixed Model Compression

Authors: Park, Baeseong, Kwon, Se Jung, Oh, Daehwan, Kim, Byeongwook, Lee, Dongsoo

Subject Terms: Computer Science - Machine Learning, Computer Science - Information Theory

Access URL: http://arxiv.org/abs/2105.01869