Learning from Imperfect Data: Towards Efficient Knowledge Distillation of Autoregressive Language Models for Text-to-SQL

Authors: Zhong, Qihuang, Chen, Kunfeng, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language, Computer Science - Databases

Access URL: http://arxiv.org/abs/2410.11371

Iterative Data Generation with Large Language Models for Aspect-based Sentiment Analysis

Authors: Zhong, Qihuang, Li, Haiyun, Zhuang, Luyao, Liu, Juhua, Du, Bo

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2407.00341

Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems

Authors: Zhong, Qihuang, Wang, Kang, Xu, Ziyang, Liu, Juhua, Ding, Liang, Du, Bo

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2404.14963

Revisiting Knowledge Distillation for Autoregressive Language Models

Authors: Zhong, Qihuang, Ding, Liang, Shen, Li, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2402.11890

ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decoding

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2402.11889

Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models

Authors: Zhu, Miaoxi, Zhong, Qihuang, Shen, Li, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2310.13315

Self-Evolution Learning for Discriminative Language Model Pretraining

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2305.15275

Revisiting Token Dropping Strategy in Efficient BERT Pretraining

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Liu, Xuebo, Zhang, Min, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2305.15273

Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks

Authors: Zheng, Haoqi, Zhong, Qihuang, Ding, Liang, Tian, Zhiliang, Niu, Xin, Li, Dongsheng, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language, Computer Science - Networking and Internet Architecture

Access URL: http://arxiv.org/abs/2305.13547

Towards Making the Most of ChatGPT for Machine Translation

Authors: Peng, Keqin, Ding, Liang, Zhong, Qihuang, Shen, Li, Liu, Xuebo, Zhang, Min, Ouyang, Yuanxin, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2303.13780

AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks

Authors: Sun, Hao, Shen, Li, Zhong, Qihuang, Ding, Liang, Chen, Shixiang, Sun, Jingwei, Li, Jing, Sun, Guangzhong, Tao, Dacheng

Subject Terms: Computer Science - Machine Learning, Computer Science - Distributed, Parallel, and Cluster Computing, Mathematics - Optimization and Control

Access URL: http://arxiv.org/abs/2303.00565

Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2302.10198

Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE

Authors: Zhong, Qihuang, Ding, Liang, Peng, Keqin, Liu, Juhua, Du, Bo, Shen, Li, Zhan, Yibing, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2302.09268

Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE

Authors: Zhong, Qihuang, Ding, Liang, Zhan, Yibing, Qiao, Yu, Wen, Yonggang, Shen, Li, Liu, Juhua, Yu, Baosheng, Du, Bo, Chen, Yixin, Gao, Xinbo, Miao, Chunyan, Tang, Xiaoou, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2212.01853

Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models

Authors: Zhong, Qihuang, Ding, Liang, Shen, Li, Mi, Peng, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2210.05497

PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2208.10160

E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2205.14912

A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis

Authors: Wang, Bing, Ding, Liang, Zhong, Qihuang, Li, Ximing, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language

Access URL: http://arxiv.org/abs/2204.07832

Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis

Authors: Zhong, Qihuang, Ding, Liang, Liu, Juhua, Du, Bo, Jin, Hua, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2201.04831

Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis

Authors: Liu, Juhua, Zhong, Qihuang, Ding, Liang, Jin, Hua, Du, Bo, Tao, Dacheng

Subject Terms: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

Access URL: http://arxiv.org/abs/2110.13398