Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts F i n d i n g s - A C L

Ganesh Jawahar | Haichuan Yang | Yunyang Xiong | Zechun Liu | Dilin Wang | Fei Sun | Meng Li | Aasish Pappu | Barlas Oguz | Muhammad Abdul-Mageed | Laks Lakshmanan | Raghuraman Krishnamoorthi | Vikas Chandra |

LLM-QAT: Data-Free Quantization Aware Training for Large Language Models F i n d i n g s - A C L

Zechun Liu | Barlas Oguz | Changsheng Zhao | Ernie Chang | Pierre Stock | Yashar Mehdad | Yangyang Shi | Raghuraman Krishnamoorthi | Vikas Chandra |

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization F i n d i n g s - E M N L P

Xijie Huang | Zechun Liu | Shih-Yang Liu | Kwang-Ting Cheng |

Scaling Parameter-Constrained Language Models with Quality Data EMNLP

Ernie Chang | Matteo Paltenghi | Yang Li | Pin-Jie Lin | Changsheng Zhao | Patrick Huber | Zechun Liu | Rastislav Rabatin | Yangyang Shi | Vikas Chandra |

Target-Aware Language Modeling via Granular Data Sampling EMNLP

Ernie Chang | Pin-Jie Lin | Yang Li | Changsheng Zhao | Daeil Kim | Rastislav Rabatin | Zechun Liu | Yangyang Shi | Vikas Chandra |