LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin

Shihan Dou | Enyu Zhou | Yan Liu | Songyang Gao | Wei Shen | Limao Xiong | Yuhao Zhou | Xiao Wang | Zhiheng Xi | Xiaoran Fan | Shiliang Pu | Jiang Zhu | Rui Zheng | Tao Gui | Qi Zhang | Xuanjing Huang |