Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study

Boxin Wang | Wei Ping | Peng Xu | Lawrence McAfee | Zihan Liu | Mohammad Shoeybi | Yi Dong | Oleksii Kuchaiev | Bo Li | Chaowei Xiao | Anima Anandkumar | Bryan Catanzaro |