SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training

Nan He | Weichen Xiong | Hanwen Liu | Yi Liao | Lei Ding | Kai Zhang | Guohua Tang | Xiao Han | Yang Wei |