Improving In-Context Few-Shot Learning via Self-Supervised Training NAACL

Mingda Chen | Jingfei Du | Ramakanth Pasunuru | Todor Mihaylov | Srini Iyer | Veselin Stoyanov | Zornitsa Kozareva |

Efficient Large Scale Language Modeling with Mixtures of Experts EMNLP

Mikel Artetxe | Shruti Bhosale | Naman Goyal | Todor Mihaylov | Myle Ott | Sam Shleifer | Xi Victoria Lin | Jingfei Du | Srinivasan Iyer | Ramakanth Pasunuru | Giridharan Anantharaman | Xian Li | Shuohui Chen | Halil Akin | Mandeep Baines | Louis Martin | Xing Zhou | Punit Singh Koura | Brian O’Horo | Jeffrey Wang | Luke Zettlemoyer | Mona Diab | Zornitsa Kozareva | Veselin Stoyanov |

Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models EMNLP

Mengzhou Xia | Mikel Artetxe | Jingfei Du | Danqi Chen | Veselin Stoyanov |

On the Role of Bidirectionality in Language Model Pre-Training F i n d i n g s - E M N L P

Mikel Artetxe | Jingfei Du | Naman Goyal | Luke Zettlemoyer | Veselin Stoyanov |

Few-shot Learning with Multilingual Generative Language Models EMNLP

Xi Victoria Lin | Todor Mihaylov | Mikel Artetxe | Tianlu Wang | Shuohui Chen | Daniel Simig | Myle Ott | Naman Goyal | Shruti Bhosale | Jingfei Du | Ramakanth Pasunuru | Sam Shleifer | Punit Singh Koura | Vishrav Chaudhary | Brian O’Horo | Jeff Wang | Luke Zettlemoyer | Zornitsa Kozareva | Mona Diab | Veselin Stoyanov | Xian Li |

Self-training Improves Pre-training for Natural Language Understanding NAACL

Jingfei Du | Edouard Grave | Beliz Gunel | Vishrav Chaudhary | Onur Celebi | Michael Auli | Veselin Stoyanov | Alexis Conneau |

Larger-Scale Transformers for Multilingual Masked Language Modeling ACL IJCNLP RepL4NLP

Naman Goyal | Jingfei Du | Myle Ott | Giri Anantharaman | Alexis Conneau |

General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference EMNLP Findings

Jingfei Du | Myle Ott | Haoran Li | Xing Zhou | Veselin Stoyanov |

Pretrained Language Models for Biomedical and Clinical Tasks: Understanding and Extending the State-of-the-Art ClinicalNLP EMNLP

Patrick Lewis | Myle Ott | Jingfei Du | Veselin Stoyanov |

Knowledge-Augmented Language Model and Its Application to Unsupervised Named-Entity Recognition NAACL

Angli Liu | Jingfei Du | Veselin Stoyanov |