Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks ACL

Tristan Thrush | Kushal Tirumala | Anmol Gupta | Max Bartolo | Pedro Rodriguez | Tariq Kane | William Gaviria Rojas | Peter Mattson | Adina Williams | Douwe Kiela |

Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-Based Hate NAACL

Hannah Kirk | Bertie Vidgen | Paul Rottger | Tristan Thrush | Scott Hale |

Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants NAACL

Max Bartolo | Tristan Thrush | Sebastian Riedel | Pontus Stenetorp | Robin Jia | Douwe Kiela |

Proceedings of the First Workshop on Dynamic Adversarial Data Collection DADC NAACL

Max Bartolo | Hannah Kirk | Pedro Rodriguez | Katerina Margatina | Tristan Thrush | Robin Jia | Pontus Stenetorp | Adina Williams | Douwe Kiela |

Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements EMNLP

Leandro Von Werra | Lewis Tunstall | Abhishek Thakur | Sasha Luccioni | Tristan Thrush | Aleksandra Piktus | Felix Marty | Nazneen Rajani | Victor Mustar | Helen Ngo |

Dynabench: Rethinking Benchmarking in NLP NAACL

Douwe Kiela | Max Bartolo | Yixin Nie | Divyansh Kaushik | Atticus Geiger | Zhengxuan Wu | Bertie Vidgen | Grusha Prasad | Amanpreet Singh | Pratik Ringshia | Zhiyi Ma | Tristan Thrush | Sebastian Riedel | Zeerak Waseem | Pontus Stenetorp | Robin Jia | Mohit Bansal | Christopher Potts | Adina Williams |

Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation EMNLP

Max Bartolo | Tristan Thrush | Robin Jia | Sebastian Riedel | Pontus Stenetorp | Douwe Kiela |

Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection ACL IJCNLP

Bertie Vidgen | Tristan Thrush | Zeerak Waseem | Douwe Kiela |

Findings of the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation EMNLP WMT

Guillaume Wenzek | Vishrav Chaudhary | Angela Fan | Sahir Gomez | Naman Goyal | Somya Jain | Douwe Kiela | Tristan Thrush | Francisco Guzmán |

Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization BlackboxNLP EMNLP

Tristan Thrush | Ethan Wilcox | Roger Levy |