Proceedings of the Third Workshop on Natural Language Generation, Evaluation, and Metrics (GEM) GEM WS

Sebastian Gehrmann | Alex Wang | João Sedoc | Elizabeth Clark | Kaustubh Dhole | Khyathi Raghavi Chandu | Enrico Santus | Hooman Sedghamiz |

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them F i n d i n g s - A C L

Mirac Suzgun | Nathan Scales | Nathanael Schärli | Sebastian Gehrmann | Yi Tay | Hyung Won Chung | Aakanksha Chowdhery | Quoc Le | Ed Chi | Denny Zhou | Jason Wei |

SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation EMNLP

Elizabeth Clark | Shruti Rijhwani | Sebastian Gehrmann | Joshua Maynez | Roee Aharoni | Vitaly Nikolaev | Thibault Sellam | Aditya Siddhant | Dipanjan Das | Ankur Parikh |

TaTA: A Multilingual Table-to-Text Dataset for African Languages F i n d i n g s - E M N L P

Sebastian Gehrmann | Sebastian Ruder | Vitaly Nikolaev | Jan Botha | Michael Chavinda | Ankur Parikh | Clara Rivera |

Dialect-robust Evaluation of Generated Text ACL

Jiao Sun | Thibault Sellam | Elizabeth Clark | Tu Vu | Timothy Dozat | Dan Garrette | Aditya Siddhant | Jacob Eisenstein | Sebastian Gehrmann |

Benchmarking Large Language Model Capabilities for Conditional Generation ACL

Joshua Maynez | Priyanka Agrawal | Sebastian Gehrmann |

A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization ACL

Lining Zhang | Simon Mille | Yufang Hou | Daniel Deutsch | Elizabeth Clark | Yixin Liu | Saad Mahamood | Sebastian Gehrmann | Miruna Clinciu | Khyathi Raghavi Chandu | João Sedoc |

GEMv2: Multilingual NLG Benchmarking in a Single Line of Code EMNLP

Sebastian Gehrmann | Abhik Bhattacharjee | Abinaya Mahendiran | Alex Wang | Alexandros Papangelis | Aman Madaan | Angelina Mcmillan-major | Anna Shvets | Ashish Upadhyay | Bernd Bohnet |

Intriguing Properties of Compression on Multilingual Models EMNLP

Kelechi Ogueji | Orevaoghene Ahia | Gbemileke Onilude | Sebastian Gehrmann | Sara Hooker | Julia Kreutzer |

Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM) GEM

Antoine Bosselut | Khyathi Chandu | Kaustubh Dhole | Varun Gangal | Sebastian Gehrmann | Yacine Jernite | Jekaterina Novikova | Laura Perez-Beltrachini |

evision PDF of 'Learning Compact Metrics for MT EMNLP

Amy Pu | Hyung Won Chung | Ankur Parikh | Sebastian Gehrmann | Thibault Sellam |

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics ACL GEM IJCNLP

Sebastian Gehrmann | Tosin Adewumi | Karmanya Aggarwal | Pawan Sasanka Ammanamanchi | Anuoluwapo Aremu | Antoine Bosselut | Khyathi Raghavi Chandu | Miruna-Adriana Clinciu | Dipanjan Das | Kaustubh Dhole | Wanyu Du | Esin Durmus | Ondřej Dušek | Chris Chinenye Emezue | Varun Gangal | Cristina Garbacea | Tatsunori Hashimoto | Yufang Hou | Yacine Jernite | Harsh Jhamtani | Yangfeng Ji | Shailza Jolly | Mihir Kale | Dhruv Kumar | Faisal Ladhak | Aman Madaan | Mounica Maddela | Khyati Mahajan | Saad Mahamood | Bodhisattwa Prasad Majumder | Pedro Henrique Martins | Angelina McMillan-Major | Simon Mille | Emiel van Miltenburg | Moin Nadeem | Shashi Narayan | Vitaly Nikolaev | Andre Niyongabo Rubungo | Salomey Osei | Ankur Parikh | Laura Perez-Beltrachini | Niranjan Ramesh Rao | Vikas Raunak | Juan Diego Rodriguez | Sashank Santhanam | João Sedoc | Thibault Sellam | Samira Shaikh | Anastasia Shimorina | Marco Antonio Sobrevilla Cabezudo | Hendrik Strobelt | Nishant Subramani | Wei Xu | Diyi Yang | Akhila Yerukola | Jiawei Zhou |

Proceedings of the 1st Workshop on Natural Language Generation, Evaluation, and Metrics (GEM 2021) ACL GEM IJCNLP

Antoine Bosselut | Esin Durmus | Varun Prashant Gangal | Sebastian Gehrmann | Yacine Jernite | Laura Perez-Beltrachini | Samira Shaikh | Wei Xu |

Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards ACL GEM IJCNLP

Angelina McMillan-Major | Salomey Osei | Juan Diego Rodriguez | Pawan Sasanka Ammanamanchi | Sebastian Gehrmann | Yacine Jernite |

Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Models ACL IJCNLP

Matthew Finlayson | Aaron Mueller | Sebastian Gehrmann | Stuart Shieber | Tal Linzen | Yonatan Belinkov |

LMdiff: A Visual Diff Tool to Compare Language Models EMNLP

Hendrik Strobelt | Benjamin Hoover | Arvind Satyanaryan | Sebastian Gehrmann |

Interpretability and Analysis in Neural NLP ACL

Yonatan Belinkov | Sebastian Gehrmann | Ellie Pavlick |

exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformer Models ACL

Benjamin Hoover | Hendrik Strobelt | Sebastian Gehrmann |

Learning to Evaluate Translation Beyond English: BLEURT Submissions to the WMT Metrics 2020 Shared Task EMNLP WMT

Thibault Sellam | Amy Pu | Hyung Won Chung | Sebastian Gehrmann | Qijun Tan | Markus Freitag | Dipanjan Das | Ankur Parikh |

The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models EMNLP

Ian Tenney | James Wexler | Jasmijn Bastings | Tolga Bolukbasi | Andy Coenen | Sebastian Gehrmann | Ellen Jiang | Mahima Pushkarna | Carey Radebaugh | Emily Reif | Ann Yuan |

Proceedings of the 1st Workshop on Evaluating NLG Evaluation EvalNLGEval INLG

Shubham Agarwal | Ondřej Dušek | Sebastian Gehrmann | Dimitra Gkatzia | Ioannis Konstas | Emiel Van Miltenburg | Sashank Santhanam |

ToTTo: A Controlled Table-To-Text Generation Dataset EMNLP

Ankur Parikh | Xuezhi Wang | Sebastian Gehrmann | Manaal Faruqui | Bhuwan Dhingra | Diyi Yang | Dipanjan Das |

A Corpus for Detecting High-Context Medical Conditions in Intensive Care Patient Notes Focusing on Frequently Readmitted Patients LREC

Edward T. Moseley | Joy T. Wu | Jonathan Welt | John Foote | Patrick D. Tyler | David W. Grant | Eric T. Carlson | Sebastian Gehrmann | Franck Dernoncourt | Leo Anthony Celi |

Improving Human Text Comprehension through Semi-Markov CRF-based Neural Section Title Generation NAACL

Sebastian Gehrmann | Steven Layne | Franck Dernoncourt |

GLTR: Statistical Detection and Visualization of Generated Text ACL

Sebastian Gehrmann | Hendrik Strobelt | Alexander Rush |

LSTM Networks Can Perform Dynamic Counting ACL WS

Mirac Suzgun | Yonatan Belinkov | Stuart Shieber | Sebastian Gehrmann |

Margin Call: an Accessible Web-based Text Viewer with Generated Paragraph Summaries in the Margin INLG WS

Naba Rizvi | Sebastian Gehrmann | Lidan Wang | Franck Dernoncourt |

Generating Abstractive Summaries with Finetuned Language Models INLG WS

Sebastian Gehrmann | Zachary Ziegler | Alexander Rush |

Bottom-Up Abstractive Summarization EMNLP

Sebastian Gehrmann | Yuntian Deng | Alexander Rush |

Debugging Sequence-to-Sequence Models with Seq2Seq-Vis EMNLP WS

Hendrik Strobelt | Sebastian Gehrmann | Michael Behrisch | Adam Perer | Hanspeter Pfister | Alexander Rush |

End-to-End Content and Plan Selection for Data-to-Text Generation INLG WS

Sebastian Gehrmann | Falcon Dai | Henry Elder | Alexander Rush |

E2E NLG Challenge Submission: Towards Controllable Generation of Diverse Natural Language INLG WS

Henry Elder | Sebastian Gehrmann | Alexander O’Connor | Qun Liu |