Petals: Collaborative Inference and Fine-tuning of Large Models ACL

Alexander Borzunov | Dmitry Baranchuk | Tim Dettmers | Maksim Riabinin | Younes Belkada | Artem Chumachenko | Pavel Samygin | Colin Raffel |

An Empirical Survey of Data Augmentation for Limited Data Learning in NLP TACL

Jiaao Chen | Derek Tam | Colin Raffel | Mohit Bansal | Diyi Yang |

ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning ACL

Shachar Don-Yehiya | Elad Venezian | Colin Raffel | Noam Slonim | Leshem Choshen |

Knowledge is a Region in Weight Space for Fine-tuned Language Models F i n d i n g s - E M N L P

Almog Gueta | Elad Venezian | Colin Raffel | Noam Slonim | Yoav Katz | Leshem Choshen |

Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model EMNLP

Haikang Deng | Colin Raffel |

Evaluating the Factual Consistency of Large Language Models Through News Summarization F i n d i n g s - A C L

Derek Tam | Anisha Mascarenhas | Shiyue Zhang | Sarah Kwan | Mohit Bansal | Colin Raffel |

Efficient Methods for Natural Language Processing: A Survey TACL

Marcos Treviso | Ji-Ung Lee | Tianchu Ji | Betty van Aken | Qingqing Cao | Manuel R. Ciosici | Michael Hassid | Kenneth Heafield | Sara Hooker | Colin Raffel | Pedro H. Martins | André F. T. Martins | Jessica Zosa Forde | Peter Milder | Edwin Simpson | Noam Slonim | Jesse Dodge | Emma Strubell | Niranjan Balasubramanian | Leon Derczynski | Iryna Gurevych | Roy Schwartz |

Crosslingual Generalization through Multitask Finetuning ACL

Niklas Muennighoff | Thomas Wang | Lintang Sutawika | Adam Roberts | Stella Biderman | Teven Le Scao | M Saiful Bari | Sheng Shen | Zheng Xin Yong | Hailey Schoelkopf | Xiangru Tang | Dragomir Radev | Alham Fikri Aji | Khalid Almubarak | Samuel Albanie | Zaid Alyafeai | Albert Webson | Edward Raff | Colin Raffel |

PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts ACL

Stephen Bach | Victor Sanh | Zheng Xin Yong | Albert Webson | Colin Raffel | Nihal V. Nayak | Abheesht Sharma | Taewoon Kim | M Saiful Bari | Thibault Fevry | Zaid Alyafeai | Manan Dey | Andrea Santilli | Zhiqing Sun | Srulik Ben-david | Canwen Xu | Gunjan Chhablani | Han Wang | Jason Fries | Maged Al-shaibani | Shanya Sharma | Urmish Thakker | Khalid Almubarak | Xiangru Tang | Dragomir Radev | Mike Tian-jian Jiang | Alexander Rush |

Learning with Limited Text Data ACL

Diyi Yang | Ankur Parikh | Colin Raffel |

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models TACL

Linting Xue | Aditya Barua | Noah Constant | Rami Al-Rfou | Sharan Narang | Mihir Kale | Adam Roberts | Colin Raffel |

What Language Model to Train if You Have One Million GPU Hours? F i n d i n g s - E M N L P

Teven Le Scao | Thomas Wang | Daniel Hesslow | Stas Bekman | M Saiful Bari | Stella Biderman | Hady Elsahar | Niklas Muennighoff | Jason Phang | Ofir Press | Colin Raffel | Victor Sanh | Sheng Shen | Lintang Sutawika | Jaesung Tae | Zheng Xin Yong | Julien Launay | Iz Beltagy |

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer NAACL

Linting Xue | Noah Constant | Adam Roberts | Mihir Kale | Rami Al-Rfou | Aditya Siddhant | Aditya Barua | Colin Raffel |

Do Transformer Modifications Transfer Across Implementations and Applications? EMNLP

Sharan Narang | Hyung Won Chung | Yi Tay | Liam Fedus | Thibault Fevry | Michael Matena | Karishma Malkan | Noah Fiedel | Noam Shazeer | Zhenzhong Lan | Yanqi Zhou | Wei Li | Nan Ding | Jake Marcus | Adam Roberts | Colin Raffel |

Improving and Simplifying Pattern Exploiting Training EMNLP

Derek Tam | Rakesh R. Menon | Mohit Bansal | Shashank Srivastava | Colin Raffel |

How Much Knowledge Can You Pack Into the Parameters of a Language Model? EMNLP

Adam Roberts | Colin Raffel | Noam Shazeer |

Monotonic Infinite Lookback Attention for Simultaneous Machine Translation ACL

Naveen Arivazhagan | Colin Cherry | Wolfgang Macherey | Chung-Cheng Chiu | Semih Yavuz | Ruoming Pang | Wei Li | Colin Raffel |