GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints EMNLP

Joshua Ainslie | James Lee-Thorp | Michiel de Jong | Yury Zemlyanskiy | Federico Lebron | Sumit Sanghai |

FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference F i n d i n g s - A C L

Michiel de Jong | Yury Zemlyanskiy | Joshua Ainslie | Nicholas FitzGerald | Sumit Sanghai | Fei Sha | William Cohen |

CoLT5: Faster Long-Range Transformers with Conditional Computation EMNLP

Joshua Ainslie | Tao Lei | Michiel de Jong | Santiago Ontanon | Siddhartha Brahma | Yury Zemlyanskiy | David Uthus | Mandy Guo | James Lee-Thorp | Yi Tay | Yun-Hsuan Sung | Sumit Sanghai |

Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing COLING

Yury Zemlyanskiy | Michiel de Jong | Joshua Ainslie | Panupong Pasupat | Peter Shaw | Linlu Qiu | Sumit Sanghai | Fei Sha |

DOCENT: Learning Self-Supervised Entity Representations from Large Document Collections EACL

Yury Zemlyanskiy | Sudeep Gandhe | Ruining He | Bhargav Kanagal | Anirudh Ravula | Juraj Gottweis | Fei Sha | Ilya Eckstein |

ReadTwice: Reading Very Large Documents with Memories NAACL

Yury Zemlyanskiy | Joshua Ainslie | Michiel de Jong | Philip Pham | Ilya Eckstein | Fei Sha |

Self-Attentive, Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text ACL

Lukas Ruff | Yury Zemlyanskiy | Robert Vandermeulen | Thomas Schnake | Marius Kloft |

Aiming to Know You Better Perhaps Makes Me a More Engaging Dialogue Partner CoNLL

Yury Zemlyanskiy | Fei Sha |