Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering EACL

Wenhu Chen | Pat Verga | Michiel De Jong | John Wieting | William Cohen |

GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints EMNLP

Joshua Ainslie | James Lee-Thorp | Michiel de Jong | Yury Zemlyanskiy | Federico Lebron | Sumit Sanghai |

FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference F i n d i n g s - A C L

Michiel de Jong | Yury Zemlyanskiy | Joshua Ainslie | Nicholas FitzGerald | Sumit Sanghai | Fei Sha | William Cohen |

CoLT5: Faster Long-Range Transformers with Conditional Computation EMNLP

Joshua Ainslie | Tao Lei | Michiel de Jong | Santiago Ontanon | Siddhartha Brahma | Yury Zemlyanskiy | David Uthus | Mandy Guo | James Lee-Thorp | Yi Tay | Yun-Hsuan Sung | Sumit Sanghai |

Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing COLING

Yury Zemlyanskiy | Michiel de Jong | Joshua Ainslie | Panupong Pasupat | Peter Shaw | Linlu Qiu | Sumit Sanghai | Fei Sha |

ReadTwice: Reading Very Large Documents with Memories NAACL

Yury Zemlyanskiy | Joshua Ainslie | Michiel de Jong | Philip Pham | Ilya Eckstein | Fei Sha |