Symbol tuning improves in-context learning in language models EMNLP

Jerry Wei | Le Hou | Andrew Lampinen | Xiangning Chen | Da Huang | Yi Tay | Xinyun Chen | Yifeng Lu | Denny Zhou | Tengyu Ma | Quoc Le |

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them F i n d i n g s - A C L

Mirac Suzgun | Nathan Scales | Nathanael Schärli | Sebastian Gehrmann | Yi Tay | Hyung Won Chung | Aakanksha Chowdhery | Quoc Le | Ed Chi | Denny Zhou | Jason Wei |

Transcending Scaling Laws with 0.1% Extra Compute EMNLP

Yi Tay | Jason Wei | Hyung Chung | Vinh Tran | David So | Siamak Shakeri | Xavier Garcia | Steven Zheng | Jinfeng Rao | Aakanksha Chowdhery | Denny Zhou | Donald Metzler | Slav Petrov | Neil Houlsby | Quoc Le | Mostafa Dehghani |

Token Dropping for Efficient BERT Pretraining ACL

Le Hou | Richard Yuanzhe Pang | Tianyi Zhou | Yuexin Wu | Xinying Song | Xiaodan Song | Denny Zhou |

Extremely Small BERT Models from Mixed-Vocabulary Training EACL

Sanqiang Zhao | Raghav Gupta | Yang Song | Denny Zhou |

Fast WordPiece Tokenization EMNLP

Xinying Song | Alex Salcianu | Yang Song | Dave Dopson | Denny Zhou |

MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices ACL

Zhiqing Sun | Hongkun Yu | Xiaodan Song | Renjie Liu | Yiming Yang | Denny Zhou |