Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned ACL

Elena Voita | David Talbot | Fedor Moiseev | Rico Sennrich | Ivan Titov |