Crosslingual Generalization through Multitask Finetuning ACL

Niklas Muennighoff | Thomas Wang | Lintang Sutawika | Adam Roberts | Stella Biderman | Teven Le Scao | M Saiful Bari | Sheng Shen | Zheng Xin Yong | Hailey Schoelkopf | Xiangru Tang | Dragomir Radev | Alham Fikri Aji | Khalid Almubarak | Samuel Albanie | Zaid Alyafeai | Albert Webson | Edward Raff | Colin Raffel |

Character-Aware Models Improve Visual Text Rendering ACL

Rosanne Liu | Dan Garrette | Chitwan Saharia | William Chan | Adam Roberts | Sharan Narang | Irina Blok | Rj Mical | Mohammad Norouzi | Noah Constant |

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models TACL

Linting Xue | Aditya Barua | Noah Constant | Rami Al-Rfou | Sharan Narang | Mihir Kale | Adam Roberts | Colin Raffel |

mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer NAACL

Linting Xue | Noah Constant | Adam Roberts | Mihir Kale | Rami Al-Rfou | Aditya Siddhant | Aditya Barua | Colin Raffel |

Do Transformer Modifications Transfer Across Implementations and Applications? EMNLP

Sharan Narang | Hyung Won Chung | Yi Tay | Liam Fedus | Thibault Fevry | Michael Matena | Karishma Malkan | Noah Fiedel | Noam Shazeer | Zhenzhong Lan | Yanqi Zhou | Wei Li | Nan Ding | Jake Marcus | Adam Roberts | Colin Raffel |

How Much Knowledge Can You Pack Into the Parameters of a Language Model? EMNLP

Adam Roberts | Colin Raffel | Noam Shazeer |