XLM-E: Cross-lingual Language Model Pre-training via ELECTRA ACL

Zewen Chi | Shaohan Huang | Li Dong | Shuming Ma | Bo Zheng | Saksham Singhal | Payal Bajaj | Xia Song | Xian-Ling Mao | Heyan Huang | Furu Wei |

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training NAACL

Zewen Chi | Li Dong | Furu Wei | Nan Yang | Saksham Singhal | Wenhui Wang | Xia Song | Xian-Ling Mao | Heyan Huang | Ming Zhou |

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs EMNLP

Zewen Chi | Li Dong | Shuming Ma | Shaohan Huang | Saksham Singhal | Xian-Ling Mao | Heyan Huang | Xia Song | Furu Wei |

Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task EMNLP WMT

Jian Yang | Shuming Ma | Haoyang Huang | Dongdong Zhang | Li Dong | Shaohan Huang | Alexandre Muzio | Saksham Singhal | Hany Hassan | Xia Song | Furu Wei |

Consistency Regularization for Cross-Lingual Fine-Tuning ACL IJCNLP

Bo Zheng | Li Dong | Shaohan Huang | Wenhui Wang | Zewen Chi | Saksham Singhal | Wanxiang Che | Ting Liu | Xia Song | Furu Wei |

Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training EMNLP

Bo Zheng | Li Dong | Shaohan Huang | Saksham Singhal | Wanxiang Che | Ting Liu | Xia Song | Furu Wei |