Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Jingyang Yuan | Huazuo Gao | Damai Dai | Junyu Luo | Liang Zhao | Zhengyan Zhang | Zhenda Xie | Yuxing Wei | Lean Wang | Zhiping Xiao | Yuqing Wang | Chong Ruan | Ming Zhang | Wenfeng Liang | Wangding Zeng |