Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research

Luca Soldaini | Rodney Kinney | Akshita Bhagia | Dustin Schwenk | David Atkinson | Russell Authur | Ben Bogin | Khyathi Chandu | Jennifer Dumas | Yanai Elazar | Valentin Hofmann | Ananya Jha | Sachin Kumar | Li Lucy | Xinxi Lyu | Nathan Lambert | Ian Magnusson | Jacob Morrison | Niklas Muennighoff | Aakanksha Naik | Crystal Nam | Matthew Peters | Abhilasha Ravichander | Kyle Richardson | Zejiang Shen | Emma Strubell | Nishant Subramani | Oyvind Tafjord | Evan Walsh | Luke Zettlemoyer | Noah Smith | Hannaneh Hajishirzi | Iz Beltagy | Dirk Groeneveld | Jesse Dodge | Kyle Lo |

Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models

Junhao Zheng | Shengjie Qiu | Qianli Ma |

Distributional Inclusion Hypothesis and Quantifications: Probing for Hypernymy in Functional Distributional Semantics

Chun Lo | Wai Lam | Hong Cheng | Guy Emerson |

AI ‘News’ Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian

Giovanni Puccetti | Anna Rogers | Chiara Alzetta | Felice Dell’Orletta | Andrea Esuli |

EZ-STANCE: A Large Dataset for English Zero-Shot Stance Detection

Chenye Zhao | Cornelia Caragea |

RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization

Jaavid J | Raj Dabre | Aswanth M | Jay Gala | Thanmay Jayakumar | Ratish Puduppully | Anoop Kunchukuttan |

Guardians of the Machine Translation Meta-Evaluation: Sentinel Metrics Fall In!

Stefano Perrella | Lorenzo Proietti | Alessandro Scirè | Edoardo Barba | Roberto Navigli |

EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities

Nian Li | Chen Gao | Mingyu Li | Yong Li | Qingmin Liao |

Spiral of Silence: How is Large Language Model Killing Information Retrieval?—A Case Study on Open Domain Question Answering

Xiaoyang Chen | Ben He | Hongyu Lin | Xianpei Han | Tianshu Wang | Boxi Cao | Le Sun | Yingfei Sun |

Disinformation Capabilities of Large Language Models

Ivan Vykopal | Matúš Pikuliak | Ivan Srba | Robert Moro | Dominik Macko | Maria Bielikova |

COKE: A Cognitive Knowledge Graph for Machine Theory of Mind

Jincenzi Wu | Zhuang Chen | Jiawen Deng | Sahand Sabour | Helen Meng | Minlie Huang |

On Efficient and Statistical Quality Estimation for Data Annotation

Jan-Christoph Klie | Juan Haladjian | Marc Kirchner | Rahul Nair |

Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Model

Christian Tomani | David Vilar | Markus Freitag | Colin Cherry | Subhajit Naskar | Mara Finkelstein | Xavier Garcia | Daniel Cremers |

L-Eval: Instituting Standardized Evaluation for Long Context Language Models

Chenxin An | Shansan Gong | Ming Zhong | Xingjian Zhao | Mukai Li | Jun Zhang | Lingpeng Kong | Xipeng Qiu |

∞Bench: Extending Long Context Evaluation Beyond 100K Tokens

Xinrong Zhang | Yingfa Chen | Shengding Hu | Zihang Xu | Junhao Chen | Moo Hao | Xu Han | Zhen Thai | Shuo Wang | Zhiyuan Liu | Maosong Sun |