Mitigating Frequency Bias and Anisotropy in Language Model Pre-Training with Syntactic Smoothing EMNLP

Richard Diehl Martinez | Zebulon Goriely | Andrew Caines | Paula Buttery | Lisa Beinborn |

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes CoNLL BabyLM WS

Zébulon Goriely | Richard Diehl Martinez | Andrew Caines | Paula Buttery | Lisa Beinborn |

Less is More: Pre-Training Cross-Lingual Small-Scale Language Models with Cognitively-Plausible Curriculum Learning Strategies CoNLL BabyLM WS

Suchir Salhan | Richard Diehl Martinez | Zébulon Goriely | Paula Buttery |

CLIMB – Curriculum Learning for Infant-inspired Model Building CoNLL

Richard Diehl Martinez | Hope McGovern | Zebulon Goriely | Christopher Davis | Andrew Caines | Paula Buttery | Lisa Beinborn |