NLPExplorer

BabyLM - 2024

Total Papers:- 30

Total Papers accross all years:- 30

Total Citations :- 0

1 2 3 »

Teaching Tiny Minds: Exploring Methods to Enhance Knowledge Distillation for Small Language Models

Hong Meng Yam | Nathan Paek |

ConcreteGPT: A Baby GPT-2 Based on Lexical Concreteness and Curriculum Learning

Luca Capone | Alessandro Bondielli | Alessandro Lenci |

A surprisal oracle for when every layer counts

Xudong Hong | Sharid Loáiciga | Asad Sayeed |

Exploring Curriculum Learning for Vision-Language Tasks: A Study on Small-Scale Multimodal Training

Rohan Saha | Abrar Fahim | Alona Fyshe | Alex Murphy |

Extending the BabyLM Initiative : Promoting Diversity in Datasets and Metrics through High-Quality Linguistic Corpora

Laurent Prévot | Sheng-Fu Wang | Jou-An Chi | Shu-Kai Hsieh |

Developmentally Plausible Multimodal Language Models Are Highly Modular

Alina Klerings | Christian Bartelt | Aaron Mueller |

ELC-ParserBERT: Low-Resource Language Modeling Utilizing a Parser Network With ELC-BERT

Rufus Behr |

Choosy Babies Need One Coach: Inducing Mode-Seeking Behavior in BabyLlama with Reverse KL Divergence

Shaozhen Shi | Yevgen Matusevych | Malvina Nissim |

Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Graphemes vs. phonemes: battling it out in character-based language models

Bastian Bunzeck | Daniel Duran | Leonie Schade | Sina Zarrieß |

Using Curriculum Masking Based on Child Language Development to Train a Large Language Model with Limited Training Data

Different Ways to Forget: Linguistic Gates in Recurrent Neural Networks

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data

Are BabyLMs Second Language Learners?

Lukas Edman | Lisa Bylinina | Faeze Ghorbanpour | Alexander Fraser |