A Pretrainer’s Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity NAACL

Shayne Longpre | Gregory Yauney | Emily Reif | Katherine Lee | Adam Roberts | Barret Zoph | Denny Zhou | Jason Wei | Kevin Robinson | David Mimno | Daphne Ippolito |

Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System INLG SIGDIAL

Daphne Ippolito | Nicholas Carlini | Katherine Lee | Milad Nasr | Yun William Yu |

Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy INLG SIGDIAL

Daphne Ippolito | Florian Tramer | Milad Nasr | Chiyuan Zhang | Matthew Jagielski | Katherine Lee | Christopher Choquette Choo | Nicholas Carlini |

Deduplicating Training Data Makes Language Models Better ACL

Katherine Lee | Daphne Ippolito | Andrew Nystrom | Chiyuan Zhang | Douglas Eck | Chris Callison-Burch | Nicholas Carlini |

Analysis and Detection of Reading Miscues for Interactive Literacy Tutors COLING

Katherine Lee | Andreas Hagen | Nicholas Romanyshyn | Sean Martin | Bryan Pellom |