Reproduction of Human Evaluations in: “It’s not Rocket Science: Interpreting Figurative Language in Narratives” HumEval WS

Saad Mahamood |

evision PDF of 'Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP insights WS

Anya Belz | Craig Thomson | Ehud Reiter | Gavin Abercrombie | Jose M. Alonso-Moral | Mohammad Arvan | Anouck Braggaar | Mark Cieliebak | Elizabeth Clark | Kees van Deemter | Tanvi Dinkar | Ondřej Dušek | Steffen Eger | Qixiang Fang | Mingqi Gao | Albert Gatt | Dimitra Gkatzia | Javier González-Corbelle | Dirk Hovy | Manuela Hürlimann | Takumi Ito | John D. Kelleher | Filip Klubicka | Emiel Krahmer | Huiyuan Lai | Chris van der Lee | Yiru Li | Saad Mahamood | Margot Mieskes | Emiel van Miltenburg | Pablo Mosteiro | Malvina Nissim | Natalie Parde | Ondřej Plátek | Verena Rieser | Jie Ruan | Joel Tetreault | Antonio Toral | Xiaojun Wan | Leo Wanner | Lewis Watson | Diyi Yang |

A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization ACL

Lining Zhang | Simon Mille | Yufang Hou | Daniel Deutsch | Elizabeth Clark | Yixin Liu | Saad Mahamood | Sebastian Gehrmann | Miruna Clinciu | Khyathi Raghavi Chandu | João Sedoc |

It’s Commonsense, isn’t it? Demystifying Human Evaluations in Commonsense-Enhanced NLG Systems EACL HumEval

Miruna-Adriana Clinciu | Dimitra Gkatzia | Saad Mahamood |

Reproducing a Comparison of Hedged and Non-hedged NLG Texts INLG

Saad Mahamood |

The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics ACL GEM IJCNLP

Sebastian Gehrmann | Tosin Adewumi | Karmanya Aggarwal | Pawan Sasanka Ammanamanchi | Anuoluwapo Aremu | Antoine Bosselut | Khyathi Raghavi Chandu | Miruna-Adriana Clinciu | Dipanjan Das | Kaustubh Dhole | Wanyu Du | Esin Durmus | Ondřej Dušek | Chris Chinenye Emezue | Varun Gangal | Cristina Garbacea | Tatsunori Hashimoto | Yufang Hou | Yacine Jernite | Harsh Jhamtani | Yangfeng Ji | Shailza Jolly | Mihir Kale | Dhruv Kumar | Faisal Ladhak | Aman Madaan | Mounica Maddela | Khyati Mahajan | Saad Mahamood | Bodhisattwa Prasad Majumder | Pedro Henrique Martins | Angelina McMillan-Major | Simon Mille | Emiel van Miltenburg | Moin Nadeem | Shashi Narayan | Vitaly Nikolaev | Andre Niyongabo Rubungo | Salomey Osei | Ankur Parikh | Laura Perez-Beltrachini | Niranjan Ramesh Rao | Vikas Raunak | Juan Diego Rodriguez | Sashank Santhanam | João Sedoc | Thibault Sellam | Samira Shaikh | Anastasia Shimorina | Marco Antonio Sobrevilla Cabezudo | Hendrik Strobelt | Nishant Subramani | Wei Xu | Diyi Yang | Akhila Yerukola | Jiawei Zhou |

Underreporting of errors in NLG output, and what to do about it INLG

Emiel van Miltenburg | Miruna Clinciu | Ondřej Dušek | Dimitra Gkatzia | Stephanie Inglis | Leo Leppänen | Saad Mahamood | Emma Manning | Stephanie Schoch | Craig Thomson | Luou Wen |

Twenty Years of Confusion in Human Evaluation: NLG Needs Evaluation Sheets and Standardised Definitions INLG

David M. Howcroft | Anya Belz | Miruna-Adriana Clinciu | Dimitra Gkatzia | Sadid A. Hasan | Saad Mahamood | Simon Mille | Emiel van Miltenburg | Sashank Santhanam | Verena Rieser |

Hotel Scribe: Generating High Variation Hotel Descriptions INLG WS

Saad Mahamood | Maciej Zembrzuski |

Explainable Artificial Intelligence and its potential within Industry INLG WS

Saad Mahamood |

A Snapshot of NLG Evaluation Practices 2005 - 2014 ENLG WS

Dimitra Gkatzia | Saad Mahamood |

Generating Annotated Graphs using the NLG Pipeline Architecture INLG WS

Saad Mahamood | William Bradshaw | Ehud Reiter |

Working with Clinicians to Improve a Patient-Information NLG System INLG WS

Saad Mahamood | Ehud Reiter |

Generating Affective Natural Language for Parents of Neonatal Infants ENLG WS

Saad Mahamood | Ehud Reiter |

A Comparison of Hedged and Non-hedged NLG Texts ENLG WS

Saad Mahamood | Ehud Reiter | Chris Mellish |