Natural Language Generation

• Introduction: Eisenstein p. 457
• Gatt et al 2017 - Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation Now outdated. Goes over traditional non-neural methods.
• Yu et al 2020 - A Survey of Knowledge-Enhanced Text Generation

Data-to-text generation is generation where the input is formatted data such as tables of numbers. A typical example is generating human-readable weather reports from numbers predicted from weather simulations. An example dataset is RotoWire (paper).

• Reiter 2007 - An Architecture for Data-to-Text Systems
• Belz 2008 - Automatic Generation of Weather Forecast Texts Using Comprehensive Probabilistic Generation-Space Models
• Puduppully et al 2018 - Data-to-Text Generation with Content Selection and Planning A good baseline, used as baseline in Workshop in NLG and Translation
• Zhang et al 2022 - TempLM: Distilling Language Models into Template-Based Generators

This is generation from a meaning representation, such as AMR or slots and values - the inverse of semantic parsing. See also AMR - Generation.

• Overviews
  • Controllable Neural Text Generation

Survey: 2020 - Evaluation of Text Generation: A Survey

• Belz & Reiter 2006 - Comparing Automatic and Human Evaluation of NLG systems
• Hashimoto et al 2019 - Unifying Human and Statistical Evaluation for Natural Language Generation
• Dusek et al 2019 - Automatic Quality Estimation for Natural Language Generation:Ranting (Jointly Rating and Ranking)
• Sallam et al 2020 - BLEURT: Learning Robust Metrics for Text Generation (ACL 2020)
• Gehrmann et al 2021 - The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics (Website)
  • Freitag et al 2021 - Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation Uses Multidimensional Quality Metrics (MQM) framework. MT paper, used in WMT
• Dou et al 2021 - Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text
• Ruan et al 2024 - Defining and Detecting Vulnerability in Human Evaluation Guidelines: A Preliminary Study Towards Reliable NLG Evaluation

Papers from a while ago.

• Bohnet et al 2010 - Broad Coverage Multilingual Deep Sentence Generation with a Stochastic Multi-Level Realizer

See also NLP progress - Generation and ACL Wiki - Datasets for NLG

• AMR dataset
• E2E dataset: github or here (paper)
• MultiWOZ Data to text generation datset
• Rotowire
• WikiBio
• WebNLG
• ViGGO Dataset, created a UCSC (paper)

• Hayashi et al 2019 - Findings of the Third Workshop on Neural Generation and Translation

• Anya Belz
• Ehud Reiter

• Sequence to Sequence Models
• Watermarking