Structured Prediction

• Jan 12 of NLP 202 (slides) and the readings for that lecture
• Section 3.1-3.3 of Jeff's thesis gives an very brief overview of loss functions for structured prediction

• Structured Perceptron: Collins 2002 - Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms
• Structured SVM

Recent papers: https://paperswithcode.com/task/structured-prediction

• Andor et al 2016 - Globally Normalized Transition-Based Neural Networks Talks about the label bias problem.
• Edunov et al 2017 - Classical Structured Prediction Losses for Sequence to Sequence Learning Uses a sample (i.e. beam) size of 16 for the MT experiments. See section 6.5 and Table 2.
• Gómez-Rodríguez et al 2018 - Global Transition-based Non-projective Dependency Parsing
• Peng et al 2018 - Backpropagating through Structured Argmax using a SPIGOT (see also Backpropagating Through Discontinuities)
• Wang et al 2018 - Second-Order Semantic Dependency Parsing with End-to-End Neural Networks
• Thai et al 2018 - Embedded-State Latent Conditional Random Fields for Sequence Labeling
• Ma et al 2019 - Randomized Greedy Search for Structured Prediction: Amortized Inference and Learning
• Collobert 2019 - A Fully Differentiable Beam Search Decoder
• Goyal et al 2019 - An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search
• Tu & Gimple 2019 - Benchmarking Approximate Inference Methods for Neural Structured Prediction
• 2019 - Semantic Role Labeling with Iterative Structure Refinement Good related work section
• Zhang et al 2020 - Efficient Second-Order TreeCRF for Neural Dependency Parsing
• Rush 2020 - Torch-Struct: Deep Structured Prediction Library
• Gao & Gormley 2020 - Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors
• Fonseca & Martins 2020 - Revisiting Higher-Order Dependency Parsers
• Widmoser 2021 - Randomized Deep Structured Prediction for Discourse-Level Processing “Our experiments show that in all cases, deep structured prediction outperforms traditional shallow approaches, structured learning outperforms inference over locally trained models, and generic randomized inference performs competitively to exact inference.”

• 4th Workshop on Structured Prediction for NLP

• Structured Prediction for Language and Other Discrete Data (SPFLODD) course at CMU: 2015 2014

• Structured Prediction Energy Networks
• Label Bias Problem