====== Structured Prediction ======

===== Overviews =====
  * Jan 12 of [[https://classes.soe.ucsc.edu/nlp202/Winter21/|NLP 202]] ([[https://classes.soe.ucsc.edu/nlp202/Winter21/slides/loss-functions.pdf|slides]]) and the readings for that lecture
  * Section 3.1-3.3 of Jeff's [[https://jflanigan.github.io/flanigan-thesis.pdf|thesis]] gives an very brief overview of loss functions for structured prediction

===== Key Papers =====
  * Structured Perceptron: [[https://www.aclweb.org/anthology/W02-1001.pdf|Collins 2002 - Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms]]
  * Structured SVM

===== Recent Papers =====
Recent papers: [[https://paperswithcode.com/task/structured-prediction]]
  * [[https://arxiv.org/pdf/1603.06042.pdf|Andor et al 2016 - Globally Normalized Transition-Based Neural Networks]]  Talks about the [[label bias problem]].
  * [[https://arxiv.org/pdf/1711.04956v5.pdf|Edunov et al 2017 - Classical Structured Prediction Losses for Sequence to Sequence Learning]]  Uses a sample (i.e. beam) size of 16 for the MT experiments.  See section 6.5 and Table 2.
  *  [[https://arxiv.org/pdf/1807.01745.pdf|Gómez-Rodríguez et al 2018 - Global Transition-based Non-projective Dependency Parsing]]
  * [[https://www.aclweb.org/anthology/P18-1173.pdf|Peng et al 2018 - Backpropagating through Structured Argmax using a SPIGOT]] (see also [[ml:Optimization in Deep Learning#Backpropagating Through Discontinuities]])
  * [[https://arxiv.org/pdf/1906.07880.pdf|Wang et al 2018 - Second-Order Semantic Dependency Parsing with End-to-End Neural Networks]]
  * [[https://www.aclweb.org/anthology/K18-1001.pdf|Thai et al 2018 - Embedded-State Latent Conditional Random Fields for Sequence Labeling]]
  * [[https://par.nsf.gov/servlets/purl/10145797|Ma et al 2019 - Randomized Greedy Search for Structured Prediction: Amortized Inference and Learning]]
  * **[[http://proceedings.mlr.press/v97/collobert19a/collobert19a.pdf|Collobert 2019 - A Fully Differentiable Beam Search Decoder]]**
  * [[https://www.aclweb.org/anthology/N19-1171.pdf|Goyal et al 2019 - An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search]]
  * [[https://www.aclweb.org/anthology/N19-1335.pdf|Tu & Gimple 2019 - Benchmarking Approximate Inference Methods for Neural Structured Prediction]]
  * [[https://www.aclweb.org/anthology/D19-1099.pdf|2019 - Semantic Role Labeling with Iterative Structure Refinement]] Good related work section
  * [[https://arxiv.org/pdf/2005.00975.pdf|Zhang et al 2020 - Efficient Second-Order TreeCRF for Neural Dependency Parsing]]
  * [[https://www.aclweb.org/anthology/2020.acl-demos.38.pdf|Rush 2020 - Torch-Struct: Deep Structured Prediction Library]]
  * [[https://www.aclweb.org/anthology/2020.emnlp-main.406.pdf|Gao & Gormley 2020 - Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors]]
  * [[https://www.aclweb.org/anthology/2020.acl-main.776.pdf|Fonseca & Martins 2020 - Revisiting Higher-Order Dependency Parsers]]
  * [[https://arxiv.org/pdf/2101.10435.pdf|Widmoser 2021 - Randomized Deep Structured Prediction for Discourse-Level Processing]] "Our experiments show that in all cases, deep structured prediction outperforms traditional shallow approaches, structured learning outperforms inference over locally trained models, and generic randomized inference performs competitively to exact inference."

==== Conferences and Workshops ====
  * [[https://www.aclweb.org/anthology/2020.spnlp-1.0.pdf|4th Workshop on Structured Prediction for NLP]]

==== Courses and Tutorials ====
  * Structured Prediction for Language and Other Discrete Data (SPFLODD) course at CMU: [[http://demo.clab.cs.cmu.edu/fa2015-11763/|2015]] [[https://sites.google.com/site/spflodd/|2014]]

===== Related Pages =====
  * [[Structured Prediction Energy Networks]]
  * [[Label Bias Problem]]