• Introductions to BERT
  • Paper: Devlin et al 2018 - BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
  • Blogs
    • Demystifying BERT: A Comprehensive Guide to the Groundbreaking NLP Framework
    • The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)

• Wang & Cho 2019 - BERT has a Mouth, and It Must Speak:BERT as a Markov Random Field Language Model WARNING: Mistake in this paper, it's not an MRF.

Summary: Rogers et al 2020 - A Primer in BERTology: What we know about how BERT works. See also Neural Network Psychology.

• Wiedemann et al 2019 - Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddings
• Tenney et al 2019 - What do you learn from context? Probing for sentence structure in contextualized word representations
• Tenney et al 2019 - BERT Rediscovers the Classical NLP Pipeline
• Rogers et al 2020 - A Primer in BERTology: What we know about how BERT works

• Sun et al 2019 - How to Fine-Tune BERT for Text Classification? Exhaustive study investigating different fine-tuning methods of BERT on text classification tasks and provides a general strategy for BERT fine-tuning.
• Shi & Lin 2019 - Simple BERT Models for Relation Extraction and Semantic Role Labeling
• Liu 2019 - Fine-tune BERT for Extractive Summarization
• Zhang et al 2019 - BERTScore: Evaluating Text Generation with BERT

• Mulyar et al 2020 - MT-Clinical BERT: Scaling Clinical Information Extraction with Multitask Learning

• Shen et al 2019 - Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT

• Pretraining
• Transformers