https://jlab.soe.ucsc.edu/nlp-wiki/ 2026-05-11T09:20:35+00:00 https://jlab.soe.ucsc.edu/nlp-wiki/ https://jlab.soe.ucsc.edu/nlp-wiki/lib/tpl/dokuwiki/images/favicon.ico text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:a_tutorial_on_deep_latent_variable_models_of_natural_language&rev=1686814574&do=diff A Tutorial on Deep Latent Variable Models of Natural Language Kim et al 2019 - A Tutorial on Deep Latent Variable Models of Natural Language text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:discriminative_training_methods_for_hidden_markov_models_-_theory_and_experiments_with_perceptron_algorithms&rev=1686814574&do=diff Discriminative Training Methods for Hidden Markov Models - Theory and Experiments with Perceptron Algorithms Discriminative Training Methods for Hidden Markov Models - Theory and Experiments with Perceptron Algorithms text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:dual_learning_for_machine_translation&rev=1686814574&do=diff Dual Learning For Machine Translation Dual Learning For Machine Translation TLDR; The authors cast machine translation as a game of a primal task and a dual task, which allows two agents to teach each other in a reinforcement learning fashion (i.e., policy gradient method). With this implementation, the agent only requires monolingual English text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:etc&rev=1686814574&do=diff ETC: Encoding Long and Structured Inputs in Transformers ETC: Encoding Long and Structured Inputs in Transformers TLDR; ETC encodes long inputs using global-local attention and represents structures by combining relative position representations and flexible masking. It also employs CPC pre-training for hierarchical global tokens (structures). text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:experience_grounds_language&rev=1686814574&do=diff Experience Grounds Language Experience Grounds Language TLDR; Current models trained only from text corpora might hinder the language understanding research because the supervision from text alone can be limited, so we need to embed social context. The authors proposed the notion of World Scopes (WS), which considers not only the text corpus but also grounding, embodiment, and social interaction. text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:language_models_are_few-shot_learners&rev=1686814574&do=diff Language Models are Few-Shot Learners Language Models are Few-Shot Learners text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:language_models_are_unsupervised_multitask_learners&rev=1686814574&do=diff Language Models are Unsupervised Multitask Learners Language Models are Unsupervised Multitask Learners * Code: <https://github.com/openai/gpt-2> * Open-source clone of the dataset: OpenWebText (from <https://github.com/NVIDIA/Megatron-LM>) text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:megatron-lm_training_multi-billion_parameter_language_models_using_model_parallelism&rev=1686814574&do=diff Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism * Follow-up blog post: State-of-the-Art Language Modeling Using Megatron on the NVIDIA A100 GPU text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:mikolov_2013_-_distributed_representations_of_words_and_phrases_and_their_compositionality&rev=1686814574&do=diff Distributed Representations of Words and Phrases and their Compositionality Distributed Representations of Words and Phrases and their Compositionality text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:mikolov_2013_-_efficient_estimation_of_word_representations_in_vector_space&rev=1686814574&do=diff Efficient Estimation of Word Representations in Vector Space Efficient Estimation of Word Representations in Vector Space Follow-up work: Mikolov 2013 - Distributed Representations of Words and Phrases and their Compositionality text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:schwartz_2018_-_sopa_bridging_cnns_rnns_and_weighted_finite-state_machines&rev=1686814574&do=diff SoPa: Bridging CNNs, RNNs, and Weighted Finite-State Machines SoPa: Bridging CNNs, RNNs, and Weighted Finite-State Machines text/html 2023-06-15T07:36:14+00:00 Anonymous (anonymous@undisclosed.example.com) https://jlab.soe.ucsc.edu/nlp-wiki/doku.php?id=paper:the_sample_complexity_of_agnostic_learning_under_deterministic_labels&rev=1686814574&do=diff Paper: The sample complexity of agnostic learning under deterministic labels The sample complexity of agnostic learning under deterministic labels