Table of Contents

Data Preparation

Creating a Train/Dev/Test Split

Generally, you'll want use an existing train/dev/test split if it exists for that dataset so you can compare to previous methods. If the dataset doesn't have a split, it may not be a standard NLP dataset and it may be better to use a different dataset that is more widely used in experiments. If you need to create your train/dev/test split, here are some things to consider:

Papers

Tokenization

See Tokenization.