Perplexity can be used to evaluate the performance of language models.

To convert from loss (-log probability, aka cross entropy) to perplexity exponentiate the loss (perplexity = exp(loss)), and vice-versa (loss = ln(perplexity)). When the loss is very low (< .1) then the perplexity is roughly 1 + loss.

• Evaluation
• Language Model