Language Identification

• Jauhiainen et al 2018 - Automatic Language Identification in Texts: A Survey

• Lui & Baldwin 2012 - langid.py: An Off-the-shelf Language Identification Tool
• Palakodety et al 2020- Hope Speech Detection: A Computational Analysis of the Voice of Peace Clustering based on polyglot word embeddings is an easy method for unsupervised language detection (see section 5.1).
• Palakodety & KhudaBukhsh 2020 - Annotation Efficient Language Identification from Weak Labels

Comparison here.

• FastText Language ID
• GoogleLangID
• langid.py paper
• langdetect
• spaCy langdetect

• Code Switching
• Data Preparation