Scaling laws are used to pick optimal hyperparameters for large models.

• Kaplan et al 2020 - Scaling Laws for Neural Language Models
• Hoffmann et al 2022 - Training Compute-Optimal Large Language Models
• Tay et al 2022 - Scaling Laws vs Model Architectures: How Does Inductive Bias Influence Scaling?

• Large models are usually trained with scaling laws in mind (often compute optimal for deployment, not training). See for example:
  • Google 2023 - PaLM 2 Technical Report (see section 2)

• Schaeffer et al 2023 - Are Emergent Abilities of Large Language Models a Mirage?
• Hu et al 2023 - Predicting Emergent Abilities with Infinite Resolution Evaluation

• Hyperparameter Tuning
• Language Model
• Pretraining Methodology