Prompt Engineering

• Guides
  • A Beginner's Guide to ChatGPT Prompt Engineering Good concise intro (Accessed March 2025)
  • The Prompt Engineering Cheat Sheet Great
  • Prompt Engineering Guide This one is pretty good
  • Google Cloud - Prompt engineering: overview and guide
• Overview Papers
  • Leidinger et al 2023 - The language of prompting: What linguistic properties make a prompt successful?
• Slides
• Blogs:
  • Lil'log Prompt Engineering
  • Github: BREX's Prompt Engineering Guide
  • Github: DAIR AI's Prompt Engineering Guide
  • Courses
    • learnprompting.org
• Awesome ChatGPT Prompts

Note: some of these papers should be moved to automatic prompt engineering.

• Mishra et al 2022 - Reframing Instructional Prompts to GPTk’s Language
• Sorensen et al 2022 - An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels
• Webson & Pavlick 2021 - Do Prompt-Based Models Really Understand the Meaning of Their Prompts?
• Wang et al 2022 - Self-Instruct: Aligning Language Model with Self Generated Instructions
• Webson et al 2023 - Are Language Models Worse than Humans at Following Prompts? It's Complicated
• Cheng et al 2023 - Batch Prompting: Efficient Inference with Large Language Model APIs Batch prompting to save money
• Yin et al 2023 - Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learning
• Chen et al 2023 - Exploring Lottery Prompts for Pre-trained Language Models Searches to see if there is a prompt for each example that produces the correct answer
• Li et al 2023 - Do you really follow me? Adversarial Instructions for Evaluating the Robustness of Large Language Models
• Wang & Zhao 2023 - Metacognitive Prompting Improves Understanding in Large Language Models
• Zhang et al 2025 - Why Prompt Design Matters and Works: A Complexity Analysis of Prompt Search Space in LLMs Shows that naive CoT prompting, like “think step by step,“can severely hinder performance.
• What makes a good prompt?
  • Leidinger et al 2023 - The language of prompting: What linguistic properties make a prompt successful?
  • Gonen et al 2023 - Demystifying Prompts in Language Models via Perplexity Estimation

• Shin et al 2020 - AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts
• Deng et al 2022 - RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
• Wen et al 2023 - Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery
• Pryzant et al 2023 - Automatic Prompt Optimization with "Gradient Descent" and Beam Search
• Yang et al 2023 - Large Language Models as Optimizers Figures out a good prompt for the task
• Ferdando et al 2023 - Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
• Metaprompting
  • Blog posts, etc
    • A Complete Guide to Meta Prompting
    • Meta-prompting
    • Meta-Prompting Concept: Asking Chat-GPT for the best prompt for your desired completion, then to revise it before using it
  • 2022 - MetaPrompting: Learning to Learn Better Prompts
  • 2024 - Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding

Examples of clever or interesting prompts.

• See appendix of https://arxiv.org/pdf/2307.16789.pdf
• Yasunaga et al 2023 - Large Language Models as Analogical Reasoners
• Arora et al 2022 - Ask Me Anything: A Simple Strategy for Prompting Language Models
• Awesome ChatGPT Prompts
• Prompt to break down sentences into independent facts:

(from Min 2023)

• Xie et al 2024 - OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments See the prompts in appendix C

• Prompting