Prompt Engineering Fundamentals / Prompt Optimization

Example Selection

Intermediate [3/5]
Demonstration curation Example choice

Definition

Example selection is the process of choosing diverse, high-quality demonstrations for few-shot learning. The examples you choose significantly impact model performance—poor examples can mislead the model, while well-chosen examples dramatically improve output quality.

Effective example selection considers diversity, representativeness, clarity, and relevance to the target task.

Key Concepts

  • Diversity: Cover different cases, edge cases, and variations
  • Quality: Each example should be correct and well-formatted
  • Representativeness: Examples should reflect the distribution of real inputs
  • Relevance: Select examples similar to the target task
  • Order: Example ordering can affect model behavior

Examples

Poor Selection
Biased Examples
Task: Sentiment classification Examples (all positive): "Great product!" → Positive "Love it!" → Positive "Amazing quality!" → Positive Problem: Model may predict everything as Positive
Unbalanced examples bias the model toward one class.
Good Selection
Diverse, Balanced Examples
Task: Sentiment classification Examples (balanced): "Great product, exceeded expectations!" → Positive "Terrible quality, broke after one day" → Negative "It's okay, nothing special" → Neutral "Love the design but shipping was slow" → Mixed Coverage: Different sentiments, varying lengths, edge cases
Balanced selection teaches the model to handle various cases.

Interactive Exercise

✎
Choose Better Examples

For a task that categorizes emails as "Urgent" or "Normal", which set of examples is better?

Set A: All examples of urgent emails

Set B: Mix of urgent and normal, including edge cases

Explain why and what examples you'd include.

Pro Tips
  • Include at least one example per category or output type
  • Add edge cases that commonly cause confusion
  • Use dynamic example selection based on input similarity
  • Test different example orderings—recency bias is real

Related Terms

Few-Shot Prompting One-Shot Prompting In-Context Learning Semantic Search