Greedy Decoding

Prompt: "The best programming language is" GREEDY DECODING (temperature=0): Step 1: [Python: 0.35, JavaScript: 0.30, ...] → "Python" Step 2: [because: 0.40, for: 0.25, ...] → "because" Step 3: [it: 0.45, of: 0.30, ...] → "it" ... Output: "Python because it is versatile and easy to learn." (Always this exact output for this prompt) SAMPLING (temperature=0.7): Run 1: "JavaScript for web development and..." Run 2: "Python because of its simplicity..." Run 3: "depends on the use case..." (Different each time!) ─────────────────────────────────────────── Greedy can get stuck in local optima: "The capital of France is" → "Paris" ✓ (good) "Write a creative story" → repetitive, boring (bad) Greedy works well for factual, single-answer questions but struggles with creative or open-ended tasks.

# OpenAI - set temperature to 0 response = client.chat.completions.create( model="gpt-4", messages=[{"role": "user", "content": "What is 2+2?"}], temperature=0 # Greedy decoding ) # Claude - temperature 0 response = anthropic.messages.create( model="claude-3-opus", max_tokens=100, temperature=0, # Greedy decoding messages=[...] ) # HuggingFace - do_sample=False from transformers import pipeline generator = pipeline("text-generation") output = generator( "The answer is", do_sample=False, # Greedy decoding max_length=50 ) # When to use greedy: # ✓ Math problems (one correct answer) # ✓ Factual questions # ✓ Code that must be syntactically correct # ✓ When reproducibility is required # ✗ Creative writing # ✗ Brainstorming # ✗ When diversity is needed