Foundation Concepts / Basic Interaction

Conversation History

Essential [1/5]
Message history Context log Dialogue record

Definition

Conversation history is the record of previous messages exchanged between a user and an LLM within a session. This history provides context that allows the model to maintain coherent, contextually aware conversations across multiple turns.

Unlike humans, LLMs don't have persistent memory between API calls—the conversation history must be explicitly passed with each request to maintain continuity.

Key Concepts

  • Message roles: Typically "system", "user", and "assistant" to distinguish between instruction, input, and response
  • Context window limits: History must fit within the model's token limit, requiring truncation strategies for long conversations
  • Stateless architecture: Each API call is independent; history must be resent every time
  • Reference resolution: Enables the model to understand pronouns and references to earlier messages

Examples

API Format
Message Array Structure
messages = [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "What's the capital of Japan?"}, {"role": "assistant", "content": "The capital of Japan is Tokyo."}, {"role": "user", "content": "What's its population?"} ] # The model knows "its" refers to Tokyo because of history
The conversation history allows the model to resolve "its" to Tokyo from the previous exchange.
History Management
Truncation Strategy
# When history exceeds token limit: 1. Always keep system message 2. Keep most recent N messages 3. Optionally summarize older messages 4. Remove middle messages if needed # Priority: Recent context > Summarized old > Full old history
Strategies for managing conversation history when approaching context limits.

Interactive Exercise

✎
Identify the Reference

Given this conversation history, what does "it" refer to in the last message?

User: "I'm learning Python."
Assistant: "Great choice! Python is beginner-friendly."
User: "How long will it take to learn it?"

Pro Tips
  • Implement sliding window to keep recent messages when history grows large
  • Consider summarizing older context to preserve important information
  • Track token count to avoid unexpected truncation by the API
  • Store conversation history server-side for multi-session continuity

Related Terms

Context Window Turn Token System Prompt