SimToM | HyperKit.ai

Definition

SimToM (Simulated Theory of Mind) is a two-stage prompting technique for tasks requiring understanding what someone else knows or believes. First, filter context to only what a specific person could know, then answer from their perspective.

This addresses LLMs' tendency to use omniscient knowledge when answering questions about others' beliefs or knowledge states.

Key Concepts

Theory of Mind: Understanding others have different knowledge/beliefs
Perspective filtering: What could this person actually know?
Knowledge isolation: Separating observer vs participant knowledge
Two-stage reasoning: Filter first, then answer from perspective

Examples

Problem

The Theory of Mind Challenge

CLASSIC THEORY OF MIND PROBLEM:

Story: "Sally puts a marble in the basket and leaves.
While she's gone, Anne moves the marble to the box.
Sally comes back."

Question: "Where will Sally look for the marble?"

INCORRECT (no ToM):
"Sally will look in the box because that's where
the marble is."
❌ Uses omniscient knowledge

CORRECT (with ToM):
"Sally will look in the basket because that's
where she left it. She doesn't know Anne moved it."
✓ Uses Sally's perspective

WHY LLMS STRUGGLE:

The model "knows" the marble is in the box.
It must ignore this knowledge and reason about
what SALLY knows, not what is objectively true.

Human children develop this ability around age 4.
LLMs often fail because they don't naturally
separate "what I know" from "what they know".

Implementation

SimToM Two-Stage Process

SIMTOM IMPLEMENTATION:

STAGE 1 - PERSPECTIVE FILTERING:

prompt_stage1 = """
Story: {story}

Target person: {person}

List ONLY the events/facts that {person} directly
witnessed or was told about. Exclude anything that
happened when they weren't present.

What {person} knows:"""

STAGE 2 - ANSWER FROM PERSPECTIVE:

prompt_stage2 = """
Based ONLY on what {person} knows:
{filtered_knowledge}

Question: {question}

Answer from {person}'s perspective, using only
the knowledge listed above:"""

---

EXAMPLE APPLICATION:

Story: "Alice tells Bob she's planning a surprise
party for Carol on Saturday. Carol overhears Alice
on the phone saying 'everything is set for Saturday.'
Carol doesn't know it's a party for her."

Question: "Does Carol know about the surprise party?"

STAGE 1 - What Carol knows:
- Carol heard Alice say "everything is set for Saturday"
- Carol does NOT know what "everything" refers to
- Carol was NOT present when Alice told Bob about party

STAGE 2 - Answer from Carol's perspective:
"No, Carol does not know about the surprise party.
She only knows something is happening Saturday,
but she doesn't know it's a party or that it's for
her. From her perspective, 'everything is set'
could refer to anything."

---

PERFORMANCE:
┌─────────────────────┬──────────┬─────────────┐
│ ToM Task            │ Direct   │ + SimToM    │
├─────────────────────┼──────────┼─────────────┤
│ False belief tasks  │ 58%      │ 87% (+29%)  │
│ Perspective-taking  │ 62%      │ 83% (+21%)  │
│ Knowledge tracking  │ 55%      │ 79% (+24%)  │
└─────────────────────┴──────────┴─────────────┘

Dramatic improvements on ToM reasoning!

Interactive Exercise

✎

Apply SimToM

Story: "John buys a gift and hides it in the closet. Mary sees him hide it. Later, while both are out, their son Tim finds it and moves it to the garage. Neither John nor Mary knows Tim moved it."

Apply SimToM for the question: "Where does Mary think the gift is?"

Pro Tips

Stage 1 (filtering) is crucial - be explicit about what person witnessed
Watch for information that person "couldn't have known"
Useful for customer service (what does customer know?)
Helps with debugging user issues (user's vs developer's knowledge)

Definition

Key Concepts

Examples

Interactive Exercise

Related Terms