Advanced Reasoning Strategies / Multi-Path Reasoning

Graph of Thought

Expert [5/5]
GoT Graph-of-Thoughts Non-linear reasoning

Definition

Graph of Thought (GoT) extends Tree of Thought by allowing thoughts to form a directed graph rather than a tree. This enables merging of reasoning paths, cycles for iterative refinement, and more complex reasoning topologies.

GoT better models human reasoning patterns where multiple lines of thought can converge on insights or loop back to refine earlier conclusions.

Key Concepts

  • Thought aggregation: Merging multiple reasoning paths into one
  • Refinement loops: Iteratively improving thoughts through cycles
  • Graph operations: Generate, aggregate, refine, and score thoughts
  • Flexible topology: Arbitrary connections between thought nodes

Examples

GoT vs ToT
Structural Comparison
Tree of Thought (ToT): A /|\ B C D (branches only diverge) /| | E F G Graph of Thought (GoT): A /|\ B C D /|\ /| E F─┴─G (paths can merge!) \ / H (aggregated insight) │ H' (refined thought - cycle)
Use Case
Document Summarization with GoT
Task: Summarize a long document Graph Operations: 1. GENERATE: Create 4 parallel summaries of different sections [Section1] [Section2] [Section3] [Section4] 2. AGGREGATE: Merge pairs of summaries [Section1+2] [Section3+4] 3. AGGREGATE: Merge into single summary [Full Summary Draft] 4. REFINE: Loop back to improve coherence [Full Summary Draft] → [Refined Summary] 5. SCORE: Evaluate quality Score: 8.5/10 → Accept Graph Structure: S1 S2 S3 S4 (generate) \ / \ / S12 S34 (aggregate) \ / Summary (aggregate) ↓↺ Refined (refine loop) ↓ Final (output)

Interactive Exercise

✎
Design a GoT Flow

Design a Graph of Thought for this task:

Task: "Compare three products and write a recommendation"

Use: Generate, Aggregate, Refine, and Score operations

Pro Tips
  • Use aggregation when combining partial results
  • Add refinement loops for quality-critical outputs
  • GoT shines for tasks with natural decomposition
  • Monitor for infinite loops in refinement cycles

Related Terms

Tree of Thought Chain of Thought Self-Refine Decomposed Prompting