Model Internals / Representations

Latent Space

Advanced [4/5]
Embedding space Representation space Feature space

Definition

Latent space is the high-dimensional vector space where neural networks represent data in a compressed, meaningful form. Each point in latent space represents a possible input, and the structure of this space captures semantic relationships learned during training.

Understanding latent space is key to understanding how models encode meaning and generate outputs.

Key Concepts

  • Dimensionality: Typically hundreds to thousands of dimensions
  • Learned structure: Similar concepts cluster together
  • Interpolation: Moving smoothly between points creates meaningful outputs
  • Manifold: Real data often lies on lower-dimensional surfaces

Examples

Visualization
Latent Space Structure
LATENT SPACE OF A LANGUAGE MODEL: Imagine a 768-dimensional space where: Animals Vehicles ↑ ↑ cat • • dog car • • truck • bird • bus • fish • plane Each point is a 768-dim vector, but concepts cluster! PROPERTIES: 1. Proximity = Similarity distance(cat, dog) < distance(cat, car) 2. Directions = Relationships king → queen follows same direction as man → woman 3. Interpolation = Blending lerp(cat, dog, 0.5) might represent "generic pet" WHY IT MATTERS: - Nearest neighbors find similar concepts - Arithmetic reveals relationships - Visualization (t-SNE, UMAP) shows structure - Generation samples from this space
Application
Latent Space Interpolation
IMAGE GENERATION (Stable Diffusion, DALL-E): Latent Space Interpolation between two images: Photo A: "sunset over ocean" Photo B: "sunrise over mountains" [0.2, -0.4, 0.8, ...] [0.5, 0.1, 0.6, ...] ↘ ↙ Interpolation at t=0.5 [0.35, -0.15, 0.7, ...] ↓ "golden hour over hills" The in-between point generates a meaningful blend! TEXT GENERATION (Style Transfer): Formal: "I would appreciate your assistance" Casual: "Hey, can you help me out?" Latent interpolation might yield: "Could you help me with this?" PRACTICAL CODE: def interpolate(z1, z2, steps=10): """Linear interpolation in latent space""" results = [] for t in np.linspace(0, 1, steps): z_interp = (1 - t) * z1 + t * z2 results.append(decode(z_interp)) return results

Interactive Exercise

✎
Predict Latent Structure

If "happy", "joyful", and "ecstatic" are nearby in latent space, what would you expect to find near them? What would be far away?

Pro Tips
  • Latent spaces are not directly interpretable—they're learned abstractions
  • VAEs explicitly learn structured latent spaces with regularization
  • Diffusion models denoise through latent space iteratively
  • The "holes" in latent space can produce nonsensical outputs

Related Terms

Embedding Vector Embedding Embedding Model Dimensionality