Cover image for How Vision-Language Models Learned to Reason About Space (10 Papers, One Thread)

이승준

This is a cross-post. The original (with diagrams, code, and cheat sheets for every chapter) lives on my blog:
👉 The Evolution of Spatial VLMs

Vision-language models can describe a warehouse photo in fluent prose — and then fail to answer "how many meters is the forklift from the shelf?" They see but don't perceive. Over 2024–2026 a line of research closed that gap step by step, and the arc is remarkably coherent once you read it through a single lens:

The representational mismatch between discrete language tokens and continuous 3D geometry — and the successive attempts to close it.

I wrote a 10-chapter course tracing that arc paper by paper. Here's the map, with a one-line "why this paper had to exist" for each. Full chapter (analogies + mermaid diagrams + runnable code) is linked on each.

The 10-paper arc

  1. BLINK: See but Not Perceive — The diagnosis. Humans 95.7% vs GPT-4V 51.3% on tasks people solve "in the blink of an eye." VLMs read images as a bag of semantic concepts, not geometry.

  2. SpatialVLM: Data Is the Answer — Reframe spatial blindness as a data problem, not an architecture one. Lift 2D web images to metric 3D and synthesize 2B spatial VQA pairs.

  3. SpatialRGPT — Point at a Region, Plant the Depth — Add region grounding + a gracefully-degrading depth plugin. The first VLM to point at "this object" and reason in metric 3D.

  4. SpatialBot — Learning to Read Depth — Make depth queryable: a Depth API the model can call instead of guessing.

  5. Thinking in Space — From single images to video: does the model build a cognitive map and remember space?

  6. OmniSpatial — Draw the map of the ability itself: a comprehensive taxonomy of spatial reasoning.

  7. Perspective-Aware Reasoning (APC) — "If I stood over there…" — mental imagery to break egocentric bias.

  8. Depth-native VLMs — DepthVLM vs DepthLM — What if the VLM is the depth sensor? Two takes on baking depth in natively.

  9. Verifiable RL for Spatial Reasoning — Spatial skill isn't only more labels; a verifiable reward can activate latent ability. SVQA-R1 → Smooth Operator (50K samples rivaling 4.5M supervised).

  10. Grounded 3D-Aware — Reconstruct While You Reason — The synthesis: GR3D promotes grounding into a generation-time mechanism; VLM-3R folds 3D reconstruction into instruction tuning. Plus the six frontiers still open.

Why read it as one thread?

Because each paper is a different answer to the same question: how do you get a token-based model to respect continuous geometry? Data synthesis, depth plugins, cognitive maps, mental rotation, native depth, verifiable RL, and reconstruction-in-the-loop are all attacks on that one mismatch. Seeing them in sequence turns a pile of papers into a story with a direction.

If that framing is useful, the full course — with a visual cheat sheet, concept diagrams, and code for each chapter — is here:

👉 The Evolution of Spatial VLMs — full course

I write these paper-arc courses regularly (VLM post-training, Physical AI, agent architectures) at direcf.github.io.