SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

🤗 Upvotes: 31 | cs.CV, cs.AI

Authors:
Soohyun Ryu, Sohee Kim, Eunho Yang

Title:
SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem

Arxiv:
http://arxiv.org/abs/2609.07064v1

Abstract:
Large Vision-Language Models (LVLMs) have achieved strong performance on diverse visual tasks, yet their ability to reconstruct and reason about the 3D structure of the scene depicted in 2D images -- referred to as spatial intelligence -- remains limited. Existing approaches attempt to address this gap by using real-scene spatial question answering datasets that require dense geometric annotations. However, constructing such labels is costly, time-consuming, and often noisy due to reliance on external perception modules. In this work, we propose a novel paradigm inspired by human cognitive development: learning foundational spatial skills through structured block-manipulation tasks. We introduce SpatialBlock-15k, a synthetic dataset of 15,000 block-stacking problems covering 3D-to-2D projection, viewpoint transformation, and structural combination. The dataset further incorporates controlled color modulation as visual cues to encourage anchor-based reasoning in visually complex conditions. Experiments demonstrate that LVLMs trained on our dataset through either direct answering or reasoning-based prediction significantly outperform baselines and generalize to real-world spatial tasks, despite the dataset's synthetic and compact nature. Code and data are available at https://github.com/rsoohyun/SpatialBlock.

What is Daily Paper Cast?

We update every weekday to discuss highest-voted papers from Huggingface Daily Paper (https://huggingface.co/papers). Both the podcast scripts and audio are generated by AI. Feedback and suggestions are welcome! Email us: [e-mail]

Creator:
Jingwen Liang, 3D ML, https://www.linkedin.com/in/jingwen-liang/
Gengyu Wang, LLM ML, http://wanggengyu.com

Listen on:
Spotify: https://open.spotify.com/show/21nrhmdaA8qoBiH8q03NXL
Apple Podcast: https://podcasts.apple.com/us/podcast/daily-paper-cast/id1777620236

Cover Image by Kawen Kuang https://kawen.art

Evan: Welcome to Daily Paper Cast.

Ashley: Today, we're featuring a paper from Hugging Face's daily paper list of September 11, 2026.

This paper has garnered 31 upvotes.

Evan: The title of the paper is 'SpatialBlock: Enhancing Spatial Intelligence in LVLMs via Synthetic Block-Stacking Problem.'

Ashley: It's authored by Soohyun Ryu and Sohee Kim from KAIST, with Eunho Yang as the corresponding author representing both KAIST and AITRICS in South Korea.

Evan: Alright, let's dive into the paper's Introduction.

So Ashley, what's the primary focus of this research?

Ashley: Large Vision-Language Models, or LVLMs, have shown impressive performance across various visual tasks.

However, their ability to reconstruct and reason about the 3D structure of scenes from 2D images—what we call spatial intelligence—still remains a challenge.

Evan: Got it.

And addressing this gap seems crucial for applications requiring robust spatial reasoning, right?

Ashley: Exactly.

Applications like autonomous driving and robotics heavily rely on spatial reasoning to interpret and interact with their environments.

Existing methods to enhance this capability in LVLMs usually depend on real-scene spatial question-answering datasets.

However, these datasets necessitate dense geometric annotations, which are costly, time-consuming, and often noisy due to their reliance on external perception models like segmentation or depth estimation modules.

Evan: I see.

So how does this paper propose to solve these issues?

Ashley: The researchers drew inspiration from human cognitive development, particularly how foundational spatial skills are acquired through structured block-manipulation tasks.

They introduced SpatialBlock-15k, a synthetic dataset comprising 15,000 block-stacking problems designed to teach LVLMs crucial spatial reasoning tasks.

Evan: Interesting!

What specific tasks does this dataset include?

Ashley: The dataset includes three main categories of spatial reasoning tasks: 3D-to-2D projection, which involves predicting the 2D appearance of a 3D structure from a specific viewpoint; viewpoint transformation, which requires reasoning about how a structure looks under different perspectives; and structural combination, which involves determining the resultant structure when two 3D configurations are combined.

Evan: And how do they ensure the models can handle complex visual conditions?

Ashley: To tackle visually complex scenarios, the researchers incorporated controlled color modulation as visual cues.

These modulations serve as anchors, helping the model focus on task-relevant elements and optimize their spatial reasoning capabilities, despite dealing with intricate scenes.

Evan: Right, so using this synthetic and compact dataset, how do the experiments measure up against existing models?

Ashley: Experiments showed that LVLMs trained on SpatialBlock-15k, through either direct answering or reasoning-based prediction, significantly outperformed baseline models and effectively generalized to real-world spatial tasks.

This performance boost was notable despite the dataset being synthetic and compact.

Evan: That sounds like a significant leap forward.

In summary, what are the main contributions of this paper?

Ashley: The contributions of this paper are three-fold: firstly, it introduces a novel paradigm for enhancing spatial intelligence in LVLMs by shifting from annotation-heavy real-scene supervision to foundational spatial skill learning through structured synthetic tasks.

Secondly, it presents SpatialBlock-15k, a scalable dataset enriched with controlled color cues for effective task-relevant reasoning.

Lastly, it demonstrates that training LVLMs on such synthetic data can significantly improve spatial reasoning performance and generalize effectively to real-world scenes.

Evan: Thanks, Ashley.

That wraps up the Introduction section of the paper.

Let's move on to the methods in the next part.

Evan: Let's dive into the methods proposed in this paper.

Ashley: The researchers proposed two training strategies for enhancing spatial intelligence in LVLMs using the SpatialBlock-15k dataset.

These strategies are designed to target different aspects of the model's capabilities and are referred to as direct answer prediction and reasoning-based prediction.

Evan: Let's start with the direct answer prediction model.

What's the approach here?

Ashley: The direct answer prediction model focuses on cultivating rapid inference by mapping visual inputs directly to their corresponding answers.

This is achieved by training the model with supervision only on the ground-truth answer sequence.

Given a textual query and an image, the model parameters are optimized to predict the next token by minimizing the standard cross-entropy loss.

Evan: So it's about optimizing the model to make instantaneous spatial problem-solving more efficient.

And what about the reasoning-based prediction model?

Ashley: Exactly.

Now, the reasoning-based prediction model goes a step further by enabling the model to articulate its internal logic.

This involves a two-step process: first, the model undergoes supervised fine-tuning to establish basic task-solving ability.

Then, reinforcement learning is employed to facilitate high-order structural reasoning.

Evan: Interesting.

Can you elaborate on the fine-tuning process?

Ashley: Sure.

For model initialization, the researchers used Low-Rank Adaptation, or LoRA, instead of full-parameter fine-tuning.

This choice was made to maintain the model's Chain-of-Thought reasoning capabilities while aligning it with the tasks in the dataset.

Unlike the conventional 'cold-start' phase, which relies on synthesized trajectories from larger teacher models, the researchers found that using LoRA-based tuning preserved the model’s reasoning ability more effectively.

Evan: And how is the reinforcement learning applied after this initialization?

Ashley: Following LoRA-based initialization, the model is directly optimized using reinforcement learning via Group Relative Policy Optimization, or GRPO.

The researchers designed a multi-objective reward function to evaluate response correctness and reasoning trace quality.

This function consists of three components: accuracy reward, format reward, and length reward.

Together, these components ensure that the model's responses are correct, follow the required Chain-of-Thought structure, and are of appropriate length.

Evan: How does this reward system work in practice?

Ashley: For each question, the old policy model samples a group of candidate responses.

Each response is evaluated with the reward function, and an advantage is calculated.

The model is then updated by maximizing an objective function that incorporates the rewards, with hyper-parameters set for stability and performance.

This approach allows the model to improve its spatial reasoning and answer generation iteratively.

Evan: That sounds pretty thorough.

Now, what about the dataset itself?

How is SpatialBlock-15k constructed and utilized?

Ashley: SpatialBlock-15k is designed to systematically probe and enhance foundational spatial abilities.

It moves away from conventional annotation-heavy paradigms by utilizing block-stacking problems inspired by early stages of human spatial cognitive development.

The dataset comprises three main types of tasks: 3D-to-2D projection, viewpoint transformation, and structural combination.

Evan: Let's break these down.

What exactly do these tasks involve?

Ashley: 3D-to-2D projection tasks require predicting the 2D appearance of a 3D block structure from a specific viewpoint.

This involves reconstructing the 3D configuration and mentally transforming it to the target viewpoint.

Viewpoint transformation tasks require reasoning about how a given 3D structure appears under self-rotation or viewpoint changes, maintaining structural consistency across transformations.

Structural combination tasks involve determining how two 3D structures combine and interact to form a coherent global structure.

Evan: And they’ve introduced visual cues in this dataset as well?

Ashley: Yes, exactly.

They incorporated controlled color modulation as visual cues.

These cues serve as functional guidance, revealing depth information, structural correspondences, and contact regions.

For instance, different colors are used based on depth from a given viewpoint, facilitating depth-aware reasoning.

Anchor blocks are highlighted to maintain the same structural role before and after transformations.

Also, overlapping colored blocks indicate attachment locations for structural combinations.

Evan: So, color cues are used not just for diversity but for guiding the model's reasoning process.

How effective is this integration?

Ashley: The experiments conclusively demonstrated that models trained with these visual cues showed significant performance improvement.

Removing visual cues led to performance drops, suggesting the importance of color cues in helping models better understand complex scenes and mimic human-like spatial reasoning.

Evan: It seems like the dataset and the proposed methods are tightly integrated.

Are there any specific metrics used to evaluate the models?

Ashley: Yes, they adopted standardized evaluation metrics to assess the capability across several spatial reasoning benchmarks.

The main benchmarks included a held-out test split of SpatialBlock-15k for in-domain evaluation and four real-world spatial reasoning benchmarks for out-of-domain testing.

These benchmarks tested the models’ abilities on mental simulation, viewpoint transformations, multi-image spatial reasoning, and general visual perception.

Evan: And how did the models fare against these benchmarks?

Ashley: The models trained with SpatialBlock-15k outperformed existing spatial specialist models on most benchmarks.

Particularly, the direct answer prediction models excelled in tasks requiring canonical viewpoint transformations, while reasoning-based models performed better on tasks needing complex logical inference or multiple image reasoning.

Despite being trained on synthetic data alone, the models also maintained stable performance on the general visual understanding benchmark.

Evan: It seems like it's a well-rounded and effective approach.

That wraps up the Method section of the paper.

Evan: Now, let's delve into the experiments and results of this study.

Ashley: The researchers conducted multiple experiments to validate the effectiveness of the SpatialBlock-15k dataset in fostering spatial intelligence in LVLMs.

They first described the experimental setup, including the training details and the evaluation protocols used.

Evan: Which models did they use for these experiments?

Ashley: They used four baseline models: Qwen2.5-VL-3B, Qwen2.5-VL-7B, Qwen3-VL-4B, and InternVL3-2B.

To distinguish between the two training strategies, a suffix was added to the model name: SpatialBlock-direct for direct answer prediction and SpatialBlock-reason for reasoning-based prediction.

Both variants were trained using the entire SpatialBlock-15k dataset.

Evan: And what benchmarks did they use to evaluate the models' capabilities?

Ashley: They evaluated the spatial reasoning capabilities on five benchmarks, spanning both in-domain and out-of-domain settings.

For in-domain evaluation, they introduced SB-Bench, which is the held-out test split of SpatialBlock-15k and comprises 600 questions.

For out-of-domain evaluation, they assessed generalization to real-world scenarios on four benchmarks: MindCube, MMSI-Bench, SPBench for spatial reasoning, and MMMU for general visual perception.

All benchmarks were restricted to multiple-choice questions to align with their training answer format.

Evan: How did the models perform on these benchmarks?

Ashley: The results were impressive.

Despite being trained with only 15,000 synthetic samples, the models consistently outperformed existing spatial specialists on real-scene benchmarks.

For instance, the SpatialBlock-direct models showed substantial gains on MindCube, which required mental simulation.

Specifically, the Qwen2.5-VL-3B model outperformed the state-of-the-art SpatialLadder by 2.7%, while the Qwen2.5-VL-7B model showed an even larger gain of 17.6%.

The Qwen3-VL-4B model further improved upon its backbone by 25.1%, achieving the best open-source performance.

This pattern was observed across benchmarks, indicating strong generalizability.

Evan: And what about the reasoning-based models?

Ashley: The SpatialBlock-reason models also demonstrated strong results, particularly on MMSI-Bench, which required complex logical inference.

The Qwen2.5-VL-3B model outperformed the SpatialLadder-3B model by 3.2%, using a simpler training pipeline based solely on synthetic block-stacking data.

The Qwen2.5-VL-7B model outperformed both SpaceR and Spatial-SSRL models with only 15,000 training samples.

Beyond final-answer accuracy, these models also showed higher reasoning quality, achieving a higher alignment score with ground-truth reasoning steps compared to baselines.

Evan: It sounds like the approach was robust across different benchmarks and settings.

Did they conduct any ablation studies to further dissect the contributions of each component?

Ashley: Yes, they performed extensive ablation studies to assess the effects of each component in dataset construction and training.

They evaluated models trained on a single question type and found that combining all three types led to the best performance across benchmarks.

Removing visual cues from training led to performance drops on several benchmarks, underscoring the importance of color cues in helping models understand complex scenes.

Evan: What about the robustness tests?

Ashley: They also assessed the robustness to varying viewing angles by generating problems from different angles.

The models maintained consistent performance across these viewpoints, suggesting that they had genuinely learned the underlying 3D structure rather than merely memorizing appearance patterns.

Evan: Any other findings that stood out?

Ashley: Interestingly, even with synthetic data training, the models maintained strong performance on the general visual understanding benchmark, MMMU.

This indicates that training on carefully designed synthetic datasets could enhance specific reasoning capabilities without degrading broader visual perception skills.

Evan: So, in summary, what does this say about the effectiveness of using structured synthetic tasks for training?

Ashley: These results suggest that structured synthetic datasets, like SpatialBlock-15k, provide an effective training signal for enhancing spatial reasoning in LVLMs.

The combination of targeted tasks, controlled visual cues, and robust training strategies can significantly elevate model performance and generalizability.

Evan: That's the end of the Experiment section.

Let's turn our attention to the discussion and conclusion in the next part.

Evan: Shall we move on to the related work discussed in this paper?

Ashley: The Related Work section is divided into two main areas: Visual Spatial Reasoning in LVLMs and Training Datasets for Spatial Intelligence.

Let's start with the first part.

Evan: Great.

What does the paper say about recent efforts in visual spatial reasoning for LVLMs?

Ashley: Recent research in this area primarily focuses on either modifying the model architecture or proposing specialized training strategies.

On the architectural side, some approaches add spatial tokens into the vision encoder or incorporate depth-aware plugin modules.

For instance, Tong et al.

in 2024 and Lou et al.

in 2025, worked on introducing spatial tokens and depth-aware plugins respectively.

Evan: And on the training strategy side?

How do these differ from architectural modifications?

Ashley: The training strategies often involve designing hierarchical schemes that transition from basic spatial perception to more complex reasoning tasks.

Some methods encourage models to output cognitive maps encoding object positions and orientations.

For example, Yang et al.

in 2025 and Yin et al.

in 2025 developed methods for outputting such cognitive maps.

However, both architectural and training-based approaches typically rely on real-scene annotations or pseudo-3D signals derived from external modules, which limit their scalability and robustness.

Evan: So, the use of real-scene annotations introduces challenges related to scalability and noise in the data.

What have researchers proposed in terms of training datasets?

Ashley: That's right.

The second part of the Related Work section covers various datasets proposed to enhance the spatial intelligence of LVLMs.

These range from simple spatial relation datasets to more complex multi-step reasoning challenges.

Evan: What are some notable examples of these datasets?

Ashley: Early benchmarks involved simpler tasks using external models to extract 3D information, generating low-level question-answer pairs.

For example, Chen et al.

in 2024 and Ma et al.

in 2025b created datasets incorporating object detection and pose estimation to query spatial relations like distance and orientation.

Evan: And how have these datasets evolved to address more complex reasoning tasks?

Ashley: To foster higher-level logic, recent works use 3D annotated video data to construct more sophisticated tasks such as route planning or spatio-temporal appearance ordering through human annotation.

Substantial works by Yang et al.

in 2025 and Ouyang et al.

in 2025 have leveraged such data.

MindCube, for instance, focuses on spatial mental modeling from limited views or inferring arrangements under perspective shifts.

Evan: However, these approaches still depend heavily on dense, real-scene 3D annotations, right?

Ashley: Indeed.

The dense real-scene annotations are labor-intensive and inherently noisy, making them an expensive and sometimes impractical solution.

This has motivated researchers to explore alternative methods that can achieve robust spatial intelligence with less dependency on real-scene data.

Evan: And this paper proposes using structured synthetic tasks to overcome these hurdles?

Ashley: Exactly.

By using synthetic block-stacking problems inspired by human cognitive development, the SpatialBlock-15k dataset offers a more scalable and cleaner alternative.

This approach addresses both scalability and noise issues while providing a robust training signal for spatial reasoning.

Evan: So, in a way, this paper builds on previous research but takes a different route to tackle the underlying issues.

That wraps up the Related Work section.

Evan: Alright, let's summarize the key contributions and takeaways from this paper.

Ashley: The primary focus of this paper is enhancing spatial intelligence in Large Vision-Language Models using a novel synthetic dataset called SpatialBlock-15k.

This dataset consists of 15,000 block-stacking problems designed to improve essential spatial reasoning tasks such as 3D-to-2D projection, viewpoint transformation, and structural combination.

Evan: One of the significant advancements made by this paper is the integration of controlled color modulation as visual cues.

These cues help models focus on task-relevant elements and perform spatial reasoning even in visually complex scenarios.

Ashley: Experiments showed that LVLMs trained on SpatialBlock-15k significantly outperformed baseline models on both in-domain and real-world spatial tasks.

The models demonstrated strong generalizability and maintained robust performance across different benchmarks, highlighting the effectiveness of structured synthetic tasks as a training signal.

Evan: Furthermore, this research underscores the potential of synthetic datasets to provide a scalable and cleaner alternative to traditional real-scene data, addressing issues of scalability and data noise effectively.

Ashley: To sum up, the main contributions of this paper are the introduction of a new perspective on improving spatial intelligence, the development of the SpatialBlock-15k dataset, and empirical evidence showing the effectiveness of targeted training on fundamental spatial reasoning tasks.

Evan: That brings us to the end of today's episode.

Thanks for tuning in to Daily Paper Cast.

Ashley: We hope you found this episode informative.

Be sure to join us again for more insights into the latest research papers in AI and related fields.

Evan: Until next time, stay curious and keep exploring.