SenseNova-U1.5: Towards Native Unified Visual Intelligence

🤗 Upvotes: 73 | cs.CV

Authors:
Haiwen Diao, Jiahao Wang, Chenjing Ding, Hanming Deng, Jiangnan Chen, Ruixi Zhang, Ruohui Wang, Wenwen Tong, Xiangyu Fan, Yubo Wang, Yue Zhu, Yuwei Niu, Zhengqi Bai, Zhiqian Lin, Zhitao Yang, Zhongang Cai, Bo Yang, Chen Feng, Chengguang Lv, Guangjia Liu, Guanlin Wang, Hanyu Zhang, Haojia Yu, Hongcan Xiao, Hongli Wang, Huan Wu, Huaping Zhong, Jian Fang, Jianan Fan, Jiaqi Li, Jiefan Lu, Jing Zuo, Jingcheng Ni, Junxiang Xu, Linjun Dai, Mutian Xu, Peishen Yan, Penghao Wu, Ruijie Mao, Ruisi Wang, Shihao Bai, Shuang Yang, Shuya Yang, Shuyan Zheng, Silei Wu, Siying Li, Tao Chu, Tianbo Zhong, Tongxi Zhou, Weichao Luo, Weichen Fan, Wenhao Jia, Wenjie Gao, Xiangli Kong, Yan Li, Yang Yong, Zimo Wen, Zixuan Qian, Wenxiu Sun, Ruihao Gong, Quan Wang, Lewei Lu, Lei Yang, Ziwei Liu, Dahua Lin

Title:
SenseNova-U1.5: Towards Native Unified Visual Intelligence

Arxiv:
http://arxiv.org/abs/2609.11929v1

Abstract:
We launch SenseNova-U1.5, an 8B-MoT native unified multimodal model that understands, reasons about, and generates visual content within an encoder-free and VAE-free architecture. We strengthen its visual interface through spatially coherent patch reconstruction and scale its training with carefully curated generation and editing data, improved task formulation, structural prompt enhancement, and native resolutions of up to 4K. For post-training, we optimize specialized experts for visual aesthetics, bilingual text rendering, infographic generation, and image editing, and consolidate their capabilities through multi-expert on-policy distillation. Across extensive evaluations, SenseNova-U1.5 largely advances image fidelity, text rendering, complex composition, multi-reference editing, and interleaved generation, while improving instruction following and preserving subject identity, geometry, and unmodified regions. Despite limited exposure to structured formats in its generation data, SenseNova-U1.5 generalizes effectively to long, complex, and structured visual instructions, further proving that multimodal understanding can transfer to visual planning and creation. Together, these findings position native unified modelling as a promising path towards systems that perceive, reason and create within a fully end-to-end framework. We will open-source training code, including supervised fine-tuning, reinforcement learning, and on-policy distillation.

What is Daily Paper Cast?

We update every weekday to discuss highest-voted papers from Huggingface Daily Paper (https://huggingface.co/papers). Both the podcast scripts and audio are generated by AI. Feedback and suggestions are welcome! Email us: [e-mail]

Creator:
Jingwen Liang, 3D ML, https://www.linkedin.com/in/jingwen-liang/
Gengyu Wang, LLM ML, http://wanggengyu.com

Listen on:
Spotify: https://open.spotify.com/show/21nrhmdaA8qoBiH8q03NXL
Apple Podcast: https://podcasts.apple.com/us/podcast/daily-paper-cast/id1777620236

Cover Image by Kawen Kuang https://kawen.art

Evan: Welcome to today's episode of Daily Paper Cast.

Evan: Today's paper is from the Hugging Face daily paper list of September 11, 2026, and it has received 73 upvotes.

Ashley: The title is 'SenseNova-U1.5: Towards Native Unified Visual Intelligence.'

Evan: The first two authors are Haiwen Diao and Jiahao Wang, with the corresponding author being Quan Wang, all from SenseNova.

Evan: Alright, Ashley, why don't you kick us off with the introduction?

Ashley: Sure, Evan.

In recent years, visual generation has rapidly evolved beyond conventional image synthesis into a broader medium for visual creation.

This spans multilingual typography, information-dense design, high-resolution rendering, multi-reference composition, and fine-grained object-background editing.

Evan: That sounds pretty ambitious.

But isn't there a fundamental architectural divide that most systems struggle with?

Ashley: Exactly.

Most systems perceive images through pretrained vision encoders and generate them through variational autoencoders or VAEs.

These two components operate in distinct representational spaces: one optimized for semantic abstraction and the other for pixel-level fidelity.

Evan: Right.

How does this separation impact the seamless coordination of perception, reasoning, and generation?

Ashley: Well, this separation limits how seamlessly perception, reasoning, and generation can be coordinated within a unified visual framework across diverse forms of visual creation.

Native unified modeling aims to address this by learning directly from pixels and words.

Evan: I see.

So, what does SenseNova-U1.5 propose to overcome these challenges?

Ashley: SenseNova-U1.5 introduces two major architectural shifts.

First, it moves from independent patch prediction to spatially joint reconstruction.

Instead of mapping each visual token to pixels in isolation, tokens are projected onto a two-dimensional feature field and progressively reconstructed through spatial convolutions and Pixel Shuffle upsampling.

Evan: Interesting.

How does this approach help?

Ashley: This allows neighboring regions to exchange information before pixels are finalized, ultimately resolving color, texture, and geometry jointly rather than patch by patch.

The design preserves the efficiency of compact visual sequences while substantially improving spatial coherence and enabling native generation at resolutions up to 4K.

Evan: Got it.

And what's the second architectural advance?

Ashley: The second advance is a shift from joint reward optimization to a specialize-then-unify strategy.

Visual creation capabilities like aesthetic synthesis, bilingual text rendering, infographic design, and image editing each come with distinct reward signals, rollout dynamics, and optimization challenges.

Evan: How does this specialize-then-unify strategy work?

Ashley: Initially, dedicated reinforcement-learning experts are optimized for distinct capabilities with tailored data, rewards, sampling strategies, and regularization.

Their complementary strengths are then consolidated through multi-expert on-policy distillation.

By distilling along the student model’s own generation trajectories, expert capabilities are transferred into a single unified policy while preserving their strengths.

Evan: That's a smart approach.

So, what does the paper show about these methods?

Ashley: Extensive evaluations demonstrate that SenseNova-U1.5 significantly advances image fidelity, bilingual typography, complex composition, multi-reference editing, interleaved generation, and instruction following while preserving subject identity, geometry, and unmodified regions.

Evan: So, SenseNova-U1.5 showcases that a single native visual representation can support understanding, reasoning, generation, and editing by eliminating parallel visual pathways and repeated conversion between encoder features and VAE latents.

Ashley: Precisely.

Despite compressing each 32×32-pixel region into a single visual token, this compact representation supports efficient inference and delivers strong performance across multiple tasks.

Evan: Fascinating stuff.

And that wraps up the Introduction section of this paper.

Evan: Alright, Ashley, let's dive into the Method section.

How does SenseNova-U1.5 achieve its results?

Ashley: SenseNova-U1.5 employs a methodology that includes distinct steps in model architecture, training procedure, and reward modeling.

Let's start with the model architecture.

Ashley: The model features a near-lossless visual interface that transforms raw images or noise-corrupted visual inputs into compact token sequences without relying on an external visual encoder or VAE.

This involves convolutional projections that downsample the input by factors of 16 and 2, yielding one visual token for each 32×32 image region.

Evan: So, it ditches the conventional encoders and VAEs.

How does it maintain spatial information?

Ashley: Exactly.

To preserve spatial coordinates, two-dimensional sinusoidal positional embeddings are used.

Special and tokens delimit individual visual blocks.

These visual and textual representations are projected into a shared hidden space and jointly processed by the unified backbone.

Evan: And for the generation part?

Ashley: For generation, the model employs resolution-aware noise conditioning to factor in image resolution, effectively extending up to 4096 × 4096.

This conditioning helps the denoising process adapt to image variations across diverse resolutions and aspect ratios.

Evan: How does SenseNova-U1.5 address the issue of seams and texture discontinuities?

Ashley: Great question.

To combat seams and texture discontinuities at high resolutions, the original Multi-Layer Perceptron, or MLP, head in SenseNova-U1 is replaced with a lightweight spatially coupled decoder.

This decoder progressively reconstructs the full-resolution RGB image through Pixel Shuffle stages and convolutions, allowing pixels near patch boundaries to be determined jointly.

Evan: That's a novel approach.

What about the unified training objectives?

Ashley: SenseNova-U1.5 is trained using a unified objective that combines autoregressive language modeling, native pixel-space flow learning, and perceptual supervision.

For multimodal understanding, it optimizes the conditional likelihood of the target text sequence.

For visual generation, it employs x-prediction and learns pixel-space flow matching directly in RGB space.

Evan: How does it manage varying image resolutions during training?

Ashley: By constructing a training trajectory with resolution-adaptive noise.

Specifically, it normalizes noise scales and uses a dedicated sinusoidal multi-layer perceptron or MLP for encoding.

This method allows the model to predict clean endpoints and induce velocity estimates, optimizing them with a pixel-space flow-matching loss.

Evan: And what's the role of perceptual supervision in this process?

Ashley: Perceptual supervision helps to improve structural consistency and local visual coherence.

It uses the Learned Perceptual Image Patch Similarity or LPIPS metric to provide feature-space supervision complementary to the RGB flow objective.

The entire training process integrates these signals into a unified model.

Evan: Alright, let's move on to the Training Procedure.

How does SenseNova-U1.5 build its capabilities progressively?

Ashley: SenseNova-U1.5 builds its capabilities in three main stages: Generation Pre-Training, Unified Mid-Training, and Unified Supervised Fine-Tuning.

Let’s break it down.

Evan: Sure, let’s start with Generation Pre-Training.

Ashley: In the Generation Pre-Training stage, the model starts with a pre-trained understanding branch and randomly initializes the generation branch.

It’s then trained via pixel-space flow matching.

Training phases begin with text-to-image data at resolutions from 256 × 256 to 1024 × 1024, progressing to higher resolutions and incorporating image editing and interleaved generation tasks.

Evan: What comes next after this initial phase?

Ashley: Next is Unified Mid-Training.

Here, two branches are jointly optimized to balance multimodal competence and diverse generative capabilities.

This stage uses a mixed corpus including text-only and multimodal-understanding data, text-to-image data, image-editing data, and interleaved image-text data.

Evan: And how about the final stage?

Ashley: The final stage is Unified Supervised Fine-Tuning, which focuses on high-quality instruction-following data.

This stage consolidates the model’s capabilities, ensuring strong instruction adherence and effective capability integration.

Evan: What about the specialized experts mentioned earlier?

How are they trained?

Ashley: In Stage 4, Multi-Expert Reinforcement Learning, four experts are trained for aesthetics, text rendering, infographic generation, and image editing, each with task-specific data, rewards, and regularization.

Evan: Could you give a brief overview of how these experts are optimized?

Ashley: Certainly.

The Aesthetic Expert uses a mix of aesthetic-preference and typography data with targeted rewards.

The OCR Expert focuses on bilingual text rendering accuracy by using OCR-based rewards.

The Editing Expert balances instruction adherence, preservation of unedited content, and quality of edits.

The Infographic Expert undergoes stages of specialization for text rendering, visual quality, and aesthetic preference.

Evan: That's comprehensive.

And how is the final unified model distilled from these specialized experts?

Ashley: Stage 5 is Multi-Expert On-Policy Distillation.

Here, each expert’s capabilities are distilled into a single unified model using on-policy distillation along the model's own generation trajectories.

This approach ensures the final model retains the specialized strengths while functioning coherently as a single system.

Evan: Fascinating method.

That concludes the detailed methods section of the SenseNova-U1.5 paper.

Evan: Now, let's dive into the Experiment and Results section.

Ashley, can you walk us through the evaluations and outcomes for SenseNova-U1.5?

Ashley: Certainly, Evan.

The paper evaluates SenseNova-U1.5 across a variety of multimodal and language understanding benchmarks.

These cover areas such as STEM reasoning, general visual question answering, OCR, hallucination detection, and visual reasoning.

Evan: What are some of the key highlights from these evaluations?

Ashley: SenseNova-U1.5 shows consistently strong performance across diverse benchmarks.

For instance, it achieves competitive results on multimodal benchmarks like MMMU, MathVista, and MathVision.

In general VQA tasks, it performs well on MMBench and MMStar, indicating robust multimodal understanding.

Evan: How about OCR and visual reasoning?

Ashley: In optical character recognition or OCR, SenseNova-U1.5 performs well on benchmarks like AI2D, OCRBench, and InfoVQA.

For visual reasoning, it shows strong results on benchmarks such as BabyVision and TiR.

Evan: Great.

What about its performance in language understanding and instruction following?

Ashley: Beyond multimodal capability, SenseNova-U1.5 retains strong language understanding and instruction-following performance.

It excels in benchmarks like MMLU-Pro and C-Eval, and shows significant improvements in instruction-following with high scores in IFEval and IFBench.

Evan: Let's talk about image generation.

How does SenseNova-U1.5 measure up there?

Ashley: In image generation tasks, SenseNova-U1.5 performs impressively.

On Qwen-Image-Bench, it achieves the best overall performance among open-source models with scores of 60.22 and 60.13 for the English and Chinese subsets, respectively.

It also demonstrates strong results in text rendering and interleaved text-image generation tests across diverse benchmarks.

Evan: Can you provide specifics from some evaluations like GenEval or CVTG-2K?

Ashley: Definitely.

On GenEval, SenseNova-U1.5 achieves the best performance among open-source models, particularly excelling in counting, position, and attribute binding tasks.

For text-centric generation, it achieves the highest overall performance on CVTG-2K, with commendable word accuracy even in dense text settings.

Evan: Any notable results in long-form or structured text generation?

Ashley: Yes, on LongText-Bench, SenseNova-U1.5 leads in both English and Chinese tracks.

It maintains stability and accuracy as text length and layout complexity increase, which is crucial for structured and bilingual text rendering.

Evan: That’s impressive.

How about SenseNova-U1.5’s performance in complex infographic generation?

Ashley: On IGenBench, it achieves the strongest performance among open-source models without prompt enhancement.

Its structured visual planning and informative layouts are competitive with the best systems out there.

Evan: And how did it fare in domain-specific and reasoning-intensive tasks like BizGenEval and WISE?

Ashley: On BizGenEval, it shows substantial improvements, particularly in knowledge-intensive scenarios.

On WISE, when enabled with chain-of-thought reasoning, it achieves leading performance among open-source models, especially in cultural knowledge, biology, and chemistry.

Evan: That’s quite extensive.

What about image editing capabilities?

Ashley: In image editing, SenseNova-U1.5 displays leading performance across varied benchmarks such as ImgEdit and GEdit-Bench.

It excels in semantic consistency, text-centric edits, and even multi-reference editing, showing robustness in fulfilling different types of edit instructions while maintaining high visual fidelity.

Evan: Wow.

And does it hold up in reasoning-centric editing tasks?

Ashley: Yes, on RISEBench, it outperforms other open-source models.

Chain-of-thought reasoning further enhances its performance, particularly in tasks requiring causal and logical inferences.

Evan: How does it perform in interleaved image-text generation?

Ashley: In interleaved generation tasks, SenseNova-U1.5 achieves state-of-the-art performances.

It’s effective in VBVR-Pro-Bench, excelling in both in-domain and out-of-domain tasks, showing strong cognitive abilities and generalization.

Evan: That’s robust performance all across the board.

That wraps up our discussion on the Experiment and Results section of the SenseNova-U1.5 paper.

Evan: Let’s move on to the Related Work section.

Ashley, what does the paper say about the foundation and context of SenseNova-U1.5?

Ashley: In the Related Work section, SenseNova-U1.5 builds on several key areas, with specific focus on native multimodal unified models, reinforcement learning for diffusion models, and on-policy distillation for unified models.

Evan: Let's start with native multimodal unified models.

What are the main points there?

Ashley: Native multimodal unified models directly process visual inputs without external encoders and aim to combine understanding and generation within a single framework.

Examples include models like Fuyu-8B, EVE, Mono-InternVL, and the NEO-unify architecture which SenseNova-U1.5 is built upon.

Evan: How do these models achieve this integration?

Ashley: They unify multimodal learning through token-level autoregression for discrete models and pursue end-to-end modeling without explicit tokenizers or latent bottlenecks for continuous models.

The SenseNova-U series scales this direction by ensuring that understanding, reasoning, and generation emerge from a shared visual substrate.

Evan: What about the reinforcement learning aspect, especially for diffusion models?

Ashley: In language modeling, reinforcement learning from human feedback or RLHF has established a general post-training paradigm.

Visual generation models have adopted similar post-training paradigms with ReFL, DDPO, and DPOK optimizing diffusion models using preference rewards or policy gradients.

Evan: Could you give some examples of how online reinforcement learning has been extended to visual generation?

Ashley: Sure, online reinforcement learning in visual generation uses methods like Flow-GRPO and DanceGRPO.

These methods improve efficiency, preference alignment, compositional accuracy, text rendering, and visual quality by employing group-relative optimization and stochastic differential equation-based rollouts.

Evan: How does this apply to SenseNova-U1.5?

Ashley: In SenseNova-U1.5, task-dependent RL post-training combines coefficients-preserving sampling with interleaved or task-specific rewards to address distinct optimization requirements of generation and editing.

Specialized rewards cater to various aspects like perceptual quality and text-image alignment.

Evan: Interesting.

Let's move on to on-policy distillation.

What's new in this area?

Ashley: On-policy distillation, or OPD, mitigates the distribution mismatch of conventional knowledge distillation by training the student model on its own generated trajectories while receiving supervision from a teacher model.

Related works like Flow-OPD and DiffusionOPD distill task-specialized generators through various techniques.

Evan: And how does multi-teacher distillation fit into this?

Ashley: Multi-teacher on-policy distillation, such as what we see in MOPD, extends OPD by integrating capabilities from multiple domain experts.

Each domain expert supervises the student-generated trajectories, enabling diverse reasoning capabilities to be consolidated into a single model.

Evan: Does SenseNova-U1.5 implement something similar?

Ashley: Yes, it does.

SenseNova-U1.5 retains four task-specialized external experts and hard-routes their supervision into a single native pixel-space model.

This way, it preserves the specific guidance and policies of each expert while consolidating their capabilities into one unified model.

Evan: That’s a comprehensive integration of several advanced methods.

Any other notable related works highlighted in the paper?

Ashley: The paper also cites benchmarks and techniques from a variety of recent works such as Gemini 4, OneCAT, and many others, which have pushed the boundaries of vision-language models, pixel-space generative models, and capability-specific rewards in reinforcement learning.

Evan: That's quite an extensive body of related work covered in the paper.

That concludes the Related Work section of the SenseNova-U1.5 paper.

Evan: Alright, Ashley, let's wrap this up.

What are the key contributions and takeaways of the SenseNova-U1.5 paper?

Ashley: To summarize, SenseNova-U1.5 is a significant advancement towards native unified visual intelligence.

Its primary contributions include a shift from independent patch prediction to spatially joint reconstruction, which improves spatial coherence and supports high-resolution generation.

Evan: Yes, and it also employs a specialize-then-unify strategy in reinforcement learning, right?

Ashley: Correct.

This strategy optimizes distinct capabilities separately and then consolidates them through multi-expert on-policy distillation, preserving the strengths of each specialized expert while forming a comprehensive, unified model.

Evan: It’s impressive how this model integrates advanced techniques in training, reward modeling, and on-policy distillation to handle complex multimodal tasks effectively.

Ashley: Indeed.

The evaluations show that SenseNova-U1.5 excels across various benchmarks in multimodal understanding, image generation, text rendering, image editing, and reasoning, making it a robust solution for diverse visual generation and understanding tasks.

Evan: That’s a thorough overview of a very impactful paper.

Thanks for breaking it down, Ashley.

Ashley: My pleasure, Evan.

And thank you to our listeners for joining us today.

We hope you found this discussion insightful.

Evan: Make sure to tune in tomorrow for more deep dives into the latest papers from the world of AI and machine learning.

Until then, stay curious and keep exploring.

Ashley: Goodbye and see you next time on Daily Paper Cast!