VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning

Overview

Native visual reasoning, i.e., reasoning through visual generation, has recently emerged as a promising direction for studying visual intelligence beyond language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across six held-out visual reasoning benchmarks such as RISE-Video, MME-CoF-Pro, and BabyVision. Further analysis validates that these gains reflect visual reasoning rather than instruction-pattern fitting. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded on verifiable task-specific rules, achieve fine-grained alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image, video, and interleaved generators. Our analysis shows that video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative by externalizing intermediate visual states. Critically, ablations and probing confirm the presence of vision-native trajectories, that are a more crucial substrate than explicit linguistic chains of thought for visual reasoning. We release all data, models, scorers, and code to facilitate future research.

The models are presented in the paper VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning.

Models Zoo

Model	Base Architecture	Other Remarks
Image Generation Models
VBVR-Pro-BAGEL	BAGEL-7B-MoT	Complete model
VBVR-Pro-FLUX2-dev	FLUX.2-dev	Complete model, Diffusers format
VBVR-Pro-FLUX2-dev-diffsynth	FLUX.2-dev	LoRA model, DiffSynth format
VBVR-Pro-Qwen-Image-Edit	Qwen-Image-Edit-2511	Complete model, Diffusers format
VBVR-Pro-Qwen-Image-Edit-diffsynth	Qwen-Image-Edit-2511	LoRA model, DiffSynth format
Interleaved Image Generation Models
VBVR-Pro-ThinkMorph	ThinkMorph-7B	Complete model
VBVR-Pro-SenseNova-U1	SenseNova-U1-8B-MoT	Complete model
Video Generation Models
VBVR-Pro-LTX2.3	LTX-Video-2.3	Complete model, Diffusers format
VBVR-Pro-LTX2.3-diffsynth	LTX-Video-2.3	LoRA model, DiffSynth format
VBVR-Pro-Wan2.1-I2V-14B	Wan2.1-I2V-14B-720P	Complete model, Diffusers format
VBVR-Pro-Wan2.1-I2V-14B-diffsynth	Wan2.1-I2V-14B-720P	LoRA model, DiffSynth format
VBVR-Pro-Wan2.2-I2V-A14B	Wan2.2-I2V-A14B	Complete model, Diffusers format
VBVR-Pro-Wan2.2-I2V-A14B-diffsynth	Wan2.2-I2V-A14B	LoRA model, DiffSynth format
VBVR-Pro-Wan2.2-TI2V-5B	Wan2.2-TI2V-5B	Complete model, Diffusers format
VBVR-Pro-Wan2.2-TI2V-5B-diffsynth	Wan2.2-TI2V-5B	LoRA model, DiffSynth format
VBVR-Pro-Wan2.2-TI2V-5B-RLVR	Wan2.2-TI2V-5B	Complete model, RL with verifiable rewards
VBVR-Pro-Wan2.2-TI2V-5B-RLVLM-Qwen3.6-27B-Reward	Wan2.2-TI2V-5B	Complete model, RL with Qwen3.6-27B VLM rewards

VBVR-Pro Benchmark Results

Models	Overall	In-Domain by Category						Out-of-Domain by Category
		Avg.	Abst.	Know.	Perc.	Spat.	Trans.	Avg.	Abst.	Know.	Perc.	Spat.	Trans.
Image Generation Models
Proprietary Models
Qwen-Image-2.0	0.313	0.248	0.269	0.196	0.225	0.170	0.132	0.378	0.341	0.235	0.391	0.384	0.080
Seedream-5.0-Pro	0.557	0.485	0.518	0.312	0.509	0.401	0.217	0.629	0.507	0.455	0.661	0.559	0.202
Open-source Models
BAGEL-7B-MoT	0.089	0.066	0.039	0.085	0.067	0.046	0.027	0.111	0.201	0.031	0.073	0.028	0.121
FLUX.2-dev	0.157	0.108	0.088	0.109	0.072	0.100	0.066	0.206	0.197	0.165	0.184	0.241	0.077
Qwen-Image-Edit	0.134	0.108	0.092	0.082	0.100	0.109	0.056	0.159	0.176	0.063	0.141	0.182	0.082
Strong Baselines
VBVR-Pro-BAGEL	0.172	0.168	0.199	0.105	0.110	0.213	0.055	0.176	0.254	0.104	0.148	0.015	0.145
VBVR-Pro-FLUX.2	0.407	0.484	0.483	0.323	0.367	0.449	0.336	0.330	0.361	0.272	0.255	0.454	0.128
VBVR-Pro-Qwen-Image	0.322	0.332	0.298	0.217	0.193	0.431	0.222	0.311	0.341	0.239	0.233	0.413	0.181
Interleaved Image Generation Models
Proprietary Models
GPT-Image-2	0.507	0.428	0.456	0.318	0.428	0.206	0.300	0.587	0.398	0.413	0.633	0.480	0.303
Nano Banana Pro	0.564	0.480	0.518	0.422	0.512	0.285	0.174	0.648	0.553	0.499	0.657	0.585	0.220
Open-source Models
ThinkMorph-7B	0.154	0.113	0.100	0.082	0.101	0.148	0.031	0.195	0.176	0.166	0.163	0.253	0.103
VBVR-SenseNova-U1	0.408	0.469	0.356	0.313	0.373	0.386	0.477	0.347	0.291	0.317	0.275	0.480	0.238
SenseNova-U1-8B-MoT	0.565	0.533	0.501	0.395	0.544	0.355	0.349	0.597	0.448	0.495	0.533	0.717	0.401
Strong Baselines
VBVR-Pro-ThinkMorph	0.373	0.402	0.403	0.344	0.238	0.454	0.184	0.344	0.367	0.224	0.238	0.535	0.257
VBVR-Pro-SenseNova-U1	0.638	0.811	0.648	0.695	0.621	0.770	0.541	0.464	0.480	0.328	0.344	0.558	0.408
Video Generation Models
Proprietary Models
Veo 3.1	0.309	0.312	0.275	0.299	0.252	0.267	0.157	0.305	0.305	0.233	0.252	0.312	0.219
Kling V3	0.392	0.356	0.213	0.326	0.320	0.355	0.229	0.427	0.294	0.564	0.375	0.242	0.412
SeedDance 2.0	0.499	0.451	0.338	0.361	0.353	0.468	0.308	0.547	0.369	0.511	0.478	0.538	0.532
Open-source Models
HunyuanVideo-I2V	0.054	0.054	0.023	0.064	0.015	0.084	0.032	0.053	0.088	0.014	0.028	0.062	0.055
CogVideoX1.5-5B-I2V	0.085	0.100	0.061	0.118	0.069	0.092	0.060	0.070	0.125	0.038	0.051	0.040	0.024
Wan2.1-I2V-14B	0.100	0.105	0.052	0.125	0.091	0.102	0.052	0.095	0.112	0.073	0.071	0.123	0.044
Wan2.2-TI2V-5B	0.094	0.066	0.029	0.073	0.050	0.083	0.031	0.122	0.156	0.052	0.106	0.063	0.099
Wan2.2-I2V-14B-720P	0.182	0.157	0.082	0.131	0.110	0.161	0.156	0.207	0.224	0.139	0.140	0.195	0.273
LTX2.3-I2AV	0.112	0.106	0.062	0.109	0.070	0.133	0.055	0.119	0.161	0.135	0.086	0.091	0.050
VBVR-Wan2.2	0.517	0.548	0.237	0.499	0.334	0.566	0.591	0.486	0.310	0.343	0.345	0.732	0.684
Strong Baselines
VBVR-Pro-LTX2.3	0.425	0.527	0.409	0.510	0.346	0.460	0.390	0.324	0.381	0.108	0.201	0.477	0.386
VBVR-Pro-Wan2.1-I2V-14B	0.562	0.730	0.617	0.580	0.452	0.676	0.623	0.395	0.410	0.305	0.230	0.617	0.439
VBVR-Pro-Wan2.2-TI2V-5B	0.470	0.641	0.528	0.556	0.373	0.565	0.557	0.300	0.333	0.127	0.161	0.505	0.409
VBVR-Pro-Wan2.2-I2V-14B	0.670	0.808	0.632	0.685	0.556	0.751	0.636	0.532	0.479	0.418	0.350	0.679	0.690

Installation

We recommend using uv to manage the unified Python 3.10 environment.

uv installation guide: https://docs.astral.sh/uv/getting-started/installation/#installing-uv

git clone https://github.com/Video-Reason/VBVR-Pro.git
cd VBVR-Pro/
uv sync --extra cu124 # or one of [cu118|cu121|cu124|cu126|cu128|cu129]
source .venv/bin/activate

Select the extra for the CUDA wheel build you want; the extras are mutually exclusive. For faster attention on BAGEL and ThinkMorph, install the optional FlashAttention extension:

uv sync --extra cu124 --extra flash-attn

Download Models

Merged models (self-contained, recommended)

The merged (complete) models do not require separate base model downloads:

# Image generation
hf download Video-Reason/VBVR-Pro-FLUX2-dev --local-dir models/VBVR-Pro-FLUX2-dev
hf download Video-Reason/VBVR-Pro-Qwen-Image-Edit --local-dir models/VBVR-Pro-Qwen-Image-Edit
hf download Video-Reason/VBVR-Pro-BAGEL --local-dir models/VBVR-Pro-BAGEL

# Interleaved image generation
hf download Video-Reason/VBVR-Pro-ThinkMorph --local-dir models/VBVR-Pro-ThinkMorph
hf download Video-Reason/VBVR-Pro-SenseNova-U1 --local-dir models/VBVR-Pro-SenseNova-U1

# Video generation
hf download Video-Reason/VBVR-Pro-LTX2.3 --local-dir models/VBVR-Pro-LTX2.3
hf download Video-Reason/VBVR-Pro-Wan2.1-I2V-14B --local-dir models/VBVR-Pro-Wan2.1-I2V-14B
hf download Video-Reason/VBVR-Pro-Wan2.2-I2V-A14B --local-dir models/VBVR-Pro-Wan2.2-I2V-A14B
hf download Video-Reason/VBVR-Pro-Wan2.2-TI2V-5B --local-dir models/VBVR-Pro-Wan2.2-TI2V-5B
hf download Video-Reason/VBVR-Pro-Wan2.2-TI2V-5B-RLVR --local-dir models/VBVR-Pro-Wan2.2-TI2V-5B-RLVR
hf download Video-Reason/VBVR-Pro-Wan2.2-TI2V-5B-RLVLM-Qwen3.6-27B-Reward --local-dir models/VBVR-Pro-Wan2.2-TI2V-5B-RLVLM-Qwen3.6-27B-Reward

DiffSynth LoRA models (require base model downloads)

For -diffsynth releases, download the corresponding base models first:

export MODELS_DIR="${PWD}/models"
mkdir -p "${MODELS_DIR}"

hf download black-forest-labs/FLUX.2-dev \
  --local-dir "${MODELS_DIR}/black-forest-labs/FLUX.2-dev"
hf download Qwen/Qwen-Image-Edit-2511 \
  --local-dir "${MODELS_DIR}/Qwen/Qwen-Image-Edit-2511"
hf download DiffSynth-Studio/LTX-2.3-Repackage \
  --local-dir "${MODELS_DIR}/DiffSynth-Studio/LTX-2.3-Repackage"
hf download google/gemma-3-12b-it-qat-q4_0-unquantized \
  --local-dir "${MODELS_DIR}/google/gemma-3-12b-it-qat-q4_0-unquantized"
hf download Wan-AI/Wan2.1-I2V-14B-720P \
  --local-dir "${MODELS_DIR}/Wan-AI/Wan2.1-I2V-14B-720P"
hf download Wan-AI/Wan2.2-I2V-A14B \
  --local-dir "${MODELS_DIR}/Wan-AI/Wan2.2-I2V-A14B"
hf download Wan-AI/Wan2.2-TI2V-5B \
  --local-dir "${MODELS_DIR}/Wan-AI/Wan2.2-TI2V-5B"

Inference

All models are invoked through the unified example.py CLI:

# Image editing (FLUX.2 merged)
python example.py \
  --model_path models/VBVR-Pro-FLUX2-dev \
  --image_paths first_frame.png \
  --prompt "Move the red block to the left of the blue block." \
  --output flux_edit.png

# Video generation (Wan2.2 merged)
python example.py \
  --model_path models/VBVR-Pro-Wan2.2-I2V-A14B \
  --image_paths first_frame.png \
  --prompt "The subject walks toward the doorway." \
  --num_frames 81 --width 832 --height 480 \
  --output wan.mp4

# Audio-video generation (LTX-2.3 merged)
python example.py \
  --model_path models/VBVR-Pro-LTX2.3 \
  --image_paths first_frame.png \
  --prompt "The machine starts and makes a quiet mechanical hum." \
  --num_frames 49 --fps 24 \
  --output ltx.mp4

# Interleaved generation with reasoning (ThinkMorph)
python example.py \
  --model_path models/VBVR-Pro-ThinkMorph \
  --image_paths first_frame.png \
  --prompt "Solve the task step by step." \
  --think --max_rounds 10 --num_images 2 \
  --output thinkmorph_outputs

# DiffSynth LoRA (downloads base on first use)
python example.py \
  --model_path models/VBVR-Pro-Qwen-Image-Edit-diffsynth \
  --image_paths first_frame.png \
  --prompt "Place the cup on the empty shelf." \
  --output qwen_lora.png

Use python example.py --list-models for the full list of supported model types.

Training

For reproducible data preparation and distributed training, use the guide for the intended stage:

• supervised fine-tuning: TRAINING.md;
• reinforcement learning: RL_TRAINING.md.

Development

Install the repository-wide lint environment and run the same checks as CI:

uv sync --frozen --only-group dev --no-install-project --inexact
.venv/bin/ruff check --output-format=github .
.venv/bin/ruff format --check .

The checks cover Python code in the root project and rl_training/. External repositories under vbvr_pro_models/ are excluded.

Download Evaluation Data

hf download Video-Reason/VBVR-Pro-Bench --repo-type dataset --local-dir data/VBVR-Pro-Bench

After generating outputs, evaluate using VBVR-Pro-Bench and submit results to the leaderboard.

License

VBVR-Pro source code, scripts, configuration files, task-specific scoring software, and the rl_training package are licensed under the Apache License 2.0. VBVR-Pro data and benchmark materials are separately licensed under CC BY-NC 4.0. Model weights and third-party materials remain subject to their applicable model-card and upstream terms. See LICENSE.md for details.

Citation

@misc{xu2026vbvrproscalableverifiablesuite,
      title={VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning},
      author={Junxiang Xu and Ruisi Wang and Fanyi Pu and Maijunxian Wang and Ran Ji and Tongxi Zhou and Chenyang Gu and Jing Zuo and Hongcan Xiao and Yimeng Geng and Wanqi Yin and Wei Chen and Oscar Qian and Zhengan Yan and Ziqi Huang and Haiwen Diao and Liang Pan and Bo Li and Xiangyu Fan and Dezhi Luo and Fengyuan Yu and Zehong Zhao and Qingying Gao and Tinghui Zhu and Yilan Zhang and Jingqi Tong and Pinyuan Feng and Zhengze Jiang and Letian Wang and Ziyu Guo and Renrui Zhang and Jieneng Chen and Sonia Joseph and Constantin Venhoff and Saman Motamed and Mengyue Yang and Chandra Sripada and Alan Yuille and Philip Torr and Lvmin Zhang and Vikash Kumar and Daniel Khashabi and Nikolaus Kriegeskorte and Raphaël Millière and Vincent C. Müller and Anyi Rao and Quan Wang and Ziwei Liu and Dahua Lin and Lei Yang and Hokin Deng and Zhongang Cai},
      year={2026},
      eprint={2608.26105},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.26105},
}

Acknowledgements

This project includes code modified from DiffSynth-Studio, BAGEL, and ThinkMorph. We gratefully acknowledge the authors and contributors for their open research.