Abstract

Camera-controlled video diffusion models (VDMs) have recently emerged as powerful world models, enabling users to explore 3D scenes through flexible, user-defined camera trajectories. Nevertheless, current VDMs typically rely on implicit spatiotemporal representations without explicit 3D geometric grounding. Such geometry-agnostic modeling often leads to issues including geometrically implausible structures and cross-view spatial inconsistencies. To alleviate this, we present DreamWorld, a new recipe of world model that novelly bridges the strong spatial structure priors of 3D foundation models with the high-fidelity generative capabilities of video diffusion models for geometry-consistent 3D scene generation. Specifically, given the input image and camera trajectory, DreamWorld first learns a geometry video diffusion model to predict compact geometry features for the target novel views, functioning as explicit structure pivots to reflect the underlying 3D spatial layout. To achieve this, we introduce a distillation paradigm that transfers high-level structural knowledge from a pretrained 3D foundation model to the diffusion model, thereby enabling it to produce geometrically consistent and spatially coherent features. Conditioned on such geometry features, another appearance video diffusion model is then utilized to synthesize the final video, ensuring improved geometric plausibility and cross-view consistency while maintaining high visual fidelity. Extensive experiments demonstrate that DreamWorld outperforms existing methods in visual quality, 3D consistency, and camera controllability.

Overview

DreamWorld bridges the strong spatial structural priors of 3D foundation models with the high-fidelity generative capabilities of video diffusion models for 3D-consistent world modeling. Given an input image and a user-defined camera trajectory, DreamWorld first trains a geometry video diffusion model to predict complete geometry-aware representations for target novel views from incomplete warped observations. These predicted geometry features function as explicit structural pivots that reflect the underlying 3D spatial layout of the scene. Conditioned on such geometry features, an appearance video diffusion model then synthesizes the final high-fidelity RGB frames. In this way, DreamWorld triggers 3D-consistent world modeling in a geometry-appearance disentangled two-stage fashion, where the geometry stage focuses on structural consistency and camera adherence, while the appearance stage emphasizes photorealistic details and visual fidelity.

1. Qualitative Comparison Against Baselines

DreamWorld pivots on a decoupled geometry-grounded paradigm for 3D world modeling, yielding high-fidelity novel-view sequences with strong 3D consistency, while avoiding warping-induced artifacts and geometrically implausible distortions.

2. More Qualitative Results of DreamWorld

To showcase the versatility of our paradigm, we present a wide range of results spanning intricate indoor environments, expansive outdoor landscapes, and challenging out-of-domain scenarios that highlight the robust generalization ability of DreamWorld.

@inproceedings{yang2026dreamworld,
  title={DreamWorld: Geometry-Grounded Video Diffusion for 3D-Consistent World Modeling},
  author={Haibo Yang and Yang Chen and Yingwei Pan and Zhineng Chen and Ting Yao and Tao Mei},
  booktitle={ECCV},
  year={2026}
}