ArXiv

GeoT2V-Bench: Benchmarking 3D Consistency in Text-to-Video Models via 3D Reconstruction

Authors
Chenrui Fan, Paolo Favaro
Categories
cs.CV
arXiv
https://arxiv.org/abs/2606.24829v1
PDF
https://arxiv.org/pdf/2606.24829v1

Brief

GeoT2V-Bench (Chenrui Fan & Paolo Favaro, arXiv 2026-06-23) measures whether camera-prompted text-to-video outputs provide coherent multi-view evidence for a single static 3D scene. The pipeline estimates per-frame intrinsics and poses (VGGT-style), fits a flexible DeformableGS, constructs a static MedianGS via temporal-median aggregation, and renders it along the estimated camera path. Evaluating 3,840 reconstructions across 12 open-weight models and 80 GeCo-Eval prompts, the benchmark reports multi-dimensional reconstruction profiles and shows that visible motion, static-render error, flow agreement, and flexible-vs-static behavior often disagree — exposing complementary failure modes when treating generated clips as global static-scene acquisitions.

Why it matters

GeoT2V-Bench: a reconstruction-based diagnostic benchmark (Fan & Favaro, arXiv 2026-06-23) that evaluates camera-prompted text-to-video clips for explicit rigid 3D reconstructability using a pipeline of VGGT-style per-frame intrinsics/pose estimation, DeformableGS fitting, and a temporally-median aggregated static "MedianGS" proxy.

Key details

  • Large-scale evaluation: a four-seed protocol produced 3,840 completed reconstructions from 12 open-weight model configurations and 80 GeCo-Eval static-scene prompts; the benchmark reports a continuous reconstruction profile (apparent image motion, estimated trajectory behavior, MedianGS static-render error, static-render–flow agreement, and flexible-vs-static fit gap) and finds these metrics frequently disagree, revealing complementary failure modes.
Source evidence

Abstract

Camera-prompted text-to-video (T2V) models are increasingly used to synthesize virtual camera captures, such as orbiting objects or moving through static scenes. For these outputs, visual plausibility is insufficient: the generated frames should also provide coherent multi-view evidence for a single static 3D scene. We introduce GeoT2V-Bench, a reconstruction-based diagnostic benchmark for evaluating whether camera-prompted T2V clips can support explicit rigid 3D reconstruction. Our pipeline estimates per-frame camera intrinsics and poses with VGGT-style geometry estimation, fits DeformableGS, derives a static MedianGS proxy by temporal-median aggregation, and renders this proxy along the estimated camera path. Instead of producing a pass/fail label or a single scalar score, GeoT2V-Bench reports a continuous reconstruction profile covering apparent image motion, estimated trajectory behavior, MedianGS static rendering error, static-render flow agreement, and the gap between flexible and static fits. On a fair-format four-seed evaluation with 3,840 completed reconstructions from 12 open-weight model configurations and 80 GeCo-Eval static-scene prompts, we find that visible motion, static rendering error, flow agreement, and flexible-vs-static behavior often disagree. GeoT2V-Bench therefore captures complementary failure modes that emerge when generated videos are tested as global static-scene acquisitions.

Comment: 36 pages, 17 figures, 18 tables