论文
arXiv
Agent
GeoSimulation
中文标题
UrbanVerse:通过观看城市观光视频实现城市模拟的规模化
English Title
UrbanVerse: Scaling Urban Simulation by Watching City-Tour Videos
Mingxuan Liu, Honglin He, Elisa Ricci, Wayne Wu, Bolei Zhou
发布时间
2025/10/17 01:42:34
来源类型
preprint
语言
en
摘要
中文对照

从配送机器人到四足机器人的城市具身人工智能代理正日益遍布我们的城市,在复杂的街道环境中导航以提供最后一公里的连接服务。训练此类代理需要多样且高保真的城市环境以实现规模化,然而现有的人工构建或程序生成的仿真场景要么缺乏可扩展性,要么无法捕捉真实世界的复杂性。我们提出了UrbanVerse,一种数据驱动的真实世界到仿真系统,能够将众包的城市观光视频转化为具备物理感知和交互能力的仿真场景。UrbanVerse包含:(i) UrbanVerse-100K,一个包含超过10万项标注的城市三维资产库,具有语义与物理属性;(ii) UrbanVerse-Gen,一种自动化的流程,从视频中提取场景布局,并利用检索到的资产快速构建度量尺度的三维仿真环境。在IsaacSim中运行,UrbanVerse提供了来自24个国家的160个高质量构建场景,以及一组由艺术家精心设计的10个测试场景基准。实验表明,UrbanVerse场景保留了真实世界的语义与布局,其人类评估的逼真度与手工制作场景相当。在城市导航任务中,基于UrbanVerse训练的策略展现出显著的缩放规律和强泛化能力,在仿真环境中成功率提升6.3%,在零样本的仿真到现实迁移中提升30.1%,仅需两次干预即完成300米的真实世界任务。

English Original

Urban embodied AI agents, ranging from delivery robots to quadrupeds, are increasingly populating our cities, navigating chaotic streets to provide last-mile connectivity. Training such agents requires diverse, high-fidelity urban environments to scale, yet existing human-crafted or procedurally generated simulation scenes either lack scalability or fail to capture real-world complexity. We introduce UrbanVerse, a data-driven real-to-sim system that converts crowd-sourced city-tour videos into physics-aware, interactive simulation scenes. UrbanVerse consists of: (i) UrbanVerse-100K, a repository of 100k+ annotated urban 3D assets with semantic and physical attributes, and (ii) UrbanVerse-Gen, an automatic pipeline that extracts scene layouts from video and instantiates metric-scale 3D simulations using retrieved assets. Running in IsaacSim, UrbanVerse offers 160 high-quality constructed scenes from 24 countries, along with a curated benchmark of 10 artist-designed test scenes. Experiments show that UrbanVerse scenes preserve real-world semantics and layouts, achieving human-evaluated realism comparable to manually crafted scenes. In urban navigation, policies trained in UrbanVerse exhibit scaling power laws and strong generalization, improving success by +6.3% in simulation and +30.1% in zero-shot sim-to-real transfer comparing to prior methods, accomplishing a 300 m real-world mission with only two interventions.

元数据
arXiv2510.15018v2
来源arXiv
类型论文
抽取状态raw
关键词
Agent
GeoSimulation
cs.CV
cs.AI
cs.RO