论文
arXiv
RemoteSensing
EarthObservation
Multimodal
GeoMultimodal
Agent
中文标题
连接感知与行动:面向鲁棒地球观测智能体的轻量级多模态元规划框架
English Title
Bridging Perception and Action: A Lightweight Multimodal Meta-Planner Framework for Robust Earth Observation Agents
Jinghui Xu, Boyi Shangguan, Mengke Zhu, Hao Liu, Junhuan Jiang, Guangjun He, Pengming Feng, Shichao Jin, Bin Liang, Yongzhe Chang, Junbo Tan, Tiantian Zhang, Xueqian Wang
发布时间
2026/5/6 19:30:21
来源类型
preprint
语言
en
摘要
中文对照

自主地球观测(Earth Observation, EO)智能体正从被动感知转向复杂、多步骤任务执行。然而,当前将规划与执行集成于单一模型的架构在动态EO场景中常面临组合爆炸与推理错误等挑战。为应对这些问题,我们提出轻量级多模态元规划框架(Lightweight Multimodal Meta-Planner, LMMP)。LMMP引入双感知机制,使战略规划同时锚定于多模态图像特征与高层任务语义。关键在于,我们构建了元任务库(Meta Task Library),将遥感领域专家知识直接注入工作流,从而标准化领域逻辑并确保规划具备物理可行性。此外,我们设计两阶段训练流程:首先通过专家蒸馏的监督微调(Supervised Fine-Tuning)初始化元规划器,再基于执行反馈采用直接偏好优化(Direct Preference Optimization)进行精调。在源自EarthBench与ThinkGeo的数据集上开展的大量实验表明,LMMP显著提升了工具调用准确率与任务成功率。该框架还展现出优异的“即插即用”通用性,能在先前未见的EO任务中持续提升多种执行器主干网络的性能。

English Original

Autonomous Earth Observation (EO) agents are transitioning from passive perception to complex, multi-step task execution. However, current architectures that integrate planning and execution within a single model often struggle with combinatorial complexity and reasoning errors in dynamic EO scenarios. To resolve these challenges, we propose the Lightweight Multimodal Meta-Planner (LMMP) framework. LMMP incorporates a dual-awareness mechanism that grounds strategic plans in both multimodal image features and high-level task semantics. Crucially, we introduce a Meta Task Library to inject remote sensing expert knowledge directly into the workflow, which standardizes domain logic and ensures plans are physically feasible. We further implement a two-stage training pipeline, initializing the Meta-Planner via expert-distilled Supervised Fine-Tuning and refining it through Direct Preference Optimization based on execution feedback. Extensive experiments on a dataset derived from EarthBench and ThinkGeo demonstrate that LMMP significantly improves tool-calling accuracy and task success rates. Moreover, the framework exhibits strong ``plug-and-play'' versatility, consistently enhancing the performance of diverse executor backbones across previously unseen EO missions.

我的阅读记录

正在加载阅读记录…

元数据
arXiv2605.04777v1
来源arXiv
类型论文
抽取状态raw
关键词
RemoteSensing
EarthObservation
Multimodal
GeoMultimodal
Agent
cs.MA