论文
arXiv
RemoteSensing
EarthObservation
LLM
Multimodal
GeoMultimodal
中文标题
先填补,后进阶:面向场景专用遥感多模态大语言模型的能力缺口驱动式后训练
English Title
Filling Before Advancing: Capability-Gap-Driven Post-Training for Scenario-Specialized Remote Sensing MLLMs
Yuheng Zong, Minghua Wang, Xin Zhao, Zhi-Hui Zhan, Antonio Plaza, Jon Atli Benediktsson
发布时间
2026/7/24 19:21:25
来源类型
preprint
语言
en
摘要
中文对照

遥感多模态大语言模型(RS-MLLMs)已提升通用航拍图像理解能力。然而,地球观测应用需细粒度的场景专用能力,受限于高质量场景数据稀缺及能力覆盖不全。我们将该适配问题建模为能力缺口驱动的后训练任务,并提出“先填补,后进阶”(FBA)方法。FBA 不依赖于在目标领域样本上进行单阶段监督微调(SFT),而是先填补前置能力缺口,再推进至场景专用化。我们以海岸港口理解这一典型多源场景为例实例化 FBA,构建三层监督数据集 CPRS(Coastal-Port Remote Sensing),并设计三个有序阶段:(1)遥感语义锚定,实现俯视视角下的视觉-语言对齐;(2)领域桥接收敛,在不同模态下使目标场景与桥接场景共享遥感先验知识;(3)证据支撑的场景调优,提升下游任务性能。我们构建 HarborEval——一个涵盖感知、空间理解、鲁棒性与生成能力的八轨诊断基准。在相近训练预算下,HarborEval 分数在 LLaVA-v1.5 上由 Direct-SFT 的 57.95 提升至 FBA 的 70.29,在 Qwen3-VL 上由 81.09 提升至 83.37。FBA 同样优于 Collapsed-SFT,并在港口相关 VRSBench/RSVQA 子集及 OpenEval 上取得领先。分阶段与角色替换分析验证了能力缺口的渐进式填补及各阶段特定角色的有效性。CPRS、HarborEval、代码及训练权重的公开示例与更新发布于 https://github.com/Z0ngL1ng/filling-before-advancing。

English Original

Remote sensing multimodal large language models (RS-MLLMs) have improved general aerial-image understanding. However, Earth observation applications require fine-grained scenario specialization, constrained by scarce high-quality scenario data and incomplete capability coverage. We formulate this adaptation as a capability-gap-driven post-training problem and propose filling before advancing (FBA). Rather than relying on single-stage supervised fine-tuning (SFT) over target-domain samples, FBA first fills prerequisite capability gaps before advancing toward scenario specialization. We instantiate FBA for coastal harbor understanding, a representative multi-source scenario, by constructing CPRS (Coastal-Port Remote Sensing), a three-layer supervision dataset coupled with three ordered stages: (1) RS semantic anchoring for overhead-view visual-language alignment; (2) domain-bridge convergence for shared RS priors across target and bridging scenarios under different modalities; and (3) evidence-grounded scenario tuning for downstream performance. We construct HarborEval, an eight-track diagnostic benchmark covering perception, spatial understanding, robustness, and generation. Under comparable training budgets, HarborEval increases from 57.95 with Direct-SFT to 70.29 with FBA on LLaVA-v1.5, and from 81.09 to 83.37 on Qwen3-VL. FBA also outperforms Collapsed-SFT and leads on harbor-related VRSBench/RSVQA subsets and OpenEval. Stage-wise and role-replacement analyses validate progressive gap filling and stage-specific roles. Public examples and release updates for CPRS, HarborEval, code, and trained weights are available at https://github.com/Z0ngL1ng/filling-before-advancing.

我的阅读记录

正在加载阅读记录…

元数据
arXiv2607.22205v2
来源arXiv
类型论文
抽取状态raw
关键词
RemoteSensing
EarthObservation
LLM
Multimodal
GeoMultimodal
cs.CV
cs.AI