论文
arXiv
GeoAI
GIS
RemoteSensing
EarthObservation
SpatialIntelligence
LLM
GeoLargeModel
GeoFoundationModel
Multimodal
GeoMultimodal
中文标题
LISAT:面向卫星影像的语言引导分割助手
English Title
LISAT: Language-Instructed Segmentation Assistant for Satellite Imagery
Jerome Quenum, Wen-Han Hsieh, Tsung-Han Wu, Ritwik Gupta, Trevor Darrell, David M. Chan
发布时间
2025/5/6 01:56:25
来源类型
preprint
语言
en
摘要
中文对照

分割模型能够识别图像中预定义的一组对象。然而,能够对隐含涉及多个感兴趣对象的复杂用户查询进行推理的模型仍处于发展初期。近期在推理分割领域的进展——从复杂的、隐含的查询文本生成分割掩码——表明视觉-语言模型可在开放域中运行并生成合理输出。然而,我们的实验表明,此类模型在处理复杂遥感影像时表现不佳。本文提出LISAt,一种专为描述复杂遥感场景、回答相关问题以及分割感兴趣对象而设计的视觉-语言模型。我们在一个新构建的地理空间推理-分割数据集GRES上训练LISAt,该数据集包含9,205张图像上的27,615个标注,以及一个包含超过一百万个问答对的多模态预训练数据集PreGRES。在遥感图像描述任务中,LISAt相较于现有地理空间基础模型RS-GPT4V的BLEU-4指标提升超过10.04%;在推理分割任务中,相较于当前最先进的开放域模型,gIoU指标提升达143.36%。我们的模型、数据集及代码已公开于https://lisat-bair.github.io/LISAt/

English Original

Segmentation models can recognize a pre-defined set of objects in images. However, models that can reason over complex user queries that implicitly refer to multiple objects of interest are still in their infancy. Recent advances in reasoning segmentation--generating segmentation masks from complex, implicit query text--demonstrate that vision-language models can operate across an open domain and produce reasonable outputs. However, our experiments show that such models struggle with complex remote-sensing imagery. In this work, we introduce LISAt, a vision-language model designed to describe complex remote-sensing scenes, answer questions about them, and segment objects of interest. We trained LISAt on a new curated geospatial reasoning-segmentation dataset, GRES, with 27,615 annotations over 9,205 images, and a multimodal pretraining dataset, PreGRES, containing over 1 million question-answer pairs. LISAt outperforms existing geospatial foundation models such as RS-GPT4V by over 10.04 % (BLEU-4) on remote-sensing description tasks, and surpasses state-of-the-art open-domain models on reasoning segmentation tasks by 143.36 % (gIoU). Our model, datasets, and code are available at https://lisat-bair.github.io/LISAt/

元数据
arXiv2505.02829v1
来源arXiv
类型论文
抽取状态raw
关键词
GeoAI
GIS
RemoteSensing
EarthObservation
SpatialIntelligence
LLM
GeoLargeModel
GeoFoundationModel
Multimodal
GeoMultimodal
cs.AI