结合SAM视觉分割模型与随机森林机器学习的无人机影像盐沼植被“精灵圈”提取

周若彤; 谭凯; 杨建儒; 韩江涛; 张卫国

doi:10.12284/hyxb2024048

结合SAM视觉分割模型与随机森林机器学习的无人机影像盐沼植被“精灵圈”提取

doi: 10.12284/hyxb2024048

华东师范大学河口海岸学国家重点实验室，上海 200241

基金项目: 国家自然科学基金项目（4217010220,41901399）；上海市自然科学基金项目（22ZR1420900）；重庆市自然科学基金项目（CSTB2022NSCQ-MSX1254）；测绘遥感信息工程湖南省重点实验室开放基金项目（E22335）；上海市科委社发研究项目（20DZ1204700）。

详细信息

作者简介:
周若彤（2000—），女，山东省菏泽市人，主要从事海岸带遥感研究。E-mail：51253904045@stu.ecnu.edu.cn

通讯作者:
谭凯（1987—），男，湖南省娄底市人，副研究员，主要从事海岸带多源遥感及应用研究。E-mail：ktan@sklec.ecnu.edu.cn

中图分类号: P237
计量
- 文章访问数: 57
- HTML全文浏览量: 41
- PDF下载量: 16
- 被引次数: 0
出版历程
- 收稿日期: 2023-09-29
- 录用日期: 2024-03-01
- 修回日期: 2023-12-28
- 网络出版日期: 2024-03-11

Extraction of salt-marsh vegetation "fairy circles" from UAV images by the combination of SAM visual segmentation model and random forest machine learning algorithm

State Key Laboratory of Estuarine and Coastal Research, East China Normal University, Shanghai 200241, China

摘要

摘要: “精灵圈”是海岸带盐沼植被生态系统中的一种“空间自组织”结构，对盐沼湿地的生产力、稳定性和恢复力有重要影响。无人机影像是实现“精灵圈”空间位置高精度识别及解译其时空演化趋势与规律的重要数据源，但“精灵圈”像素与背景像素在色彩信息和外形特征上差异较小，如何从二维影像中智能精准地识别“精灵圈”像素并对识别的单个像素形成个体“精灵圈”是目前的技术难点。本文提出了一种结合分割万物模型（Segment Anything Model，SAM）视觉分割模型与随机森林机器学习的无人机影像“精灵圈”分割及分类方法，实现了单个“精灵圈”的识别和提取。首先，通过构建索伦森-骰子系数（Sørensen-Dice coefficient，Dice）和交并比（Intersection over Union，IOU）评价指标，从SAM中筛选预训练模型并对其参数进行优化，实现全自动影像分割，得到无属性信息的分割掩码/分割类；然后，利用红、绿、蓝（RGB）三通道信息及空间二维坐标将分割掩码与原图像进行信息匹配，构造分割掩码的特征指标，并根据袋外数据（Out of Bag，OOB）误差减少及特征分布规律对特征进行分析和筛选；最后，利用筛选的特征对随机森林模型进行训练，实现“精灵圈”植被、普通植被和光滩的自动识别与分类。实验结果表明：本文方法“精灵圈”平均正确提取率96.1%，平均错误提取率为9.5%，为精准刻画“精灵圈”时空格局及海岸带无人机遥感图像处理提供了方法和技术支撑。
- 盐沼植被 /
- 精灵圈 /
- segment anything model (SAM) /
- 无人机影像 /
- 机器学习
Abstract: The “fairy circle” represents a unique form of spatial self-organization found within coastal salt marsh ecosystems, profoundly influencing the productivity, stability, and resilience of these wetlands. Unmanned Aerial Vehicle (UAV) imagery plays a pivotal role in precisely pinpointing the “fairy circle” locations and deciphering their temporal and spatial development trends. However, identifying “fairy circle” pixels within two-dimensional images poses a considerable technical challenge due to the subtle differences in color and shape characteristics between these pixels and their surroundings. Therefore, intelligently and accurately identify “fairy circle” pixels from two-dimensional images and form individual “fairy circle” for the identified pixels are the current technical difficulties. This paper introduces an innovative approach to extract “fairy circle” from UAV images by integrating the SAM (Segment Anything Model) visual segmentation model with random forest machine learning. This novel method accomplishes the recognition and extraction of individual “fairy circle” through a two-step process: segmentation followed by classification. Initially, we establish Dice (Sørensen-Dice coefficient) and IOU (Intersection Over Union) evaluation metrics, and optimize SAM’s pre-trained model parameters, which produces segmentation mask devoid of attribute information by fully automated image segmentation. Subsequently, we align the segmentation mask with the original image, and utilizes RGB (red, green, and blue) color channels and spatial coordinates to construct a feature index for the segmentation mask. These features undergo analysis and selection based on Out-of-Bag (OOB) error reduction and feature distribution patterns. Ultimately, the refined features are employed to train a random forest model, enabling the automatic identification and classification of “fairy circle” vegetation, common vegetation, and bare flat areas. The experimental results show that the average correct extraction rate of "fairy circle" is 96.1%, and the average wrong extraction rate is 9.5%, which provides methodological and technological support for the accurate depiction of the spatial and temporal pattern of "fairy circle" as well as the processing of coastal remote sensing images by UAVs.
- salt marsh vegetation /
- fairy circle /
- segment anything model (SAM) /
- unmanned Aerial Vehicle images /
- machine learning

HTML全文

图 1 研究区域概况

Fig. 1 Overview of the study area

下载: 全尺寸图片幻灯片

图 2 I区域正射影像（a）、整个研究区域正射影像（b）和II区域正射影像（c）

Fig. 2 Orthophoto of Region I (a), orthophoto of the entire study area (b), and orthophoto of Region II (c)

下载: 全尺寸图片幻灯片

图 3 无人机摄像系统（a）和数据采集飞行路线（b）

Fig. 3 UAV camera system (a) and data collection flight path (b)

下载: 全尺寸图片幻灯片

图 4 研究方法流程

Fig. 4 Research method flow

下载: 全尺寸图片幻灯片

图 5 ViT 3种模型变体的IOU与Dice箱式图

Fig. 5 Box chart of IOU and Dice for the three variants of ViT model

下载: 全尺寸图片幻灯片

图 6 特征指标重要性情况

Fig. 6 Importance of the selected indicators

下载: 全尺寸图片幻灯片

图 7 位置与形态指标分布（a, b）和RGB指数指标分布（c, d）

横坐标为样本序号，纵坐标为特征值，红色星号代表掩码的类型（从左到右分别为“精灵圈”植被、普通植被和光滩），彩色折线代表特征值的分布情况

Fig. 7 Distributions of position and shape indexes (a, b), and distributions of RGB indexes (c, d)

Where the horizontal coordinate is the sample serial number, the vertical coordinate is the feature index value, the red asterisk represents the type of mask (from left to right are "fairy circle" vegetation, common vegetation, and bare mudflat), and the color line represents the distribution of feature value

下载: 全尺寸图片幻灯片

图 8 研究区I提取结果

a. 原始正射影像；b. SAM分割结果RGB显示；c. RF分类结果；d. “精灵圈”提取结果叠加原始正射影像；e. 删除特征bbox-x0、bbox-y0、IKAW和MVARI 4个特征前“精灵圈”提取结果

Fig. 8 Extraction results of Region I

a. Original orthophoto; b. RGB display of SAM segmentation results; c. random forest classification result; d. “ fairy circle” extraction result superimposed on original orthophoto; e. "fairy circle" extraction result superimposed on original orthophoto before removing the features of bbox-x0, bbox-y0, IKAW, and MVARI

下载: 全尺寸图片幻灯片

图 9 研究区II提取结果

Fig. 9 Extraction results of Region II

下载: 全尺寸图片幻灯片

表 1 RGB植被指数

Tab. 1 RGB vegetation index

指数	公式
EXG	2 × G − R − B
GCC or Green Ratio	G/B + G + R
GRVI	(G − R)/(G + R)
IKAW	(R − B)/(R + B)
MGRVI	(G² − R²)/(G² + R²)
MVARI	(G − B)/(G + R − B)
RGBVI	(G² − B × R)/(G² + B × R)
TGI	G − (0.39 × R) − (0.61 × B)
VARI	(G − R)/(G + R − B)
VDVI or GLA	(2 × G − R − B)/(2 × G + R + B)

下载: 导出CSV

表 2 “精灵圈”提取混淆矩阵，类别1、2、3分别代表“精灵圈”、背景植被和光滩

Tab. 2 “ Fairy circle” extraction confusion matrix, and categories 1, 2 and 3 represent fairy circle, background vegetation, and bare flat, respectively

研究区I（13个特征）				研究区II（13个特征）
实际类别	预测类别			实际类别	预测类别
实际类别	1	2	3	实际类别	1	2	3
1	172	11	1	1	359	4	1
2	27	157	5	2	18	35	0
3	1	1	112	3	1	1	39
正确识别率	93.5%			正确识别率	98.6%
错误识别率	14%			错误识别率	5.0%
研究区I（17个特征）				研究区II（17个特征）
实际类别	预测类别			实际类别	预测类别
实际类别	1	2	1	实际类别	1	2	3
1	170	14	0	1	354	4	1
2	35	151	3	2	31	35	0
3	3	1	110	3	3	1	39
正确识别率	92.4%			正确识别率	97.3%
错误识别率	18.3%			错误识别率	8.8%

下载: 导出CSV

参考文献(28)

[1]	韩倩倩, 牛振国, 吴孟泉, 等. 基于潮位校正的中国潮间带遥感监测及变化[J]. 科学通报, 2019, 64(4): 456−473. doi: 10.1360/N972018-00723 Han Qianqian, Niu Zhenguo, Wu Mengquan, et al. Remote-sensing monitoring and analysis of China intertidal zone changes based on tidal correction[J]. Chinese Science Bulletin, 2019, 64(4): 456−473. doi: 10.1360/N972018-00723
[2]	陈一宁, 陈鹭真. 滨海蓝碳生态系统的碳库间相互作用研究进展及展望[J]. 海洋学研究, 2023, 41(1): 3−13. Chen Yining, Chen Luzhen. Interactions between vegetation and sediment carbon pools within coastal blue carbon ecosystems: a review and perspective[J]. Journal of Marine Sciences, 2023, 41(1): 3−13.
[3]	Zhao Lixia, Zhang Kang, Siteur K, et al. Fairy circles reveal the resilience of self-organized salt marshes[J]. Science Advances, 2021, 7(6): eabe1100. doi: 10.1126/sciadv.abe1100
[4]	Rietkerk M, Bastiaansen R, Banerjee S, et al. Evasion of tipping in complex systems through spatial pattern formation[J]. Science, 2021, 374(6564): eabj0359. doi: 10.1126/science.abj0359
[5]	Ruiz-Reynés D, Gomila D, Sintes T, et al. Fairy circle landscapes under the sea[J]. Science Advances, 2017, 3(8): e1603262. doi: 10.1126/sciadv.1603262
[6]	de Paoli H, van der Heide T, van den Berg A, et al. Behavioral self-organization underlies the resilience of a coastal ecosystem[J]. Proceedings of the National Academy of Sciences of the United States of America, 2017, 114(30): 8035−8040.
[7]	Tao Pengjie, Tan Kai, Ke Tao, et al. Recognition of ecological vegetation fairy circles in intertidal salt marshes from UAV LiDAR point clouds[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 114: 103029. doi: 10.1016/j.jag.2022.103029
[8]	Liu Qingsheng, Huang Chong, Liu Gaohuan, et al. Comparison of CBERS-04, GF-1, and GF-2 satellite panchromatic images for mapping quasi-circular vegetation patches in the Yellow River Delta, China[J]. Sensors, 2018, 18(8): 2733. doi: 10.3390/s18082733
[9]	Shi Lei, Liu Qingsheng, Huang Chong, et al. Mapping quasi-circular vegetation patch dynamics in the Yellow River Delta, China, between 1994 and 2016[J]. Ecological Indicators, 2021, 126: 107656. doi: 10.1016/j.ecolind.2021.107656
[10]	Zhang Yunjie, Liu Qingsheng, Liu Gaohuan, et al. Mapping of circular or elliptical vegetation community patches: a comparative use of SPOT-5, ALOS and ZY-3 imagery[C]//Proceedings of the 8th International Congress on Image and Signal Processing. Shenyang: IEEE, 2015.
[11]	Zhang Xianlong, Zhang Fei, Qi Yaxiao, et al. New research methods for vegetation information extraction based on visible light remote sensing images from an unmanned aerial vehicle (UAV)[J]. International Journal of Applied Earth Observation and Geoinformation, 2019, 78: 215−226. doi: 10.1016/j.jag.2019.01.001
[12]	Kirillov A , Mintun E , Ravi N , et al. Segment Anything[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). 0[2024-02-29]. DOI: 10.1109/ICCV51070.2023.00371.
[13]	Ning Guochen, Liang Hanyin, Jiang Zhongliang, et al. The potential of 'Segment Anything' (SAM) for universal intelligent ultrasound image guidance[J]. Bioscience Trends, 2023, 17(3): 230−233. doi: 10.5582/bst.2023.01119
[14]	Chen Fang, Chen Lingyu, Han Haojie, et al. The ability of Segmenting Anything Model (SAM) to segment ultrasound images[J]. Bioscience Trends, 2023, 17(3): 211−218. doi: 10.5582/bst.2023.01128
[15]	Shi Peilun, Qiu Jianing, Abaxi S M D, et al. Generalist vision foundation models for medical imaging: a case study of segment anything model on zero-shot medical segmentation[J]. Diagnostics, 2023, 13(11): 1947. doi: 10.3390/diagnostics13111947
[16]	Maxwell A E, Warner T A, Fang Fang. Implementation of machine-learning classification in remote sensing: an applied review[J]. International Journal of Remote Sensing, 2018, 39(9): 2784−2817. doi: 10.1080/01431161.2018.1433343
[17]	Pádua L, Adão T, Hruška J, et al. Vineyard classification using machine learning techniques applied to RGB-UAV imagery[C]//Proceedings of 2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa: IEEE, 2020.
[18]	Juel A, Groom G B, Svenning J C, et al. Spatial application of Random Forest models for fine-scale coastal vegetation classification using object based analysis of aerial orthophoto and DEM data[J]. International Journal of Applied Earth Observation and Geoinformation, 2015, 42: 106−114. doi: 10.1016/j.jag.2015.05.008
[19]	周小成, 郑磊, 黄洪宇. 基于多特征优选的无人机可见光遥感林分类型分类[J]. 林业科学, 2021, 57(6): 24−36. Zhou Xiaocheng, Zheng Lei, Huang Hongyu. Classification of forest stand based on multi-feature optimization of UAV visible light remote sensing[J]. Scientia Silvae Sinicae, 2021, 57(6): 24−36.
[20]	Yang Shuting, Gu Lingjia, Li Xiaofeng, et al. Crop classification method based on optimal feature selection and hybrid CNN-RF networks for multi-temporal remote sensing imagery[J]. Remote Sensing, 2020, 12(19): 3119. doi: 10.3390/rs12193119
[21]	Fu Bolin, Liu Man, He Hongchang, et al. Comparison of optimized object-based RF-DT algorithm and SegNet algorithm for classifying Karst wetland vegetation communities using ultra-high spatial resolution UAV data[J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 104: 102553. doi: 10.1016/j.jag.2021.102553
[22]	Han Kai, Wang Yunhe, Chen Hanting, et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(1): 87−110. doi: 10.1109/TPAMI.2022.3152247
[23]	Liaw A, Wiener M. Classification and regression by randomForest[J]. R News, 2002, 2(3): 18−22.
[24]	Breiman L. Random forests[J]. Machine Learning, 2001, 45(1): 5−32. doi: 10.1023/A:1010933404324
[25]	Rodriguez-Galiano V F, Ghimire B, Rogan J, et al. An assessment of the effectiveness of a random forest classifier for land-cover classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2012, 67: 93−104. doi: 10.1016/j.isprsjprs.2011.11.002
[26]	Nguyen M H, de la Torre F. Optimal feature selection for support vector machines[J]. Pattern Recognition, 2010, 43(3): 584−591. doi: 10.1016/j.patcog.2009.09.003
[27]	Morgan G R, Wang Cuizhen, Morris J T. RGB indices and canopy height modelling for mapping tidal marsh biomass from a small unmanned aerial system[J]. Remote Sensing, 2021, 13(17): 3406. doi: 10.3390/rs13173406
[28]	Congalton R G, Green K. Assessing the Accuracy of Remotely Sensed Data: Principles and Practices[M]. 3rd ed. Boca Raton: CRC Press, 2019.