Sea ice segmentation on optical images empowered by multi-dimensional attention within a U-Net architecture
-
摘要: 本文基于Sentinel-2光学遥感影像，提出一种U-Net架构下多维度注意力赋能的海冰分割算法，该算法在经典U-Net的基础上，创新性地在编码路径末端引入时间感知多头注意力模块，通过可学习的空间位置编码增强空间感知，并以时间编码（年份采用最小−最大归一化处理，月、日通过正弦余弦函数编码）为查询向量，对深层图像特征进行全局时间关联推理，并在解码路径中嵌入轻量化三重注意力模块，即通道−空间−时间，计算三者权重，以逐像素乘积形式融合特征信息，有效增强关键特征，聚焦细节。为验证本文算法的准确性和有效性，选取经典VIT、DeepLabV3+、U-Net模型为对照方法，并进行消融试验。试验结果表明，本文算法在OA、Kappa系数、Mean IoU系数指标上最优，分别为92.11%、0.846和0.574。两个注意力模块的联合作用使得模型在避免全局偏差的同时提升局部分类置信度，特别是将30%～50%冰密集度与固定冰的分类正确率分别提升了48.8%与31.95%。Abstract: Based on Sentinel-2 optical remote sensing imagery, this paper proposes a sea ice segmentation algorithm empowered by multi-dimensional attention within a U-Net architecture. Building upon the classical U-Net, the algorithm innovatively introduces a temporal-aware multi-head attention module at the end of the encoder path. This module enhances spatial perception using learnable spatial positional encodings and utilizes temporal encodings (where the year is processed by min-max normalization, and the month and day are encoded via sine-cosine functions) as query vectors to perform global temporal correlation reasoning on deep image features. Furthermore, a lightweight triple attention module (channel-spatial-temporal) is embedded within the decoder path. This module calculates the weights for these three dimensions and fuses feature information via element-wise multiplication, effectively enhancing key features and focusing on details. To validate the accuracy and effectiveness of the proposed algorithm, classical VIT, DeepLabV3+ and U-Net models were selected as comparative methods, and ablation studies were conducted. Experimental results demonstrate that the proposed algorithm achieves the best performance in terms of OA (Overall Accuracy), Kappa coefficient, and Mean IoU (Intersection over Union) coefficients, reaching 92.11%, 0.846, and 0.574, respectively. The combined effect of the two attention modules enables the model to avoid global bias while improving local classification confidence. Notably, the classification accuracy for 30%−50% ice concentration and fast ice was significantly improved by 48.8% and 31.95%, respectively.
-
Key words:
- northeast passage /
- sea ice segmentation /
- multi-dimensional attention /
- U-Net
-
图 1 17景Sentinel-2影像足迹（a）；同景不同时间卫星图像（b, c）
Fig. 1 The footprint of 17 scenes of Sentinel-2 imagery (a); S2 images of the same scene at different times (b, c)
图 2 数据集标签类别像素级分布
横坐标数字代表不同类别冰密集度，其具体定义为：0：＜10%冰密集度, 1：10%～30%冰密集度, 2：30%～50%冰密集度, 3：50%～70%冰密集度, 4：70%～90%冰密集度, 5：90%～100%冰密集度, 6：固定冰，7：陆地
Fig. 2 Pixel-level distribution of label categories in the dataset
The numbers on the horizontal axis represent sea ice concentration category, defined as follows: 0: ice concentration ＜10%, 1: 10%−30%, 2: 30%−50%, 3: 50%−70%, 4: 70%−90%, 5: 90%−100%, 6: fast ice, 7: land
图 3 基于Sentinel-2海冰样本构建标签示例
Fig. 3 Examples of labels constructed from Sentinel-2 sea ice samples
图 4 U-Net架构下多维度注意力赋能的海冰分割模型
Fig. 4 Multi-dimensional attention-empowered sea ice segmentation model under U-Net architecture
图 6 消融试验模型混淆矩阵
横纵坐标数字代表不同类别冰密集度，其具体定义为：0：＜10%冰密集度, 1：10%～30%冰密集度, 2：30%～50%冰密集度, 3：50%～70%冰密集度, 4：70%～90%冰密集度, 5：90%～100%冰密集度, 6：固定冰，7：陆地
Fig. 6 Confusion matrix of ablation test models
The numbers on the horizontal axis represent sea ice concentration category, defined as follows: 0: ice concentration ＜10%, 1: 10%−30%, 2: 30%−50%, 3: 50%−70%, 4: 70%−90%, 5: 90%−100%, 6: fast ice, 7: land
图 7 各模型针对类别2识别情况对比
Fig. 7 Comparison of category 2 recognition performance across models
图 8 各模型针对类别6识别情况对比
Fig. 8 Comparison of category 6 recognition performance across models
表 1 用于定义标签的冰密集度
Tab. 1 Ice concentration used to define labels
定义 类别编码 浓度掩膜 无冰 55 0: <10%冰密集度 海冰密集度不足1/10（开阔水域） 01 碎浮冰水域 02 1/10 10 1: 10%～30%冰密集度 1/10～2/10 12 1/10～3/10 13 2/10 20 2/10～3/10 23 2/10～4/10 24 2: 30%～50%冰密集度 3/10 30 3/10～4/10 34 3/10～5/10 35 4/10 40 4/10～5/10 45 4/10～6/10 46 3: 50%～70%冰密集度 5/10 50 5/10～6/10 56 5/10～7/10 57 6/10 60 6/10～7/10 67 6/10～8/10 68 4: 70%～90%冰密集度 7/10 70 7/10～8/10 78 7/10～9/10 79 8/10 80 8/10～9/10 89 8/10～10/10 81 5: 90%～100%冰密集度 9/10 90 9/10～10/10 或 (9+)/10 91 10/10 92 6: 固定冰
（附着于海岸线的厚冰）陆地 100 7: 陆地 未确定/未知 99 
下载: 导出CSV
表 2 所有比较模型的评估结果
Tab. 2 Evaluation results of all compared models
模型 OA/% Mean IoU Kappa系数 参数量 训练时间/h 本文模型 92.11 0.574 0.846 14660232 3.58 U-Net 90.92 0.455 0.823 8630728 1.136 VIT 84.30 0.409 0.724 1266472 2.67 DeepLabV3+ 86.80 0.481 0.806 17003112 4.42
下载: 导出CSV
表 3 消融试验结果精度
Tab. 3 Accuracy of ablation test results
模型 TA LWA OA/% Mean IoU Kappa G U-Net × × 90.92 0.455 0.823 - U-Net-TA √ × 91.14 0.460 0.827 13.36% U-Net-LWA × √ 85.80 0.440 0.792 / 本文模型 √ √ 92.11 0.574 0.846 86.64% 注：“×”表示未加入该模块；“√”表示加入该模块；“-”表示基准模型（U-Net）自身比较；“/”表示模型性能未超越基准，贡献度不适用。
下载: 导出CSV
-
[1] National Snow and Ice Data Center. Arctic Sea ice decline stalls out at second lowest minimum[EB/OL]. Arctic Sea Ice News & Analysis(2020−09−21)[2025−06−07].https://nsidc.org/sea-ice-today/analyses/arctic-sea-ice-decline-stalls-out-second-lowest-minimum. [2] Barnhart K R, Miller C R, Overeem I, et al. Mapping the future expansion of Arctic open water[J]. Nature Climate Change, 2016, 6(3): 280−285. doi: 10.1038/nclimate2848 [3] Wu A, Che Tao, Xu Qingchao, et al. Assessing the economic viability of the Arctic Northeast Passage from 2021 to 2065[J]. International Journal of Digital Earth, 2024, 17(1): 2323182. doi: 10.1080/17538947.2024.2323182 [4] Iqrah J M, Koo Y, Wang Wei, et al. Toward polar sea-ice classification using color-based segmentation and auto-labeling of sentinel-2 imagery to train an efficient deep learning model[EB/OL]. (2023−03−08)[2025−06−08]. https://arxiv.org/abs/2303.12719. [5] Meraner A, Ebel P, Zhu Xiaoxiang, et al. Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 333−346. doi: 10.1016/j.isprsjprs.2020.05.013 [6] Haverkamp D, Soh L K, Tsatsoulis C. A dynamic local thresholding technique for sea ice classification[C]//Proceedings of IGARSS’93-IEEE International Geoscience and Remote Sensing Symposium. Tokyo: IEEE, 1993: 638−640. [7] Stofa M M, Abdani S R, Moubark A M, et al. Recent developments of artificial intelligence methods for sea ice concentration monitoring using high-resolution imaging datasets[J]. Ecological Informatics, 2025, 87: 103132. doi: 10.1016/j.ecoinf.2025.103132 [8] 孙绍哲, 邝慧妍, 叶玉芳, 等. 基于MODIS影像的北极冰水分类方法对比研究[J]. 极地研究, 2025, 37(1): 39−54.Sun Shaozhe, Kuang Huiyan, Ye Yufang, et al. Comparative study of five methods for Arctic sea ice and water classification based on MODIS images[J]. Chinese Journal of Polar Research, 2025, 37(1): 39−54. [9] Tan Weikai, Li J, Xu Linlin, et al. Semiautomated segmentation of Sentinel-1 SAR imagery for mapping sea ice in Labrador coast[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(5): 1419−1432. doi: 10.1109/JSTARS.2018.2806640 [10] 王志勇, 孙培蕾, 刘健. 一种联合多特征的极化SAR海冰类型提取方法[J]. 遥感信息, 2020, 35(4): 23−29.Wang Zhiyong, Sun Peilei, Liu Jian. A sea ice classification method of polarimetric SAR data by multi-feature combination[J]. Remote Sensing Information, 2020, 35(4): 23−29. [11] Ji Wei, Fang Zhou, Feng Decai, et al. Semantic segmentation of arctic sea ice in summer from remote sensing satellite images based on BAU-NET[J]. Journal of Applied Remote Sensing, 2022, 16(4): 046514. [12] Ren Yibin, Xu Huan, Liu Bin, et al. Sea ice and open water classification of SAR images using a deep learning model[C]//IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium. Waikoloa: IEEE, 2020: 3051−3054. [13] 郑付强, 匡定波, 胡勇, 等. 基于U-ASPP-Net的北极独立海冰精细识别方法[J]. 红外与毫米波学报, 2021, 40(6): 798−808.Zheng Fuqiang, Kuang Dingbo, Hu Yong, et al. Refined segmentation method based on U-ASPP-Net for Arctic independent sea ice[J]. Journal of Infrared and Millimeter Waves, 2021, 40(6): 798−808. [14] 杨浩. 基于Res-Unet模型的辽东湾遥感影像海冰灾害风险分析[D]. 大连: 大连理工大学, 2024.Yang Hao. Analysis of sea ice disaster risk in the Liaodong Bay remote sensing images based on Res-UNet model[D]. Dalian: Dalian University of Technology, 2024. [15] Liu Ze, Lin Yutong, Cao Yue, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 9992−10002. [16] 邵志远, 赵杰臣, 解龙翔, 等. 基于深度学习和Sentinel-1卫星影像的北极海冰分类精度和影响因素[J]. 海洋学研究, 2024, 42(3): 119−130.Shao Zhiyuan, Zhao Jiechen, Xie Longxiang, et al. Classification accuracy and influencing factors of Arctic sea ice based on deep learning and Sentinel-1 satellite imagery[J]. Journal of Marine Sciences, 2024, 42(3): 119−130. [17] Borisova J, Kuznetsov A, Solovev G, et al. Understanding the limitations of deep transformer models for sea ice forecasting[C]//25th International Conference on Computational Science. Singapore: Springer, 2025: 104−118. [18] Ren Yibin, Li Xiaofeng, Yang Xiaofeng, et al. Development of a dual-attention U-Net model for sea ice and open water classification on SAR images[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 4010205. [19] Petty A A, Bagnardi M, Kurtz N T, et al. Assessment of ICESat-2 sea ice surface classification with Sentinel-2 imagery: implications for freeboard and new estimates of lead and floe geometry[J]. Earth and Space Science, 2021, 8(3): e2020EA001491. doi: 10.1029/2020EA001491 [20] 郑付强, 匡定波, 胡勇, 等. 基于Multiloss-SAM-ConvLSTM的北极航道独立海冰运动预测[J]. 红外与毫米波学报, 2022, 41(5): 894−904.Zheng Fuqiang, Kuang Dingbo, Hu Yong, et al. Prediction of independent sea ice motion in Arctic channel based on Multiloss-SAM-ConvLSTM[J]. Journal of Infrared and Millimeter Waves, 2022, 41(5): 894−904. [21] National Snow and Ice Data Center. U. S. national ice center arctic and Antarctic sea ice charts in SIGRID-3 format, Version 1[EB/OL]. (2023−02)[2024−12−06]. https://nsidc.org/data/g10013/versions/1. DOI: 10.7265/4B7S-RN93. [22] World Meteorological Organization (WMO). WMO Sea-Ice Nomenclature[EB/OL]. WMO-No. 259(2014−03)[2024−12−06]. https://library.wmo.int/zh/records/item/41953-wmo-sea-ice-nomenclature?offset=50. [23] Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation[C]//18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234−241. [24] Çiçek Ö, Abdulkadir A, Lienkamp S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation[C]//19th International Conference on Medical Image Computing and Computer-Assisted Intervention. Athens: Springer, 2016: 424−432. [25] 孙晓宇, 孙启振, 沈辉, 等. 1979−2022年北极海冰范围变化特征及趋势分析[J]. 海洋预报, 2023, 40(5): 49−55.Sun Xiaoyu, Sun Qizhen, Shen Hui, et al. Analysis of characteristics and trends of Arctic sea ice extent changes from 1979 to 2022[J]. Marine Forecasts, 2023, 40(5): 49−55. [26] 苏海洋. 基于深度学习的北极海冰密集度降尺度分析预报方法研究[D]. 哈尔滨: 哈尔滨工程大学, 2024.Su Haiyang. Arctic sea ice concentration downscaling analysis and prediction based on deep learning[D]. Harbin: Harbin Engineering University, 2024. [27] 魏立新, 张占海. 北极海冰变化特征分析[J]. 海洋预报, 2007, 24(4): 42−48.Wei Lixin, Zhang Zhanhai. Analysis of Arctic sea ice variability[J]. Marine Forecasts, 2007, 24(4): 42−48. [28] Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: transformers for image recognition at scale[C]//International Conference on Learning Representations. Vienna: OpenReview. net, 2021. -
计量
- 文章访问数: 108
- HTML全文浏览量: 60
- PDF下载量: 9
- 被引次数: 0
