基于Swin-Transformer和时空融合注意力机制的ENSO预测

张霄智; 方巍; 王淏西

doi:10.12284/hyxb2024127

基于Swin-Transformer和时空融合注意力机制的ENSO预测

doi: 10.12284/hyxb2024127

张霄智^1,,
方巍^{1, 2, 3, ,},
王淏西¹

1.
南京信息工程大学计算机学院/软件学院，江苏南京 210044
2.
中国气象局武汉暴雨研究所中国气象局流域强降水重点开放实验室/暴雨监测预警湖北省重点实验室，湖北武汉 430205
3.
中国气象科学研究院灾害天气国家重点实验室，北京 100081

基金项目: 国家自然科学基金面上项目(No.42475149)；中国气象局流域强降水重点开放实验室开放研究基金(No.2023BHR−Y14)；南京气象科技创新研究院北极阁开放研究基金(BJG202306)；灾害天气国家重点实验室开放课题(2024LASW-B19)；江苏省研究生科研与实践创新计划项目（NO.KYCX24_1533）。

详细信息

作者简介:
张霄智（1998—），男，江苏省南通市人，研究方向为深度学习、ENSO预测。E-mail：2201083714@qq.com

通讯作者:
方巍，教授，博士生导师，研究方向为人工智能、大数据分析、机器学习和计算机视觉。E-mail：fangwei@nuist.edu.cn

中图分类号: P467
计量
- 文章访问数: 46
- HTML全文浏览量: 11
- PDF下载量: 6
- 被引次数: 0
出版历程
- 收稿日期: 2024-07-24
- 修回日期: 2024-10-30
- 刊出日期: 2024-12-06

ENSO prediction based on Swin-Transformer and spatio-temporal fusion attention mechanism

Zhang Xiaozhi^1
,,
Fang Wei^{1, 2, 3
, ,},
Wang Haoxi¹

1.
School of Computer Science & School of Software , Nanjing University of Information Science and Technology, Nanjing 210044, China
2.
China Meteorological Administration, China Meteorological Administration Basin Heavy Rainfall Key Laboratory/Hubei Key Laboratory for Heavy Rain Monitoring and Warning Research, Institute of Heavy Rain, Wuhan 430205, China
3.
State Key Laboratory of Severe Weather, Chinese Academy of Meteorological Sciences, Beijing 100081, China

摘要

摘要: 厄尔尼诺−南方涛动预测是气候变化研究的热点问题之一。本文将Swin-Transformer模型与时空融合注意力机制相结合，采用1850−2014年CMIP6多模式模拟历史数据、1871−1979年SODA同化数据和1980−2023年GODAS同化数据，构建厄尔尼诺−南方涛动预测模型，即ENSO-STformer。该模型通过在CMIP6和SODA数据集上进行充分的训练，并在GODAS数据上进行评估，结果表明：本文模型在提前11个月的Niño3.4指数相关技巧的平均值上分别比CanCM4、CCSM3、GFDLaer04动力预报系统高出5.1%、21.6%和12.4%，同时，在中长期的Niño3.4指数相关技巧上显著优于其他深度学习模型，并可以进行长达24个月的有效ENSO预测，此外，在对2015−2016年厄尔尼诺事件模拟中表现出较强的应对春季预报障碍能力。
- 深度学习 /
- ENSO预测 /
- 时空融合注意力机制 /
- 卷积神经网络 /
- Niño3.4指数
Abstract: The prediction of El Niño-Southern Oscillation is one of the hot issues in climate change research. This paper combines swin-transformer model with spatio-temporal fusion attention mechanism, and uses CMIP6 multi-model simulation historical data from 1850 to 2014, SODA assimilated data from 1871 to 1979 and GODAS assimilated data from 1980 to 2023 to construct El Niño-Southern Oscillation prediction model—ENSO-STformer. The model was fully trained on CMIP6 and SODA datasets and evaluated on GODAS data. The results show that the average skill of this model in predicting the Niño3.4 index at 11-month lead times exceeds those of CanCM4, CCSM3, and GFDLaer04 by 5.1%, 21.6%, and 12.4% respectively. Meanwhile, the Niño3.4 index related skills of the proposed model are significantly better than other deep learning models in the medium and long term. Effective ENSO forecasts can be made for up to 24 months, and the 2015−2016 El Niño event simulation shows strong ability to cope with spring forecast obstacles.
- deep learning /
- ENSO predicting /
- spatio-temporal fusion attention mechanism /
- convolutional neural network /
- Niño3.4 index

HTML全文

图 1 本文模型结构

Fig. 1 Model structure of this paper

下载: 全尺寸图片幻灯片

图 2 气候数据下采样与映射

Fig. 2 Climate data downsampling and mapping

下载: 全尺寸图片幻灯片

图 3 Niño3.4指数预测器模块

Fig. 3 Niño3.4 Index Predictor module

下载: 全尺寸图片幻灯片

图 4 ENSO-STformer、CNN^[23]、ENSOTR^[26]、GCNTR^[33]和动力预报系统CanCM4^[32]、CCSM3^[32]、GFDL-aer04^[32]与SINTEX-F^[33]的ONI相关技巧情况

Fig. 4 ONI related skills of ENSO-STformer, CNN^[23], ENSOTR^[26], GCNTR^[33], CanCM4^[32], CCSM3^[32], GFDL-aer04^[32] and SINTEX-F^[33]

下载: 全尺寸图片幻灯片

图 5 ENSO-STformer模型在GODAS数据集上不同提前期的预测效果以及对应的相关系数

Fig. 5 Prediction effect of ENSO-STformer model in different lead times and corresponding correlation coefficient on GODAS data set

下载: 全尺寸图片幻灯片

图 6 ENSO-STformer和CMC1-CanCM3在不同时期下ONI的预测值与真实值比较

实线和虚线分别表示本文模型和CMC1-CanCM3的预测结果，不同颜色表示不同提前时期的情况

Fig. 6 ONI predicted values of ENSO-STformer and CMC1-CanCM3 in different periods are compared with the real values

Solid and dashed lines represent the predicted results of the model in this paper and CMC1-CanCM3 respectively, and different colors represent the situations in different advance periods

下载: 全尺寸图片幻灯片

表 1 CMIP6数据信息

Tab. 1 CMIP6 data information

序号	模式名称	所属国家	研发机构
1	ACCESS-CM2	澳大利亚	CSIRO-ARCCSS
2	ACCESS-ESM1-5	澳大利亚	CSIRO
3	CAMS-CSM1-O	中国	CAMS
4	CanESM5-CanOE	加拿大	CCCma
5	E3SM-1-O	美国	E3SM-Project LLNL UCI
6	BCC-CSM2-HR	中国	BCC
7	FGOALS-f3-L	中国	CAS
8	FGOALS-g3	中国	CAS
9	FIO-ESM-2-0	中国	FIO-QLNM
10	AWI-CM-1-1-MR	德国	AWI
11	IPSL-CM6A-LR	法国	IPSL
12	MIROC6	日本	MIROC
13	MIROC-ES2L	日本	MIROC
14	MRI-ESM2-0	日本	MRI
15	NESM3	中国	NUIST
16	NorESM2-MM	挪威	NCC

下载: 导出CSV

表 2 SODA和GODAS数据信息

Tab. 2 Data information of SODA and GODAS

序号	再分析数据	所属国家	研发机构
1	SODA	美国	UMD
2	GODAS	美国	NCEP

下载: 导出CSV

表 3 CMIP6和SODA数据划分

Tab. 3 Data division between CMIP6 and SODA

数据集	训练样本数	验证样本数
CMIP6	25321	6331
SODA	1302	325

下载: 导出CSV

表 4 各模型评价指标对比结果

Tab. 4 Comparison results of evaluation indicators of each model

模型	RMSE ↓	MAE↓	PCC↑
Swin-Transformer	1.11	0.95	0.52
Swin(w/下采样)	0.91	0.77	0.64
ENSO-STformer	0.76	0.63	0.73
本文(w/时空融合注意力机制)	0.83	0.68	0.69
本文(w/Niño3.4指数预测器)	0.88	0.72	0.66
注：表中RMSE、MAE和PCC数据为验证集中所有样本在相应模型预测下的平均值，向下的箭头表示指标值越低模型性能越好，向上的箭头表示指标值越高模型性能越好，所加粗的数据是该指标下最优的情况。

下载: 导出CSV

参考文献(33)

[1]	石世玮, 智海, 林鹏飞, 等. 热带太平洋盐度年际变化对海表温度异常作用比较: 1997/1998、2014/2015和2015/2016年El Niño事件[J]. 大气科学, 2020, 44(5): 1057−1075. Shi Shiwei, Zhi Hai, Lin Pengfei, et al. Contrasting salinity interannual variations in the tropical pacific and their effects on recent El Niño events: 1997/1998, 2014/2015, and 2015/2016[J]. Chinese Journal of Atmospheric Sciences, 2020, 44(5): 1057−1075.
[2]	祁莉, 毛欣, 张文君. 北太平洋海温Victoria模态与ENSO年际关系的非对称特征[J]. 大气科学学报, 2022, 45(2): 280−291. Qi Li, Mao Xin, Zhang Wenjun. Asymmetric characteristics of interannual relationship between Victoria mode of North Pacific SST and ENSO[J]. Transactions of Atmospheric Sciences, 2022, 45(2): 280−291.
[3]	万云霞, 晏红明, 金燕, 等. 两次不同ENSO背景下云南冬季极端冷事件的成因分析[J]. 大气科学学报, 2023, 46(4): 575−586. Wan Yunxia, Yan Hongming, Jin Yan, et al. Causes of two extremely cold events in Yunnan during different ENSO events[J]. Transactions of Atmospheric Sciences, 2023, 46(4): 575−586.
[4]	Zhang Ronghua, Zhou Lu, Gao Chuan, et al. A transformer-based coupled ocean-atmosphere model for ENSO studies[J]. Science Bulletin, 2024, 69(15): 2323−2327. doi: 10.1016/j.scib.2024.04.048
[5]	Wang Gaige, Cheng Honglei, Zhang Yiming, et al. ENSO analysis and prediction using deep learning: a review[J]. Neurocomputing, 2023, 520: 216−229. doi: 10.1016/j.neucom.2022.11.078
[6]	Cai Wenju, Santoso A, Gollins M, et al. Changing El Niño-Southern Oscillation in a warming climate[J]. Nature Reviews Earth & Environment, 2021, 2(9): 628−644.
[7]	Anttila-Hughes J K, Jina A S, Mccord G C. ENSO impacts child undernutrition in the global tropics[J]. Nature Communications, 2021, 12(1): 5785. doi: 10.1038/s41467-021-26048-7
[8]	Cai Wenju, Ng B, Geng Tao, et al. Anthropogenic impacts on twentieth-century ENSO variability changes[J]. Nature Reviews Earth & Environment, 2023, 4(6): 407−418.
[9]	任宏利, 郑飞, 罗京佳, 等. 中国热带海−气相互作用与ENSO动力学及预测研究进展[J]. 气象学报, 2020, 78(3): 351−369. doi: 10.11676/qxxb2020.023 Ren Hongli, Zheng Fei, Luo Jingjia, et al. A review of research on tropical air-sea interaction, ENSO dynamics, and ENSO prediction in China[J]. Acta Meteorologica Sinica, 2020, 78(3): 351−369. doi: 10.11676/qxxb2020.023
[10]	Dong Changming, Xu Guangjun, Han Guoqing, et al. Recent developments in artificial intelligence in oceanography[J]. Ocean-Land-Atmosphere Research, 2022, 2022: 9870950.
[11]	Zhu Yuchao, Zhang Ronghua, Moum J N, et al. Physics-informed deep-learning parameterization of ocean vertical mixing improves climate simulations[J]. National Science Review, 2022, 9(8): nwac044.
[12]	Zheng Gang, Li Xiaofeng, Zhang Ronghua, et al. Purely satellite data-driven deep learning forecast of complicated tropical instability waves[J]. Science Advances, 2020, 6(29): eaba1482. doi: 10.1126/sciadv.aba1482
[13]	Ling Fenghua, Luo Jingjia, Li Yue, et al. Multi-task machine learning improves multi-seasonal prediction of the Indian Ocean Dipole[J]. Nature Communications, 2022, 13(1): 7681.
[14]	Zhao Sen, Jin Feifei, Stuecker M F, et al. Explainable El Niño predictability from climate mode interactions[J]. Nature, 2024, 630(8018): 891−898. doi: 10.1038/s41586-024-07534-6
[15]	黄刚, 胡开明, 唐颢苏, 等. 从能量学角度理解气候背景场对ENSO热带和热带外遥相关的影响[J]. 大气科学, 2024, 48(1): 218−227 Huang Gang, Hu Kaiming, Tang Haosu, et al. Understanding the influence of background mean-state field on ENSO tropical and extratropical teleconnection from an energetic perspective[J]. Chinese Journal of Atmospheric Sciences, 2024, 48(1): 218−227
[16]	Liu Ze, Lin Yutong, Cao Yue, et al. Swin Transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 9992−10002.
[17]	Fekadu K. Ethiopian seasonal rainfall variability and prediction using canonical correlation analysis (CCA)[J]. Earth Sciences, 2015, 4(3): 112−119. doi: 10.11648/j.earth.20150403.14
[18]	Biabanaki M, Eslamian S S, Koupai J A, et al. A principal components/singular spectrum analysis approach to ENSO and PDO influences on rainfall in western Iran[J]. Hydrology Research, 2014, 45(2): 250−262.
[19]	Zhou Lu, Zhang Ronghua. A hybrid neural network model for ENSO prediction in combination with principal oscillation pattern analyses[J]. Advances in Atmospheric Sciences, 2022, 39(6): 889−902. doi: 10.1007/s00376-021-1368-4
[20]	Kido S, Richter I, Tozuka T, et al. Understanding the interplay between ENSO and related tropical SST variability using linear inverse models[J]. Climate Dynamics, 2023, 61(3/4): 1029−1048.
[21]	杨淑贤, 零丰华, 应武杉, 等. 人工智能技术气候预测应用简介[J]. 大气科学学报, 2022, 45(5): 641−659. Yang Shuxian, Ling Fenghua, Ying Wushan, et al. A brief overview of the application of artificial intelligence to climate pre-diction[J]. Transactions of Atmospheric Sciences, 2022, 45(5): 641−659.
[22]	Fang Wei, Sha Yu, Sheng V S. Survey on the application of artificial intelligence in ENSO forecasting[J]. Mathematics, 2022, 10(20): 3793. doi: 10.3390/math10203793
[23]	Ham Y G, Kim J H, Luo Jingjia. Deep learning for multi-year ENSO forecasts[J]. Nature, 2019, 573(7775): 568−572. doi: 10.1038/s41586-019-1559-7
[24]	周佩, 黄颖婕, 胡冰逸, 等. 基于LSTM深度学习的ENSO预测及其春季预报障碍研究[J]. 北京大学学报(自然科学版), 2021, 57(6): 1071−1078. Zhou Pei, Huang Yingjie, Hu Bingyi, et al. Spring predictability barrier phenomenon in ENSO prediction model based on LSTM Deep learning algorithm[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(6): 1071−1078.
[25]	Geng Huantong, Wang Tianlei. Spatiotemporal model based on deep learning for ENSO forecasts[J]. Atmosphere, 2021, 12(7): 810. doi: 10.3390/atmos12070810
[26]	Ye Min, Nie Jie, Liu Anan, et al. Multi-year ENSO forecasts using parallel convolutional neural networks with heterogeneous architecture[J]. Frontiers in Marine Science, 2021, 8: 717184.
[27]	Rui Chuang, Sun Zhengya, Zhang Wensheng, et al. Enhancing ENSO predictions with self-attention ConvLSTM and temporal embeddings[J]. Frontiers in Marine Science, 2024, 11: 1334210. doi: 10.3389/fmars.2024.1334210
[28]	Lu Yunlong, FENG Junqiao, HU Dunxin. CMIP6 models simulation of the connection between North/South Pacific Meridional Mode and ENSO[J]. Journal of Oceanology and Limnology, 2024, 42(2): 439−453. doi: 10.1007/s00343-023-3024-6
[29]	Lyu Pumeng, Tang Tao, Ling Fenghua, et al. ResoNet: robust and explainable ENSO forecasts with hybrid convolution and transformer networks[J]. Advances in Atmospheric Sciences, 2024, 41(7): 1289−1298. doi: 10.1007/s00376-024-3316-6
[30]	Behringer D, Xue Yan. Evaluation of the global ocean data assimilation system at NCEP: the Pacific Ocean[C]//Eighth Symposium on Integrated Observing and Assimilation Systems for Atmosphere, Oceans, and LandSurface, AMS 84th Annual Meeting. Seattle, 2004.
[31]	Ouyang Daliang, He Su, Zhang Guozhong, et al. Efficient multi-scale attention module with cross-spatial learning[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Rhodes Island: IEEE, 2023: 1−5.
[32]	Kirtman B P, Min D, Infanti J M, et al. The North American multimodel ensemble: phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction[J]. Bulletin of the American Meteorological Society, 2014, 95(4): 585−601.
[33]	杜先君, 李河. 基于集成GCN-Transformer网络的ENSO预测模型[J]. 海洋学报, 2023, 45(12): 156−165. Du Xianjun, Li He. ENSO prediction model based on integrated GCN-Transformer network[J]. Haiyang Xuebao, 2023, 45(12): 156−165.