ENSO prediction based on Swin-Transformer and spatio-temporal fusion attention mechanism

Zhang Xiaozhi; Fang Wei; Wang Haoxi

doi:10.12284/hyxb2024127

Volume 46 Issue 12

Dec. 2024

Turn off MathJax

Article Contents

Article Navigation > Haiyang Xuebao > 2024 > 46(12): 111-121

Zhang Xiaozhi，Fang Wei，Wang Haoxi. ENSO prediction based on Swin-Transformer and spatio-temporal fusion attention mechanism[J]. Haiyang Xuebao，2024, 46(12)：111–121 doi: 10.12284/hyxb2024127

Citation:

Zhang Xiaozhi，Fang Wei，Wang Haoxi. ENSO prediction based on Swin-Transformer and spatio-temporal fusion attention mechanism[J]. Haiyang Xuebao，2024, 46(12)：111–121 doi: 10.12284/hyxb2024127

Citation:

Zhang Xiaozhi，Fang Wei，Wang Haoxi. ENSO prediction based on Swin-Transformer and spatio-temporal fusion attention mechanism[J]. Haiyang Xuebao，2024, 46(12)：111–121 doi: 10.12284/hyxb2024127

PDF( 2467 KB)

ENSO prediction based on Swin-Transformer and spatio-temporal fusion attention mechanism

doi: 10.12284/hyxb2024127

Zhang Xiaozhi^1
,,
Fang Wei^{1, 2, 3
,
,},
Wang Haoxi¹

1.
School of Computer Science & School of Software , Nanjing University of Information Science and Technology, Nanjing 210044, China
2.
China Meteorological Administration, China Meteorological Administration Basin Heavy Rainfall Key Laboratory/Hubei Key Laboratory for Heavy Rain Monitoring and Warning Research, Institute of Heavy Rain, Wuhan 430205, China
3.
State Key Laboratory of Severe Weather, Chinese Academy of Meteorological Sciences, Beijing 100081, China

Received Date: 2024-07-24
Rev Recd Date: 2024-10-30
Publish Date: 2024-12-06

Abstract

Abstract

The prediction of El Niño-Southern Oscillation is one of the hot issues in climate change research. This paper combines swin-transformer model with spatio-temporal fusion attention mechanism, and uses CMIP6 multi-model simulation historical data from 1850 to 2014, SODA assimilated data from 1871 to 1979 and GODAS assimilated data from 1980 to 2023 to construct El Niño-Southern Oscillation prediction model—ENSO-STformer. The model was fully trained on CMIP6 and SODA datasets and evaluated on GODAS data. The results show that the average skill of this model in predicting the Niño3.4 index at 11-month lead times exceeds those of CanCM4, CCSM3, and GFDLaer04 by 5.1%, 21.6%, and 12.4% respectively. Meanwhile, the Niño3.4 index related skills of the proposed model are significantly better than other deep learning models in the medium and long term. Effective ENSO forecasts can be made for up to 24 months, and the 2015−2016 El Niño event simulation shows strong ability to cope with spring forecast obstacles.
- deep learning,
- ENSO predicting,
- spatio-temporal fusion attention mechanism,
- convolutional neural network,
- Niño3.4 index

FullText(HTML)

References(33)

References

[1]	石世玮, 智海, 林鹏飞, 等. 热带太平洋盐度年际变化对海表温度异常作用比较: 1997/1998、2014/2015和2015/2016年El Niño事件[J]. 大气科学, 2020, 44(5): 1057−1075. Shi Shiwei, Zhi Hai, Lin Pengfei, et al. Contrasting salinity interannual variations in the tropical pacific and their effects on recent El Niño events: 1997/1998, 2014/2015, and 2015/2016[J]. Chinese Journal of Atmospheric Sciences, 2020, 44(5): 1057−1075.
[2]	祁莉, 毛欣, 张文君. 北太平洋海温Victoria模态与ENSO年际关系的非对称特征[J]. 大气科学学报, 2022, 45(2): 280−291. Qi Li, Mao Xin, Zhang Wenjun. Asymmetric characteristics of interannual relationship between Victoria mode of North Pacific SST and ENSO[J]. Transactions of Atmospheric Sciences, 2022, 45(2): 280−291.
[3]	万云霞, 晏红明, 金燕, 等. 两次不同ENSO背景下云南冬季极端冷事件的成因分析[J]. 大气科学学报, 2023, 46(4): 575−586. Wan Yunxia, Yan Hongming, Jin Yan, et al. Causes of two extremely cold events in Yunnan during different ENSO events[J]. Transactions of Atmospheric Sciences, 2023, 46(4): 575−586.
[4]	Zhang Ronghua, Zhou Lu, Gao Chuan, et al. A transformer-based coupled ocean-atmosphere model for ENSO studies[J]. Science Bulletin, 2024, 69(15): 2323−2327. doi: 10.1016/j.scib.2024.04.048
[5]	Wang Gaige, Cheng Honglei, Zhang Yiming, et al. ENSO analysis and prediction using deep learning: a review[J]. Neurocomputing, 2023, 520: 216−229. doi: 10.1016/j.neucom.2022.11.078
[6]	Cai Wenju, Santoso A, Gollins M, et al. Changing El Niño-Southern Oscillation in a warming climate[J]. Nature Reviews Earth & Environment, 2021, 2(9): 628−644.
[7]	Anttila-Hughes J K, Jina A S, Mccord G C. ENSO impacts child undernutrition in the global tropics[J]. Nature Communications, 2021, 12(1): 5785. doi: 10.1038/s41467-021-26048-7
[8]	Cai Wenju, Ng B, Geng Tao, et al. Anthropogenic impacts on twentieth-century ENSO variability changes[J]. Nature Reviews Earth & Environment, 2023, 4(6): 407−418.
[9]	任宏利, 郑飞, 罗京佳, 等. 中国热带海−气相互作用与ENSO动力学及预测研究进展[J]. 气象学报, 2020, 78(3): 351−369. doi: 10.11676/qxxb2020.023 Ren Hongli, Zheng Fei, Luo Jingjia, et al. A review of research on tropical air-sea interaction, ENSO dynamics, and ENSO prediction in China[J]. Acta Meteorologica Sinica, 2020, 78(3): 351−369. doi: 10.11676/qxxb2020.023
[10]	Dong Changming, Xu Guangjun, Han Guoqing, et al. Recent developments in artificial intelligence in oceanography[J]. Ocean-Land-Atmosphere Research, 2022, 2022: 9870950.
[11]	Zhu Yuchao, Zhang Ronghua, Moum J N, et al. Physics-informed deep-learning parameterization of ocean vertical mixing improves climate simulations[J]. National Science Review, 2022, 9(8): nwac044.
[12]	Zheng Gang, Li Xiaofeng, Zhang Ronghua, et al. Purely satellite data-driven deep learning forecast of complicated tropical instability waves[J]. Science Advances, 2020, 6(29): eaba1482. doi: 10.1126/sciadv.aba1482
[13]	Ling Fenghua, Luo Jingjia, Li Yue, et al. Multi-task machine learning improves multi-seasonal prediction of the Indian Ocean Dipole[J]. Nature Communications, 2022, 13(1): 7681.
[14]	Zhao Sen, Jin Feifei, Stuecker M F, et al. Explainable El Niño predictability from climate mode interactions[J]. Nature, 2024, 630(8018): 891−898. doi: 10.1038/s41586-024-07534-6
[15]	黄刚, 胡开明, 唐颢苏, 等. 从能量学角度理解气候背景场对ENSO热带和热带外遥相关的影响[J]. 大气科学, 2024, 48(1): 218−227 Huang Gang, Hu Kaiming, Tang Haosu, et al. Understanding the influence of background mean-state field on ENSO tropical and extratropical teleconnection from an energetic perspective[J]. Chinese Journal of Atmospheric Sciences, 2024, 48(1): 218−227
[16]	Liu Ze, Lin Yutong, Cao Yue, et al. Swin Transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 9992−10002.
[17]	Fekadu K. Ethiopian seasonal rainfall variability and prediction using canonical correlation analysis (CCA)[J]. Earth Sciences, 2015, 4(3): 112−119. doi: 10.11648/j.earth.20150403.14
[18]	Biabanaki M, Eslamian S S, Koupai J A, et al. A principal components/singular spectrum analysis approach to ENSO and PDO influences on rainfall in western Iran[J]. Hydrology Research, 2014, 45(2): 250−262.
[19]	Zhou Lu, Zhang Ronghua. A hybrid neural network model for ENSO prediction in combination with principal oscillation pattern analyses[J]. Advances in Atmospheric Sciences, 2022, 39(6): 889−902. doi: 10.1007/s00376-021-1368-4
[20]	Kido S, Richter I, Tozuka T, et al. Understanding the interplay between ENSO and related tropical SST variability using linear inverse models[J]. Climate Dynamics, 2023, 61(3/4): 1029−1048.
[21]	杨淑贤, 零丰华, 应武杉, 等. 人工智能技术气候预测应用简介[J]. 大气科学学报, 2022, 45(5): 641−659. Yang Shuxian, Ling Fenghua, Ying Wushan, et al. A brief overview of the application of artificial intelligence to climate pre-diction[J]. Transactions of Atmospheric Sciences, 2022, 45(5): 641−659.
[22]	Fang Wei, Sha Yu, Sheng V S. Survey on the application of artificial intelligence in ENSO forecasting[J]. Mathematics, 2022, 10(20): 3793. doi: 10.3390/math10203793
[23]	Ham Y G, Kim J H, Luo Jingjia. Deep learning for multi-year ENSO forecasts[J]. Nature, 2019, 573(7775): 568−572. doi: 10.1038/s41586-019-1559-7
[24]	周佩, 黄颖婕, 胡冰逸, 等. 基于LSTM深度学习的ENSO预测及其春季预报障碍研究[J]. 北京大学学报(自然科学版), 2021, 57(6): 1071−1078. Zhou Pei, Huang Yingjie, Hu Bingyi, et al. Spring predictability barrier phenomenon in ENSO prediction model based on LSTM Deep learning algorithm[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2021, 57(6): 1071−1078.
[25]	Geng Huantong, Wang Tianlei. Spatiotemporal model based on deep learning for ENSO forecasts[J]. Atmosphere, 2021, 12(7): 810. doi: 10.3390/atmos12070810
[26]	Ye Min, Nie Jie, Liu Anan, et al. Multi-year ENSO forecasts using parallel convolutional neural networks with heterogeneous architecture[J]. Frontiers in Marine Science, 2021, 8: 717184.
[27]	Rui Chuang, Sun Zhengya, Zhang Wensheng, et al. Enhancing ENSO predictions with self-attention ConvLSTM and temporal embeddings[J]. Frontiers in Marine Science, 2024, 11: 1334210. doi: 10.3389/fmars.2024.1334210
[28]	Lu Yunlong, FENG Junqiao, HU Dunxin. CMIP6 models simulation of the connection between North/South Pacific Meridional Mode and ENSO[J]. Journal of Oceanology and Limnology, 2024, 42(2): 439−453. doi: 10.1007/s00343-023-3024-6
[29]	Lyu Pumeng, Tang Tao, Ling Fenghua, et al. ResoNet: robust and explainable ENSO forecasts with hybrid convolution and transformer networks[J]. Advances in Atmospheric Sciences, 2024, 41(7): 1289−1298. doi: 10.1007/s00376-024-3316-6
[30]	Behringer D, Xue Yan. Evaluation of the global ocean data assimilation system at NCEP: the Pacific Ocean[C]//Eighth Symposium on Integrated Observing and Assimilation Systems for Atmosphere, Oceans, and LandSurface, AMS 84th Annual Meeting. Seattle, 2004.
[31]	Ouyang Daliang, He Su, Zhang Guozhong, et al. Efficient multi-scale attention module with cross-spatial learning[C]//ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Rhodes Island: IEEE, 2023: 1−5.
[32]	Kirtman B P, Min D, Infanti J M, et al. The North American multimodel ensemble: phase-1 seasonal-to-interannual prediction; phase-2 toward developing intraseasonal prediction[J]. Bulletin of the American Meteorological Society, 2014, 95(4): 585−601.
[33]	杜先君, 李河. 基于集成GCN-Transformer网络的ENSO预测模型[J]. 海洋学报, 2023, 45(12): 156−165. Du Xianjun, Li He. ENSO prediction model based on integrated GCN-Transformer network[J]. Haiyang Xuebao, 2023, 45(12): 156−165.