Two-stage storm entity prediction based on multiscale and attention

doi:10.11996/JG.j.2095-302X.2024040696

Abstract

Abstract:

Storms are a type of natural phenomenon characterized by a short life cycle, sudden occurrence, and small spatial scale. Radar echo backpropagation methods are commonly employed for prediction.However, time series prediction models find it difficult to locate the key information of storms among numerous features, leading to low prediction accuracy. The models cannot fully learn the high-frequency information in images, resulting in missing details in the predictions and blurry results. To enhance prediction performance, we proposed a two-stage framework for single storm forecasting. In the first stage, a multi-scale module extracted multi-scale information, while an attention mechanism mined important features impacting prediction. Spatiotemporal long-term and short-term memory units were utilized for sequence prediction. The second stage performed bias correction on the results of the first stage. Frequency domain loss enriched prediction details. Experimental results showed that on the radar echo dataset, compared with the mainstream PredRNN-V2 model, the mean squared error was reduced by 11.4% and SSIM was improved by 4.3%, showing superior performance in single storm forecasting tasks. On the Moving MNIST dataset, the mean squared error was reduced by 4.95%, the perceptual loss was reduced by 12.67%, and the SSIM was improved to 0.898, demonstrating strong time series prediction capabilities.

Key words: attention mechanism, dilated convolution, frequency domain loss, long short-term memory, time series prediction

CLC Number:

WEI Min, YAO Xin. Two-stage storm entity prediction based on multiscale and attention[J]. Journal of Graphics, 2024, 45(4): 696-704.

Figures/Tables 14

References 32

[1]	金锐, 方璘王昊. 基于频谱分解和注意力机制的雷达回波外推算法[J]. 科技和产业, 2023, 23(8): 259-267.
	JIN R, FANG L W H. Radar echo extrapolation algorithm based on spectrum decomposition and spatiotemporal attention mechanism[J]. Science Technology and Industry, 2023, 23(8): 259-267 (in Chinese).
[2]	RAHAMAN N, BARATIN A, ARPIT D, et al. On the spectral bias of neural networks[EB/OL]. [2023-12-10]. http://arxiv.org/abs/1806.08734.
[3]	ZOU H B, WU S S, SHAN J S, et al. A method of radar echo extrapolation based on TREC and Barnes filter[J]. Journal of Atmospheric and Oceanic Technology, 2019, 36(9): 1713-1727. DOI
[4]	鹿天柱, 钱晓超, 何舒, 等. 一种基于深度学习的时间序列预测方法[J]. 控制与决策, 2021, 36(3): 645-652.
	LU T Z, QIAN X C, HE S, et al. A time series prediction method based on deep learning[J]. Control and Decision, 2021, 36(3): 645-652 (in Chinese).
[5]	BAI S J, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. [2023-12-10]. http://arxiv.org/abs/1803.0127.1
[6]	韩丰, 龙明盛, 李月安, 等. 循环神经网络在雷达临近预报中的应用[J]. 应用气象学报, 2019, 30(1): 61-69.
	HAN F, LONG M S, LI Y A, et al. The application of recurrent neural network to nowcasting[J]. Journal of Applied Meteorological Science, 2019, 30(1): 61-69 (in Chinese).
[7]	ZENG A L, CHEN M X, ZHANG L, et al. Are transformers effective for time series forecasting? [EB/OL]. [2023-12-10]. http://arxiv.org/abs/2205.13504.
[8]	WU H X, XU J H, WANG J M, et al. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting[EB/OL]. [2023-12-10]. http://arxiv.org/abs/2106.13008.
[9]	ZHOU T, MA Z Q, WEN Q S, et al. FEDformer: frequency enhanced decomposed transformer for long-term series forecasting[EB/OL]. [2023-12-10]. http://arxiv.org/abs/2201.12740.
[10]	LI J C, WANG Y J, MCAULEY J. Time interval aware self-attention for sequential recommendation[C]// The 13th International Conference on Web Search and Data Mining. New York: ACM, 2020: 322-330.
[11]	王英伟, 马树才. 基于ARIMA和LSTM混合模型的时间序列预测[J]. 计算机应用与软件, 2021, 38(2): 291-298.
	WANG Y W, MA S C. Time series forecasting based on arima_dlstm hybrid model[J]. Computer Applications and Software, 2021, 38(2): 291-298 (in Chinese).
[12]	SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[EB/OL]. [2023-12-10]. http://arxiv.org/abs/1506.04214.
[13]	王天雷. 基于长短时记忆网络的雷达回波外推关键技术研究[D]. 南京: 南京信息工程大学, 2022.
	WANG T L. Research on key technologies of radar echo extrapolation based on long-short memory network[D]. Nanjing: Nanjing University of Information Science & Technology, 2022 (in Chinese).
[14]	XU Z, DU J, WANG J J, et al. Satellite image prediction relying on GAN and LSTM neural networks[C]// 2019 IEEE International Conference on Communications. New York: IEEE Press, 2019: 1-6.
[15]	WANG Y, WU H, ZHANG J, et al. PredRNN: a recurrent neural network for spatiotemporal predictive learning[J]. IEEE Trans Pattern Anal Mach Intell, 2023, 45(2): 2208-2225.
[16]	罗健文, 邹茂扬, 杨昊, 等. 面向降雨预报的雷达回波预测序列外推方法[J]. 计算机应用研究, 2024, 41(4): 1138-1142.
	LUO J W, ZOU M Y, YANG H, et al. Research on extrapolation of radar echo prediction sequence for rainfall prediction[J]. Application Research of Computers, 2024, 41(4): 1138-1142 (in Chinese).
[17]	徐泽鑫, 杨磊, 李康顺. 较短的长序列时间序列预测模型[EB/OL]. [2023-12-10]. https://link.cnki.net/urlid/51.1307.TP.20230817.1343.005.
	XU Z., YANG L., LI K. Short-term long sequence time series prediction model[EB/OL]. [2023-12-10]. https://link.cnki.net/urlid/51.1307.TP.20230817.1343.005 (in Chinese).
[18]	ZHENG J B, WANG Y L, TAN C, et al. CVT-SLR: contrastive visual-textual transformation for sign language recognition with variational alignment[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 23141-23150.
[19]	李克文, 于明洋. 基于双重注意力机制改进的DA-LSTM时间序列预测模型[J]. 计算机应用与软件, 2023, 40(2): 253-258, 349.
	LI K W, YU M Y. An improved da-lstm time series prediction model based on dual attention mechanism[J]. Computer Applications and Software, 2023, 40(2): 253-258, 349 (in Chinese).
[20]	LI J F, WEN Y, HE L H. SCConv: spatial and channel reconstruction convolution for feature redundancy[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 6153-6162.
[21]	程勇, 钱坤, 康志明, 等. 一种雷达回波外推的注意力融合和信息回忆的LSTM方法[J]. 热带气象学报, 2023, 39(5): 653-663.
	CHENG Y, QIAN K, KANG Z M, et al. An attention fusion and information recall lstm method for radar echo extrapolation[J]. Journal of Tropical Meteorology, 2023, 39(5): 653-663 (in Chinese).
[22]	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Trans Pattern Anal Mach Intell, 2020, 42(8): 2011-2023.
[23]	WOO S, PARK J, LEE J-Y, et al. CBAM: convolutional block attention module[C]// The 15th European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19.
[24]	WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 11531-11539.
[25]	JING Y H, LIN L P, LI X H, et al. An attention mechanism based convolutional network for satellite precipitation downscaling over China[J]. Journal of Hydrology, 2022, 613: 128388.
[26]	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2023-12-10]. https://arxiv.org/abs/1706.05587v3.
[27]	孔震, 张华鲁, 岳圣凯, 等. 基于时域卷积网络的多尺度双线性天气预测模型[J]. 图学学报, 2020, 41(5): 764-770. DOI
	KONG Z, ZHANG H L, YUE S K, et al. Multi-scale and bilinear models based on temporal convolutional network[J]. Journal of Graphics, 2020, 41(5): 764-770 (in Chinese). DOI
[28]	YAO J L, XU F F, QIAN Z, et al. A forecast-refinement neural network based on DyConvGRU and U-net for radar echo extrapolation[J]. IEEE Access, 2023, 11: 53249-53261.
[29]	GAO Z Y, TAN C, WU L R, et al. SimVP: simpler yet better video prediction[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 3160-3170.
[30]	TAN C, GAO Z Y, WU L R, et al. Temporal attention unit: towards efficient spatiotemporal predictive learning[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 18770-18782.
[31]	WANG Y B, ZHANG J J, ZHU H Y, et al. Memory in memory: a predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9146-9154.
[32]	贺琪, 李汶龙, 宋巍, 等. 结合残差时空注意力机制的海面温度预测算法[J]. 图学学报, 2022, 43(4): 677-684.
	HE Q, LI W L, SONG W, et al. Sea surface temperature prediction algorithm combined with residual spatial-temporal attention mechanism[J]. Journal of Graphics, 2022, 43(4): 677-684 (in Chinese).

模型	MSE	SSIM	LPIPS
SimVP	3 629.6	0.913	0.147
TAU	2 656.9	0.924	0.147
PredRNN-V2	2 962.4	0.892	0.127
本文模型	2 623.8	0.932	0.127

模型	MSE	SSIM	LPIPS
SimVP	3 629.6	0.913	0.147
TAU	2 656.9	0.924	0.147
PredRNN-V2	2 962.4	0.892	0.127
本文模型	2 623.8	0.932	0.127

模型	MSE	SSIM	LPIPS
ConvLSTM	103.3	0.707	0.156
PredRNN-V2	48.4	0.891	0.071
本文模型	46.0	0.898	0.062

模型	MSE	SSIM	LPIPS
ConvLSTM	103.3	0.707	0.156
PredRNN-V2	48.4	0.891	0.071
本文模型	46.0	0.898	0.062

不同尺度组合	MSE	SSIM	LPIPS
单尺度	2 903.2	0.918	0.101
rate=1,6	2 800.1	0.923	0.105
rate=1,6,12	2 710.5	0.925	0.116
rate=1,6,12,18	2 623.8	0.932	0.127