基于多尺度与注意力机制的两阶段风暴单体外推研究

doi:10.11996/JG.j.2095-302X.2024040696

摘要/Abstract

摘要：

风暴是一种生命周期短、发生突然、空间尺度小的自然现象，常用雷达回波外推方法进行预测，但时序预测模型难以在众多特征中定位风暴关键信息，导致预测精度低，模型无法充分学习图像高频信息，导致预测细节缺失，结果模糊。为了提升预测性能，提出两阶段风暴单体外推框架。第一阶段使用多尺度模块提取多尺度信息，注意力机制挖掘影响预测的重要特征，使用时空长短期记忆单元进行序列预测。第二阶段对一阶段结果进行偏差矫正，使用频域损失丰富外推细节。实验结果表明，在雷达回波数据集上，与主流模型PredRNN-V2相比，该模型均方误差降低11.4%，SSIM提升4.3%，在风暴单体外推任务中表现优越。在Moving MNIST数据集上，均方误差降低4.95%，感知损失降低12.67%，SSIM提升至0.898，具有良好的时序预测能力。

关键词: 注意力机制, 空洞卷积, 频域损失, 长短期记忆, 时空序列预测

Abstract:

Storms are a type of natural phenomenon characterized by a short life cycle, sudden occurrence, and small spatial scale. Radar echo backpropagation methods are commonly employed for prediction.However, time series prediction models find it difficult to locate the key information of storms among numerous features, leading to low prediction accuracy. The models cannot fully learn the high-frequency information in images, resulting in missing details in the predictions and blurry results. To enhance prediction performance, we proposed a two-stage framework for single storm forecasting. In the first stage, a multi-scale module extracted multi-scale information, while an attention mechanism mined important features impacting prediction. Spatiotemporal long-term and short-term memory units were utilized for sequence prediction. The second stage performed bias correction on the results of the first stage. Frequency domain loss enriched prediction details. Experimental results showed that on the radar echo dataset, compared with the mainstream PredRNN-V2 model, the mean squared error was reduced by 11.4% and SSIM was improved by 4.3%, showing superior performance in single storm forecasting tasks. On the Moving MNIST dataset, the mean squared error was reduced by 4.95%, the perceptual loss was reduced by 12.67%, and the SSIM was improved to 0.898, demonstrating strong time series prediction capabilities.

Key words: attention mechanism, dilated convolution, frequency domain loss, long short-term memory, time series prediction

中图分类号:

魏敏, 姚鑫. 基于多尺度与注意力机制的两阶段风暴单体外推研究[J]. 图学学报, 2024, 45(4): 696-704.

WEI Min, YAO Xin. Two-stage storm entity prediction based on multiscale and attention[J]. Journal of Graphics, 2024, 45(4): 696-704.

图/表 14

图1 算法整体实现框架

Fig. 1 The overall implementation framework of the algorithm

图2 多尺度模块

Fig. 2 Multi-scale module

图3 注意力模块

Fig. 3 Attention module

图4 ST-LSTM单元

Fig. 4 ST-LSTM unit

图5 4层ST-LSTM

Fig. 5 4-layer ST-LSTM

图6 偏差矫正模块

Fig. 6 Bias correction module

图7 对比实验结果((a)预测图像；(b)预测图像与真实图像的差值)

Fig. 7 Comparative experimental results ((a) Predicted image; (b) Difference between predicted image and ground truth)

表1 不同模型在雷达回波数据集上的客观评价指标

Table 1 Objective evaluation metrics of different models on radar echo datasets

模型	MSE	SSIM	LPIPS
SimVP	3 629.6	0.913	0.147
TAU	2 656.9	0.924	0.147
PredRNN-V2	2 962.4	0.892	0.127
本文模型	2 623.8	0.932	0.127

图8 不同模型在Moving MNIST数据集上的预测效果对比

Fig. 8 Comparison of predictive performance of different models on the Moving MNIST dataset

表2 不同模型在Moving MNIST数据集上的客观评价指标

Table 2 Objective evaluation metrics of different models on the moving mnist dataset

模型	MSE	SSIM	LPIPS
ConvLSTM	103.3	0.707	0.156
PredRNN-V2	48.4	0.891	0.071
本文模型	46.0	0.898	0.062

表3 不同尺度组合的评价指标

Table 3 The evaluation indicators of combination of different scales

不同尺度组合	MSE	SSIM	LPIPS
单尺度	2 903.2	0.918	0.101
rate=1,6	2 800.1	0.923	0.105
rate=1,6,12	2 710.5	0.925	0.116
rate=1,6,12,18	2 623.8	0.932	0.127

表4 使用不同注意力机制时的评价指标

Table 4 Evaluation metrics for different attention mechanisms

不同注意力	MSE	SSIM	LPIPS
无注意力	2 919.1	0.895	0.103
仅通道注意力	2 810.7	0.920	0.117
仅空间注意力	2 812.8	0.925	0.116
通道与空间注意力	2 623.8	0.932	0.127

表5 单阶段和两阶段模型的评价指标

Table 5 Evaluation metrics for single-stage and two-stage models

参数	MSE	SSIM	LPIPS
单阶段	2 754.2	0.924	0.115
两阶段	2 623.8	0.932	0.127

表6 整体架构的消融实验

Table 6 Ablation study of the overall architecture

参数	MSE	SSIM	LPIPS
基线	2 962.4	0.892	0.127
基线+多尺度	2 912.5	0.913	0.104
基线+注意力	2 913.1	0.921	0.101
基线+多尺度+注意力	2 754.2	0.924	0.115
基线+多尺度+注意力+二阶段	2 623.8	0.932	0.127

参考文献 32

[1]	金锐, 方璘王昊. 基于频谱分解和注意力机制的雷达回波外推算法[J]. 科技和产业, 2023, 23(8): 259-267.
	JIN R, FANG L W H. Radar echo extrapolation algorithm based on spectrum decomposition and spatiotemporal attention mechanism[J]. Science Technology and Industry, 2023, 23(8): 259-267 (in Chinese).
[2]	RAHAMAN N, BARATIN A, ARPIT D, et al. On the spectral bias of neural networks[EB/OL]. [2023-12-10]. http://arxiv.org/abs/1806.08734.
[3]	ZOU H B, WU S S, SHAN J S, et al. A method of radar echo extrapolation based on TREC and Barnes filter[J]. Journal of Atmospheric and Oceanic Technology, 2019, 36(9): 1713-1727. DOI
[4]	鹿天柱, 钱晓超, 何舒, 等. 一种基于深度学习的时间序列预测方法[J]. 控制与决策, 2021, 36(3): 645-652.
	LU T Z, QIAN X C, HE S, et al. A time series prediction method based on deep learning[J]. Control and Decision, 2021, 36(3): 645-652 (in Chinese).
[5]	BAI S J, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. [2023-12-10]. http://arxiv.org/abs/1803.0127.1
[6]	韩丰, 龙明盛, 李月安, 等. 循环神经网络在雷达临近预报中的应用[J]. 应用气象学报, 2019, 30(1): 61-69.
	HAN F, LONG M S, LI Y A, et al. The application of recurrent neural network to nowcasting[J]. Journal of Applied Meteorological Science, 2019, 30(1): 61-69 (in Chinese).
[7]	ZENG A L, CHEN M X, ZHANG L, et al. Are transformers effective for time series forecasting? [EB/OL]. [2023-12-10]. http://arxiv.org/abs/2205.13504.
[8]	WU H X, XU J H, WANG J M, et al. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting[EB/OL]. [2023-12-10]. http://arxiv.org/abs/2106.13008.
[9]	ZHOU T, MA Z Q, WEN Q S, et al. FEDformer: frequency enhanced decomposed transformer for long-term series forecasting[EB/OL]. [2023-12-10]. http://arxiv.org/abs/2201.12740.
[10]	LI J C, WANG Y J, MCAULEY J. Time interval aware self-attention for sequential recommendation[C]// The 13th International Conference on Web Search and Data Mining. New York: ACM, 2020: 322-330.
[11]	王英伟, 马树才. 基于ARIMA和LSTM混合模型的时间序列预测[J]. 计算机应用与软件, 2021, 38(2): 291-298.
	WANG Y W, MA S C. Time series forecasting based on arima_dlstm hybrid model[J]. Computer Applications and Software, 2021, 38(2): 291-298 (in Chinese).
[12]	SHI X J, CHEN Z R, WANG H, et al. Convolutional LSTM network: a machine learning approach for precipitation nowcasting[EB/OL]. [2023-12-10]. http://arxiv.org/abs/1506.04214.
[13]	王天雷. 基于长短时记忆网络的雷达回波外推关键技术研究[D]. 南京: 南京信息工程大学, 2022.
	WANG T L. Research on key technologies of radar echo extrapolation based on long-short memory network[D]. Nanjing: Nanjing University of Information Science & Technology, 2022 (in Chinese).
[14]	XU Z, DU J, WANG J J, et al. Satellite image prediction relying on GAN and LSTM neural networks[C]// 2019 IEEE International Conference on Communications. New York: IEEE Press, 2019: 1-6.
[15]	WANG Y, WU H, ZHANG J, et al. PredRNN: a recurrent neural network for spatiotemporal predictive learning[J]. IEEE Trans Pattern Anal Mach Intell, 2023, 45(2): 2208-2225.
[16]	罗健文, 邹茂扬, 杨昊, 等. 面向降雨预报的雷达回波预测序列外推方法[J]. 计算机应用研究, 2024, 41(4): 1138-1142.
	LUO J W, ZOU M Y, YANG H, et al. Research on extrapolation of radar echo prediction sequence for rainfall prediction[J]. Application Research of Computers, 2024, 41(4): 1138-1142 (in Chinese).
[17]	徐泽鑫, 杨磊, 李康顺. 较短的长序列时间序列预测模型[EB/OL]. [2023-12-10]. https://link.cnki.net/urlid/51.1307.TP.20230817.1343.005.
	XU Z., YANG L., LI K. Short-term long sequence time series prediction model[EB/OL]. [2023-12-10]. https://link.cnki.net/urlid/51.1307.TP.20230817.1343.005 (in Chinese).
[18]	ZHENG J B, WANG Y L, TAN C, et al. CVT-SLR: contrastive visual-textual transformation for sign language recognition with variational alignment[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 23141-23150.
[19]	李克文, 于明洋. 基于双重注意力机制改进的DA-LSTM时间序列预测模型[J]. 计算机应用与软件, 2023, 40(2): 253-258, 349.
	LI K W, YU M Y. An improved da-lstm time series prediction model based on dual attention mechanism[J]. Computer Applications and Software, 2023, 40(2): 253-258, 349 (in Chinese).
[20]	LI J F, WEN Y, HE L H. SCConv: spatial and channel reconstruction convolution for feature redundancy[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 6153-6162.
[21]	程勇, 钱坤, 康志明, 等. 一种雷达回波外推的注意力融合和信息回忆的LSTM方法[J]. 热带气象学报, 2023, 39(5): 653-663.
	CHENG Y, QIAN K, KANG Z M, et al. An attention fusion and information recall lstm method for radar echo extrapolation[J]. Journal of Tropical Meteorology, 2023, 39(5): 653-663 (in Chinese).
[22]	HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Trans Pattern Anal Mach Intell, 2020, 42(8): 2011-2023.
[23]	WOO S, PARK J, LEE J-Y, et al. CBAM: convolutional block attention module[C]// The 15th European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19.
[24]	WANG Q L, WU B G, ZHU P F, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 11531-11539.
[25]	JING Y H, LIN L P, LI X H, et al. An attention mechanism based convolutional network for satellite precipitation downscaling over China[J]. Journal of Hydrology, 2022, 613: 128388.
[26]	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL]. [2023-12-10]. https://arxiv.org/abs/1706.05587v3.
[27]	孔震, 张华鲁, 岳圣凯, 等. 基于时域卷积网络的多尺度双线性天气预测模型[J]. 图学学报, 2020, 41(5): 764-770. DOI
	KONG Z, ZHANG H L, YUE S K, et al. Multi-scale and bilinear models based on temporal convolutional network[J]. Journal of Graphics, 2020, 41(5): 764-770 (in Chinese). DOI
[28]	YAO J L, XU F F, QIAN Z, et al. A forecast-refinement neural network based on DyConvGRU and U-net for radar echo extrapolation[J]. IEEE Access, 2023, 11: 53249-53261.
[29]	GAO Z Y, TAN C, WU L R, et al. SimVP: simpler yet better video prediction[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 3160-3170.
[30]	TAN C, GAO Z Y, WU L R, et al. Temporal attention unit: towards efficient spatiotemporal predictive learning[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 18770-18782.
[31]	WANG Y B, ZHANG J J, ZHU H Y, et al. Memory in memory: a predictive neural network for learning higher-order non-stationarity from spatiotemporal dynamics[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9146-9154.
[32]	贺琪, 李汶龙, 宋巍, 等. 结合残差时空注意力机制的海面温度预测算法[J]. 图学学报, 2022, 43(4): 677-684.
	HE Q, LI W L, SONG W, et al. Sea surface temperature prediction algorithm combined with residual spatial-temporal attention mechanism[J]. Journal of Graphics, 2022, 43(4): 677-684 (in Chinese).