A progressive spatiotemporal detail enhancement algorithm for video dehazing

doi:10.11996/JG.j.2095-302X.2026030576

Abstract

Abstract:

To address insufficient texture detail restoration and inadequate inter-frame consistency modeling in existing video dehazing methods, a progressive spatiotemporal detail-enhanced video dehazing algorithm (Progressive Spatiotemporal Detail Enhancement Network，PSTD-net) was proposed, comprising two modules: a spatial detail enhancement and a cross-frame temporal modeling. First, to alleviate insufficient high-frequency texture modeling in existing methods, a progressive detail-enhancement encoder was proposed to effectively extract and restore texture details damaged by haze. Second, to improve the temporal consistency between video frames, a cross-frame detai- enhancement module was designed to model inter-frame dependencies through a spatiotemporal attention mechanism, enhancing image details while suppressing artifact generation. Experiments conducted on multiple synthetic and real hazy video datasets showed that PSTD-net improved both dehazing effect and visual consistency, providing a new solution for video dehazing tasks.

Key words: video dehazing, progressive detail enhancement, temporal attention, differential convolution, video restoration

CLC Number:

ZHANG Yi, WANG Zhen, LIU Yanli, XING Guanyu. A progressive spatiotemporal detail enhancement algorithm for video dehazing[J]. Journal of Graphics, 2026, 47(3): 576-588.

Figures/Tables 12

Fig. 1 Overall structure of PSTD-net

Fig. 2 Architecture of the cross-frame detail enhancement module

Fig. 3 Architecture of the multi-scale neighborhood RWKV module

Table 1 Quantitative results on HazeWorld and REVIDE

方法	年份	类型	HazeWorld		REVIDE
方法	年份	类型	PSNR/dB	SSIM	PSNR/dB	SSIM
DCP^[3]	2011	图像去雾	16.49	0.812 6	11.03	0.728 5
Griddehazenet^[6]	2019	图像去雾	22.80	0.921 7	19.69	0.854 5
FFA^[41]	2020	图像去雾	22.11	0.900 7	16.65	0.813 3
MSDBN^[42]	2020	图像去雾	23.70	0.885 8	22.01	0.875 9
Dehamer^[43]	2022	图像去雾	22.92	0.904 4	19.43	0.700 8
Dehazeformer^[17]	2023	图像去雾	25.44	0.928 6	19.92	0.814 9
DEA-Net^[29]	2024	图像去雾	28.29	0.918 8	20.28	0.778 8
VDH^[8]	2018	视频去雾	17.97	0.778 0	16.64	0.813 3
EDVR^[44]	2019	视频去雾	22.91	0.903 6	21.22	0.870 7
CG-IDN^[10]	2021	视频去雾	25.25	0.915 5	23.21	0.883 6
BasicVSR++^[45]	2022	视频去雾	26.06	0.920 7	21.68	0.872 6
MAP-Net^[11]	2023	视频去雾	27.12	0.934 9	24.03	0.892 7
DVD-Net^[46]	2024	视频去雾	28.33	0.935 6	24.34	0.892 1
PSTD-net	—	视频去雾	28.66	0.938 7	23.65	0.896 7

Fig. 4 Qualitative comparison of restoration performance on sky regions under natural scenarios ((a) Input image; (b) EDVR; (c) Dehamer; (d) Dehazeformer; (e) DEA-net; (f) MAP-Net; (g) PSTD-net; (h) Ground truth)

Fig. 5 Qualitative comparison of sky region restoration in natural scenarios ((a) Input image; (b) EDVR; (c) Dehamer; (d) Dehazeformer; (e) DEA-net; (f) MAP-Net; (g) PSTD-net; (h) Ground truth)

Fig. 6 Qualitative comparison of dehazing performance on the REVIDE dataset ((a) Input image; (b) EDVR; (c) Dehamer; (d) Dehazeformer; (e) DEA-net; (f) MAP-Net; (g) PSTD-net; (h) Ground truth)

Table 2 Results of ablation experiments for the spatial detail enhancement module

模型	PSNR/dB	SSIM	FLOPS/G	参数量/M
Base	26.69	0.882 4	63.43	6.81
Base-SDE	27.32	0.911 0	63.12	6.80
PSTD-net	27.61	0.917 2	63.42	6.81

Fig. 7 Influence of the spatial detail enhancement module on dehazing performance ((a) Base; (b) Base-SDE; (c) PSTD-net; (d) Ground truth)

Table 3 Results of ablation experiments for the cross-frame detail enhancement module

模型	PSNR/dB	SSIM	FLOPS/G	参数量/ M
Base-Stack	26.53	0.8768	57.96	6.45
Base-SDE	27.43	0.9135	58.49	6.44
PSTD-net	27.61	0.9172	63.42	6.81

Fig. 8 Influence of the spatial detail enhancement module on dehazing performance ((a) Base-satck; (b) Base-nodetail; (c) PSTD-net; (d) Ground truth)

Table 4 Results of ablation experiments for different loss terms

损失	PSNR	SSIM	迭代次数/ K
L₁	26.57	0.893 9	40
L₂	26.02	0.879 3	40
L₁+L₂	26.86	0.903 2	40
L₁+L_CR	27.61	0.917 2	40

References 46

[1]	MCCARTNEY E J. Optics of the atmosphere: scattering by molecules and particles[M]. New York: John Wiley & Sons, 1976: 176-215.
[2]	TAN R T. Visibility in bad weather from a single image[C]// 2008 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2008: 1-8.
[3]	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. DOI PMID
[4]	FATTAL R. Single image dehazing[J]. ACM Transactions on Graphics, 2008, 27(3): 1-9.
[5]	PANG Y W, XIE J, LI X L. Visual haze removal by a unified generative adversarial network[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(11): 3211-3221. DOI URL
[6]	LIU X H, MA Y R, SHI Z H, et al. GridDehazeNet: attention-based multi-scale network for image dehazing[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 7313-7322.
[7]	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198. DOI PMID
[8]	REN W Q, ZHANG J G, XU X Y, et al. Deep video dehazing with semantic segmentation[J]. IEEE Transactions on Image Processing, 2019, 28(4): 1895-1908. DOI URL
[9]	LI B Y, PENG X L, WANG Z Y, et al. End-to-end united video dehazing and detection[C]//The 32nd AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2018: 859.
[10]	ZHANG X Y, DONG H, PAN J S, et al. Learning to restore hazy video: a new real-world dataset and a new method[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 9235-9244.
[11]	XU J Q, HU X W, ZHU L, et al. Video dehazing via a multi-range temporal alignment network with physical prior[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 18053-18062.
[12]	BERMAN D, TREIBITZ T, AVIDAN S. Non-local image dehazing[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 1674-1682.
[13]	ZHENG Z R, REN W Q, CAO X C, et al. Ultra-high-definition image dehazing via multi-guided bilateral learning[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 16180-16189.
[14]	REN W Q, MA L, ZHANG J W, et al. Gated fusion network for single image dehazing[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 3253-3261.
[15]	ZHOU Y, CHEN Z H, LI P, et al. FSAD-Net: feedback spatial attention dehazing network[J]. IEEE Transactions on Neural Networks and Learning Systems, 2023, 34(10): 7719-7733. DOI URL
[16]	LI B Y, PENG X L, WANG Z Y, et al. AOD-Net: all-in-one dehazing network[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 4780-4788.
[17]	SONG Y D, HE Z Q, QIAN H, et al. Vision transformers for single image dehazing[J]. IEEE Transactions on Image Processing, 2023, 32: 1927-1941. DOI URL
[18]	庞宇, 吴天次, 王元发, 等. 快速视频去雾改进算法的FPGA实现[J]. 计算机应用研究, 2024, 41(6): 1803-1807.
	PANG Y, WU T C, WANG Y F, et al. FPGA implementation of improved algorithm for fast video de-fogging[J]. Application Research of Computers, 2024, 41(6): 1803-1807(in Chinese).
[19]	姚婷婷, 梁越, 柳晓鸣, 等. 基于雾线先验的时空关联约束视频去雾算法[J]. 电子与信息学报, 2020, 42(11): 2796-2804.
	YAO T T, LIANG Y, LIU X M, et al. Video dehazing algorithm via haze-line prior with spatiotemporal correlation constraint[J]. Journal of Electronics & Information Technology, 2020, 42(11): 2796-2804(in Chinese).
[20]	VAN NGUYEN T, VIEN A G, LEE C. Real-time image and video dehazing based on multiscale guided filtering[J]. Multimedia Tools and Applications, 2022, 81(25): 36567-36584. DOI
[21]	ZHANG J W, LI L, ZHANG Y, et al. Video dehazing with spatial and temporal coherence[J]. The Visual Computer, 2011, 27(6/8): 749-757. DOI URL
[22]	KIM J H, JANG W D, SIM J Y, et al. Optimized contrast enhancement for real-time image and video dehazing[J]. Journal of Visual Communication and Image Representation, 2013, 24(3): 410-425. DOI URL
[23]	SHAIKH J A H, MUKANE S M, RANDIVE S N. Lightweight progressive recurrent network for video de-hazing in adverse weather conditions[J]. The Visual Computer, 2025, 41(7): 4661-4672. DOI
[24]	LIU Y, WAN L, FU H Z, et al. Phase-based memory network for video dehazing[C]// The 30th ACM International Conference on Multimedia. New York: ACM, 2022: 5427-5435.
[25]	张万绪, 袁永德, 闫阳, 等. 基于暗原色先验的快速视频去雾优化算法[J]. 西北大学学报(自然科学版), 2016, 46(1): 43-47.
	ZHANG W X, YUAN Y D, YAN Y, et al. Fasting video haze removal algorithm using dark channel prior[J]. Journal of Northwest University (Natural Science Edition), 2016, 46(1): 43-47(in Chinese).
[26]	SHENG B, LI P, ALI R, et al. Improving video temporal consistency via broad learning system[J]. IEEE Transactions on Cybernetics, 2022, 52(7): 6662-6675. DOI URL
[27]	ZHU J, ZHANG Q W, FEI L K, et al. FFFN: frame-by-frame feedback fusion network for video super-resolution[J]. IEEE Transactions on Multimedia, 2023, 25: 6821-6835. DOI URL
[28]	WOO S, DEBNATH S, HU R H, et al. ConvNeXt V2: co-designing and scaling ConvNets with masked autoencoders[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 16133-16142.
[29]	CHEN Z X, HE Z W, LU Z M. DEA-Net: single image dehazing based on detail-enhanced convolution and content-guided attention[J]. IEEE Transactions on Image Processing, 2024, 33: 1002-1015. DOI PMID
[30]	SU Z, LIU W Z, YU Z T, et al. Pixel difference networks for efficient edge detection[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 5097-5107.
[31]	YU Z T, ZHAO C X, WANG Z Z, et al. Searching central difference convolutional networks for face anti-spoofing[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 5294-5304.
[32]	PENG B, ALCAIDE E, ANTHONY Q, et al. RWKV: reinventing RNNs for the transformer era[EB/OL]. [2025-08-27]. https://aclanthology.org/2023.findings-emnlp.936/.
[33]	DUAN Y C, WANG W Y, CHEN Z, et al. efficient and scalable visual perception with RWKV-like architectures[EB/OL]. [2025-03-04]. https://iclr.cc/virtual/2025/poster/28412.
[34]	LI X, WANG W H, HU X L, et al. Selective kernel networks[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 510-519.
[35]	WU H Y, QU Y Y, LIN S H, et al. Contrastive learning for compact single image dehazing[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 10546-10555.
[36]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2025-04-04]. https://dblp.org/rec/journals/corr/SimonyanZ14a.html.
[37]	LUCAS A, LOPEZ-TAPIA S, MOLINA R, et al. Generative adversarial networks and perceptual losses for video super-resolution[J]. IEEE Transactions on Image Processing, 2019, 28(7): 3312-3327. DOI PMID
[38]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. DOI PMID
[39]	PASZKE A, GROSS S, MASSA F, et al. PyTorch:an imperative style, high-performance deep learning library[C]// The 33rd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2019: 721.
[40]	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2025-04-22]. https://dblp.org/rec/journals/corr/KingmaB14.html.
[41]	QIN X, WANG Z L, BAI Y C, et al. FFA-Net: feature fusion attention network for single image dehazing[C]// The 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 11908-11915.
[42]	DONG H, PAN J S, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2154-2164.
[43]	GUO C L, YAN Q X, ANWAR S, et al. Image dehazing transformer with transmission-aware 3D position embedding[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5802-5810.
[44]	WANG X T, CHAN K C K, YU K, et al. EDVR: video restoration with enhanced deformable convolutional networks[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2019: 1954-1963.
[45]	CHAN K C K, ZHOU S C, XU X Y, et al. BasicVSR++: improving video super-resolution with enhanced propagation and alignment[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5962-5971.
[46]	FAN J K, WENG J W, WANG K, et al. Driving-video dehazing with non-aligned regularization for safety assistance[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 26109-26119.