基于YUV颜色空间GAN网络的图像去雾算法研究

doi:10.11996/JG.j.2095-302X.2023050928

图学学报 ›› 2023, Vol. 44 ›› Issue (5): 928-936.DOI: 10.11996/JG.j.2095-302X.2023050928

• 图像处理与计算机视觉 • 上一篇下一篇

基于YUV颜色空间GAN网络的图像去雾算法研究

徐祯东(), 张天宇, 张世恒, 姚从荣, 王道累()

上海电力大学能源与机械工程学院，上海 201306

收稿日期:2023-04-23 接受日期:2023-06-18 出版日期:2023-10-31 发布日期:2023-10-31
通讯作者: 王道累(1981-)，男，教授，博士。主要研究方向为机器学习、机器视觉和图像处理等。E-mail：alfredwdl@shiep.edu.cn
作者简介:徐祯东(1999-)，男，硕士研究生。主要研究方向为数字图像处理和图像三维重建。E-mail：935510065@qq.com
基金资助:
国家自然科学基金项目(61502297)

Image defogging algorithm based on YUV color space GAN network

XU Zhen-dong(), ZHANG Tian-yu, ZHANG Shi-heng, YAO Cong-rong, WANG Dao-lei()

College of Energy and Mechanical Engineering, Shanghai University of Electric Power, Shanghai 201306, China

Received:2023-04-23 Accepted:2023-06-18 Online:2023-10-31 Published:2023-10-31
Contact: WANG Dao-lei (1981-), professor, Ph.D. His main research interests cover machine learning, machine vision and image processing, etc. E-mail：alfredwdl@shiep.edu.cn
About author:XU Zhen-dong (1999-), master student. His main research interests coverdigital image processing and image reconstruction in 3D. E-mail：935510065@qq.com
Supported by:
National Natural Science Foundation of China(61502297)

摘要/Abstract

摘要：

针对目前单幅图像去雾算法存在有色差，去雾效果不理想等问题，提出了一种基于YUV颜色空间的单幅图像去雾算法。该方法应用了GAN图像着色任务的思想，从正向的角度对雾霾图像实现重新上色。将雾霾图像转换至YUV颜色空间，在Y通道采用密集残差模块采集图片的亮度特征，根据特征对雾霾图像的亮度信息进行调整，降低雾霾对图像的影响。在UV通道上采用4个残差模块对图像颜色信息进行多次提取，根据提取的颜色信息通过模型预测对图像进行重上色。采用包含跳跃连接结构的特征融合网络将底层特征与高层特征进行融合，在融合过程中加入注意力模块以实现更加精细的去雾。实验结果表明，该算法在合成雾霾图像数据集上，RMSE，SSIM和PSNR 3种指标均达到了较高的水平，在真实雾霾图像上，对浓雾和薄雾图像均表现出了优异的去雾效果，具有良好的稳定性。

关键词: 生成对抗网络, 图像去雾, 重上色, YUV颜色空间, 跳跃连接

Abstract:

To address the current problems of chromatic aberration and unsatisfactory defogging effects in the single-image defogging algorithm, we proposed a single-image defogging algorithm based on YUV color space. This method applied the idea of GAN image coloring task to recolor haze images from a positive perspective. The haze image was converted to the YUV color space, and the dense residual module was employed to collect the brightness features of the image from the Y channel. Additionally, the brightness information of the haze image was adjusted according to the characteristics, mitigating the impact of haze on the image. Four residual modules were used on the UV channel to extract image color information multiple times, and recolored the image through model prediction based on the extracted color information. A feature fusion network, including a skip connection structure, was utilized to fuse low-level features with high-level ones. Furthermore, the addition of an attention module during the fusion process led to more refined dehazing. The experimental results demonstrated the algorithm’s efficacy, showcasing remarkable performance in terms of RMSE, SSIM, and PSNR on the synthetic haze image datasets. On the real haze image, the algorithm displayed excellent performance on dense fog and thin fog images, ultimately leading to an outstanding defogging effect and ensuring a high level of stability.

Key words: GAN, image dehazing, recolor, YUV color space, jump connection

中图分类号:

TP391

徐祯东, 张天宇, 张世恒, 姚从荣, 王道累. 基于YUV颜色空间GAN网络的图像去雾算法研究[J]. 图学学报, 2023, 44(5): 928-936.

XU Zhen-dong, ZHANG Tian-yu, ZHANG Shi-heng, YAO Cong-rong, WANG Dao-lei. Image defogging algorithm based on YUV color space GAN network[J]. Journal of Graphics, 2023, 44(5): 928-936.

图/表 10

图1 消融试验模型

Fig. 1 Ablation test model

图2 消融试验结果((a)清晰图；(b)雾霾图；(c) Y通道；(d) UV通道；(e) YUV通道)

Fig. 2 Ablation test results ((a) Clear diagram; (b) Foggy diagram; (c) Brightness channel; (d) Colour difference channel; (e) Full channel)

图3 GAN模型结构

Fig. 3 GAN model structure

图4 ResBlock结构图

Fig. 4 ResBlock structure diagram

图5 加入了卷积层的Residual dense block结构图

Fig. 5 Residual dense block structure diagram with convolutional layers

图6 带有注意力机制的ConvBlock结构图

Fig. 6 ConvBlock structure diagram with attention mechanism

表1 SOTS数据集的评价结果

Table 1 Evaluation results of SOTS dataset

Method	PSNR (dB)	SSIM	RMSE
Ours	28.130	0.958	7.706
GCANet	23.049	0.927	8.831
FFA-Net	21.976	0.913	9.056
MSBDN-DFF	32.631	0.976	5.615
gUNet	25.412	0.934	8.326
DehazeNet	22.376	0.877	9.574
PSD	26.742	0.947	7.923

图7 SOTS数据集部分实验结果((a)清晰图；(b)雾霾图；(c) Ours；(d) GCANet；(e) FFA-Net；(f) MSBDN-DFF；(g) gUNet；(h) DehazeNet；(i) PSD)

Fig. 7 Evaluation results of SOTS dataset ((a) Clear diagram; (b) Foggy diagram; (c) Ours; (d) GCANet; (e) FFA-Net; (f) MSBDN-DFF; (g) gUNet; (h) DehazeNet; (i) PSD)

图8 OTS数据集部分实验结果((a)雾霾图；(b) Ours；(c) GCANet；(d) FFA-Net；(e) MSBDN-DFF；(f) gUNet；(g) DehazeNet；(h) PSD)

Fig. 8 Evaluation results of OTS dataset ((a) Foggy diagram; (b) Ours; (c) GCANet; (d) FFA-Net; (e) MSBDN-DFF; (f) gUNet; (g) DehazeNet; (h) PSD)

图9 真实数据集部分实验结果((a)雾霾图；(b) Ours；(c) GCANet；(d) FFA-Net；(e) MSBDN-DFF；(f) gUNet；(g) DehazeNet；(h) PSD)

Fig. 9 Partial experimental results of real datasets ((a) Foggy diagram; (b) Ours; (c) GCANet; (d) FFA-Net; (e) MSBDN-DFF; (f) gUNet; (g) DehazeNet; (h) PSD)

参考文献 31

[1]	陈清江, 张雪. 基于并联卷积神经网络的图像去雾[J]. 自动化学报, 2021, 47(7): 1739-1748.
	CHEN Q J, ZHANG X. Single image dehazing based on multiple convolutional neural networks[J]. Acta Automatica Sinica, 2021, 47(7): 1739-1748. (in Chinese)
[2]	杨爱萍, 刘瑾, 邢金娜, 等. 基于内容特征和风格特征融合的单幅图像去雾网络[J]. 自动化学报, 2023, 49(4): 769-777.
	YANG A P, LIU J, XING J N, et al. Content feature and style feature fusion network for single image dehazing[J]. Acta Automatica Sinica, 2023, 49(4): 769-777. (in Chinese)
[3]	王道累, 张天宇. 图像去雾算法的综述及分析[J]. 图学学报, 2020, 41(6): 861-870.
	WANG D L, ZHANG T Y. Review and analysis of image defogging algorithm[J]. Journal of Graphics, 2020, 41(6): 861-870. (in Chinese)
[4]	ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2015, 24(11): 3522-3533. DOI URL
[5]	KIM J H, JANG W D, SIM J Y, et al. Optimized contrast enhancement for real-time image and video dehazing[J]. Journal of Visual Communication and Image Representation, 2013, 24(3): 410-425. DOI URL
[6]	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. DOI PMID
[7]	LI B Y, PENG X L, WANG Z Y, et al. An all-in-one network for dehazing and beyond[EB/OL]. [2023-03-09]. https://arxiv.org/abs/1707.06543.
[8]	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2016, 25(11): 5187-5198. DOI URL
[9]	ZHU H Y, PENG X, CHANDRASEKHAR V, et al. DehazeGAN: when image dehazing meets differential programming[C]// The 27th International Joint Conference on Artificial Intelligence. New York: ACM, 2018: 1234-1240.
[10]	YANG X T, XU Z, LUO J B. Towards perceptual image dehazing by physics-based disentanglement and adversarial training[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 7485-7492.
[11]	ENGIN D, GENC A, EKENEL H K. Cycle-dehaze: enhanced CycleGAN for single image dehazing[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2018: 938-946.
[12]	赵建堂. 基于深度学习的单幅图像去雾算法[J]. 激光与光电子学进展, 2019, 56(11): 146-153.
	ZHAO J T. Single-image defogging algorithm based on deep learning[J]. Laser & Optoelectronics Progress, 2019, 56(11): 146-153. (in Chinese)
[13]	SHAO Y J, LI L, REN W Q, et al. Domain adaptation for image dehazing[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2805-2814.
[14]	SHIMOBABA T, NAGAHAMA Y, KAKUE T, et al. Calculation reduction method for color digital holography and computer-generated hologram using color space conversion[J]. Optical Engineering, 2014, 53(2): 024108. DOI URL
[15]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
[16]	向方明, 朱遵义, 许敬, 等. YUV到RGB颜色空间转换算法研究[J]. 现代电子技术, 2012, 35(22): 65-68.
	XIANG F M, ZHU Z Y, XU J, et al. Research on algorithms of color space conversion from YUV to RGB[J]. Modern Electronics Technique, 2012, 35(22): 65-68. (in Chinese)
[17]	LI B Y, REN W Q, FU D P, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505. DOI URL
[18]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[19]	ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 2472-2481.
[20]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13708-13717.
[21]	ZHANG Q L, YANG Y B. SA-net: shuffle attention for deep convolutional neural networks[C]// ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE Press, 2021: 2235-2239.
[22]	QIN X, WANG Z L, BAI Y C, et al. FFA-net: feature fusion attention network for single image dehazing[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11908-11915. DOI URL
[23]	CHEN D D, HE M M, FAN Q N, et al. Gated context aggregation network for image dehazing and deraining[C]// 2019 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2019: 1375-1383.
[24]	DONG H, PAN J S, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2154-2164.
[25]	CHEN Z Y, WANG Y C, YANG Y, et al. PSD: principled synthetic-to-real dehazing guided by physical priors[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 7176-7185.
[26]	SONG Y D, ZHOU Y, QIAN H, et al. Rethinking performance gains in image dehazing networks[EB/OL]. [2023-03-09]. https://arxiv.org/abs/2209.11448.
[27]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612. DOI URL
[28]	SHEIKH H R, SABIR M F, BOVIK A C. A statistical evaluation of recent full reference image quality assessment algorithms[J]. IEEE Transactions on Image Processing, 2006, 15(11): 3440-3451. PMID
[29]	韩翰, 卓力, 张菁, 等. 基于深度学习的无参考图像质量评价综述[J]. 测控技术, 2022, 41(4): 1-10.
	HAN H, ZHUO L, ZHANG J, et al. Summary of No-reference image quality assessment based on deep learning[J]. Measurement & Control Technology, 2022, 41(4): 1-10. (in Chinese)
[30]	GUERRA J C V, KHANAM Z, EHASAN S, et al. Weather classification: a new multi-class dataset, data augmentation approach and comprehensive evaluations of convolutional neural networks[C]// 2018 NASA/ESA Conference on Adaptive Hardware and Systems. New York: IEEE Press, 2018: 305-310.
[31]	LIN D, LU C W, HUANG H, et al. RSCM: region selection and concurrency model for multi-class weather recognition[C]// IEEE Transactions on Image Processing. New York: IEEE Press, 2017: 4154-4167.

基于YUV颜色空间GAN网络的图像去雾算法研究

Image defogging algorithm based on YUV color space GAN network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 31

相关文章 12

编辑推荐

Metrics

本文评价

[1]	邵俊棋, 钱文华, 徐启豪. 基于条件残差生成对抗网络的风景图生成[J]. 图学学报, 2023, 44(4): 710-717.
[2]	廖仕敏, 刘仰川, 朱叶晨, 王艳玲, 高欣 . 一种基于 CycleGAN 改进的低剂量 CT 图像增强网络[J]. 图学学报, 2022, 43(4): 570-578.
[3]	方洪波, 万广, 陈忠辉, 黄以卫, 张文勇, 谢本亮. 基于改进 YOLOv5s 的离线手写数学符号识别[J]. 图学学报, 2022, 43(3): 387-395.
[4]	汪玉金, 谢诚, 余蓓蓓, 向鸿鑫, 柳青. 属性语义与图谱语义融合增强的零次学习图像识别[J]. 图学学报, 2021, 42(6): 899-907.
[5]	林森 , 刘旭 . 门控融合对抗网络的水下图像增强 [J]. 图学学报, 2021, 42(6): 948-956.
[6]	满开亮, 汪友生, 刘继荣. 基于稠密残差网络的图像超分辨率重建算法[J]. 图学学报, 2021, 42(4): 556-562.
[7]	王道累, 张天宇. 图像去雾算法的综述及分析[J]. 图学学报, 2020, 41(6): 861-870.
[8]	杨勇，刘惠义. 极端低光情况下的图像增强方法[J]. 图学学报, 2020, 41(4): 520-528.
[9]	李桂，李腾. 基于姿态引导的场景保留人物视频生成[J]. 图学学报, 2020, 41(4): 539-547.
[10]	罗琪彬 1,2，蔡强 1,2 . 采用双框架生成对抗网络的图像运动模糊盲去除[J]. 图学学报, 2019, 40(6): 1056-1063.
[11]	马时平，李权合，马红强，毕笃彦. 基于视觉物理模型的去雾算法[J]. 图学学报, 2018, 39(2): 269-277.
[12]	冷建伟. 基于高阶马尔科夫随机场的单幅图像去雾算法[J]. 图学学报, 2017, 38(5): 745-753.