Image defogging algorithm based on YUV color space GAN network

doi:10.11996/JG.j.2095-302X.2023050928

Abstract

Abstract:

To address the current problems of chromatic aberration and unsatisfactory defogging effects in the single-image defogging algorithm, we proposed a single-image defogging algorithm based on YUV color space. This method applied the idea of GAN image coloring task to recolor haze images from a positive perspective. The haze image was converted to the YUV color space, and the dense residual module was employed to collect the brightness features of the image from the Y channel. Additionally, the brightness information of the haze image was adjusted according to the characteristics, mitigating the impact of haze on the image. Four residual modules were used on the UV channel to extract image color information multiple times, and recolored the image through model prediction based on the extracted color information. A feature fusion network, including a skip connection structure, was utilized to fuse low-level features with high-level ones. Furthermore, the addition of an attention module during the fusion process led to more refined dehazing. The experimental results demonstrated the algorithm’s efficacy, showcasing remarkable performance in terms of RMSE, SSIM, and PSNR on the synthetic haze image datasets. On the real haze image, the algorithm displayed excellent performance on dense fog and thin fog images, ultimately leading to an outstanding defogging effect and ensuring a high level of stability.

Key words: GAN, image dehazing, recolor, YUV color space, jump connection

CLC Number:

TP391

XU Zhen-dong, ZHANG Tian-yu, ZHANG Shi-heng, YAO Cong-rong, WANG Dao-lei. Image defogging algorithm based on YUV color space GAN network[J]. Journal of Graphics, 2023, 44(5): 928-936.

Figures/Tables 10

Fig. 1 Ablation test model

Fig. 2 Ablation test results ((a) Clear diagram; (b) Foggy diagram; (c) Brightness channel; (d) Colour difference channel; (e) Full channel)

Fig. 3 GAN model structure

Fig. 4 ResBlock structure diagram

Fig. 5 Residual dense block structure diagram with convolutional layers

Fig. 6 ConvBlock structure diagram with attention mechanism

Table 1 Evaluation results of SOTS dataset

Method	PSNR (dB)	SSIM	RMSE
Ours	28.130	0.958	7.706
GCANet	23.049	0.927	8.831
FFA-Net	21.976	0.913	9.056
MSBDN-DFF	32.631	0.976	5.615
gUNet	25.412	0.934	8.326
DehazeNet	22.376	0.877	9.574
PSD	26.742	0.947	7.923

Fig. 7 Evaluation results of SOTS dataset ((a) Clear diagram; (b) Foggy diagram; (c) Ours; (d) GCANet; (e) FFA-Net; (f) MSBDN-DFF; (g) gUNet; (h) DehazeNet; (i) PSD)

Fig. 8 Evaluation results of OTS dataset ((a) Foggy diagram; (b) Ours; (c) GCANet; (d) FFA-Net; (e) MSBDN-DFF; (f) gUNet; (g) DehazeNet; (h) PSD)

Fig. 9 Partial experimental results of real datasets ((a) Foggy diagram; (b) Ours; (c) GCANet; (d) FFA-Net; (e) MSBDN-DFF; (f) gUNet; (g) DehazeNet; (h) PSD)

References 31

[1]	陈清江, 张雪. 基于并联卷积神经网络的图像去雾[J]. 自动化学报, 2021, 47(7): 1739-1748.
	CHEN Q J, ZHANG X. Single image dehazing based on multiple convolutional neural networks[J]. Acta Automatica Sinica, 2021, 47(7): 1739-1748. (in Chinese)
[2]	杨爱萍, 刘瑾, 邢金娜, 等. 基于内容特征和风格特征融合的单幅图像去雾网络[J]. 自动化学报, 2023, 49(4): 769-777.
	YANG A P, LIU J, XING J N, et al. Content feature and style feature fusion network for single image dehazing[J]. Acta Automatica Sinica, 2023, 49(4): 769-777. (in Chinese)
[3]	王道累, 张天宇. 图像去雾算法的综述及分析[J]. 图学学报, 2020, 41(6): 861-870.
	WANG D L, ZHANG T Y. Review and analysis of image defogging algorithm[J]. Journal of Graphics, 2020, 41(6): 861-870. (in Chinese)
[4]	ZHU Q S, MAI J M, SHAO L. A fast single image haze removal algorithm using color attenuation prior[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2015, 24(11): 3522-3533. DOI URL
[5]	KIM J H, JANG W D, SIM J Y, et al. Optimized contrast enhancement for real-time image and video dehazing[J]. Journal of Visual Communication and Image Representation, 2013, 24(3): 410-425. DOI URL
[6]	HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353. DOI PMID
[7]	LI B Y, PENG X L, WANG Z Y, et al. An all-in-one network for dehazing and beyond[EB/OL]. [2023-03-09]. https://arxiv.org/abs/1707.06543.
[8]	CAI B L, XU X M, JIA K, et al. DehazeNet: an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2016, 25(11): 5187-5198. DOI URL
[9]	ZHU H Y, PENG X, CHANDRASEKHAR V, et al. DehazeGAN: when image dehazing meets differential programming[C]// The 27th International Joint Conference on Artificial Intelligence. New York: ACM, 2018: 1234-1240.
[10]	YANG X T, XU Z, LUO J B. Towards perceptual image dehazing by physics-based disentanglement and adversarial training[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32(1): 7485-7492.
[11]	ENGIN D, GENC A, EKENEL H K. Cycle-dehaze: enhanced CycleGAN for single image dehazing[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2018: 938-946.
[12]	赵建堂. 基于深度学习的单幅图像去雾算法[J]. 激光与光电子学进展, 2019, 56(11): 146-153.
	ZHAO J T. Single-image defogging algorithm based on deep learning[J]. Laser & Optoelectronics Progress, 2019, 56(11): 146-153. (in Chinese)
[13]	SHAO Y J, LI L, REN W Q, et al. Domain adaptation for image dehazing[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2805-2814.
[14]	SHIMOBABA T, NAGAHAMA Y, KAKUE T, et al. Calculation reduction method for color digital holography and computer-generated hologram using color space conversion[J]. Optical Engineering, 2014, 53(2): 024108. DOI URL
[15]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
[16]	向方明, 朱遵义, 许敬, 等. YUV到RGB颜色空间转换算法研究[J]. 现代电子技术, 2012, 35(22): 65-68.
	XIANG F M, ZHU Z Y, XU J, et al. Research on algorithms of color space conversion from YUV to RGB[J]. Modern Electronics Technique, 2012, 35(22): 65-68. (in Chinese)
[17]	LI B Y, REN W Q, FU D P, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505. DOI URL
[18]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[19]	ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 2472-2481.
[20]	HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 13708-13717.
[21]	ZHANG Q L, YANG Y B. SA-net: shuffle attention for deep convolutional neural networks[C]// ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE Press, 2021: 2235-2239.
[22]	QIN X, WANG Z L, BAI Y C, et al. FFA-net: feature fusion attention network for single image dehazing[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11908-11915. DOI URL
[23]	CHEN D D, HE M M, FAN Q N, et al. Gated context aggregation network for image dehazing and deraining[C]// 2019 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2019: 1375-1383.
[24]	DONG H, PAN J S, XIANG L, et al. Multi-scale boosted dehazing network with dense feature fusion[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2154-2164.
[25]	CHEN Z Y, WANG Y C, YANG Y, et al. PSD: principled synthetic-to-real dehazing guided by physical priors[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 7176-7185.
[26]	SONG Y D, ZHOU Y, QIAN H, et al. Rethinking performance gains in image dehazing networks[EB/OL]. [2023-03-09]. https://arxiv.org/abs/2209.11448.
[27]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2004, 13(4): 600-612. DOI URL
[28]	SHEIKH H R, SABIR M F, BOVIK A C. A statistical evaluation of recent full reference image quality assessment algorithms[J]. IEEE Transactions on Image Processing, 2006, 15(11): 3440-3451. PMID
[29]	韩翰, 卓力, 张菁, 等. 基于深度学习的无参考图像质量评价综述[J]. 测控技术, 2022, 41(4): 1-10.
	HAN H, ZHUO L, ZHANG J, et al. Summary of No-reference image quality assessment based on deep learning[J]. Measurement & Control Technology, 2022, 41(4): 1-10. (in Chinese)
[30]	GUERRA J C V, KHANAM Z, EHASAN S, et al. Weather classification: a new multi-class dataset, data augmentation approach and comprehensive evaluations of convolutional neural networks[C]// 2018 NASA/ESA Conference on Adaptive Hardware and Systems. New York: IEEE Press, 2018: 305-310.
[31]	LIN D, LU C W, HUANG H, et al. RSCM: region selection and concurrency model for multi-class weather recognition[C]// IEEE Transactions on Image Processing. New York: IEEE Press, 2017: 4154-4167.