A night traffic scene enhancement algorithm based on double fusion Unet light suppression curve estimation

doi:10.11996/JG.j.2095-302X.2023020241

Abstract

Abstract:

The existing enhancement methods were found to be inadequate for dealing with night traffic images characterized by multiple and complex light sources and uneven brightness distribution, and were prone to overexposure and image blurring. To address this problem, a night traffic image enhancement algorithm for light suppression curve estimation based on double fusion Unet was proposed. First, the glow decomposition model was introduced to suppress the light of the input image, suppressing the noise of the image while removing the influence of artificial light sources. Secondly, the double fusion Unet network was utilized, where the designed double fusion module could integrate more layers in the encoding and decoding process. The feature information preserved more image details when extracting illumination information, thereby predicting the illumination distribution map better suited for the input image. Finally, the suppression image, the original night traffic image, and the illumination parameter map extracted by the network served as input, and the improved curve estimation algorithm was applied, thus enhancing the input night traffic image and improving visual quality of the image. Experimental results showed that the proposed algorithm could outperform its counterparts in both subjective and objective comparisons, proving its effectiveness, particularly in the cases of many light sources with uneven distribution.

Key words: image enhancement, curve estimation, dual fusion Unet, glow decomposition, artificial light source

CLC Number:

TP391

GAO Tao, WANG Dui-e, CHEN Ting, WANG Xiao. A night traffic scene enhancement algorithm based on double fusion Unet light suppression curve estimation[J]. Journal of Graphics, 2023, 44(2): 241-248.

Figures/Tables 10

Fig. 1 The framework of the double-fusion Unet suppression curve enhancement algorithm ((a) Conv-BNR module; (b) The overall framework)

Fig. 2 Fusion skip connection module (fusion module)

Fig. 3 Bottom fusion module

Fig. 4 Example of night traffic dataset ((a) Expressway; (b) Residential area; (c) Urban traffic road; (d) Underground parking lot)

Table 1 The comparative results of ablation study

算法	SSIM	PSNR
Base网络	0.290 1	7.442 2
Base+抑光预处理	0.292 0	7.771 4
改进双融合Unet网络	0.655 9	16.170 7
本文	0.679 8	16.921 3

Table 2 Weight result comparison

权重				评价指标
ω₁	ω₂	ω₃	ω₄	SSIM	PSNR
0.4	0.2	0.2	0.2	0.327 7	19.994 7
0.2	0.4	0.2	0.2	0.347 6	19.918 7
0.2	0.2	0.4	0.2	0.679 8	16.921 3
0.2	0.2	0.2	0.4	0.531 2	14.229 5

Fig. 5 Comparison of image enhancement effects in monochrome light source scene ((a) Original image; (b) Literature [26]; (c) Literature [21]; (d) Literature [25]; (e) Literature [27]; (f) Ours)

Fig. 6 Comparison of image enhancement effects in multi-color light source scenes ((a) Original image; (b) Literature [26]; (c) Literature [21]; (d) Literature [25]; (e) Literature [27]; (f) Ours)

Table 3 Objective evaluation results based on the enhancement effect of five algorithms

算法	SSIM	PSNR	UQI	ILNIQE
文献[26]	0.239 4	6.884 7	0.208 1	29.414 3
文献[21]	0.290 1	7.442 2	0.219 1	29.334 0
文献[25]	0.202 4	5.157 7	0.152 2	30.247 8
文献[27]	0.222 9	5.935 6	0.132 7	34.469 3
本文	0.679 8	16.921 3	0.554 0	27.270 2

Fig. 7 The failure cases of the proposed model ((a) Urban street; (b) Expressway; (c) Suburban street; (d) Highway bridge)

References 27

[1]	WANG S M, LI X G, ZHANG H, et al. A gradient-domain image enhancement method for traffic signs in nighttime surveillance[C]// The 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics. New York: IEEE Press, 2018: 1-6.
[2]	YANG K F, ZHANG X S, LI Y J. A biological vision inspired framework for image enhancement in poor visibility conditions[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2019, 29: 1493-1506. DOI URL
[3]	YING Z Q, LI G, GAO W. A bio-inspired multi-exposure fusion framework for low-light image enhancement[EB/OL]. [2022-03-17]. https://arxiv.org/abs/1711.00591.
[4]	WANG S H, ZHENG J, HU H M, et al. Naturalness preserved enhancement algorithm for non-uniform illumination images[J]. IEEE Transactions on Image Processing, 2013, 22(9): 3538-3548. DOI PMID
[5]	ZHANG F, LI Y, YOU S D, et al. Learning temporal consistency for low light video enhancement from single images[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 4965-4974.
[6]	LV F F, LI Y, LU F. Attention guided low-light image enhancement with a large scale low-light simulation dataset[J]. International Journal of Computer Vision, 2021, 129(7): 2175-2193. DOI
[7]	WEI X X, ZHANG X S, WANG S S, et al. BLNet: a fast deep learning framework for low-light image enhancement with noise removal and color restoration[EB/OL]. [2022-07-21]. https://arxiv.org/abs/2106.15953.
[8]	JOBSON D J, RAHMAN Z, WOODELL G A. Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 1997, 6(3): 451-462. DOI URL
[9]	JOBSON D J, RAHMAN Z, WOODELL G A. A multiscale retinex for bridging the gap between color images and the human observation of scenes[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 1997, 6(7): 965-976. DOI URL
[10]	YING Z Q, LI G, REN Y R, et al. A new low-light image enhancement algorithm using camera response model[C]// 2017 IEEE International Conference on Computer Vision Workshops. New York: IEEE Press, 2018: 3015-3022.
[11]	REN Y R, YING Z Q, LI T H, et al. LECARM: low-light image enhancement using the camera response model[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(4): 968-981. DOI URL
[12]	LI M D, LIU J Y, YANG W H, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2018, 27(6): 2828-2841. DOI URL
[13]	杨勇, 刘惠义. 极端低光情况下的图像增强方法[J]. 图学学报, 2020, 41(4): 520-528.
	YANG Y, LIU H Y. The method of image enhancement under extremely low-light conditions[J]. Journal of Graphics, 2020, 41(4): 520-528. (in Chinese)
[14]	李华基, 程江华, 刘通, 等. 基于U-net++网络的弱光图像增强方法[J]. 计算机科学, 2021, 48(S2): 278-282.
	LI H J, CHENG J H, LIU T, et al. Low-light image enhancement method based on U-net++ network[J]. Computer Science, 2021, 48(S2): 278-282. (in Chinese)
[15]	LORE K G, AKINTAYO A, SARKAR S. LLNet: a deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650-662. DOI URL
[16]	LV F, LU F, WU J, et al. MBLLEN: Low-light image/video enhancement using CNNs[C]// British Machine Vision Conference. Newcastle: IJCV, 2018: 4.
[17]	REN W Q, LIU S F, MA L, et al. Low-light image enhancement via a deep hybrid network[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2019, 28(9): 4364-4375. DOI URL
[18]	CHEN Y S, WANG Y C, KAO M H, et al. Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 6306-6314.
[19]	CHEN C, CHEN Q F, XU J, et al. Learning to see in the dark[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 3291-3300.
[20]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]// Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[21]	GUO C L, LI C Y, GUO J C, et al. Zero-reference deep curve estimation for low-light image enhancement[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 1777-1786.
[22]	胡韦伟, 汪荣贵, 方帅, 等. 基于双边滤波的Retinex图像增强算法[J]. 工程图学学报, 2010, 31(2): 104-109.
	HU W W, WANG R G, FANG S, et al. Retinex algorithm for image enhancement based on bilateral filtering[J]. Journal of Engineering Graphics, 2010, 31(2): 104-109. (in Chinese)
[23]	张杰, 周浦城, 张谦. 基于迭代多尺度引导滤波Retinex的低照度图像增强[J]. 图学学报, 2018, 39(1): 1-11.
	ZHANG J, ZHOU P C, ZHANG Q. Low-light image enhancement based on iterative multi-scale guided filter retinex[J]. Journal of Graphics, 2018, 39(1): 1-11. (in Chinese)
[24]	LI Y, TAN R T, BROWN M S. Nighttime haze removal with glow and multiple light colors[C]// 2015 IEEE International Conference on Computer Vision. New York: IEEE Press, 2016: 226-234.
[25]	LI C Y, GUO C L, LOY C C. Learning to enhance low-light image via zero-reference deep curve estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8): 4225-4238.
[26]	ZHANG Y, DI X G, ZHANG B, et al. Self-supervised image enhancement network: training with low light images only[EB/OL]. [2022-07-21]. https://arxiv.org/abs/2002.11300.
[27]	ZHENG S, GUPTA G. Semantic-guided zero-shot learning for low-light image/video enhancement[C]// 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops. New York: IEEE Press, 2022: 581-590.