基于双融合Unet抑光曲线估计的夜间交通场景增强算法

doi:10.11996/JG.j.2095-302X.2023020241

图学学报 ›› 2023, Vol. 44 ›› Issue (2): 241-248.DOI: 10.11996/JG.j.2095-302X.2023020241

• 图像处理与计算机视觉 • 上一篇下一篇

基于双融合Unet抑光曲线估计的夜间交通场景增强算法

高涛¹(), 王对娥¹, 陈婷¹, 王潇²

1.长安大学信息工程学院，陕西西安 710064
2.长安大学运输工程学院，陕西西安 710064

收稿日期:2022-08-21 接受日期:2022-10-29 出版日期:2023-04-30 发布日期:2023-05-01
作者简介:高涛(1981-)，男，教授，博士。主要研究方向为图像处理和计算机视觉研究。E-mail：gtnwpu@126.com
基金资助:
国家重点研发计划项目(2019YFE0108300);国家自然科学基金项目(52172379);长安大学中央高校基本科研业务费专项资金资助项目(300102242901)

A night traffic scene enhancement algorithm based on double fusion Unet light suppression curve estimation

GAO Tao¹(), WANG Dui-e¹, CHEN Ting¹, WANG Xiao²

1. School of Information Engineering, Chang'an University, Xi'an Shaanxi 710064, China
2. School of Transportation Engineering, Chang'an University, Xi'an Shaanxi 710064, China

Received:2022-08-21 Accepted:2022-10-29 Online:2023-04-30 Published:2023-05-01
About author:GAO Tao (1981-), professor, Ph.D. His main research interests cover image processing and computer vision research. E-mail：gtnwpu@126.com
Supported by:
National Key R&D Program(2019YFE0108300);National Natural Science Foundation of China(52172379);Funded by the Special Funds for Fundamental Research Funds of the Central Universities of Chang'an University(300102242901)

摘要/Abstract

摘要：

为解决现有增强方法在处理光源多而杂，亮度分布不均匀的夜间交通图像时易出现过度曝光、图像模糊的问题，提出了一种基于双融合Unet抑光曲线估计的夜间交通图像增强算法。首先，引入辉光分解模型对输入图像进行抑光操作，去除人造光源影响的同时会抑制图像噪声；其次，使用双融合Unet网络，设计的双融合模块在编解码过程中能够融合更多层次的特征信息，在提取光照信息时保留了更多的图像细节，预测出更贴切于输入图像的光照分布图；最后，将抑光图像，原始夜间交通图像以及其网络提取的光照参数图作为输入，利用改进的曲线估计算法对输入的夜间交通图像进行增强迭代，使得图像得到更好的视觉增强。实验结果表明，该算法在主观和客观比较下均取得最好的效果，证明了本文方法的有效性，尤其是在光源多且分布不均匀的情况下。

关键词: 图像增强, 曲线估计, 双融合Unet, 辉光分解, 人造光源

Abstract:

The existing enhancement methods were found to be inadequate for dealing with night traffic images characterized by multiple and complex light sources and uneven brightness distribution, and were prone to overexposure and image blurring. To address this problem, a night traffic image enhancement algorithm for light suppression curve estimation based on double fusion Unet was proposed. First, the glow decomposition model was introduced to suppress the light of the input image, suppressing the noise of the image while removing the influence of artificial light sources. Secondly, the double fusion Unet network was utilized, where the designed double fusion module could integrate more layers in the encoding and decoding process. The feature information preserved more image details when extracting illumination information, thereby predicting the illumination distribution map better suited for the input image. Finally, the suppression image, the original night traffic image, and the illumination parameter map extracted by the network served as input, and the improved curve estimation algorithm was applied, thus enhancing the input night traffic image and improving visual quality of the image. Experimental results showed that the proposed algorithm could outperform its counterparts in both subjective and objective comparisons, proving its effectiveness, particularly in the cases of many light sources with uneven distribution.

Key words: image enhancement, curve estimation, dual fusion Unet, glow decomposition, artificial light source

中图分类号:

TP391

高涛, 王对娥, 陈婷, 王潇. 基于双融合Unet抑光曲线估计的夜间交通场景增强算法[J]. 图学学报, 2023, 44(2): 241-248.

GAO Tao, WANG Dui-e, CHEN Ting, WANG Xiao. A night traffic scene enhancement algorithm based on double fusion Unet light suppression curve estimation[J]. Journal of Graphics, 2023, 44(2): 241-248.

图/表 10

图1 双融合Unet抑光曲线增强算法框架图((a) Conv-BNR模块；(b)整体框架)

Fig. 1 The framework of the double-fusion Unet suppression curve enhancement algorithm ((a) Conv-BNR module; (b) The overall framework)

图2 融合跳跃连接模块(融合模块)

Fig. 2 Fusion skip connection module (fusion module)

图3 Bottom融合模块

Fig. 3 Bottom fusion module

图4 夜间交通数据集示例((a)高速公路；(b)住宅区；(c)城市交通道路；(d)地下停车场)

Fig. 4 Example of night traffic dataset ((a) Expressway; (b) Residential area; (c) Urban traffic road; (d) Underground parking lot)

表1 消融实验对比结果

Table 1 The comparative results of ablation study

算法	SSIM	PSNR
Base网络	0.290 1	7.442 2
Base+抑光预处理	0.292 0	7.771 4
改进双融合Unet网络	0.655 9	16.170 7
本文	0.679 8	16.921 3

表2 权重结果对比

Table 2 Weight result comparison

权重				评价指标
ω₁	ω₂	ω₃	ω₄	SSIM	PSNR
0.4	0.2	0.2	0.2	0.327 7	19.994 7
0.2	0.4	0.2	0.2	0.347 6	19.918 7
0.2	0.2	0.4	0.2	0.679 8	16.921 3
0.2	0.2	0.2	0.4	0.531 2	14.229 5

图5 单色光源场景下图像增强效果对比图((a)原图；(b)文献[26]；(c)文献[21]；(d)文献[25]；(e)文献[27]；(f)本文)

Fig. 5 Comparison of image enhancement effects in monochrome light source scene ((a) Original image; (b) Literature [26]; (c) Literature [21]; (d) Literature [25]; (e) Literature [27]; (f) Ours)

图6 多色光源场景下图像增强效果对比图((a)原图；(b)文献[26]；(c)文献[21]；(d)文献[25]；(e)文献[27]；(f)本文)

Fig. 6 Comparison of image enhancement effects in multi-color light source scenes ((a) Original image; (b) Literature [26]; (c) Literature [21]; (d) Literature [25]; (e) Literature [27]; (f) Ours)

表3 基于5种算法增强效果的客观评比结果

Table 3 Objective evaluation results based on the enhancement effect of five algorithms

算法	SSIM	PSNR	UQI	ILNIQE
文献[26]	0.239 4	6.884 7	0.208 1	29.414 3
文献[21]	0.290 1	7.442 2	0.219 1	29.334 0
文献[25]	0.202 4	5.157 7	0.152 2	30.247 8
文献[27]	0.222 9	5.935 6	0.132 7	34.469 3
本文	0.679 8	16.921 3	0.554 0	27.270 2

图7 本文模型的失败样例((a)城市街道；(b)高速公路；(c)城郊街道；(d)公路桥)

Fig. 7 The failure cases of the proposed model ((a) Urban street; (b) Expressway; (c) Suburban street; (d) Highway bridge)

参考文献 27

[1]	WANG S M, LI X G, ZHANG H, et al. A gradient-domain image enhancement method for traffic signs in nighttime surveillance[C]// The 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics. New York: IEEE Press, 2018: 1-6.
[2]	YANG K F, ZHANG X S, LI Y J. A biological vision inspired framework for image enhancement in poor visibility conditions[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2019, 29: 1493-1506. DOI URL
[3]	YING Z Q, LI G, GAO W. A bio-inspired multi-exposure fusion framework for low-light image enhancement[EB/OL]. [2022-03-17]. https://arxiv.org/abs/1711.00591.
[4]	WANG S H, ZHENG J, HU H M, et al. Naturalness preserved enhancement algorithm for non-uniform illumination images[J]. IEEE Transactions on Image Processing, 2013, 22(9): 3538-3548. DOI PMID
[5]	ZHANG F, LI Y, YOU S D, et al. Learning temporal consistency for low light video enhancement from single images[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 4965-4974.
[6]	LV F F, LI Y, LU F. Attention guided low-light image enhancement with a large scale low-light simulation dataset[J]. International Journal of Computer Vision, 2021, 129(7): 2175-2193. DOI
[7]	WEI X X, ZHANG X S, WANG S S, et al. BLNet: a fast deep learning framework for low-light image enhancement with noise removal and color restoration[EB/OL]. [2022-07-21]. https://arxiv.org/abs/2106.15953.
[8]	JOBSON D J, RAHMAN Z, WOODELL G A. Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 1997, 6(3): 451-462. DOI URL
[9]	JOBSON D J, RAHMAN Z, WOODELL G A. A multiscale retinex for bridging the gap between color images and the human observation of scenes[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 1997, 6(7): 965-976. DOI URL
[10]	YING Z Q, LI G, REN Y R, et al. A new low-light image enhancement algorithm using camera response model[C]// 2017 IEEE International Conference on Computer Vision Workshops. New York: IEEE Press, 2018: 3015-3022.
[11]	REN Y R, YING Z Q, LI T H, et al. LECARM: low-light image enhancement using the camera response model[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 29(4): 968-981. DOI URL
[12]	LI M D, LIU J Y, YANG W H, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2018, 27(6): 2828-2841. DOI URL
[13]	杨勇, 刘惠义. 极端低光情况下的图像增强方法[J]. 图学学报, 2020, 41(4): 520-528.
	YANG Y, LIU H Y. The method of image enhancement under extremely low-light conditions[J]. Journal of Graphics, 2020, 41(4): 520-528. (in Chinese)
[14]	李华基, 程江华, 刘通, 等. 基于U-net++网络的弱光图像增强方法[J]. 计算机科学, 2021, 48(S2): 278-282.
	LI H J, CHENG J H, LIU T, et al. Low-light image enhancement method based on U-net++ network[J]. Computer Science, 2021, 48(S2): 278-282. (in Chinese)
[15]	LORE K G, AKINTAYO A, SARKAR S. LLNet: a deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650-662. DOI URL
[16]	LV F, LU F, WU J, et al. MBLLEN: Low-light image/video enhancement using CNNs[C]// British Machine Vision Conference. Newcastle: IJCV, 2018: 4.
[17]	REN W Q, LIU S F, MA L, et al. Low-light image enhancement via a deep hybrid network[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2019, 28(9): 4364-4375. DOI URL
[18]	CHEN Y S, WANG Y C, KAO M H, et al. Deep photo enhancer: unpaired learning for image enhancement from photographs with GANs[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 6306-6314.
[19]	CHEN C, CHEN Q F, XU J, et al. Learning to see in the dark[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 3291-3300.
[20]	RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]// Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[21]	GUO C L, LI C Y, GUO J C, et al. Zero-reference deep curve estimation for low-light image enhancement[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 1777-1786.
[22]	胡韦伟, 汪荣贵, 方帅, 等. 基于双边滤波的Retinex图像增强算法[J]. 工程图学学报, 2010, 31(2): 104-109.
	HU W W, WANG R G, FANG S, et al. Retinex algorithm for image enhancement based on bilateral filtering[J]. Journal of Engineering Graphics, 2010, 31(2): 104-109. (in Chinese)
[23]	张杰, 周浦城, 张谦. 基于迭代多尺度引导滤波Retinex的低照度图像增强[J]. 图学学报, 2018, 39(1): 1-11.
	ZHANG J, ZHOU P C, ZHANG Q. Low-light image enhancement based on iterative multi-scale guided filter retinex[J]. Journal of Graphics, 2018, 39(1): 1-11. (in Chinese)
[24]	LI Y, TAN R T, BROWN M S. Nighttime haze removal with glow and multiple light colors[C]// 2015 IEEE International Conference on Computer Vision. New York: IEEE Press, 2016: 226-234.
[25]	LI C Y, GUO C L, LOY C C. Learning to enhance low-light image via zero-reference deep curve estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(8): 4225-4238.
[26]	ZHANG Y, DI X G, ZHANG B, et al. Self-supervised image enhancement network: training with low light images only[EB/OL]. [2022-07-21]. https://arxiv.org/abs/2002.11300.
[27]	ZHENG S, GUPTA G. Semantic-guided zero-shot learning for low-light image/video enhancement[C]// 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops. New York: IEEE Press, 2022: 581-590.

基于双融合Unet抑光曲线估计的夜间交通场景增强算法

A night traffic scene enhancement algorithm based on double fusion Unet light suppression curve estimation

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 27

相关文章 12

编辑推荐

Metrics

本文评价

[1]	范溢华 , 王永振 , 燕雪峰 , 宫丽娜 , 郭延文 , 魏明强 . 人脸识别任务驱动的低光照图像增强算法 [J]. 图学学报, 2022, 43(6): 1170-1181.
[2]	廖仕敏, 刘仰川, 朱叶晨, 王艳玲, 高欣 . 一种基于 CycleGAN 改进的低剂量 CT 图像增强网络[J]. 图学学报, 2022, 43(4): 570-578.
[3]	杨淼 , 王海文 , 胡珂 , 殷歌 , 胡金通 . 一种基于色彩补偿的水下图像综合增强算法[J]. 图学学报, 2021, 42(1): 59-64.
[4]	王道累, 张天宇. 图像去雾算法的综述及分析[J]. 图学学报, 2020, 41(6): 861-870.
[5]	杨勇，刘惠义. 极端低光情况下的图像增强方法[J]. 图学学报, 2020, 41(4): 520-528.
[6]	朱振杰1，王红茹 1,2. 基于暗原色先验原理的水下图像增强[J]. 图学学报, 2018, 39(3): 453-462.
[7]	宋瑞霞1，李达1，王小春2. 基于HSI 色彩空间的低照度图像增强算法[J]. 图学学报, 2017, 38(2): 217-223.
[8]	赵仁涛，郭彩乔，李华德，崔佳星，张志芳，铁军. 基于RBF 神经网络的低对比度图像自适应增强算法[J]. 图学学报, 2015, 36(3): 432-437.
[9]	叶仕通. 结合阈值去噪与边缘优化的图像增强算法[J]. 图学学报, 2014, 35(4): 571-576.
[10]	张新龙，申永，张璇，汪荣贵. 离散余弦变换域内的Retinex图像增强模型研究[J]. 图学学报, 2011, 32(6): 33-42.
[11]	李志球，梁双华. 改进的灰度级彩色变换法在B超图像中的应用[J]. 图学学报, 2010, 31(4): 87-93.
[12]	胡韦伟，汪荣贵，方帅，胡琼. 基于双边滤波的Retinex图像增强算法[J]. 图学学报, 2010, 31(2): 104-109.