A lightweight image flare removal method for night vision assisted driving

doi:10.11996/JG.j.2095-302X.2026010057

Abstract

Abstract:

In night-vision environments, image quality was significantly degraded by glare from intense light sources, impairing the performance of night-vision assisted driving systems. Existing flare-removal algorithms suffer from limited robustness, high computational complexity, and loss of light-source information. To address these challenges, a lightweight image flare-removal method, Night Flare Removal Network+ (NFR-Net+), was proposed to enhance image clarity while meeting the real-time computational demands of mobile devices. The approach first incorporated a feature-filtering mechanism combined with residual connection strategies to strengthen feature extraction capabilities, effectively mitigating overfitting and ensuring robust flare removal across diverse lighting conditions and flare types. Additionally, a nonlinear, activation-free feature attention module was introduced. Via a lightweight design, an efficient attention mechanism was constructed that significantly improved image-detail reconstruction while reducing model parameters by approximately 8.28% and runtime memory by about 11.1%, thereby optimizing computational efficiency. To tackle the issue of diminished image naturalness due to excessive light-source removal in traditional methods, an enhanced light-source extraction module was developed within the segmentation network. This module employed an improved light-source separation strategy to accurately preserve brightness and texture details in light-source regions, ensuring the authenticity and naturalness of output images. Experimental results demonstrated that NFR-Net+ surpassed state-of-the-art methods on image quality metrics such as Structural Similarity Index Measure (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Learned Perceptual Image Patch Similarity (LPIPS), exhibiting superior flare-removal performance and detail preservation. The method also demonstrated strong adaptability across various night-vision scenarios and hardware devices, fulfilling the efficiency requirements for real-time processing. Ablation studies further validated the effectiveness of individual components, highlighting the critical role of feature filtering and attention mechanisms in balancing performance and resource consumption. This approach provided an efficient, lightweight solution for applications such as nighttime autonomous driving and intelligent surveillance.

Key words: flare removal, night vision environment, light source information, nonlinear no activation, lightweight

CLC Number:

TP391
U463

LI Ye, JIA Junyang, HUANG Guan, LI Yujie, QI Wenting, LIU Yan. A lightweight image flare removal method for night vision assisted driving[J]. Journal of Graphics, 2026, 47(1): 57-67.

Figures/Tables 11

Fig. 1 Overall network framework diagram

Fig. 2 Locally enhanced window transformer block before and after improvement((a) Before improvement；(b) Improved)

Fig. 3 Feature filtering mechanism

Fig. 4 Simplified channels and pixel attention mechanism((a) CA；(b) SCA；(c) PA；(d) SPA)

Fig. 5 The structure diagram of the light source extraction

Fig. 6 The transformed BDD100K dataset((a), (c) Before treatment；(b), (d) Processed)

Fig. 7 Flare synthesis method

Fig. 8 This paper compares the experimental results with those of other method ((a) Input; (b) Literature [37]; (c) Literature [38]; (d) Literature [13]; (e) Literature [20]; (f) Literature [16]; (g) Ours；(h) GT)

Fig. 9 Example of generalization of experimental results of NFR-Net+ algorithm ((a) Input; (b) Output)

Table 1 Performance comparison of flare removal algorithms

方法	PSNR/dB↑	SSIM↑	LPIPS↓	推理时间/s↓
文献[37]	24.97	0.897	0.137	0.098 8
文献[38]	27.45	0.904	0.034	0.096 4
文献[13]	28.76	0.924	0.029	1.358 4
文献[20]	30.60	0.967	0.021	3.945 4
文献[16]	30.61	0.769	0.018	1.574 1
本文方法	30.68	0.973	0.020	0.076 1

Table 2 Ablation experimental results of nonlinear activation free feature attention module

方法	运行内存/G↓	参数/百万↓	PSNR/dB↑
CA	10.14	1.65	30.53
SCA	9.67	1.61	30.58
PA	9.56	1.65	30.52
SPA	9.14	1.61	30.56
FA	10.63	2.07	30.60
NAFFA	9.75	1.84	30.65

References 38

[1]	NUSSBERGER A, GRABNER H, VAN GOOL L. Robust aerial object tracking in images with lens flare[C]// 2015 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2015: 6380-6387.
[2]	YE S T, YIN J L, CHEN B H, et al. Single image glare removal using deep convolutional networks[C]// 2020 IEEE International Conference on Image Processing. New York: IEEE Press, 2020: 201-205.
[3]	WEN L H, JO K H. Deep learning-based perception systems for autonomous driving: a comprehensive survey[J]. Neurocomputing, 2022, 489: 255-270. DOI URL
[4]	BOYNTON P A, KELLEY E F. Liquid-filled camera for the measurement of high-contrast images[C]// The SPIE 5080, Cockpit Displays X. Bellingham: SPIE, 2003: 370-378.
[5]	RASKAR R, AGRAWAL A, WILSON C A, et al. Glare aware photography: 4D ray sampling for reducing glare effects of camera lenses[J]. ACM Transactions on Graphics, 2008, 27(3): 1-10.
[6]	MACLEOD H A. Thin-film optical filters[M]. Boca Raton: Taylor & Francis, 2010.
[7]	VITORIA P, BALLESTER C. Automatic flare spot artifact detection and removal in photographs[J]. Journal of Mathematical Imaging and Vision, 2019, 61(4): 515-533. DOI
[8]	ASHA C S, BHAT S K, NAYAK D, et al. Auto removal of bright spot from images captured against flashing light source[C]// 2019 IEEE International Conference on Distributed Computing, VLSI, Electrical Circuits and Robotics. New York: IEEE Press, 2019: 1-6.
[9]	时妙文, 范琳伟, 王桦, 等. 基于四元数组稀疏的彩色图像去噪[J]. 图学学报, 2023, 44(2): 298-303. DOI
	SHI M W, FAN L W, WANG H, et al. Quaternion patch-group sparse coding for color image denoising[J]. Journal of Graphics, 2023, 44(2): 298-303 (in Chinese).
[10]	徐祯东, 张天宇, 张世恒, 等. 基于YUV颜色空间GAN网络的图像去雾算法研究[J]. 图学学报, 2023, 44(5): 928-936. DOI
	XU Z D, ZHANG T Y, ZHANG S H, et al. Image defogging algorithm based on YUV color space GAN network[J]. Journal of Graphics, 2023, 44(5): 928-936 (in Chinese).
[11]	CHEN X, PAN J S, DONG J X. Bidirectional multi-scale implicit neural representations for image deraining[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 25627-25636.
[12]	QIAN X T, HANCKE G P, LAU R W H. Light source guided single-image flare removal from unpaired data[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 4177-4185.
[13]	WU Y C, HE Q R, XUE T F, et al. How to train neural networks for flare removal[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 2239-2247.
[14]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// The 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2015: 234-241.
[15]	DAI Y K, LI C Y, ZHOU S C, et al. Flare7K: a phenomenological nighttime flare removal dataset[C]// The 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2022: 284.
[16]	JIANG Y G, CHEN X H, PUN C M, et al. MFDNet: multi-Frequency Deflare Network for efficient nighttime flare removal[J]. The Visual Computer, 2024, 40(11): 7575-7588. DOI
[17]	LIANG J Y, CAO J Z, SUN G L, et al. SwinIR: image restoration using swin transformer[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 1833-1844.
[18]	陈冠豪, 徐丹, 贺康建, 等. 基于转置注意力和CNN的图像超分辨率重建网络[J]. 图学学报, 2025, 46(1): 35-46. DOI
	CHEN G H, XU D, HE K J, et al. TSA-SFNet: transpose self-attention and CNN based stereoscopic fusion network for image super-resolution[J]. Journal of Graphics, 2025, 46(1): 35-46 (in Chinese). DOI
[19]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// The 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[20]	WANG Z D, CUN X D, BAO J M, et al. Uformer: a general U-shaped transformer for image restoration[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 17662-17672.
[21]	SINHA D, EL-SHARKAWY M. Thin MobileNet: an enhanced MobileNet architecture[C]// 2019 IEEE 10th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference. New York: IEEE Press, 2019: 280-285.
[22]	KOONCE B. EfficientNet[M]// Convolutional Neural Networks with Swift for Tensorflow:Image Recognition and Dataset Categorization. Berkeley: Apress, 2021: 109-123.
[23]	FU X Y, LIANG B R, HUANG Y, et al. Lightweight pyramid networks for image deraining[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(6): 1794-1807. DOI PMID
[24]	LI B Y, PENG X L, WANG Z Y, et al. An all-in-one network for dehazing and beyond[EB/OL]. [2025-01-04]. https://arxiv.org/abs/1707.06543.
[25]	WANG T, ZHANG K H, SHEN T R, et al. Ultra-high- definition low-light image enhancement: a benchmark and transformer-based method[C]// The 37th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2023: 2654-2662.
[26]	YANG S Z, ZHANG X Y, WANG Y H, et al. DiffLLE: diffusion-based domain calibration for weak supervised low-light image enhancement[J]. International Journal of Computer Vision, 2025, 133(5): 2527-2546. DOI
[27]	SHEN Z, XU H Y, LUO T, et al. UDAformer: underwater image enhancement based on dual attention transformer[J]. Computers & Graphics, 2023, 111: 77-88. DOI URL
[28]	HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7132-7141.
[29]	QIN X, WANG Z L, BAI Y C, et al. FFA-Net: feature fusion attention network for single image dehazing[C]// The 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 11908-11915.
[30]	JIANG K, WANG Z Y, YI P, et al. Multi-scale progressive fusion network for single image deraining[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 8346-8355.
[31]	LIU X H, MA Y G, SHI Z H, et al. GridDehazeNet: attention-based multi-scale network for image Dehazing[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 7314-7323.
[32]	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2025-01-04]. ttps://dblp.uni-trier.de/db/conf/iclr/iclr2016.html#YuK15.
[33]	XU H L, LIU S H, SHU Y. Uformer++: light uformer for image restoration[C]// The 30th International Conference on Neural Information Processing. Cham: Springer, 2024: 365-376.
[34]	CHEN L Y, CHU X J, ZHANG X Y, et al. Simple baselines for image restoration[C]// The 17th European Conference on Computer Vision. Cham: Springer, 2022: 13-33.
[35]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2025-01-04]. https://dblp.uni-trier.de/db/conf/iclr/iclr2015.html#SimonyanZ14.
[36]	YU F, CHEN H F, WANG X, et al. BDD100K: a diverse driving dataset for heterogeneous multitask learning[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2636-2645.
[37]	ZHANG J, CAO Y, ZHA Z J, et al. Nighttime dehazing with a synthetic benchmark[C]// The 28th ACM International Conference on Multimedia. New York: ACM, 2020: 2355-2363.
[38]	JIN Y Y, YANG W H, TAN R T. Unsupervised night image enhancement: when layer decomposition meets light-effects suppression[C]// The 17th European Conference on Computer Vision. Cham: Springer, 2022: 404-421.