细节增强的多曝光图像融合方法

doi:10.11996/JG.j.2095-302X.2023061130

图学学报 ›› 2023, Vol. 44 ›› Issue (6): 1130-1139.DOI: 10.11996/JG.j.2095-302X.2023061130

• 图像处理与计算机视觉 • 上一篇下一篇

细节增强的多曝光图像融合方法

夏晓华¹^,²(), 刘希恒¹, 岳鹏举¹, 邹易清³, 蒋立军³

1.长安大学工程机械学院，陕西西安 710064
2.哈尔滨工业大学机器人技术与系统国家重点实验室，黑龙江哈尔滨 150001
3.柳州欧维姆机械股份有限公司，广西柳州 545006

收稿日期:2023-05-29 接受日期:2023-09-08 出版日期:2023-12-31 发布日期:2023-12-17
作者简介:第一联系人：
夏晓华(1987-)，男，副教授，博士。主要研究方向为机器视觉与光机电一体化。E-mail：xhxia@chd.edu.cn
基金资助:
国家自然科学基金项目(61901056);中国博士后科学基金项目(2020M683631XB);机器人技术与系统国家重点实验室开放基金项目(SKLRS-2021-KF-10)

Detail-enhanced multi-exposure image fusion method

XIA Xiao-hua¹^,²(), LIU Xi-heng¹, YUE Peng-ju¹, ZOU Yi-qing³, JIANG Li-jun³

1. School of Construction Machinery, Chang'an University, Xi'an Shaanxi 710064, China
2. State Key Laboratory of Robotics and System, Harbin Institute of Technology, Harbin Heilongjiang 150001, China
3. Liuzhou OVM Machinery Co., Ltd. Liuzhou Guangxi 545006, China

Received:2023-05-29 Accepted:2023-09-08 Online:2023-12-31 Published:2023-12-17
About author:First author contact:
XIA Xiao-hua (1987-), associate professor, Ph.D. His main research interests cover machine vision and opto-mechatronics.
E-mail：xhxia@chd.edu.cn
Supported by:
National Natural Science Foundation of China(61901056);China Postdoctoral Science Foundation(2020M683631XB);State Key Laboratory of Robotics and System Open Foundation(SKLRS-2021-KF-10)

摘要/Abstract

摘要：

针对基于权重图的多曝光图像融合方法存在序列图像中偏亮与偏暗区域获得的权重较低，导致融合图像高亮与黑暗区域细节丢失的问题，提出了一种细节增强的多曝光图像融合方法。将序列图像与基于权重图的融合图像进行小波分解，提取融合图像的低频分量和边缘区域高频分量并与序列图像非边缘区域高频分量融合，经小波逆变换得到细节增强的融合图像。实验选取9组经典多曝光图像序列，分别从主观比较和客观评价2个方面与9种多曝光图像融合算法进行对比。结果表明：该方法将空间域与频率域两类图像融合方法相结合，能有效解决融合图像高亮与黑暗区域细节丢失的问题，避免频率域图像融合方法易出现振铃现象的问题，融合图像真实自然、颜色饱满，使用本文方法得到融合图像的图像信息熵均值与图像梯度均值分别为7.655 5和7.027 3，在10种多曝光图像融合算法中分别排名第一和第二。综合主观和客观评价结果，提出的方法优于9种对比方法。

关键词: 多曝光图像, 图像融合, 细节增强, 小波变换, 高动态范围

Abstract:

A detail-enhanced multi-exposure image fusion method was proposed to address the problem of low weights obtained in the light and dark regions in the sequence image, resulting in the loss of details in the bright and dark regions of the fused image. Wavelet decomposition was conducted on the sequence image and the fused image based on the weight map. This process involved extracting the low-frequency components from the fused image and the high-frequency components from the edge regions, and they were fused with the high-frequency components from the non-edge regions of the sequence image. Finally, the detail-enhanced fused image was obtained through wavelet inverse transform. Experimentally, nine sets of classical multi-exposure image sequences were selected for comparison with nine multi-exposure image fusion algorithms in terms of subjective comparison and objective evaluation, respectively. The results demonstrated that the proposed method, which combined the spatial and frequency domain image fusion methods, could effectively solve the problem of detail loss in the bright and dark areas of fused images, while avoiding the problem of ringing phenomenon often encountered in frequency domain image fusion methods. The fused images were realistic, natural, and exhibited vibrant colors. The mean image information entropy value and the mean image gradient value of the fused images obtained through the proposed method were 7.655 5 and 7.027 3, respectively, ranking first and second among the ten multi-exposure image fusion algorithms. Considering both subjective and objective evaluation results, the proposed method outperformed the nine comparative methods.

Key words: multi-exposure images, image fusion, detail enhancement, wavelet transform, high dynamic range

中图分类号:

TP391

夏晓华, 刘希恒, 岳鹏举, 邹易清, 蒋立军. 细节增强的多曝光图像融合方法[J]. 图学学报, 2023, 44(6): 1130-1139.

XIA Xiao-hua, LIU Xi-heng, YUE Peng-ju, ZOU Yi-qing, JIANG Li-jun. Detail-enhanced multi-exposure image fusion method[J]. Journal of Graphics, 2023, 44(6): 1130-1139.

图/表 12

参考文献 21

[1]	CHAURASIYA R K, RAMAKRISHNAN K R. High dynamic range imaging[C]// 2013 International Conference on Communication Systems and Network Technologies. New York: IEEE Press, 2013: 83-89.
[2]	李卫中, 易本顺, 邱康, 等. 细节保留的多曝光图像融合[J]. 光学精密工程, 2016, 24(9): 2283-2292.
	LI W Z, YI B S, QIU K, et al. Detail preserving multi-exposure image fusion[J]. Optics and Precision Engineering, 2016, 24(9): 2283-2292 (in Chinese). DOI URL
[3]	杨艳春, 高晓宇, 党建武, 等. 基于WEMD和生成对抗网络重建的红外与可见光图像融合[J]. 光学精密工程, 2022, 30(3): 320-330.
	YANG Y C, GAO X Y, DANG J W, et al. Infrared and visible image fusion based on WEMD and generative adversarial network reconstruction[J]. Optics and Precision Engineering, 2022, 30(3): 320-330 (in Chinese). DOI URL
[4]	NEJATI M, KARIMI M, SOROUSHMEHR S M R, et al. Fast exposure fusion using exposedness function[C]// 2017 IEEE International Conference on Image Processing. New York: IEEE Press, 2017: 2234-2238.
[5]	ULUCAN O, ULUCAN D, TURKAN M. Ghosting-free multi-exposure image fusion for static and dynamic scenes[J]. Signal Processing, 2023, 202: 108774. DOI URL
[6]	MERTENS T, KAUTZ J, VAN REETH F. Exposure fusion: a simple and practical alternative to high dynamic range photography[J]. Computer Graphics Forum, 2009, 28(1): 161-171. DOI URL
[7]	CHAUDHARY V, KUMAR V. Fusion of multi-exposure images using recursive and Gaussian filter[J]. Multidimensional Systems and Signal Processing, 2020, 31(1): 157-172. DOI
[8]	MA K D, LI H, YONG H W, et al. Robust multi-exposure image fusion: a structural patch decomposition approach[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2017, 26(5): 2519-2532. DOI URL
[9]	HAYAT N, IMRAN M. Ghost-free multi exposure image fusion technique using dense SIFT descriptor and guided filter[J]. Journal of Visual Communication and Image Representation, 2019, 62: 295-308. DOI URL
[10]	LI H, MA K D, YONG H W, et al. Fast multi-scale structural patch decomposition for multi-exposure image fusion[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2020: 29: 5805-5816. DOI URL
[11]	LI H, CHAN T N, QI X B, et al. Detail-preserving multi- exposure fusion with edge-preserving structural patch decomposition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(11): 4293-4304. DOI URL
[12]	ZHANG J C, LUO Y D, HUANG J B, et al. Multi-exposure image fusion via perception enhanced structural patch decomposition[J]. Information Fusion, 2023, 99: 101895. DOI URL
[13]	李明明, 王新赛, 冯小二. 基于K-means聚类与NSCT的图像融合算法[J]. 火力与指挥控制, 2022, 47(2): 146-151.
	LI M M, WANG X S, FENG X E. Research on image fusion algorithm based on K-means clustering and NSCT[J]. Fire Control & Command Control, 2022, 47(2): 146-151 (in Chinese).
[14]	HU Y X, XU C, LI Z P, et al. Detail enhancement multi- exposure image fusion based on homomorphic filtering[J]. Electronics, 2022, 11(8): 1211. DOI URL
[15]	黄慧, 董林鹭, 刘小芳, 等. 改进Retinex的低光照图像增强[J]. 光学精密工程, 2020, 28(8): 1835-1849.
	HUANG H, DONG L L, LIU X F, et al. Improved retinex low light image enhancement method[J]. Optics and Precision Engineering, 2020, 28(8): 1835-1849 (in Chinese).
[16]	HE K M, SUN J, TANG X O. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409. DOI PMID
[17]	王园园, 张娜, 韩美林. 基于Canny算子边缘检测的小波变换多聚焦图像融合方法[J]. 电子测量技术, 2019, 42(17): 148-151.
	WANG Y Y, ZHANG N, HAN M L. Wavelet transform multi-focus image fusion method based on Canny operator edge detection[J]. Electronic Measurement Technology, 2019, 42(17): 148-151 (in Chinese).
[18]	王晓柱, 钮赛赛, 张凯, 等. 基于小波变换与特征提取的红外弱小目标图像融合[J]. 西北工业大学学报, 2020, 38(4): 723-732.
	WANG X Z, NIU S S, ZHANG K, et al. Image fusion of infrared weak-small target based on wavelet transform and feature extraction[J]. Journal of Northwestern Polytechnical University, 2020, 38(4): 723-732 (in Chinese). DOI URL
[19]	常霞, 焦李成, 贾建华. 基于非下采样Contourlet的多传感器图像自适应融合[J]. 计算机学报, 2009, 32(11): 2229-2238.
	CHANG X, JIAO L C, JIA J H. Multisensor image adaptive fusion based on nonsubsampled contourlet[J]. Chinese Journal of Computers, 2009, 32(11): 2229-2238 (in Chinese).
[20]	MA K D, ZENG K, WANG Z. Perceptual quality assessment for multi-exposure image fusion[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2015, 24(11): 3345-3356. DOI URL
[21]	LEE S H, PARK J S, CHO N I. A multi-exposure image fusion based on the adaptive weights reflecting the relative pixel intensity and global gradient[C]// The 25th IEEE International Conference on Image Processing. New York: IEEE Press, 2018: 1737-1741.

Image Sequence	文献[6]	文献[4]	文献[8]	文献[21]	文献[9]	文献[10]	文献[11]	文献[12]	FDM	Proposed
Balloons	7.186 8	7.446 9	7.496 1	7.836 4	7.848 9	7.433 9	7.661 2	7.418 9	6.981 7	7.538 1
Belgium house	7.333 6	7.649 6	7.632 6	7.592 5	7.615 6	7.482 6	7.607 7	7.432 5	7.079 5	7.673 6
Cadik lamp	7.544 4	7.769 3	7.752 6	7.740 2	7.584 9	7.793 4	7.684 7	7.427 1	7.157 8	7.804 8
Cave	6.739 3	7.550 5	7.579 3	7.589 2	7.212 5	7.450 7	7.647 3	7.217 6	6.584 5	7.564 3
House	7.462 4	7.635 5	7.475 9	7.693 3	7.685 7	7.680 9	7.733 5	7.580 7	7.549 8	7.714 5
Kluki	7.480 3	7.734 7	7.821 5	7.825 8	7.739 2	7.759 3	7.738 8	7.687 0	7.913 7	7.789 3
Landscape	7.344 5	7.680 8	7.645 0	7.410 7	7.339 3	7.463 7	7.573 6	7.246 7	7.435 1	7.681 6
Lighthouse	7.401 9	7.526 1	7.589 7	7.548 5	7.559 4	7.571 2	7.507 4	7.412 2	7.755 9	7.590 1
Office	6.960 1	7.523 0	7.525 1	7.514 6	7.466 7	7.437 9	7.533 5	7.365 3	7.305 3	7.543 0
Average	7.272 6	7.612 9	7.613 1	7.639 0	7.561 4	7.563 7	7.632 0	7.420 9	7.307 0	7.655 5

Image Sequence	文献[6]	文献[4]	文献[8]	文献[21]	文献[9]	文献[10]	文献[11]	文献[12]	FDM	Proposed
Balloons	7.186 8	7.446 9	7.496 1	7.836 4	7.848 9	7.433 9	7.661 2	7.418 9	6.981 7	7.538 1
Belgium house	7.333 6	7.649 6	7.632 6	7.592 5	7.615 6	7.482 6	7.607 7	7.432 5	7.079 5	7.673 6
Cadik lamp	7.544 4	7.769 3	7.752 6	7.740 2	7.584 9	7.793 4	7.684 7	7.427 1	7.157 8	7.804 8
Cave	6.739 3	7.550 5	7.579 3	7.589 2	7.212 5	7.450 7	7.647 3	7.217 6	6.584 5	7.564 3
House	7.462 4	7.635 5	7.475 9	7.693 3	7.685 7	7.680 9	7.733 5	7.580 7	7.549 8	7.714 5
Kluki	7.480 3	7.734 7	7.821 5	7.825 8	7.739 2	7.759 3	7.738 8	7.687 0	7.913 7	7.789 3
Landscape	7.344 5	7.680 8	7.645 0	7.410 7	7.339 3	7.463 7	7.573 6	7.246 7	7.435 1	7.681 6
Lighthouse	7.401 9	7.526 1	7.589 7	7.548 5	7.559 4	7.571 2	7.507 4	7.412 2	7.755 9	7.590 1
Office	6.960 1	7.523 0	7.525 1	7.514 6	7.466 7	7.437 9	7.533 5	7.365 3	7.305 3	7.543 0
Average	7.272 6	7.612 9	7.613 1	7.639 0	7.561 4	7.563 7	7.632 0	7.420 9	7.307 0	7.655 5

Image Sequence	文献[6]	文献[4]	文献[8]	文献[21]	文献[9]	文献[10]	文献[11]	文献[12]	FDM	Proposed
Balloons	4.128 7	4.438 3	4.535 7	4.365 7	4.510 0	4.482 8	4.281 0	5.178 5	3.165 0	4.576 4
Belgium house	6.486 3	7.629 4	7.530 0	6.876 1	7.246 6	7.660 1	7.389 9	7.946 2	5.895 5	7.794 7
Cadik lamp	5.223 1	5.785 0	5.784 0	5.350 5	5.590 8	5.841 6	5.677 2	5.493 5	4.216 0	6.022 0
Cave	9.335 0	11.236 2	11.176 3	10.396 3	10.423 5	10.948 3	10.975 5	11.525 9	7.571 8	11.299 5
House	7.848 7	8.915 4	10.073 4	8.316 4	9.177 7	9.565 5	8.857 7	9.376 0	6.483 3	9.025 6
Kluki	5.803 2	7.689 2	7.516 3	6.817 8	7.877 0	7.687 0	7.201 7	8.256 8	6.203 7	7.735 4
Landscape	3.825 9	4.414 3	4.125 1	3.786 9	4.147 1	4.155 1	4.144 2	3.993 3	3.322 9	4.392 1
Lighthouse	3.552 2	4.274 5	4.187 1	3.844 2	4.498 7	4.403 4	4.206 0	4.536 0	3.636 2	4.385 4
Office	6.210 7	7.863 1	8.025 5	7.432 9	7.854 8	7.980 7	7.602 7	7.993 4	6.274 9	8.014 7
Average	5.823 8	6.916 2	6.994 8	6.354 1	6.814 0	6.969 4	6.704 0	7.144 4	5.196 6	7.027 3

Image Sequence	文献[6]	文献[4]	文献[8]	文献[21]	文献[9]	文献[10]	文献[11]	文献[12]	FDM	Proposed
Balloons	4.128 7	4.438 3	4.535 7	4.365 7	4.510 0	4.482 8	4.281 0	5.178 5	3.165 0	4.576 4
Belgium house	6.486 3	7.629 4	7.530 0	6.876 1	7.246 6	7.660 1	7.389 9	7.946 2	5.895 5	7.794 7
Cadik lamp	5.223 1	5.785 0	5.784 0	5.350 5	5.590 8	5.841 6	5.677 2	5.493 5	4.216 0	6.022 0
Cave	9.335 0	11.236 2	11.176 3	10.396 3	10.423 5	10.948 3	10.975 5	11.525 9	7.571 8	11.299 5
House	7.848 7	8.915 4	10.073 4	8.316 4	9.177 7	9.565 5	8.857 7	9.376 0	6.483 3	9.025 6
Kluki	5.803 2	7.689 2	7.516 3	6.817 8	7.877 0	7.687 0	7.201 7	8.256 8	6.203 7	7.735 4
Landscape	3.825 9	4.414 3	4.125 1	3.786 9	4.147 1	4.155 1	4.144 2	3.993 3	3.322 9	4.392 1
Lighthouse	3.552 2	4.274 5	4.187 1	3.844 2	4.498 7	4.403 4	4.206 0	4.536 0	3.636 2	4.385 4
Office	6.210 7	7.863 1	8.025 5	7.432 9	7.854 8	7.980 7	7.602 7	7.993 4	6.274 9	8.014 7
Average	5.823 8	6.916 2	6.994 8	6.354 1	6.814 0	6.969 4	6.704 0	7.144 4	5.196 6	7.027 3

Image Sequence	文献[6]	文献[4]	文献[8]	文献[21]	文献[9]	文献[10]	文献[11]	文献[12]	FDM	Proposed
Balloons	0.274 9	0.374 1	4.474 4	0.981 5	1.545 5	0.502 2	0.298 9	0.869 7	0.770 9	1.946 7
Belgium house	0.327 2	0.247 9	4.262 9	0.591 8	1.471 6	0.524 6	0.483 6	0.790 8	0.654 7	1.062 3
Cadik lamp	0.475 8	0.375 6	7.436 9	0.864 9	2.541 6	0.850 8	0.425 0	1.315 0	1.151 1	1.525 0
Cave	0.147 8	0.174 2	2.119 8	0.312 3	0.691 2	0.260 6	0.340 0	0.364 4	0.464 1	0.663 3
House	0.123 8	0.162 8	2.001 7	0.313 5	0.586 7	0.239 1	0.236 7	0.416 9	0.390 6	0.620 1
Kluki	0.099 9	0.130 6	1.625 5	0.240 8	0.467 4	0.183 1	0.238 7	0.301 9	0.361 2	0.523 7
Landscape	0.115 7	0.144 1	1.561 7	0.225 9	0.441 4	0.199 4	0.232 1	0.306 8	0.351 9	0.527 4
Lighthouse	0.107 9	0.122 5	1.667 1	0.282 4	0.440 9	0.204 2	0.289 0	0.334 2	0.416 4	0.508 9
Office	0.211 3	0.196 3	2.577 7	0.394 8	0.898 2	0.336 7	0.245 9	0.490 5	0.613 4	0.817 5
Average	0.209 4	0.214 2	3.080 9	0.467 5	1.009 4	0.366 7	0.340 0	0.576 7	0.574 9	0.910 5

细节增强的多曝光图像融合方法

Detail-enhanced multi-exposure image fusion method

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 21

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王欣雨, 刘慧, 朱积成, 盛玉瑞, 张彩明. 基于高低频特征分解的深度多模态医学图像融合网络 [J]. 图学学报, 2024, 45(1): 65-77.
[2]	范腾, 杨浩, 尹稳, 周冬明. 基于神经辐射场的多尺度视图合成研究[J]. 图学学报, 2023, 44(6): 1140-1148.
[3]	张晨阳, 曹艳华, 杨晓忠. 基于分数阶小波与引导滤波的多聚焦图像融合方法[J]. 图学学报, 2023, 44(1): 77-87.
[4]	谷昱良，羿旭明 . 基于小波变换的权重自适应图像分割模型[J]. 图学学报, 2020, 41(5): 733-739.
[5]	李鹍，甘甜. 高动态范围图像在建筑空间非视觉光环境中的运用[J]. 图学学报, 2020, 41(5): 771-778.
[6]	刘可佳，马荣生，庞钰宁 . 一种基于双导向滤波的高动态红外图像细节增强与去噪算法[J]. 图学学报, 2018, 39(6): 1048-1054.
[7]	刘建新 1，曾嫱 1，徐可 2，王亚威 1 . 基于形态学和小波变换的烟叶病斑分割[J]. 图学学报, 2018, 39(5): 933-938.
[8]	盛佳佳1,2，杨学志 1,2，董张玉 1,2，焦玮 1,2. 基于NSST-IHS 变换稀疏表示的SAR 与可见光图像融合[J]. 图学学报, 2018, 39(2): 201-208.
[9]	杨程，徐晓刚，王红军. 基于改进加权平均融合算法的图像拼接技术[J]. 图学学报, 2017, 38(增刊): 25-29.
[10]	纪峰，李泽仁，常霞，吴之亮. 基于PCA 和NSCT 变换的遥感图像融合方法[J]. 图学学报, 2017, 38(2): 247-252.
[11]	刘鑫，车翔玖，林森乔. 基于倾角校正的地震层位追踪算法[J]. 图学学报, 2015, 36(3): 418-424.
[12]	孙正. 血管内超声图像后处理综述[J]. 图学学报, 2015, 36(3): 468-476.
[13]	崔汉国，刘健鑫，李彬. 基于DD-DT CWT 和SIFT 的体数据数字水印算法[J]. 图学学报, 2015, 36(2): 148-151.
[14]	朱晓临，李雪艳，邢燕，陈嫚，朱园珠. 基于小波和奇异值分解的图像边缘检测[J]. 图学学报, 2014, 35(4): 563-570.
[15]	刘晙，茹庆云. 基于快速小波变换的高适应性图像检索技术研究[J]. 图学学报, 2014, 35(2): 262-267.