Multi-focus image fusion method based on fractional wavelet combined with guided filtering

doi:10.11996/JG.j.2095-302X.2023010077

Abstract

Abstract:

To address the problems of losing details and producing artifacts at image edges in multi-focus image fusion, a new multi-focus image fusion method was proposed based on discrete fractional wavelet transform (DFRWT) combined with guided filtering. First, DFRWT was utilized to decompose the source images at multiple scales, obtaining the low frequency part and high frequency part. Then, in light of the energy distribution traits of wavelet modulus coefficients at different orders, the most suitable fractional order was selected. In the low frequency part, the initial decision was obtained by applying the Laplacian energy sum, and then the fusion rule was yielded by modifying the decision graph with guided filtering. For the high frequency part, the fractional spatial frequency fusion rule was adopted. Such rules, which can improve the efficiency of the image information on fusion. Lastly, the inverse DFRWT processing was carried out to obtain the composite image. The new method was compared with the existing five algorithms for visual comparison experiments and quantitative evaluation. The simulation experiments show that the proposed method could effectively suppress Gibbs effects and edge artifacts effects. The validity and advantages of the proposed method were verified in terms of the visual effect and objective evaluation, outperforming several classical algorithms in the quality of image fusion.

Key words: multi-focus image fusion, discrete fractional wavelet transform, guided filtering, fractional spatial frequency

CLC Number:

TP391.41

ZHANG Chen-yang, CAO Yan-hua, YANG Xiao-zhong. Multi-focus image fusion method based on fractional wavelet combined with guided filtering[J]. Journal of Graphics, 2023, 44(1): 77-87.

Figures/Tables 9

Fig. 1 Two-level DFRWT decomposition coefficient diagram (p=1) ((a) Approximate component of the first layer; (b) Detail component of the first layer; (c) Approximate component of the second layer; (d) Detail component of the second layer)

Fig. 2 Two-layer DFRWT decomposition coefficient diagram (p=0.5) ((a) Approximate component of the first layer; (b) Detail component of the first layer; (c) Approximate component of the second layer; (d) Detail component of the second layer)

Fig. 3 DFRWT decomposition results of the first-layer about Lena image ((a) p=1; (b) p=0.9; (c) p=0.7; (d) p=0.5; (e) p=0.1; (f) Original image and reconstructed image)

Fig. 4 Smoothing effect images of guided filtering ((a) Original image; (b) ε=0.01; (c) ε=0.04; (d) ε=0.16)

Fig. 5 Source images ((a) Clock left focused image; (b) Clock right focused image; (c) Pepsi left focuesd image; (d) Pepsi right focuesd image; (e) Peppers pre-focused image; (f) Peppers post-focused image)

Fig. 6 Results of experiment 1 ((a) L_P; (b) DWT; (c) PCNN; (d) NSCT; (e) DFRWT; (f) Ours)

Fig. 7 Results of experiment 2 ((a) L_P; (b) DWT; (c) PCNN; (d) NSCT; (e) DFRWT; (f) Ours)

Fig. 8 Results of experiment 3 ((a) L_P; (b) DWT; (c) PCNN; (d) NSCT; (e) DFRWT; (f) Ours)

Table 1 Objective evaluation results of image fusion

实验及指标				融合方法
实验及指标	L_P^[1]	DWT^[8]	PCNN^[25]	NSCT^[26]	DFRWT_var^[27]	本文方法
实验1					p=0.4	p=0.5
MI	4.533 1	4.653 8	4.742 9	4.813 9	4.756 4	5.120 8
Q_AB/F	0.520 6	0.607 2	0.654 0	0.675 2	0.693 7	0.740 8
SF	5.983 6	6.535 0	7.687 8	7.976 2	8.877 9	9.097 8
实验2					p=0.6	p=0.5
MI	4.7323	4.949 5	4.707 4	5.015 2	4.838 4	5.103 7
Q_AB/F	0.551 0	0.651 5	0.621 9	0.698 1	0.654 8	0.736 1
SF	5.468 7	6.434 9	6.242 3	7.034 6	7.259 1	8.216 9
实验3					p=0.6	p=0.4
MI	4.767 4	4.810 2	4.623 0	4.922 8	4.773 7	5.018 6
Q_AB/F	0.601 9	0.669 4	0.611 7	0.681 7	0.643 3	0.698 4
SF	7.598 3	7.328 4	8.610 3	9.033 1	10.316 2	12.658 8

References 28

[1]	MITCHELL H B. Image fusion: theories, techniques and applications[M]. Cham: Springer International Publishin, 2010: 64-91.
[2]	敬忠良, 肖刚, 李振华. 图像融合——理论与应用[M]. 北京: 高等教育出版社, 2007: 9-22.
	JING Z L, XIAO G, LI Z H. Image fusion: theory and applications[M]. Beijing: Higher Education Press, 2007: 9-22 (in Chinese).
[3]	王敏, 周树道. 数字图像预处理技术及应用[M]. 北京: 科学出版社, 2021: 135-147.
	WANG M, ZHOU S D. Digital image preprocessing technology and its application[M]. Beijing: Science Press, 2021: 135-147 (in Chinese).
[4]	CHAO Z, DUAN X G, JIA S F, et al. Medical image fusion via discrete stationary wavelet transform and an enhanced radial basis function neural network[J]. Applied Soft Computing, 2022, 118: 108542. DOI URL
[5]	PANGULURI S K, MOHAN L. An effective fuzzy logic and particle swarm optimization based thermal and visible-light image fusion framework using curvelet transform[J]. Optik, 2021, 243: 167529. DOI URL
[6]	余腾, 胡伍生, 孙小荣, 等. 基于非下采样Contourlet变换耦合能量相似制约的遥感图像融合算法[J]. 电子测量与仪器学报, 2021, 35(6): 71-78.
	YU T, HU W S, SUN X R, et al. Remote sensing image fusion algorithm based on nonsubsampled contourlet transform combined with energy similarity restriction[J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(6): 71-78 (in Chinese).
[7]	纪峰, 李泽仁, 常霞, 等. 基于PCA和NSCT变换的遥感图像融合方法[J]. 图学学报, 2017, 38(2): 247-252.
	JI F, LI Z R, CHANG X, et al. Remote sensing image fusion method based on PCA and NSCT transform[J]. Journal of Graphics, 2017, 38(2): 247-252 (in Chinese).
[8]	MALLAT S. A wavelet tour of signal processing[M]. 2nd ed. New York: Academic Press, 1999: 15-28.
[9]	孙延奎. 小波变换与图像、图形处理技术[M]. 2版. 北京: 清华大学出版社, 2018: 18-44.
	SUN Y K. Wavelet transform and image and graphic processing technology[M]. 2nd ed. Beijing: Tsinghua University Press, 2018: 18-44 (in Chinese).
[10]	WANG M, ZHOU S D. Image denoising using block-rotation-based SVD filtering in wavelet domain[J]. IEICE Transactions on Information and Systems, 2018, E101.D(6): 1621-1628. DOI URL
[11]	王珏, 张秀英, 蔡玉芳, 等. 联合小波变换和RSF模型的CT图像分割方法[J]. 光学学报, 2020, 40(21): 57-65.
	WANG J, ZHANG X Y, CAI Y F, et al. CT image segmentation method combining wavelet transform and RSF model[J]. Acta Optica Sinica, 2020, 40(21): 57-65 (in Chinese).
[12]	谷昱良, 羿旭明. 基于小波变换的权重自适应图像分割模型[J]. 图学学报, 2020, 41(5): 733-739.
	GU Y L, YI X M. Adaptive weights image segmentation model based on wavelet transform[J]. Journal of Graphics, 2020, 41(5): 733-739 (in Chinese).
[13]	BHAT S, KOUNDAL D. Multi-focus image fusion using neutrosophic based wavelet transform[J]. Applied Soft Computing, 2021, 106: 107307. DOI URL
[14]	MENDLOVIC D, ZALEVSKY Z, MAS D, et al. Fractional wavelet transform[J]. Applied Optics, 1997, 36(20): 4801-4806. PMID
[15]	史军, 沙学军, 张钦宇. 分数阶信号处理理论与方法[M]. 哈尔滨: 哈尔滨工业大学出版社, 2017: 112-218.
	SHI J, SHA X J, ZHANG Q Y. Fractional order signal processing theory and methods[M]. Harbin: Harbin Institute of Technology Press, 2017: 112-218 (in Chinese).
[16]	史军, 张乃通, 刘晓萍. 一种新型分数阶小波变换及其应用[J]. 中国科学: 信息科学, 2012, 42(2): 125-135.
	SHI J, ZHANG N T, LIU X P. A novel fractional wavelet transform and its applications[J]. Scientia Sinica: Informationis, 2012, 42(2): 125-135 (in Chinese).
[17]	SHI J, LIU X P, ZHANG N T. Multiresolution analysis and orthogonal wavelets associated with fractional wavelet transform[J]. Signal, Image and Video Processing, 2015, 9(1): 211-220. DOI URL
[18]	XU X J, WANG Y R, CHEN S. Medical image fusion using discrete fractional wavelet transform[J]. Biomedical Signal Processing and Control, 2016, 27: 103-111. DOI URL
[19]	LI C M, YANG X Z. Multifocus image fusion method using discrete fractional wavelet transform and improved fusion rules[J]. Journal of Modern Optics, 2021, 68(5): 246-258. DOI URL
[20]	HE K M, SUN J, TANG X O. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409. DOI PMID
[21]	LI S T, KANG X D, HU J W. Image fusion with guided filtering[J]. IEEE Transactions on Image Processing: A Publication of the IEEE Signal Processing Society, 2013, 22(7): 2864-2875. DOI URL
[22]	刘明葳, 王任华, 李静, 等. 各向异性导向滤波的红外与可见光图像融合[J]. 中国图象图形学报, 2021, 26(10): 2421-2432.
	LIU M W, WANG R H, LI J, et al. Infrared and visible image fusion with multi-scale anisotropic guided filtering[J]. Journal of Image and Graphics, 2021, 26(10): 2421-2432 (in Chinese).
[23]	周激流, 蒲亦非, 廖科. 分数阶微积分原理及其在现代信号分析与处理中的应用[M]. 北京: 科学出版社, 2010: 14-32.
	ZHOU J L, PU Y F, LIAO K. Fractional calculus principle and its application in modern signal analysis and processing[M]. Beijing: Science Press, 2010: 14-32 (in Chinese).
[24]	HUANG W, JING Z L. Evaluation of focus measures in multi-focus image fusion[J]. Pattern Recognition Letters, 2007, 28(4): 493-500. DOI URL
[25]	荣传振, 贾永兴, 杨宇, 等. NSCT域内基于自适应PCNN的图像融合新方法[J]. 信号处理, 2017, 33(3): 280-287.
	RONG C Z, JIA Y X, YANG Y, et al. A novel image fusion algorithm using adaptive PCNN in NSCT domain[J]. Journal of Signal Processing, 2017, 33(3): 280-287 (in Chinese).
[26]	LI B, PENG H, WANG J. A novel fusion method based on dynamic threshold neural P systems and nonsubsampled contourlet transform for multi-modality medical images[J]. Signal Processing, 2021, 178: 107793. DOI URL
[27]	徐小军, 王友仁, 陈帅. 基于下采样分数阶小波变换的图像融合新方法[J]. 仪器仪表学报, 2014, 35(9): 2061-2069.
	XU X J, WANG Y R, CHEN S. Novel image fusion method based on downsampling fractional wavelet transform[J]. Chinese Journal of Scientific Instrument, 2014, 35(9): 2061-2069 (in Chinese).
[28]	XYDEAS C S, PETROVIĆ V. Objective image fusion performance measure[J]. Electronics Letters, 2000, 36(4): 308. DOI URL