基于非局部信息的大气偏振模式生成方法

doi:10.11996/JG.j.2095-302X.2023030551

图学学报 ›› 2023, Vol. 44 ›› Issue (3): 551-559.DOI: 10.11996/JG.j.2095-302X.2023030551

• 图像处理与计算机视觉 • 上一篇下一篇

基于非局部信息的大气偏振模式生成方法

严圆¹^,²(), 高欣健¹(), 高隽¹^,², 王昕¹^,², 程前¹^,²

1.合肥工业大学计算机与信息学院，安徽合肥 230009
2.合肥工业大学图像信息处理研究室，安徽合肥 230009

收稿日期:2022-10-08 接受日期:2023-01-12 出版日期:2023-06-30 发布日期:2023-06-30
通讯作者: 高欣健(1990-)，男，副教授，博士。主要研究方向为图像处理、深度学习、人工智能和机器学习等。E-mail：gaoxinjian@hfut.edu.cn
作者简介:
严圆(1997-)，男，硕士研究生。主要研究方向为深度学习与偏振图像信息处理。E-mail：1784615175@qq.com
基金资助:
国家自然科学基金面上项目(61971177);国家自然科学基金面上项目(62171178);国家自然科学基金面上项目(62272141);中央高校基本业务经费项目(JZ2021HGTB0083)

A generative network based on non-local information for atmospheric polarization modelling

YAN Yuan¹^,²(), GAO Xin-jian¹(), GAO Jun¹^,², WANG Xin¹^,², CHENG Qian¹^,²

1. School of Computer and Information, Hefei University of Technology, Hefei Anhui 230009, China
2. Image Information Processing Laboratory, Hefei University of Technology, Hefei Anhui 230009, China

Received:2022-10-08 Accepted:2023-01-12 Online:2023-06-30 Published:2023-06-30
Contact: GAO Xin-jian (1990-), associate professor, Ph.D. His main research interests cover image processing, deep learning, artificial intelligence and machine learning, etc. E-mail：gaoxinjian@hfut.edu.cn
About author:
YAN Yuan (1997-), master student. His main research interests cover deep learning and polarization image information processing. E-mail：1784615175@qq.com
Supported by:
National Natural Science Foundation of China(61971177);National Natural Science Foundation of China(62171178);National Natural Science Foundation of China(62272141);The Fundamental Research Funds for the Central Universities(JZ2021HGTB0083)

摘要/Abstract

摘要：

大气偏振模式作为一种稳定的自然属性，因其包含具有方向信息的∞字形特征和子午线特征，在导航、探测等领域具有广泛地应用。针对实际天气条件下，大气偏振信息获取方式受限于动态云层的干扰，导致大气偏振模式的分布规律遭到破坏和部分信息缺失的问题，提出一种基于非局部信息的大气偏振模式生成方法，并设计了一种非局部信息修复模块进行两个阶段的修复，第一阶段通过挖掘不同区域的大气偏振模式空间分布的关联性和全局性，实现对缺失区域空间维度上偏振信息的修复；第二阶段利用大气偏振信息在不同时刻的特征映射关系和分布连续性，实现对云层干扰区域时间维度上特征信息的修复。在Temporal Polarization 1072偏振数据集上的实验结果定性和定量的表明，该方法能够有效地去除大气偏振模式中的云层干扰噪声，修复缺失区域的偏振信息，同时生成的结果具有更高的结构一致性和语义一致性。

关键词: 大气偏振模式, 云层干扰, 非局部信息, 空间维度, 时间维度

Abstract:

As a stable natural attribute, the atmospheric polarization mode is widely used in various fields such as navigation and detection because it contains the ∞ shape feature and meridian feature with directional information. However, obtaining atmospheric polarization information in real weather conditions is a challenging task due to the limitations imposed by dynamic cloud interference. This limitation causes the distribution law of atmospheric polarization information to be destroyed and, in turn, leads to the loss of some information. To address this issue, we proposed an atmospheric polarization mode generation method based on non-local information. Then, a non-local information inpainting block was designed for two-stage repair. In the first stage, the non-local spatial continuity information of the atmospheric polarization mode was mined to enhance the global structure between feature information and realize spatial information repair. In the second stage, the feature mapping relationship was established between atmospheric polarization information at different times, and the time continuity of the non-local atmospheric polarization information distribution was employed to repair feature information of superimposed noise regions in the time dimension. The experimental results on the Temporal Polarization 1072 polarization dataset qualitatively and quantitatively demonstrated the efficacy of this method in effectively removing cloud interference noise in the atmospheric polarization mode and repairing polarization information of missing areas, and higher structural and semantic consistency of the generated results.

Key words: atmospheric polarization mode, cloud interference, non-local information, spatial dimension, time dimension

中图分类号:

TP391

严圆, 高欣健, 高隽, 王昕, 程前. 基于非局部信息的大气偏振模式生成方法[J]. 图学学报, 2023, 44(3): 551-559.

YAN Yuan, GAO Xin-jian, GAO Jun, WANG Xin, CHENG Qian. A generative network based on non-local information for atmospheric polarization modelling[J]. Journal of Graphics, 2023, 44(3): 551-559.

图/表 10

图1 不同天气下采集大气偏振信息对比图((a)采集数据；(b)处理后数据)

Fig. 1 The comparison chart of atmospheric polarization information under different weather conditions ((a) Collected data; (b) Processed data)

图2 整体网络框架图

Fig. 2 The overall network framework

表1 编码器组成

Table 1 Encoder composition

Kernel	Stride	Dilation	Outputs
5×5	1×1	1	64
3×3	2×2	1	128
3×3	2×2	1	256
3×3	1×1	1	256
3×3	1×1	2	256
3×3	1×1	4	256
3×3	1×1	8	256

图3 非局部信息修复模块示意图

Fig. 3 The diagram of non-local information inpainting block

图4 时间信息修复过程可视化示意图

Fig. 4 The visual schematic of time information repair process

图5 不同天气下本文重构结果示意图((a)晴朗天气数据重构结果；(b)有云天气数据重构结果)

Fig. 5 The diagram of the reconstructed results in different weather ((a) The reconstruction results of sunny weather data; (b) The reconstruction results of cloudy weather data)

图6 不同面积云层干扰条件下重构结果((a)原始信息；(b)输入信息；(c)重构信息；(d)差异图)

Fig. 6 The reconstruction results under different area cloud interference conditions ((a) Original information; (b) Input information; (c) Reconstructed information; (d) Difference image)

图7 有云天气下各种模型重构结果示意图((a)原始信息；(b)输入信息；(c) CPN；(d) CRA；(e) CDS-VQ-VAE；(f) GNAPM；(g)本文网络)

Fig. 7 The schematic diagram of the results of various model reconstructions in cloud weather ((a) Original information; (b) Input information; (c) CPN; (d) CRA; (e) CDS-VQ-VAE; (f) GNAPM; (g) Ours)

表2 各方法重构结果PSNR与SSIM对比

Table 2 The comparison of PSNR and SSIM of each method reconstruction result

指标	Method	Proportion of cloud area (%)
指标	Method	Under 20	20~40	41~60	61~80	Over 80
PSNR	CPN	25.21	21.34	16.86	11.55	11.07
	CRA	24.11	20.26	15.57	12.24	11.76
	CDS-VQ-VAE	24.49	21.03	16.02	13.21	10.95
	GNAPM	26.00	22.61	18.46	15.35	12.98
	Ours	25.36	23.47	20.32	18.21	16.53
SSIM	CPN	0.970	0.864	0.804	0.699	0.638
	CRA	0.946	0.822	0.773	0.704	0.674
	CDS-VQ-VAE	0.964	0.862	0.795	0.724	0.631
	GNAPM	0.973	0.876	0.817	0.731	0.703
	Ours	0.974	0.899	0.861	0.823	0.802

表3 消融实验对比结果

Table 3 The result of ablation experiments

云层干扰面积	无时间信息修复层的生成网络		有时间信息修复层的生成网络
云层干扰面积	PSNR	SSIM	PSNR	SSIM
(0~20%)	24.12	0.965	25.34⬆5%	0.972⬆1%
(20%~40%)	21.86	0.858	23.61⬆8%	0.903⬆5%
(40%~60%)	15.43	0.769	21.02⬆36%	0.884⬆15%
(60%~80%)	13.46	0.701	18.24⬆36%	0.841⬆19%
(80%~100%)	12.35	0.686	16.91⬆37%	0.789⬆15%

参考文献 30

[1]	CHOI H, LEE K M, SEO J, et al. The influence of atmospheric composition on polarization in the GEMS spectral region[J]. Asia-Pacific Journal of Atmospheric Sciences, 2021, 57(3): 587-603. DOI
[2]	FOSTER J J, TEMPLE S E, HOW M J, et al. Polarisation vision: overcoming challenges of working with a property of light we barely see[J]. The Science of Nature, 2018, 105(3): 27. DOI
[3]	田柳, 高隽, 范之国, 等. 基于大气偏振模式分布规律的导航方向角计算方法[J]. 电子学报, 2012, 40(1): 141-146. DOI
	TIAN L, GAO J, FAN Z G, et al. The method of the navigation direction angle calculation with the distribution of the atmospheric polarization pattern[J]. Acta Electronica Sinica, 2012, 40(1): 141-146. (in Chinese) DOI
[4]	褚金奎, 关乐, 李世奇, 等. 大气偏振模式图分布及仿生偏振导航技术[J]. 遥感学报, 2018, 22(6): 969-979.
	CHU J K, GUAN L, LI S Q, et al. Atmospheric polarization field pattern distribution and polarization navigation technology[J]. Journal of Remote Sensing, 2018, 22(6): 969-979. (in Chinese)
[5]	范之国, 徐超, 吴川, 等. 大气偏振模式特征及其在自主导航中的应用[J]. 现代防御技术, 2017, 45(3): 1-7.
	FAN Z G, XU C, WU C, et al. Characteristics of atmospheric polarization pattern and its application in autonomous navigation[J]. Modern Defence Technology, 2017, 45(3): 1-7. (in Chinese)
[6]	XU Q, GUO Z Y, TAO Q Q, et al. Multi-spectral characteristics of polarization retrieve in various atmospheric conditions[J]. Optics Communications, 2015, 339: 167-170. DOI URL
[7]	YAN L, LI Y F, CHANDRASEKAR V, et al. General review of optical polarization remote sensing[J]. International Journal of Remote Sensing, 2020, 41(13): 4853-4864. DOI URL
[8]	晏磊, 关桂霞, 陈家斌, 等. 基于天空偏振光分布模式的仿生导航定向机理初探[J]. 北京大学学报: 自然科学版, 2009, 45(4): 616-620.
	YAN L, GUAN G X, CHEN J B, et al. The bionic orientation mechanism in the skylight polarization pattern[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2009, 45(4): 616-620. (in Chinese)
[9]	吴良海, 张骏, 范之国, 等. 多次散射因素影响下天空偏振光模式的解析模型[J]. 物理学报, 2014, 63(11): 114201.
	WU L H, ZHANG J, FAN Z G, et al. An analytical model for skylight polarization pattern with multiple scattering[J]. Acta Physica Sinica, 2014, 63(11): 114201. (in Chinese) DOI URL
[10]	赵开春, 卢皓, 尤政. 天空光偏振模式自动探测装置[J]. 光学精密工程, 2013, 21(2): 239-245.
	ZHAO K C, LU H, YOU Z. Automatic detection system for skylight polarized pattern[J]. Optics and Precision Engineering, 2013, 21(2): 239-245. (in Chinese) DOI URL
[11]	陈伟, 李延飞, 吴太夏, 等. 全天空偏振模式及其影响因素初探[J]. 遥感学报, 2018, 22(6): 989-995.
	CHEN W, LI Y F, WU T X, et al. Preliminary research on sky polarization and influential factors[J]. Journal of Remote Sensing, 2018, 22(6): 989-995. (in Chinese)
[12]	KONG Z, MA T, CHEN K, et al. Three-wavelength polarization Scheimpflug lidar system developed for remote sensing of atmospheric aerosols[J]. Applied Optics, 2019, 58(31): 8612-8621. DOI URL
[13]	魏昊明, 赵威, 戴星灿. 雾和气溶胶前向散射对消光的影响[J]. 光学精密工程, 2018, 26(6): 1354-1361.
	WEI H M, ZHAO W, DAI X C. Influence of fog and aerosol particles’ forward-scattering on light extinction[J]. Optics and Precision Engineering, 2018, 26(6): 1354-1361. (in Chinese) DOI URL
[14]	UGOLNIKOV O S, POSTYLYAKOV O V, MASLOV I A. Effects of multiple scattering and atmospheric aerosol on the polarization of the twilight sky[J]. Journal of Quantitative Spectroscopy and Radiative Transfer, 2004, 88(1-3): 233-241. DOI URL
[15]	强振平, 何丽波, 陈旭, 等. 深度学习图象修复方法综述[J]. 中国图象图形学报, 2019, 24(3): 447-463.
	QIANG Z P, HE L B, CHEN X, et al. Survey on deep learning image inpainting methods[J]. Jurnal of Image and Graphics, 2019, 24(3): 447-463. (in Chinese)
[16]	PATHAK D, KRÄHENBÜHL P, DONAHUE J, et al. Context encoders: feature learning by inpainting[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 2536-2544.
[17]	LIU G L, REDA F A, SHIH K J, et al. Image inpainting for irregular holes using partial convolutions[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 89-105.
[18]	YU J H, LIN Z, YANG J M, et al. Generative image inpainting with contextual attention[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 5505-5514.
[19]	LI J Y, WANG N, ZHANG L F, et al. Recurrent feature reasoning for image inpainting[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 7757-7765.
[20]	IIZUKA S, SIMO-SERRA E, ISHIKAWA H. Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 2017, 36(4): 1-14.
[21]	甘鑫, 高欣健, 钟彬彬, 等. 基于有限样本的大气偏振模式生成方法[J]. 光电工程, 2021, 48(5): 200331.
	GAN X, GAO X J, ZHONG B B, et al. A few-shot learning based generative method for atmospheric polarization modelling[J]. Opto-Electronic Engineering, 2021, 48(5): 13-27. (in Chinese)
[22]	赵露露, 沈玲, 洪日昌. 图像修复研究进展综述[J]. 计算机科学, 2021, 48(3): 14-26. DOI
	ZHAO L L, SHEN L, HONG R C. Survey on image inpainting research progress[J]. Computer Science, 2021, 48(3): 14-26. (in Chinese)
[23]	程前, 高欣健, 高隽, 等. 基于邻域约束的大气偏振模式生成网络[J]. 光电工程, 2022, 49(6): 55-69.
	CHENG Q, GAO X J, GAO J, et al. A generative method for atmospheric polarization modelling based on neighborhood constraint[J]. Opto-Electronic Engineering, 2022, 49(6): 55-69. (in Chinese)
[24]	LEE S, OH S W, WON D, et al. Copy-and-paste networks for deep video inpainting[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2020: 4412-4420.
[25]	YI Z L, TANG Q, AZIZI S, et al. Contextual residual aggregation for ultra high-resolution image inpainting[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 7505-7514.
[26]	HUANG T, LI S J, JIA X, et al. Neighbor2Neighbor: self-supervised denoising from single noisy images[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 14776-14785.
[27]	LI A, ZHAO S S, MA X J, et al. Short-term and long-term context aggregation network for video inpainting[C]// European Conference on Computer Vision - ECCV 2020. Cham: Springer International Publishing, 2020: 728-743.
[28]	PENG J L, LIU D, XU S C, et al. Generating diverse structure for image inpainting with hierarchical VQ-VAE[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 10770-10779.
[29]	LAVIGNE D A, BRETON M, FOURNIER G, et al. A new passive polarimetric imaging system collecting polarization signatures in the visible and infrared bands[EB/OL]. [2022-08-20]. https:spie.org/Publications/Proceedings/Paper/10.1117/12.819011?ss0=1.
[30]	DANG T Y, LIU Y T, GAO X J, et al. Multi-scale spatial transform network for atmospheric polarization prediction[C]// International Conference on Image and Graphics - ICIG 2021. Cham: Springer International Publishing, 2021: 479-490.

基于非局部信息的大气偏振模式生成方法

A generative network based on non-local information for atmospheric polarization modelling

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 30

相关文章 1

编辑推荐

Metrics

本文评价