面向高光子通量环境的目标深度估计方法

doi:10.11996/JG.j.2095-302X.2025040756

图学学报 ›› 2025, Vol. 46 ›› Issue (4): 756-762.DOI: 10.11996/JG.j.2095-302X.2025040756

• 图像处理与计算机视觉 • 上一篇下一篇

面向高光子通量环境的目标深度估计方法

杨佳熙¹(), 于乐天¹, 包骐瑞¹, 毕胜², 麻晓斗¹, 杨晟琦³, 姜雨彤⁴, 方建儒⁵, 魏小鹏¹, 杨鑫¹()

1.大连理工大学计算机学院社会计算与认知智能教育部重点实验室，辽宁大连 116024
2.大连理工大学机械工程学院，辽宁大连 116024
3.中国航空工业集团公司沈阳飞机设计研究所，辽宁沈阳 110035
4.中国北方车辆研究所先进越野系统技术全国重点实验室，北京 100072
5.大连亚明汽车部件股份有限公司，辽宁大连 116024

收稿日期:2024-07-05 接受日期:2025-03-22 出版日期:2025-08-30 发布日期:2025-08-11
通讯作者:杨鑫(1984-)，男，教授，博士。主要研究方向为计算机图形学与视觉、机器学习与大数据技术、机器人技术。E-mail：xinyang@dlut.edu.cn
第一作者:杨佳熙(1999-)，男，硕士研究生。主要研究方向为计算机视觉。E-mail：517542583@qq.com
基金资助:
国家重点研发项目(2022ZD0210500)

Object depth estimation methods for high photon flux environments

YANG Jiaxi¹(), YU Letian¹, BAO Qirui¹, BI Sheng², MA Xiaodou¹, Yang Shengqi³, JIANG Yutong⁴, FANG Jianru⁵, WEI Xiaopeng¹, YANG Xin¹()

1. Key Laboratory of Social Computing and Cognitive Intelligence, School of Computer Science, Dalian University of Technology, Dalian Liaoning 116024, China
2. School of Mechanical Engineering, Dalian University of Technology, Dalian Liaoning 116024, China
3. Shenyang Aircraft Design and Research Institute, Aviation Industry Corporation of China, Shenyang Liaoning 110035, China
4. Chinese Scholartree Ridge State Key Laboratory, China North Vehicle Research Institute, Beijing 100072, China
5. Dalian Yaming Automotive Parts Co., Ltd., Dalian Liaoning 116024, China

Received:2024-07-05 Accepted:2025-03-22 Published:2025-08-30 Online:2025-08-11
First author：YANG Jiaxi (1999-), master student. His main research interest covers computer vision. E-mail：517542583@qq.com
Supported by:
National Key Research and Development Program of China(2022ZD0210500)

摘要/Abstract

摘要：

单光子雪崩二极管(SPAD)的高时间分辨率特性及高精度特性为其开辟了广泛的应用空间，尤其是在对算法性能要求日益增长的计算机视觉、计算成像等领域。SPAD能对各种常见目标进行精确度较高的深度估计，但SPAD每次探测到光子后会进入一段无法探测的猝灭期。这导致在环境中光子数量较多时，同一脉冲周期内更早到达SPAD的光子有更大概率被采集，使得最终形成的光子数量统计曲线明显向时间轴短的方向偏移，且偏移程度随着光子通量(即单位时间内探测光子数量)的增加而扩大。该现象被称为堆积效应(Pileup effect)，其降低了深度估计算法的准确性。对于这一问题，搭建了用于采集SPAD光子数据的单光子探测系统，并在几种不同光子通量下采集了一个针对SPAD深度估计任务中堆积效应进行研究的目标深度数据集。在此基础上，设计了一个将光子通量作为全局特征进行学习的深度估计网络，其融合了SPAD探测结果中的局部空间特征和全局光子通量特征，在几种存在堆积效应的光子通量下均取得了较高的深度估计性能。

关键词: 单光子雪崩二极管, 光子通量, 堆积效应, 深度估计, 自注意力机制

Abstract:

The high temporal-resolution and high-precision characteristics of single-photon avalanche diode (SPAD) have opened up a wide range of applications, especially in fields such as computer vision and computational imaging with increasing algorithmic performance demands. Accurate depth estimation can be achieved for various targets using SPAD measurements, however, every time when SPAD device detects a photon, it will enter an undetectable quenching period. When there are a large number of photons in the environment, photon arrivals are more likely to be recorded in earlier bins than later bins, resulting in an obvious histogram distortion towards the shorter temporal axis, while the degree of distortion exacerbates with the increase of photon flux (the number of photons detected per unit time). This phenomenon, known as the Pileup Effect, reduces the accuracy of depth estimation algorithms. In this paper, a SPAD-based prototype was first constructed to collect single-photon measurements under several different photon-flux settings, and a single-photon based dataset was developed to study the pileup effect for depth estimation vision tasks. Based on our dataset, a depth estimation network was then designed to learn photon-flux as global features, simultaneously integrating the local spatial features and global flux-based features from SPAD measurements. Extensive experiments demonstrated that our network significantly achieved superior depth estimation performance under several different photon-flux settings with pileup effects.

Key words: single-photon avalanche diode, photon flux, pileup effect, depth estimation, self-attention mechanism

中图分类号:

TN312.7

杨佳熙, 于乐天, 包骐瑞, 毕胜, 麻晓斗, 杨晟琦, 姜雨彤, 方建儒, 魏小鹏, 杨鑫. 面向高光子通量环境的目标深度估计方法[J]. 图学学报, 2025, 46(4): 756-762.

YANG Jiaxi, YU Letian, BAO Qirui, BI Sheng, MA Xiaodou, Yang Shengqi, JIANG Yutong, FANG Jianru, WEI Xiaopeng, YANG Xin. Object depth estimation methods for high photon flux environments[J]. Journal of Graphics, 2025, 46(4): 756-762.

图/表 10

图1 基于SPAD测量值的深度估计网络整体结构

Fig. 1 Network architecture for depth estimation based on SPAD measurements

图2 局部特征提取模块

Fig. 2 Local feature extraction module

图3 包含全局注意力引导块的全局特征提取模块

Fig. 3 Global feature extraction module with global self-attention guidance block

图4 特征融合上采样解码模块

Fig. 4 Feature fusion and upsample decoder module

图5 数据集的10个目标及基准深度图

Fig. 5 Ten objects and corresponding ground truth map in our dataset

图6 光子通量每秒约500 K时的实验结果

Fig. 6 Experimental results at photon flux of approximately 500 K per second ((a) RGB; (b) GT; (c) Shin; (d) SP3I; (e) PE3I; (f) BPE; (g) Ours)

图7 光子通量每秒约1 M时的实验结果

Fig. 7 Experimental results at photon flux of approximately 1 M per second ((a) RGB; (b) GT; (c) Shin; (d) SP3I; (e) PE3I; (f) BPE; (g) Ours)

图8 光子通量每秒约5 M时的实验结果

Fig. 8 Experimental results at photon flux of approximately 5 M per second ((a) RGB; (b) GT; (c) Shin; (d) SP3I; (e) PE3I; (f) BPE; (g) Ours)

表1 目标深度数据集上的量化结果

Table 1 Quantitative Results on target depth estimation dataset

光子通量	模型
光子通量	Shin^[14]	SP3I^[7]	PE3I^[9]	BFE^[10]	Ours
每秒约500 K	0.061	0.053	0.037	0.029	0.024
每秒约1 M	0.092	0.098	0.074	0.065	0.033
每秒约5 M	0.141	0.147	0.118	0.110	0.052

表2 全局特征提取模块消融分析

Table 2 Ablation analysis on global feature extraction module

光子通量	测试结果
光子通量	不包含全局特征	包含全局特征
每秒约500 K	0.048	0.024
每秒约1 M	0.092	0.033
每秒约5 M	0.141	0.052

参考文献 14

[1]	COVA S, GHIONI M, LOTITO A, et al. Evolution and prospects for single-photon avalanche diodes and quenching circuits[J]. Journal of Modern Optics, 2004, 51(9/10): 1267-1288.
[2]	VELTEN A, WU D, JARABO A, et al. Femto-photography[J]. ACM Transactions on Graphics, 2013, 32(4): 1-8.
[3]	VILLA F, BRONZI D, ZOU Y, et al. CMOS SPADs with up to 500 μm diameter and 55% detection efficiency at 420 nm[J]. Journal of Modern Optics, 2014, 61(2): 102-115.
[4]	GAO L, LIANG J Y, LI C Y, et al. Single-shot compressed ultrafast photography at one hundred billion frames per second[J]. Nature, 2014, 516(7529): 74-77.
[5]	PUSZKA A, DI SIENO L, MORA A D, et al. Spatial resolution in depth for time-resolved diffuse optical tomography using short source-detector separations[J]. Biomedical Optics Express, 2015, 6(1): 1-10. DOI PMID
[6]	HEIDE F, DIAMOND S, LINDELL D B, et al. Sub- picosecond photon-efficient 3D imaging using single-photon sensors[J]. Scientific Reports, 2018, 8(1): 17726.
[7]	LINDELL D B, O'TOOLE M, Wetzstein G. Single-photon 3D imaging with deep sensor fusion[J]. ACM Transactions on Graphics, 2018, 37(4): 1-12.
[8]	TAN H, PENG J Y, XIONG Z W, et al. Deep learning based single-photon 3D imaging with multiple returns[C]// 2020 International Conference on 3D Vision. New York: IEEE Press, 2020: 1196-1205.
[9]	PENG J Y, XIONG Z W, HUANG X, et al. Photon-efficient 3D imaging with a non-local neural network[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 225-241.
[10]	PENG J Y, XIONG Z W, TAN H, et al. Boosting photon- efficient image reconstruction with a unified deep neural network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(4): 4180-4197.
[11]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// The 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[12]	WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7794-7803.
[13]	BAR-DAVID I. Communication under the Poisson regime[J]. IEEE Transactions on Information Theory, 1969, 15(1): 31-37.
[14]	SHIN D, KIRMANI A, GOYAL V K, et al. Photon-efficient computational 3-D and reflectivity imaging with single-photon detectors[J]. IEEE Transactions on Computational Imaging, 2015, 1(2): 112-125.

面向高光子通量环境的目标深度估计方法

Object depth estimation methods for high photon flux environments

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 14

相关文章 14

编辑推荐

Metrics

本文评价

[1]	陈冠豪, 徐丹, 贺康建, 施洪贞, 张浩. 基于转置注意力和CNN的图像超分辨率重建网络[J]. 图学学报, 2025, 46(1): 35-46.
[2]	苑朝, 赵明雪, 张丰羿, 冯晓勇, 李冰, 陈瑞. 基于点云特征增强的复杂室内场景3D目标检测[J]. 图学学报, 2025, 46(1): 59-69.
[3]	彭文, 林金炜. 基于空间信息关注和纹理增强的短小染色体分类方法[J]. 图学学报, 2024, 45(5): 1017-1029.
[4]	刘宗明, 洪唯, 龙睿, 祝越, 张小宇. 基于自注意机制的乳源瑶绣自动生成与应用研究[J]. 图学学报, 2024, 45(5): 1096-1105.
[5]	李滔, 胡婷, 武丹丹. 结合金字塔结构和注意力机制的单目深度估计[J]. 图学学报, 2024, 45(3): 454-463.
[6]	余伟群, 刘佳涛, 张亚萍. 融合注意力的拉普拉斯金字塔单目深度估计[J]. 图学学报, 2023, 44(4): 728-738.
[7]	廖志伟, 金兢, 张超凡, 杨学志. 基于分层压缩激励的 ASPP 网络单目深度估计[J]. 图学学报, 2022, 43(2): 214-222.
[8]	牟琦, 张寒, 何志强, 李占利 . 基于深度估计和特征融合的尺度自适应目标跟踪算法[J]. 图学学报, 2021, 42(4): 563-571.
[9]	蒋素琴, 张梦骏, 李蔚清, 苏智勇 . 基于投票决策的实时遮挡处理技术[J]. 图学学报, 2021, 42(4): 629-635.
[10]	常东良 , 尹军辉 , 谢吉洋 , 孙维亚 , 马占宇 . 面向图像分类的基于注意力引导的 Dropout[J]. 图学学报, 2021, 42(1): 32-36.
[11]	张永鹏, 张春梅, 白静. 基于 DenseNet-Attention 模型的高光谱图像分类[J]. 图学学报, 2020, 41(6): 897-904.
[12]	何也, 张旭东, 吴迪. 特征融合网络：多通道信息融合的光场深度估计 [J]. 图学学报, 2020, 41(6): 922-929.
[13]	贾瑞明，刘立强，刘圣杰，崔家礼 . 基于编解码卷积神经网络的单张图像深度估计[J]. 图学学报, 2019, 40(4): 718-724.
[14]	温静，安国艳，梁宇栋 . 基于 CNN 特征提取和加权深度迁移的单目图像深度估计[J]. 图学学报, 2019, 40(2): 248-255.