物理增强-深度协同的自由式三维超声重建

doi:10.11996/JG.j.2095-302X.2026040726

摘要/Abstract

摘要：

超声作为无辐射、便携的影像工具，其三维重建通过整合二维切面提供了更丰富的空间上下文信息，极大地扩展了传统二维超声用途。然而，现有的无传感器自由式三维超声重建研究，或受限于传统外部定位的操作妨碍，或基于深度学习方法存在缺乏对长程空间依赖的显式建模、低纹理区鲁棒性欠缺等问题。为此，提出物理增强-深度协同的自由式三维超声重建方法(PD²BNet)。该方法有机地实现了“内部物理可解释、外部深度泛化”的互补融合。输入端设计三路并行特征解耦编码器，对灰度纹理、Canny几何边缘与光流运动矢量进行独立建模，规避异质物理先验在底层卷积中的语义冲突。时空融合阶段嵌入CNN-ConvLSTM架构，递归关联局部拓扑与时序演化以抑制瞬时跳变。构建复合物理正则损失，联合速度平滑与物理先验正则损失，显著抑制帧间抖动与累积漂移，实现低纹理超声回扫场景的鲁棒性。通过显式物理约束与深度网络建模的互补融合，实现高精度无传感器6-DoF自定位，突破传统方法在物理可解释性与泛化能力上的双重局限。实验结果表明，PD²BNet在Freehand_US_data与TUS-REC-Challenge数据集上将最终漂移率分别降至16.97%和18.24%，平均角误差仅3.01°和3.27°，均优于现有最佳结果，在前臂扫查等典型临床扫描协议下实现了毫米级定位精度。量化与可视化结果证实，物理增强与深度建模的协同作用是性能跃升的关键，PD²BNet为临床无传感器自由式三维超声重建提供了高精度、高鲁棒性的算法框架支撑。

关键词: 自由式超声定位, 三维超声重建, 多特征物理先验, 物理-深度双驱动, 深度学习

Abstract:

Ultrasound, as a radiation-free and portable imaging modality, extends traditional 2D ultrasound utility through 3D reconstruction by integrating 2D cross-sectional slices to provide richer spatial contextual information. Existing sensor-free 3D freehand reconstruction either relies on cumbersome external localization or on deep-learning surrogates that overlook long-range spatial dependencies and poorly-textured regions. Therefore, PD²BNet, a physically-augmented, depth-synergized framework was proposed, which organically achieved complementary fusion of “internal physical interpretability and external deep generalization.” At the input stage, a three-branch parallel feature-decoupled encoder was designed to independently model grayscale texture, Canny geometric edges, and optical-flow motion vectors, thereby circumventing semantic conflicts among heterogeneous physical priors in low-level convolutions. During spatiotemporal fusion, a CNN-ConvLSTM architecture was embedded to recursively associate local topology with temporal evolution, suppressing instantaneous pose jumps. A composite physics-regularized loss was constructed, jointly incorporating velocity smoothing and physical prior regularization to significantly mitigate inter-frame jitter and accumulated drift, achieving robustness in low-texture ultrasound pullback scenarios. By combining explicit physical constraints with deep representations, accurate, sensor-free 6-DoF pose estimation was achieved, overcoming the interpretability and generalization limits of either paradigm alone. On Freehand_US_data and TUS-REC-Challenge, PD²BNet reduced final drift rate to 16.97 % and 18.24 %, respectively, with mean angular errors of merely 3.01° and 3.27°, outperforming state-of-the-art methods and achieving millimeter-level localization accuracy under typical clinical scanning protocols such as forearm examination. Quantitative and qualitative analyses confirmed that the synergy between physics augmentation and deep modeling drove the performance gains, establishing PD²BNet as a high-precision, robust algorithmic framework for clinical sensorless freehand 3D ultrasound reconstruction.

Key words: freehand ultrasound localization, 3D ultrasound reconstruction, multi-feature physical priors, physics-deep dual drive, deep learning

中图分类号:

TP391

李煜华, 姜杉, 杨志永, 王禹泽, 周泽洋. 物理增强-深度协同的自由式三维超声重建[J]. 图学学报, 2026, 47(4): 726-735.

LI Yuhua, JIANG Shan, YANG Zhiyong, WANG Yuze, ZHOU Zeyang. Physically-augmented and depth synergized freehand 3D ultrasound reconstruction[J]. Journal of Graphics, 2026, 47(4): 726-735.

图/表 12

参考文献 22

[1]	ZHOU R, GUO F M, AZARPAZHOOH M R, et al. A voxel-based fully convolution network and continuous max-flow for carotid vessel-wall-volume segmentation from 3D ultrasound images[J]. IEEE Transactions on Medical Imaging, 2020, 39(9): 2844-2855. DOI PMID
[2]	YANG X, DOU H R, HUANG R B, et al. Agent with warm start and adaptive dynamic termination for plane localization in 3D ultrasound[J]. IEEE Transactions on Medical Imaging, 2021, 40(7): 1950-1961. DOI URL
[3]	LI Y H, JIANG S, YANG Z Y, et al. Deformable brain pMRI-iUS registration based on higher-order cliques Markov random field[J]. Biomedical Signal Processing and Control, 2026, 113: 109146. DOI URL
[4]	PREVOST R, SALEHI M, JAGODA S, et al. 3D freehand ultrasound without external tracking using deep learning[J]. Medical Image Analysis, 2018, 48: 187-202. DOI PMID
[5]	MOZAFFARI M H, LEE W S. Freehand 3-D ultrasound imaging: a systematic review[J]. Ultrasound in Medicine & Biology, 2017, 43(10): 2099-2124. DOI URL
[6]	PU G, JIANG S, YANG Z Y, et al. A novel ultrasound probe calibration method for multimodal image guidance of needle placement in cervical cancer brachytherapy[J]. Physica Medica, 2022, 100: 81-89. DOI URL
[7]	GAO H T, HUANG Q H, XU X M, et al. Wireless and sensorless 3D ultrasound imaging[J]. Neurocomputing, 2016, 195: 159-171. DOI URL
[8]	MIURA K, ITO K, AOKI T, et al. Pose estimation of 2D ultrasound probe from ultrasound image sequences using CNN and RNN[C]// The 2nd International Workshop on Advances in Simplifying Medical Ultrasound. Cham: Springer, 2021: 96-105.
[9]	MERCIER L, LANG T, LINDSETH F, et al. A review of calibration techniques for freehand 3-D ultrasound systems[J]. Ultrasound in Medicine & Biology, 2005, 31(2): 143-165. DOI URL
[10]	SUN R, LIU C B, WANG W S, et al. UltrasOM: a mamba-based network for 3D freehand ultrasound reconstruction using optical flow[J]. Computer Methods and Programs in Biomedicine, 2025, 268: 108843. DOI URL
[11]	PREVOST R, SALEHI M, SPRUNG J, et al. Deep learning for sensorless 3D freehand ultrasound imaging[C]// The 20th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2017: 628-636.
[12]	GUO H T, CHAO H Q, XU S, et al. Ultrasound volume reconstruction from freehand scans without tracking[J]. IEEE Transactions on Biomedical Engineering, 2023, 70(3): 970-979. DOI URL
[13]	MOHAMED F, SIANG C V. A survey on 3D ultrasound reconstruction techniques[M]//ACEVES-FERNANDEZ M A. Artificial Intelligence-Applications in Medicine and Biology. London: IntechOpen, 2019: 73-92.
[14]	FARNEBÄCK G. Two-frame motion estimation based on polynomial expansion[C]// The 13th Scandinavian Conference on Image Analysis. Cham: Springer, 2003: 363-370.
[15]	CIPOLLA R, GAL Y, KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 7482-7491.
[16]	LI Q, SHEN Z Y, LI Q, et al. Long-term dependency for 3D reconstruction of freehand ultrasound without external tracker[J]. IEEE Transactions on Biomedical Engineering, 2024, 71(3): 1033-1042. DOI URL
[17]	LI Q, SHEN Z Y, YANG Q Y, et al. Nonrigid reconstruction of freehand ultrasound without a tracker[C]// The 27th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2024: 689-699.
[18]	LI Q, SHEN Z Y, LI Q, et al. Trackerless freehand ultrasound with sequence modelling and auxiliary transformation over past and future frames[C]// The 20th International Symposium on Biomedical Imaging. New York: IEEE Press, 2023: 1-5.
[19]	LUO M Y, YANG X, WANG H Z, et al. RecON: Online learning for sensorless freehand 3D ultrasound reconstruction[J]. Medical Image Analysis, 2023, 87: 102810. DOI URL
[20]	NING G C, LIANG H Y, ZHOU L, et al. Spatial position estimation method for 3D ultrasound reconstruction based on hybrid transfomers[C]// The 19th International Symposium on Biomedical Imaging. New York: IEEE Press, 2022: 1-5.
[21]	YAN Z N, YANG X, LUO M Y, et al. Fine-grained context and multi-modal alignment for freehand 3D ultrasound reconstruction[C]// The 27th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2024: 340-349.
[22]	DOU Y M, MU F Z, LI Y, et al. Sensorless end-to-end freehand 3-D ultrasound reconstruction with physics-guided deep learning[J]. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2024, 71(11): 1514-1525. DOI URL

编码器分支	输入特征(维度)	层级	操作	输出通道	特征图尺寸
分支A：灰度流	原始灰度图	1	Conv2D(1→32, k=3, p=1) + BN + ReLU + MaxPool2D(2)	32	H/2×W/2
		2	Conv2D(32→64, k=3, p=1) + BN + ReLU + MaxPool2D(2)	64	H/4×W/4
		3	Conv2D(64→128, k=3, p=1) + BN + ReLU + MaxPool2D(2)	128	H/8×W/8
分支B：边缘流	Canny距离变换	1	Conv2D(1→32, k=3, p=1) + BN + ReLU + MaxPool2D(2)	32	H/2×W/2
		2	Conv2D(32→64, k=3, p=1) + BN + ReLU + MaxPool2D(2)	64	H/4×W/4
		3	Conv2D(64→128, k=3, p=1) + BN + ReLU + MaxPool2D(2)	128	H/8×W/8
分支C：运动流	双向光流张量	1	Conv2D(4→32, k=3, p=1) + BN + ReLU + MaxPool2D(2)	32	H/2×W/2
		2	Conv2D(32→64, k=3, p=1) + BN + ReLU + MaxPool2D(2)	64	H/4×W/4
		3	Conv2D(64→128, k=3, p=1) + BN + ReLU + MaxPool2D(2)	128	H/8×W/8

编码器分支	输入特征(维度)	层级	操作	输出通道	特征图尺寸
分支A：灰度流	原始灰度图	1	Conv2D(1→32, k=3, p=1) + BN + ReLU + MaxPool2D(2)	32	H/2×W/2
		2	Conv2D(32→64, k=3, p=1) + BN + ReLU + MaxPool2D(2)	64	H/4×W/4
		3	Conv2D(64→128, k=3, p=1) + BN + ReLU + MaxPool2D(2)	128	H/8×W/8
分支B：边缘流	Canny距离变换	1	Conv2D(1→32, k=3, p=1) + BN + ReLU + MaxPool2D(2)	32	H/2×W/2
		2	Conv2D(32→64, k=3, p=1) + BN + ReLU + MaxPool2D(2)	64	H/4×W/4
		3	Conv2D(64→128, k=3, p=1) + BN + ReLU + MaxPool2D(2)	128	H/8×W/8
分支C：运动流	双向光流张量	1	Conv2D(4→32, k=3, p=1) + BN + ReLU + MaxPool2D(2)	32	H/2×W/2
		2	Conv2D(32→64, k=3, p=1) + BN + ReLU + MaxPool2D(2)	64	H/4×W/4
		3	Conv2D(64→128, k=3, p=1) + BN + ReLU + MaxPool2D(2)	128	H/8×W/8

模型编号	CNN层数	物理增强通道	物理先验正则损失	Speed loss	${{\varepsilon }_{frame}}$↓	${{\varepsilon }_{acc}}$↓	${{\varepsilon }_{drift}}$↓	FDR/%↓	MEA(°)↓
1	1	√	√	√	0.68±0.21	32.90±10.18	56.76±21.08	41.62±14.62	7.52±3.86
2	3	×	×	√	0.64±0.21	31.32±10.60	52.72±17.65	38.41±12.47	8.15±5.44
3	3	√	√	×	0.52±0.19	25.52±9.13	47.20±17.93	35.12±12.31	5.13±2.85
4	3	√	×	√	0.31±0.20	16.11±9.94	29.33±18.63	21.17±13.10	3.32±1.40
5	3	√	√	√	0.27±0.06	13.13±10.51	23.75±10.61	16.97±4.38	3.01±0.96

模型编号	CNN层数	物理增强通道	物理先验正则损失	Speed loss	${{\varepsilon }_{frame}}$↓	${{\varepsilon }_{acc}}$↓	${{\varepsilon }_{drift}}$↓	FDR/%↓	MEA(°)↓
1	1	√	√	√	0.68±0.21	32.90±10.18	56.76±21.08	41.62±14.62	7.52±3.86
2	3	×	×	√	0.64±0.21	31.32±10.60	52.72±17.65	38.41±12.47	8.15±5.44
3	3	√	√	×	0.52±0.19	25.52±9.13	47.20±17.93	35.12±12.31	5.13±2.85
4	3	√	×	√	0.31±0.20	16.11±9.94	29.33±18.63	21.17±13.10	3.32±1.40
5	3	√	√	√	0.27±0.06	13.13±10.51	23.75±10.61	16.97±4.38	3.01±0.96

输入帧数/N	FDR/%↓	MEA/°↓	单帧耗时/ms
5	21.34±9.52	4.52±2.15	8
10	16.97±4.38	3.01±0.96	12
15	16.21±5.10	2.85±1.02	25