Monocular depth estimation method with hierarchical dual-stream attention

doi:10.11996/JG.j.2095-302X.2026030553

Abstract

Abstract:

Monocular depth estimation has attracted sustained attention due to its broad application prospects in autonomous driving, 3D reconstruction, and related fields. However, existing methods still exhibit deficiencies in multi-scale feature fusion and depth modeling, which makes it difficult to simultaneously capture fine-grained local geometric details and maintain global structural consistency, and have limited adaptability to depth scale distributions across different scenes. To address these issues, a monocular depth estimation framework integrating hierarchical dual-stream attention and adaptive depth discretization was developed. In the encoding stage, a Swin Transformer was employed to construct pyramid multi-scale feature representations, enhancing the joint modeling of local and global information. In the decoding stage, a hierarchical dual-stream attention fusion network was designed to model local detail perception and global contextual semantics in parallel during progressive reconstruction, in which adaptive feature fusion was achieved through dynamic weight modulation and cross-attention mechanisms. Meanwhile, a depth recovery module was introduced to formulate depth estimation as a joint classification-regression task. By predicting a discrete depth distribution and adaptively learning the bin centers, continuous depth values were generated via probability-weighted aggregation, which ensured the continuity of depth prediction while effectively maintaining the consistency of depth relationships within the scene. Experimental results demonstrated that the proposed method achieved state-of-the-art performance on the KITTI dataset, with an AbsRel of 0.048, a SqRel of 0.147, a log10 error of 0.020, and a δ? accuracy of 0.980, and exhibited favorable generalization capability on the NYU Depth V2 dataset, validating its effectiveness and robustness under complex scenes and multi-scale depth distributions.

Key words: deep learning, monocular depth estimation, pyramid structure, attention mechanism, feature fusion

CLC Number:

TP391

WU Wenhuan, WANG Wenshu, WANG Shuao. Monocular depth estimation method with hierarchical dual-stream attention[J]. Journal of Graphics, 2026, 47(3): 553-563.

Figures/Tables 8

References 37

[1]	LEE Y, JEON J, KO Y, et al. Task-driven deep image enhancement network for autonomous driving in bad weather[C]//2021 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2021: 13746-13753.
[2]	CHO J, RAHIMPOUR S, CUTLER A, et al. Enhancing reality: a systematic review of augmented reality in neuronavigation and education[J]. World Neurosurgery, 2020, 139: 186-195. DOI PMID
[3]	BROOKS S P. Markov chain Monte Carlo method and its application[J]. Journal of the Royal Statistical Society. Series D (The Statistician), 1998, 47(1): 69-100.
[4]	EIGEN D, PUHRSCH C, FERGUS R. Depth map prediction from a single image using a multi-scale deep network[C]//The 28th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2366-2374.
[5]	LAINA I, RUPPRECHT C, BELAGIANNIS V, et al. Deeper depth prediction with fully convolutional residual networks[C]//2016 Fourth international conference on 3D vision. New York: IEEE Press, 2016: 239-248.
[6]	ARAMPATZAKIS V, PAVLIDIS G, MITIANOUDIS N, et al. Monocular depth estimation: a thorough review[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024, 46(4): 2396-2414. DOI URL
[7]	RANFTL R, BOCHKOVSKIY A, KOLTUN V. Vision transformers for dense prediction[C]//2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 12159-12168.
[8]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words:transformers for image recognition at scale[EB/OL]. (2020-10-22) [2025-01-29]. https://arxiv.org/pdf/2010.11929.pdf.
[9]	LI Z Y, CHEN Z H, LIU X M, et al. Depthformer: exploiting long-range correlation and local information for accurate monocular depth estimation[J]. Machine Intelligence Research, 2023, 20(6): 837-854. DOI
[10]	RANFTL R, LASINGER K, HAFNER D, et al. Towards robust monocular depth estimation: mixing datasets for zero-shot cross-dataset transfer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(3): 1623-1637. DOI URL
[11]	BHAT S F, ALHASHIM I, WONKA P. AdaBins: depth estimation using adaptive bins[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 4008-4017.
[12]	AGARWAL A, ARORA C. Attention attention everywhere: monocular depth prediction with skip attention[C]//2023 IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2023: 5850-5859.
[13]	YANG L H, KANG B Y, HUANG Z L, et al. Depth anything: unleashing the power of large-scale unlabeled data[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 10371-10381.
[14]	YANG L H, KANG B Y, HUANG Z L, et al. Depth anything V2[C]//The 38th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2024: 688.
[15]	CHEN S L, GUO H K, ZHU S N, et al. Video depth anything: consistent depth estimation for super-long videos[C]//2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2025: 22831-22840.
[16]	ZHANG N, NEX F, VOSSELMAN G, et al. Lite-mono: a lightweight CNN and transformer architecture for self-supervised monocular depth estimation[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 18537-18546.
[17]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 9992-10002.
[18]	OUYANG D L, HE S, ZHANG G Z, et al. Efficient multi-scale attention module with cross-spatial learning[C]//IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE Press, 2023: 1-5.
[19]	DONG X Y, BAO J N, CHEN D D, et al. CSWin transformer: a general vision transformer backbone with cross-shaped windows[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 12114-12124.
[20]	EIGEN D, FERGUS R. Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture[C]//2015 IEEE International Conference on Computer Vision. New York: IEEE Press, 2015: 2650-2658.
[21]	GEIGER A, LENZ P, STILLER C, et al. Vision meets robotics: the kitti dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237. DOI URL
[22]	SILBERMAN N, HOIEM D, KOHLI P, et al. Indoor segmentation and support inference from rgbd images[C]//The 12th European Conference on Computer Vision. Cham: Springer, 2012: 746-760.
[23]	PASZKE A, GROSS S, MASSA F, et al. PyTorch: an imperative style, high-performance deep learning library[EB/OL]. (2019-12-03) [2025-01-29]. https://arxiv.org/pdf/1912.01703.pdf.
[24]	KINGMA D P, BA J L. Adam: a method for stochastic optimization[EB/OL]. (2014-12-22) [2025-01-29]. https://arxiv.org/pdf/1412.6980.pdf.
[25]	FU H, GONG M M, WANG C H, et al. Deep ordinal regression network for monocular depth estimation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 2002-2011.
[26]	LEE J H, HAN M K, KO D W, et al. From big to small: multi-scale local planar guidance for monocular depth estimation[EB/OL]. (2019-07-25) [2025-01-29]. https://arxiv.org/pdf/1907.10326.pdf.
[27]	YANG G L, TANG H, DING M L, et al. Transformer-based attention networks for continuous pixel-wise prediction[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 16249-16259.
[28]	SHAO S W, LI R, PEI Z C, et al. NENet: monocular depth estimation via neural ensembles[EB/OL]. (2021-11-16) [2025-01-29]. https://arxiv.org/abs/2111.08313v1.
[29]	SHAO S W, LI R, PEI Z C, et al. Towards comprehensive monocular depth estimation: multiple heads are better than one[J]. IEEE Transactions on Multimedia, 2023, 25: 7660-7671. DOI URL
[30]	PATIL V, SAKARIDIS C, LINIGER A, et al. P3Depth: monocular depth estimation with a piecewise planarity prior[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 1600-1611.
[31]	YUAN W H, GU X D, DAI Z Z, et al. Neural window fully-connected crfs for monocular depth estimation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 3906-3915.
[32]	BAE J, MOON S, IM S. Deep digging into the generalization of self-supervised monocular depth estimation[C]//The 37th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2023: 187-196.
[33]	SHIM D, KIM H J. SwinDepth: unsupervised depth estimation using monocular sequences via swin transformer and densely cascaded network[C]//2023 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2023: 4983-4990.
[34]	WANG B Y, WANG S, YE D, et al. Deep neighbor layer aggregation for lightweight self-supervised monocular depth estimation[C]//ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE Press, 2024: 4405-4409.
[35]	HUYNH L, NGUYEN-HA P, MATAS J, et al. Guiding monocular depth estimation using depth-attention volume[C]// The 16th European Conference on Computer Vision. Cham: Springer, 2020: 581-597.
[36]	YIN W, LIU Y F, SHEN C H, et al. Enforcing geometric constraints of virtual normal for depth prediction[C]//2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 5683-5692.
[37]	LV C, HAN C G, LANG J, et al. GDM-depth: leveraging global dependency modelling for self-supervised indoor depth estimation[J]. Image and Vision Computing, 2024, 149: 105160. DOI URL

Method	SqRel↓	AbsRel↓	RMSE↓	δ<1.25↑	δ<1.25²↑	δ<1.25³↑
Eigen^[20]	1.548	0.203	6.307	0.769	0.950	0.988
DORN^[25]	0.307	0.072	2.727	0.932	0.984	0.994
BTS^[26]	0.245	0.059	2.756	0.956	0.993	0.998
Adabins^[11]	0.190	0.058	2.360	0.964	0.995	0.999
TransDepth^[27]	0.252	0.064	2.755	0.956	0.994	0.998
NENet^[28]	0.213	0.059	2.543	0.961	0.995	0.999
TEDepth^[29]	0.174	0.056	2.223	0.968	0.996	0.999
P3Depth^[30]	0.270	0.071	2.842	0.953	0.993	0.998
NeWCRFs^[31]	0.155	0.052	2.129	0.974	0.997	0.999
MonoFormer^[32]	0.846	0.104	4.580	0.891	0.962	0.982
SwinDepth^[33]	0.739	0.106	4.510	0.890	0.964	0.984
DepthFormer^[9]	0.158	0.052	2.143	0.975	0.997	0.999
DNA-Depth^[34]	0.682	0.097	4.357	0.902	0.968	0.984
Ours	0.147	0.048	2.094	0.980	0.997	0.999

Method	SqRel↓	AbsRel↓	RMSE↓	δ<1.25↑	δ<1.25²↑	δ<1.25³↑
Eigen^[20]	1.548	0.203	6.307	0.769	0.950	0.988
DORN^[25]	0.307	0.072	2.727	0.932	0.984	0.994
BTS^[26]	0.245	0.059	2.756	0.956	0.993	0.998
Adabins^[11]	0.190	0.058	2.360	0.964	0.995	0.999
TransDepth^[27]	0.252	0.064	2.755	0.956	0.994	0.998
NENet^[28]	0.213	0.059	2.543	0.961	0.995	0.999
TEDepth^[29]	0.174	0.056	2.223	0.968	0.996	0.999
P3Depth^[30]	0.270	0.071	2.842	0.953	0.993	0.998
NeWCRFs^[31]	0.155	0.052	2.129	0.974	0.997	0.999
MonoFormer^[32]	0.846	0.104	4.580	0.891	0.962	0.982
SwinDepth^[33]	0.739	0.106	4.510	0.890	0.964	0.984
DepthFormer^[9]	0.158	0.052	2.143	0.975	0.997	0.999
DNA-Depth^[34]	0.682	0.097	4.357	0.902	0.968	0.984
Ours	0.147	0.048	2.094	0.980	0.997	0.999

Method	AbsRel↓	RMSE↓	log10↓	δ<1.25↑	δ<1.25²↑	δ<1.25³↑
Eigen^[20]	0.158	0.641	-	0.769	0.950	0.988
DORN^[25]	0.115	0.509	0.051	0.828	0.965	0.992
BTS^[26]	0.110	0.392	0.047	0.885	0.978	0.994
TransDepth^[27]	0.106	0.365	0.045	0.900	0.983	0.996
Adabins^[11]	0.103	0.364	0.044	0.903	0.984	0.997
DAV^[35]	0.108	0.412	-	0.882	0.980	0.996
VNL^[36]	0.108	0.416	0.048	0.875	0.976	0.994
TEDepth^[29]	0.100	0.349	0.043	0.907	0.987	0.998
P3Depth^[30]	0.104	0.356	0.043	0.898	0.981	0.996
NENet^[28]	0.100	0.349	0.043	0.907	0.987	0.998
NeWCRFs^[31]	0.095	0.334	0.041	0.922	0.992	0.998
DepthFormer^[9]	0.096	0.339	0.041	0.921	0.989	0.998
PixelFormer^[12]	0.090	0.322	0.039	0.929	0.991	0.998
GDM-Depth^[37]	0.113	0.439	0.049	0.872	0.972	0.993
Ours	0.087	0.316	0.037	0.935	0.992	0.998

Method	AbsRel↓	RMSE↓	log10↓	δ<1.25↑	δ<1.25²↑	δ<1.25³↑
Eigen^[20]	0.158	0.641	-	0.769	0.950	0.988
DORN^[25]	0.115	0.509	0.051	0.828	0.965	0.992
BTS^[26]	0.110	0.392	0.047	0.885	0.978	0.994
TransDepth^[27]	0.106	0.365	0.045	0.900	0.983	0.996
Adabins^[11]	0.103	0.364	0.044	0.903	0.984	0.997
DAV^[35]	0.108	0.412	-	0.882	0.980	0.996
VNL^[36]	0.108	0.416	0.048	0.875	0.976	0.994
TEDepth^[29]	0.100	0.349	0.043	0.907	0.987	0.998
P3Depth^[30]	0.104	0.356	0.043	0.898	0.981	0.996
NENet^[28]	0.100	0.349	0.043	0.907	0.987	0.998
NeWCRFs^[31]	0.095	0.334	0.041	0.922	0.992	0.998
DepthFormer^[9]	0.096	0.339	0.041	0.921	0.989	0.998
PixelFormer^[12]	0.090	0.322	0.039	0.929	0.991	0.998
GDM-Depth^[37]	0.113	0.439	0.049	0.872	0.972	0.993
Ours	0.087	0.316	0.037	0.935	0.992	0.998

Dataset	Method	Abs Rel ↓	RMSE ↓	Sq Rel ↓	δ<1.25↑
KITTI	B	0.060	2.430	0.180	0.960
	B+E	0.056	2.239	0.163	0.967
	B+C	0.054	2.225	0.161	0.971
	B+ E+C	0.056	2.205	0.158	0.970
	B+E+C+F	0.050	2.160	0.152	0.974
	B+ E+C+F+M (Ours)	0.048	2.094	0.147	0.980
NYU	B	0.103	0.342	0.054	0.910
	B+E	0.098	0.333	0.050	0.917
	B+C	0.095	0.334	0.047	0.922
	B+ E+C	0.094	0.331	0.047	0.923
	B+E+C+F	0.091	0.325	0.044	0.929
	B+ E+C+F+M (Ours)	0.087	0.316	0.040	0.934