基于显著性检测的三维室内场景加载策略

doi:10.11996/JG.j.2095-302X.2026051103

摘要/Abstract

摘要：

面向虚拟博物馆、数字展馆等Web端三维室内场景的实时漫游需求，针对复杂模型与高分辨率纹理导致的初始加载时间长、视角切换响应慢和有限带宽下资源分配效率低等问题，提出一种融合网格显著性分析的细粒度动态加载方法。其以用户视觉关注特征为引导，将显著性信息贯穿于三维场景预处理、多分辨率结构构建与在线加载调度全过程，在保障关键区域视觉质量的同时提升传输与渲染效率。首先，针对室内漫游过程中视点分布受路径约束、关注区域相对集中等特点，采用基于引导路径的多视角投影采样方式获取场景图像；在显著性预测阶段，结合多输入编码器-解码器结构与对抗训练机制，对投影图进行显著性回归，并加入纹理增强模块以强化高分辨率纹理的表征能力；随后将多视角二维显著性结果映射回三维网格，并通过邻域平滑优化生成空间分布连续的网格显著性。基于所获得的显著性分布，在V-Partitions多分辨率建模框架中引入显著性优先级边界，使高关注度且语义连续的区域尽可能保留在同一个三角补丁中，构建带有显著性属性的层次化DAG结构。在线运行阶段，设计双层调度机制：在场景层依据物体与用户视点的空间关系计算全局重要性，在物体层面综合显著性与屏幕空间误差计算局部重要性，并通过全局优先队列实现补丁级动态调度框架。实验结果表明，首次加载时间和视角切换响应延迟显著降低，同时提升了整体画面的PSNR以及显著区域的ROI-PSNR，在线运行系统支持500 MB以上的三维场景交互，为Web端三维场景可视化提供了有效的解决方案。

关键词: 网格显著性, 三维场景加载, 渐进传输, 动态加载, 多分辨率模型

Abstract:

To support real-time walkthroughs in Web-based 3D indoor scenes such as virtual museums and digital exhibition halls, a fine-grained dynamic loading method integrating mesh saliency analysis was proposed to address long initial loading time, slow response to viewpoint changes, and low resource allocation efficiency under limited bandwidth caused by complex models and high-resolution textures. Guided by users’ visual-attention characteristics, the method incorporated saliency information throughout the entire pipeline of 3D scene preprocessing, multiresolution-structure construction, and online loading scheduling, thereby improving transmission and rendering efficiency while preserving the visual quality of key regions. First, considering that viewpoints during indoor roaming are constrained by navigation paths and that regions of interest are relatively concentrated, a guided-path-based multi-view projection sampling strategy was adopted to acquire scene images. During saliency prediction, a multi-input encoder-decoder architecture was combined with adversarial training to regress saliency maps for the projected images, and a texture-enhancement module was introduced to strengthen the representation of high-resolution textures. The resulting multi-view 2D saliency maps were then mapped back onto the 3D mesh, and neighborhood-smoothing optimization was performed to generate mesh saliency with spatially coherent distribution. Based on this distribution, saliency-priority boundaries were introduced into the V-Partitions multiresolution modeling framework, so that regions with high visual attention and semantic continuity preserved within the same triangular patch as much as possible, forming a hierarchical Directed Acyclic Graph (DAG) structure with saliency attributes. During online execution, a two-level scheduling mechanism was designed: At the scene level, global importance was computed according to the spatial relationship between objects and the user viewpoint; at the object level, local importance was determined by jointly considering saliency and screen-space error. A global priority queue is further employed to realize a patch-level dynamic scheduling framework. Experimental results showed that the method significantly reduced initial loading time and viewpoint-switching response latency, while improving both the overall PSNR of rendered images and the ROI-PSNR of salient regions. The online system supported interactive rendering of 3D scenes larger than 500 MB, providing an effective solution for Web-based 3D scene visualization.

Key words: mesh saliency, 3D scene loading, progressive transmission, dynamic loading, multi-resolution model

中图分类号:

TP 391.41

李欣怡, 沈旭昆, 胡勇, 吴珊. 基于显著性检测的三维室内场景加载策略[J]. 图学学报, 2026, 47(5): 1103-1114.

LI Xinyi, SHEN Xukun, HU Yong, WU Shan. Saliency-based 3D indoor-scene loading strategy[J]. Journal of Graphics, 2026, 47(5): 1103-1114.

图/表 12

图1 三维室内场景显著性检测流程

Fig. 1 Overview of the proposed saliency detection framework for 3D indoor scenes

图2 纹理增强模块设计

Fig. 2 Architecture of the texture enhancement module

图3 模型的多分辨率结构(DAG)

Fig. 3 Multi-resolution structure (DAG) of the model

图4 三维场景动态加载流程

Fig. 4 Three-dimensional scene dynamic loading process

图5 场景二维投影图的显著性检测结果((a) 二维投影图片；(b) 对应的显著性预测结果)

Fig. 5 The saliency results of the projection images of the scene ((a) Two-dimensional projection images; (b) The corresponding saliency prediction results)

图6 三维场景网格显著性预测结果对比(有/无纹理增强模块)((a) 未引入纹理增强模块的预测结果；(b) 引入纹理增强模块后的预测结果；(c) 场景原始渲染视图)

Fig. 6 Comparison of mesh saliency prediction results in a 3D scene with and without the texture enhancement module ((a) Predictions without the texture enhancement module; (b) Predictions with the texture enhancement module; (c) Original rendered views of the scene for reference)

图7 SCRREAM数据集网格显著性预测结果

Fig. 7 Mesh saliency prediction results on the SCRREAM dataset

图8 引入显著性信息前后三角补丁划分结果对比

Fig. 8 Comparison of triangular patch partitioning results before and after introducing saliency information

表1 4种加载策略综合数据对比

Table 1 Comprehensive data comparison of four loading strategies

策略	首次加载时间/s			视角切换响应延迟/s			PSNR/dB
网络带宽/Mbps	5	10	15	5	10	15	5	10	15
策略A	4.7	4.3	3.3	4.7	3.8	3.0	22.92	23.39	24.33
策略B	4.8	4.2	3.5	4.3	3.4	2.7	23.14	23.63	24.50
策略C	5.0	4.4	3.1	7.1	4.6	3.4	22.39	23.87	24.35
策略D	4.4	3.5	2.8	3.6	2.2	1.7	24.17	25.12	26.31

图9 4种策略在10 Mbps网络条件下的漫游关键帧对比((a) 初始阶段；(b)~(d) 快速视角切换阶段；(e) 后期阶段)

Fig. 9 Comparison of four strategies in roaming keyframes under a 10 Mbps network condition ((a) Initial stage; (b)~(d) Rapid view-switching stage; (e) Later stage)

图10 10 Mbps测试环境下4种策略对应的PSNR

Fig. 10 PSNR comparison of the four strategies under a 10 Mbps condition

表2 不同Top-p%下的ROI-PSNR/dB

Table 2 ROI-PSNR at Different Top-p%/dB

ROI区域	策略A	策略B	策略C	策略D
Top-5%	23.88	23.86	23.59	25.63
Top-10%	23.70	23.90	23.41	25.44
Top-15%	23.68	23.78	23.53	25.36
Top-20%	23.66	23.73	23.38	25.20

参考文献 26

[1]	HOPPE H. View-dependent refinement of progressive meshes[C]// The 24th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1997: 189-198.
[2]	FORGIONE T, CARLIER A, MORIN G, et al. DASH for 3D networked virtual environment[C]// The 26th ACM International Conference on Multimedia. New York: ACM, 2018: 1910-1918.
[3]	YANG B L, LI F W B, WANG X, et al. Visual saliency guided textured model simplification[J]. The Visual Computer, 2016, 32(11): 1415-1432. DOI URL
[4]	NOURI A, CHARRIER C, LEZORAY O. Global visual saliency: geometric and colorimetrie saliency fusion and its applications for 3D colored meshes[C]// 2017 Seventh International Conference on Image Processing Theory, Tools and Applications. New York: IEEE Press, 2017: 1-6.
[5]	SONG R, LIU Y H, ROSIN P L. Mesh saliency via weakly supervised classification-for-saliency CNN[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(1): 151-164. DOI URL
[6]	SONG R, ZHANG W, ZHAO Y T, et al. Mesh saliency: an independent perceptual measure or a derivative of image saliency?[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 8849-8858.
[7]	MARTIN D, FANDOS A, MASIA B, et al. SAL3D: a model for saliency prediction in 3D meshes[J]. The Visual Computer, 2024, 40(11): 7761-7771. DOI
[8]	SONG R, ZHANG W, ZHAO Y T, et al. 3D visual saliency: an independent perceptual measure or a derivative of 2D image saliency?[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 13083-13099. DOI URL
[9]	ZHANG K W, ZHU D D, MIN X K, et al. Unified approach to mesh saliency: evaluating textured and non-textured meshes through VR and multifunctional prediction[J]. IEEE Transactions on Visualization and Computer Graphics, 2025, 31(5): 3151-3160. DOI URL
[10]	ZHANG K W, ZHU D D, MIN X K, et al. Textured mesh saliency:bridging geometry and texture for human perception in 3D graphics[EB/OL]. [2025-08-27]. https://ojs.aaai.org/index.php/AAAI/article/view/33082/35237.
[11]	CHIM J, LAU R W H, LEONG H V, et al. CyberWalk: a web-based distributed virtual walkthrough environment[J]. IEEE Transactions on Multimedia, 2003, 5(4): 453-465. DOI URL
[12]	LIEVENS H, WIJNANTS M, VANDERSANDEN M, et al. Adaptive web-based VR streaming of multi-LoD 3D scenes via author-provided relevance scores[C]// 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops. New York: IEEE Press, 2021: 488-489.
[13]	JIA J Y, WANG P, WANG S, et al. An integer incremental AOI algorithm for progressive downloading of large scale VRML environments[C]// The Second International Conference on Technologies for E-Learning and Digital Entertainment. Cham: Springer, 2007: 711-722.
[14]	WANG W, JIA J Y. An incremental SMLAOI algorithm for progressive downloading large scale WebVR scenes[C]// The 14th International Conference on 3D Web Technology. New York: ACM, 2009: 55-60.
[15]	ALJAAFREH M, MAAMAR H R, BOUKERCHE A. An efficient object discovery and selection protocol in 3D streaming-based systems over thin mobile devices[C]// 2013 IEEE Wireless Communications and Networking Conference. New York: IEEE Press, 2013: 2393-2398.
[16]	SONG K W, ZENG X Y, REN C Q, et al. City-on-web: real-time neural rendering of large-scale scenes on the web[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 385-402.
[17]	REN J, CHEMERYS P, SHAKHRAI V, et al. Real-time neural light field on mobile devices: EP23844312.1[P]. 2025-10-22.
[18]	WAN Z Y, RICHARDT C, BOŽIČ A, et al. Learning neural duplex radiance fields for real-time view synthesis[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8307-8316.
[19]	CHEN Z Q, FUNKHOUSER T, HEDMAN P, et al. MobileNeRF: exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 16569-16578.
[20]	TANCIK M, CASSER V, YAN X C, et al. Block-NeRF: scalable large scene neural view synthesis[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 8238-8248.
[21]	CIGNONI P, GANOVELLI F, GOBBETTI E, et al. Batched multi triangulation[C]// VIS 05. IEEE Visualization, 2005. New York: IEEE Press, 2005: 207-214.
[22]	CHANG A X, FUNKHOUSER T, GUIBAS L, et al. ShapeNet:an information-rich 3D model repository[EB/OL]. [2025-08-27]. https://arxiv.org/abs/1512.03012.
[23]	JIANG M, HUANG S S, DUAN J Y, et al. SALICON: saliency in context[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 1072-1080.
[24]	JUNG H, LI W H, WU S C, et al. SCRREAM: scan, register, render and map: a framework for annotating accurate and dense 3D indoor scenes with a benchmark[C]// The 38th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2024: 1401.
[25]	PONCHIO F, DELLEPIANE M. Fast decompression for web-based view-dependent 3D rendering//The 20th International Conference on 3D Web Technology. New York: ACM, 2015: 199-207.
[26]	FARRUGIA J P, BILLAUD L, LAVOUE G. Adaptive streaming of 3D content for web-based virtual reality: an open-source prototype including several metrics and strategies[C]// The 14th ACM Multimedia Systems Conference. New York: ACM, 2023: 430-436.