Saliency-based 3D indoor-scene loading strategy

doi:10.11996/JG.j.2095-302X.2026051103

Abstract

Abstract:

To support real-time walkthroughs in Web-based 3D indoor scenes such as virtual museums and digital exhibition halls, a fine-grained dynamic loading method integrating mesh saliency analysis was proposed to address long initial loading time, slow response to viewpoint changes, and low resource allocation efficiency under limited bandwidth caused by complex models and high-resolution textures. Guided by users’ visual-attention characteristics, the method incorporated saliency information throughout the entire pipeline of 3D scene preprocessing, multiresolution-structure construction, and online loading scheduling, thereby improving transmission and rendering efficiency while preserving the visual quality of key regions. First, considering that viewpoints during indoor roaming are constrained by navigation paths and that regions of interest are relatively concentrated, a guided-path-based multi-view projection sampling strategy was adopted to acquire scene images. During saliency prediction, a multi-input encoder-decoder architecture was combined with adversarial training to regress saliency maps for the projected images, and a texture-enhancement module was introduced to strengthen the representation of high-resolution textures. The resulting multi-view 2D saliency maps were then mapped back onto the 3D mesh, and neighborhood-smoothing optimization was performed to generate mesh saliency with spatially coherent distribution. Based on this distribution, saliency-priority boundaries were introduced into the V-Partitions multiresolution modeling framework, so that regions with high visual attention and semantic continuity preserved within the same triangular patch as much as possible, forming a hierarchical Directed Acyclic Graph (DAG) structure with saliency attributes. During online execution, a two-level scheduling mechanism was designed: At the scene level, global importance was computed according to the spatial relationship between objects and the user viewpoint; at the object level, local importance was determined by jointly considering saliency and screen-space error. A global priority queue is further employed to realize a patch-level dynamic scheduling framework. Experimental results showed that the method significantly reduced initial loading time and viewpoint-switching response latency, while improving both the overall PSNR of rendered images and the ROI-PSNR of salient regions. The online system supported interactive rendering of 3D scenes larger than 500 MB, providing an effective solution for Web-based 3D scene visualization.

Key words: mesh saliency, 3D scene loading, progressive transmission, dynamic loading, multi-resolution model

CLC Number:

TP 391.41

LI Xinyi, SHEN Xukun, HU Yong, WU Shan. Saliency-based 3D indoor-scene loading strategy[J]. Journal of Graphics, 2026, 47(5): 1103-1114.

Figures/Tables 12

Fig. 1 Overview of the proposed saliency detection framework for 3D indoor scenes

Fig. 2 Architecture of the texture enhancement module

Fig. 3 Multi-resolution structure (DAG) of the model

Fig. 4 Three-dimensional scene dynamic loading process

Fig. 5 The saliency results of the projection images of the scene ((a) Two-dimensional projection images; (b) The corresponding saliency prediction results)

Fig. 6 Comparison of mesh saliency prediction results in a 3D scene with and without the texture enhancement module ((a) Predictions without the texture enhancement module; (b) Predictions with the texture enhancement module; (c) Original rendered views of the scene for reference)

Fig. 7 Mesh saliency prediction results on the SCRREAM dataset

Fig. 8 Comparison of triangular patch partitioning results before and after introducing saliency information

Table 1 Comprehensive data comparison of four loading strategies

策略	首次加载时间/s			视角切换响应延迟/s			PSNR/dB
网络带宽/Mbps	5	10	15	5	10	15	5	10	15
策略A	4.7	4.3	3.3	4.7	3.8	3.0	22.92	23.39	24.33
策略B	4.8	4.2	3.5	4.3	3.4	2.7	23.14	23.63	24.50
策略C	5.0	4.4	3.1	7.1	4.6	3.4	22.39	23.87	24.35
策略D	4.4	3.5	2.8	3.6	2.2	1.7	24.17	25.12	26.31

Fig. 9 Comparison of four strategies in roaming keyframes under a 10 Mbps network condition ((a) Initial stage; (b)~(d) Rapid view-switching stage; (e) Later stage)

Fig. 10 PSNR comparison of the four strategies under a 10 Mbps condition

Table 2 ROI-PSNR at Different Top-p%/dB

ROI区域	策略A	策略B	策略C	策略D
Top-5%	23.88	23.86	23.59	25.63
Top-10%	23.70	23.90	23.41	25.44
Top-15%	23.68	23.78	23.53	25.36
Top-20%	23.66	23.73	23.38	25.20

References 26

[1]	HOPPE H. View-dependent refinement of progressive meshes[C]// The 24th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1997: 189-198.
[2]	FORGIONE T, CARLIER A, MORIN G, et al. DASH for 3D networked virtual environment[C]// The 26th ACM International Conference on Multimedia. New York: ACM, 2018: 1910-1918.
[3]	YANG B L, LI F W B, WANG X, et al. Visual saliency guided textured model simplification[J]. The Visual Computer, 2016, 32(11): 1415-1432. DOI URL
[4]	NOURI A, CHARRIER C, LEZORAY O. Global visual saliency: geometric and colorimetrie saliency fusion and its applications for 3D colored meshes[C]// 2017 Seventh International Conference on Image Processing Theory, Tools and Applications. New York: IEEE Press, 2017: 1-6.
[5]	SONG R, LIU Y H, ROSIN P L. Mesh saliency via weakly supervised classification-for-saliency CNN[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(1): 151-164. DOI URL
[6]	SONG R, ZHANG W, ZHAO Y T, et al. Mesh saliency: an independent perceptual measure or a derivative of image saliency?[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 8849-8858.
[7]	MARTIN D, FANDOS A, MASIA B, et al. SAL3D: a model for saliency prediction in 3D meshes[J]. The Visual Computer, 2024, 40(11): 7761-7771. DOI
[8]	SONG R, ZHANG W, ZHAO Y T, et al. 3D visual saliency: an independent perceptual measure or a derivative of 2D image saliency?[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 13083-13099. DOI URL
[9]	ZHANG K W, ZHU D D, MIN X K, et al. Unified approach to mesh saliency: evaluating textured and non-textured meshes through VR and multifunctional prediction[J]. IEEE Transactions on Visualization and Computer Graphics, 2025, 31(5): 3151-3160. DOI URL
[10]	ZHANG K W, ZHU D D, MIN X K, et al. Textured mesh saliency:bridging geometry and texture for human perception in 3D graphics[EB/OL]. [2025-08-27]. https://ojs.aaai.org/index.php/AAAI/article/view/33082/35237.
[11]	CHIM J, LAU R W H, LEONG H V, et al. CyberWalk: a web-based distributed virtual walkthrough environment[J]. IEEE Transactions on Multimedia, 2003, 5(4): 453-465. DOI URL
[12]	LIEVENS H, WIJNANTS M, VANDERSANDEN M, et al. Adaptive web-based VR streaming of multi-LoD 3D scenes via author-provided relevance scores[C]// 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops. New York: IEEE Press, 2021: 488-489.
[13]	JIA J Y, WANG P, WANG S, et al. An integer incremental AOI algorithm for progressive downloading of large scale VRML environments[C]// The Second International Conference on Technologies for E-Learning and Digital Entertainment. Cham: Springer, 2007: 711-722.
[14]	WANG W, JIA J Y. An incremental SMLAOI algorithm for progressive downloading large scale WebVR scenes[C]// The 14th International Conference on 3D Web Technology. New York: ACM, 2009: 55-60.
[15]	ALJAAFREH M, MAAMAR H R, BOUKERCHE A. An efficient object discovery and selection protocol in 3D streaming-based systems over thin mobile devices[C]// 2013 IEEE Wireless Communications and Networking Conference. New York: IEEE Press, 2013: 2393-2398.
[16]	SONG K W, ZENG X Y, REN C Q, et al. City-on-web: real-time neural rendering of large-scale scenes on the web[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2024: 385-402.
[17]	REN J, CHEMERYS P, SHAKHRAI V, et al. Real-time neural light field on mobile devices: EP23844312.1[P]. 2025-10-22.
[18]	WAN Z Y, RICHARDT C, BOŽIČ A, et al. Learning neural duplex radiance fields for real-time view synthesis[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 8307-8316.
[19]	CHEN Z Q, FUNKHOUSER T, HEDMAN P, et al. MobileNeRF: exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 16569-16578.
[20]	TANCIK M, CASSER V, YAN X C, et al. Block-NeRF: scalable large scene neural view synthesis[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 8238-8248.
[21]	CIGNONI P, GANOVELLI F, GOBBETTI E, et al. Batched multi triangulation[C]// VIS 05. IEEE Visualization, 2005. New York: IEEE Press, 2005: 207-214.
[22]	CHANG A X, FUNKHOUSER T, GUIBAS L, et al. ShapeNet:an information-rich 3D model repository[EB/OL]. [2025-08-27]. https://arxiv.org/abs/1512.03012.
[23]	JIANG M, HUANG S S, DUAN J Y, et al. SALICON: saliency in context[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 1072-1080.
[24]	JUNG H, LI W H, WU S C, et al. SCRREAM: scan, register, render and map: a framework for annotating accurate and dense 3D indoor scenes with a benchmark[C]// The 38th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2024: 1401.
[25]	PONCHIO F, DELLEPIANE M. Fast decompression for web-based view-dependent 3D rendering//The 20th International Conference on 3D Web Technology. New York: ACM, 2015: 199-207.
[26]	FARRUGIA J P, BILLAUD L, LAVOUE G. Adaptive streaming of 3D content for web-based virtual reality: an open-source prototype including several metrics and strategies[C]// The 14th ACM Multimedia Systems Conference. New York: ACM, 2023: 430-436.