基于语义引导区域自适应的三维高斯场景局部纹理编辑

doi:10.11996/JG.j.2095-302X.2026051072

摘要/Abstract

摘要：

三维场景局部编辑是内容生成的重要任务，在影视后期处理、数字孪生及游戏开发等领域具有显著的应用价值。三维高斯泼溅(3DGS)作为代表技术，在实时渲染方面取得关键突破，为高效编辑提供了有力支撑。现有基于扩散模型的三维高斯场景局部纹理编辑方法普遍存在的语义泄露与背景漂移问题，制约编辑质量。为此，提出一种基于语义引导区域自适应的三维场景局部纹理编辑方法。该方法旨在实现高质量纹理编辑的同时，保持非编辑区域的几何与外观一致性。首先，提出高置信度三维语义掩膜生成模块，引入视觉-语言大模型，利用LangSAM 与置信度过滤策略，通过多视角投票机制构建鲁棒的三维语义场，从源头消除语义泄露。其次，提出显著性引导的锚点视角传播模块，通过语义显著性评分自动选择最优锚点视角，配合多视角注意力一致性机制有效抑制时序闪烁。最后，提出区域自适应梯度更新策略，通过解耦前景编辑损失与背景锚定损失，对场景进行精细化优化。在Instruct-NeRF2NeRF和Mip-NeRF 360等数据集的5个编辑任务上的实验表明，与GaussianEditor和DGE等基线方法相比，该方法在CLIP Score，Background LPIPS，Background PSNR和Temporal LPIPS等定量指标上综合表现更优。在定性评估中，该方法不仅在复杂语义解耦与背景保护任务上呈现出综合性优势，还在纹理质量与几何完整性之间取得了较好的平衡。

关键词: 三维高斯泼溅, 三维场景编辑, 扩散模型, 语义引导, 区域自适应

Abstract:

Local editing of 3D scenes constitutes a fundamental task in content generation, with substantial application value in domains such as post-production, digital twins, and game development. 3D Gaussian Splatting (3DGS), as a representative technique, has achieved pivotal breakthroughs in real-time rendering, thereby providing robust support for efficient editing workflows. Nevertheless, existing diffusion-based methods for local texture editing of 3D Gaussian scenes are commonly plagued by semantic leakage and background drift, which limit editing fidelity. To tackle these challenges, a semantically guided region-adaptive approach was proposed for local texture editing of 3D scenes. The proposed method was designed to deliver high-quality texture editing while rigorously preserving the geometric and appearance consistency of non-edited regions. Concretely, a high-confidence 3D semantic mask generation module was first introduced, which incorporates a vision-language foundation model, employs LangSAM in conjunction with a confidence-filtering strategy, and constructs a robust 3D semantic field via multi-view voting, thereby eliminating semantic leakage at its source. Subsequently, a saliency-guided anchor view propagation module was devised, which automatically selects optimal anchor views based on semantic saliency scores and integrates a multi-view attention consistency mechanism to effectively mitigate temporal flickering. Finally, a region-adaptive gradient update strategy was formulated, which decouples the foreground editing loss from the background anchoring loss, enabling fine-grained scene optimization. Experiments conducted on five editing tasks across the Instruct-NeRF2NeRF and Mip-NeRF 360 datasets demonstrated that, when compared with baseline methods such as GaussianEditor and DGE, the proposed method performed better overal on quantitative metrics such as CLIP Score, Background LPIPS, Background PSNR, and Temporal LPIPS. Qualitative evaluation further showed that the proposed method not only exhibited overall advantages in complex semantic decoupling and background preservation tasks, but also achieved a favorable balance between texture quality and geometric integrity.

Key words: 3D Gaussian splatting, 3D scene editing, diffusion models, semantic guidance, region- adaptation

中图分类号:

TP 391

于冰, 金桢桢, 狄悠娴, 孙昊睿, 丁友东, 黄东晋. 基于语义引导区域自适应的三维高斯场景局部纹理编辑[J]. 图学学报, 2026, 47(5): 1072-1083.

YU Bing, JIN Zhenzhen, DI Youxian, SUN Haorui, DING Youdong, HUANG Dongjin. SGRA: semantically-guided region-adaptive local texture editing in 3D Gaussian splatting editing[J]. Journal of Graphics, 2026, 47(5): 1072-1083.

图/表 8

图1 本文方法流程图

Fig. 1 Pipeline of the proposed method

图2 在IN2N数据集的Person场景下，对于“Make the person wear clothes with a pineapple pattern”指令的编辑结果对比

Fig. 2 Comparison of editing results for the instruction “Make the person wear clothes with a pineapple pattern” in the Person scene of the IN2N dataset

图3 在IN2N数据集的Person场景下，对于“Make the person wear a black leather jacket”指令的编辑结果对比

Fig. 3 Comparison of the editing results of the prompt “Make the person wear a black leather jacket” in the Person scene of the IN2N dataset

图4 在IN2N数据集的Face场景下，对于“Turn him into a clown”指令的编辑结果对比

Fig. 4 Comparison of the editing results of the prompt “Turn him into a clown” in the Face scene of the IN2N dataset

图5 在IN2N数据集的Bear场景下，对于“Turn the bear into a real furry brown bear”指令的编辑结果对比

Fig. 5 Comparison of the editing results of the prompt “Turn the bear into a real furry brown bear” in the Bear scene of the IN2N dataset

图6 在Mip-NeRF 360数据集的Bicycle场景下，对于“Turn the white bicycle blue”指令的编辑结果对比

Fig. 6 Comparison of the editing results of the prompt “Turn the white bicycle blue” in the Bicycle scene of the Mip-NeRF 360 dataset

表1 五项编辑任务的量化评估

Table 1 Quantitative assessment of the five tasks

场景任务	方法	CLIP Score ↑	Background LPIPS ↓	Background PSNR↑	Temporal LPIPS ↓
Person (Pineapple)	GaussianEditor	0.234 9	0.133 5	14.40	0.093 4
	DGE	0.246 6	0.469 7	13.83	0.096 3
	本文	0.261 5	0.193 3	15.15	0.063 4
Person (Jacket)	GaussianEditor	0.260 2	0.429 1	11.49	0.031 3
	DGE	0.221 3	0.441 3	11.19	0.029 1
	本文	0.222 0	0.390 7	12.62	0.030 2
Face (Clown)	GaussianEditor	0.245 8	0.396 2	14.76	0.198 3
	DGE	0.243 1	0.413 1	13.39	0.185 6
	本文	0.251 7	0.383 1	15.45	0.191 4
Bear (Brown)	GaussianEditor	0.253 3	0.456 6	11.99	0.166 5
	DGE	0.258 1	0.525 5	11.55	0.161 4
	本文	0.263 3	0.450 8	12.14	0.160 6
Bicycle (Blue)	GaussianEditor	0.250 5	0.523 7	14.72	0.170 3
	DGE	0.140 6	0.534 6	15.27	0.195 8
	本文	0.248 9	0.496 8	17.67	0.124 8

表2 消融实验定量对比

Table 2 Quantitative comparison of ablation experiments

Method	Person(Pineapple)			Bear(Brown)
Method	CLIP Score↑	Background LPIPS↓	Temporal LPIPS↓	CLIP Score↑	Background LPIPS↓	Temporal LPIPS↓
w/o Background Anchor	0.264 7	0.315 2	0.095 6	0.269 8	0.642 1	0.185 6
w/o Multi-View Attention	0.268 0	0.312 1	0.098 5	0.275 9	0.560 1	0.241 2
w/o Geometry Lock	0.278 8	0.313 3	0.097 1	0.281 2	0.698 7	0.189 7
Full Method (Ours)	0.261 5	0.193 3	0.063 4	0.263 3	0.450 8	0.160 6

参考文献 33

[1]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: Representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2022, 65(1): 99-106.
[2]	KERBL B, KOPANAS G, LEIMKÜEHLER T, et al. 3D Gaussian splatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 139.
[3]	BROOKS T, HOLYNSKI A, EFROS A A. InstructPix2Pix: learning to follow image editing instructions[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 18392-18402.
[4]	ZHANG L M, RAO A Y, AGRAWALA M. Adding conditional control to text-to-image diffusion models[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 3836-3847.
[5]	POOLE B, JAIN A, BARRON J T, et al. DreamFusion:text-to-3D using 2D diffusion[EB/OL]. [2025-10-15]. https://dblp.uni-trier.de/db/conf/iclr/iclr2023.html#conf/iclr/PooleJBM23.
[6]	HAQUE A, TANCIK M, EFROS A A, et al. Instruct- NeRF2NeRF: editing 3D scenes with instructions[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 19740-19750.
[7]	KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 4015-4026.
[8]	LIU S L, ZENG Z Y, REN T H, et al. Grounding DINO: marrying DINO with grounded pre-training for open-set object detection[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2025: 38-55.
[9]	LIN C H, GAO J, TANG L M, et al. Magic3D: high-resolution text-to-3D content creation[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 300-309.
[10]	HONG S, AHN D, KIM S. Debiasing scores and prompts of 2D diffusion for view-consistent text-to-3D generation[C]// The 37th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2023: 523.
[11]	SHI Y C, WANG P, YE J L, et al. MVDream:multi-view diffusion for 3D generation[EB/OL]. [2025-10-15]. https://dblp.uni-trier.de/db/conf/iclr/iclr2024.html#conf/iclr/ShiWYMLY24.
[12]	BARRON J T, MILDENHALL B, VERBIN D, et al. Mip-NeRF 360: unbounded anti-aliased neural radiance fields[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5470-5479.
[13]	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 586-595.
[14]	黄志勇, 佘雅丽, 华喜锋, 等. DCSplat: 一种深度约束的稀疏视角三维重建方法[J]. 图学学报, 2025, 46(3): 510-519. DOI
	HUANG Z Y, SHE Y L, HUA X F, et al. DCSplat: Gaussian splatting with depth information constraints under sparse viewpoints[J]. Journal of Graphics, 2025, 46(3): 510-519 (in Chinese). DOI
[15]	卢丽华, 张晓辉, 魏辉, 等. 以神经辐射场和三维高斯泼溅为基础的文本指导三维编辑综述[J]. 中国图象图形学报, 2025, 30(5): 1238-1256.
	LU L H, ZHANG X H, WEI H, et al. Text-guided 3D editing based on neural radiance fields and 3D Gaussian splatting: a review[J]. Journal of Image and Graphics, 2025, 30(5): 1238-1256 (in Chinese). DOI URL
[16]	HE S T, JI P L, YANG Y T, et al. A survey on 3D Gaussian splatting applications:segmentation, editing, and generation[EB/OL]. [2025-10-15]. https://arxiv.org/pdf/2508.09977.
[17]	QIN M H, LI W H, ZHOU J W, et al. LangSplat: 3D language Gaussian splatting[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 20051-20060.
[18]	ZHOU S J, CHANG H R, JIANG S C, et al. Feature 3DGS: supercharging 3D Gaussian splatting to enable distilled feature fields[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 21676-21685.
[19]	RADFORD A, KIM J W, HALLACY C, et al. Learning transferable visual models from natural language supervision[EB/OL]. [2025-10-15]. https://proceedings.mlr.press/v139/radford21a.
[20]	刘高屹, 胡瑞珍, 刘利刚. 基于2D特征蒸馏的3D高斯泼溅语义分割与编辑[J]. 图学学报, 2025, 46(2): 312-321. DOI
	LIU G Y, HU R Z, LIU L G. 3D Gaussian splatting semantic segmentation and editing based on 2D feature distillation[J]. Journal of Graphics, 2025, 46(2): 312-321 (in Chinese). DOI
[21]	YE M Q, DANELLJAN M, YU F, et al. Gaussian grouping: segment and edit anything in 3D scenes[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2025: 162-179.
[22]	CEN J Z, ZHOU Z W, FANG J M, et al. Segment anything in 3D with NeRFs[C]// The 37th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2023: 1130.
[23]	DOU B, ZHANG T Y, MA Y J, et al. CoSSegGaussians:compact and swift scene segmenting 3D Gaussians with dual feature fusion[EB/OL]. [2025-10-15]. https://arxiv.org/abs/2401.05925v3.
[24]	QIAN S H, KIRSCHSTEIN T, SCHONEVELD L, et al. GaussianAvatars: photorealistic head avatars with rigged 3D Gaussians[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 20299-20309.
[25]	CHEN Y W, CHEN Z L, ZHANG C, et al. GaussianEditor: swift and controllable 3D editing with Gaussian splatting[C]// 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2024: 21476-21485.
[26]	HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[C]// The 34th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2020: 574.
[27]	SONG J M, MENG C L, ERMON S. Denoising diffusion implicit models[EB/OL]. [2025-10-15]. https://dblp.org/db/conf/iclr/iclr2021.html#conf/iclr/SongME21.
[28]	ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 10684-10695.
[29]	KIM H, JANG J H, CHUN S Y. Robust 3D-masked part-level editing in 3D Gaussian splatting with regularized score distillation sampling[C]// 2025 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2025: 5501-5510.
[30]	CHEN M H, LAINA I, VEDALDI A. DGE: direct Gaussian 3D editing by consistent multi-view editing[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2025: 74-92.
[31]	LUO Y T, LIU Q, LIU Z C. STAN: spatio-temporal attention network for next location recommendation[C]// The Web Conference 2021. New York: ACM, 2021: 2177-2185.
[32]	WU J, BIAN J W, LI X H, et al. GaussCtrl: multi-view consistent text-driven 3D Gaussian splatting editing[C]// The 18th European Conference on Computer Vision. Cham: Springer, 2025: 55-71.
[33]	WEN M H, WU S J, WANG K K, et al. InterGSEdit: interactive 3D Gaussian splatting editing with 3D geometry-consistent attention prior[C]// 2025 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2025: 26136-26145.