一种带高光处理的无缝纹理映射方法

doi:10.11996/JG.j.2095-302X.2024010148

图学学报 ›› 2024, Vol. 45 ›› Issue (1): 148-158.DOI: 10.11996/JG.j.2095-302X.2024010148

• 计算机图形学与虚拟现实 • 上一篇下一篇

一种带高光处理的无缝纹理映射方法

石敏¹(), 王炳祺¹, 李兆歆²(), 朱登明³^,⁴

1.华北电力大学控制与计算机工程学院，北京 102206
2.中国农业科学院农业信息研究所，北京 100081
3.中国科学院计算技术研究所前瞻研究实验室，北京 100190
4.太仓中科信息技术研究院，江苏太仓 215400

收稿日期:2023-06-25 接受日期:2023-10-12 出版日期:2024-02-29 发布日期:2024-02-29
通讯作者:李兆歆(1983-)，男，助理研究员，博士。主要研究方向为三维重建和三维计算机视觉等。E-mail：cszli@hotmail.com
第一作者:石敏(1975-)，女，副教授，博士。主要研究方向为虚拟现实。E-mail：shi_min@ncepu.edu.cn
基金资助:
国家重点研发计划项目(2020YFB1710400);国家自然科学基金项目(61972379);山东省新旧动能转换重大产业攻关项目(2021-55);2023年苏州市前沿技术研究项目

A seamless texture mapping method with highlight processing

SHI Min¹(), WANG Bingqi¹, LI Zhaoxin²(), ZHU Dengming³^,⁴

1. School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
2. Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing, 100081, China
3. Prospective Research Laboratory, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
4. Taicang-CAS Institute of Information and Technology, Taicang Jiangsu 215400, China

Received:2023-06-25 Accepted:2023-10-12 Published:2024-02-29 Online:2024-02-29
First author：SHI Min (1975-), associate professor, Ph.D. Her main research interest covers virtual reality. E-mail：shi_min@ncepu.edu.cn
Supported by:
National Key Research and Development Plan Subject(2020YFB1710400);National Natural Science Foundation of China(61972379);Major Industrial Projects to Transform Old and New Growth Drivers in Shandong Province(2021-55);2023 Suzhou Cutting-edge Technology Research Project

摘要/Abstract

摘要：

纹理映射作为三维重建的重要步骤，直接关系到生成模型的视觉效果。传统的纹理映射方法通常使用简单的混合方法来获取表面纹理。通过将模型表面投影到每个纹理图像上来获取相应的纹理区域，然后将它们混合在一起以获取表面纹理。然而，由于相机位姿的不准确性，纹理映射结果存在明显的模糊和鬼影等问题。此外，在高光环境下获取的纹理图像很容易包含高光区域，最终导致纹理颜色的丢失，降低纹理的真实性。为了解决这些问题，提出了一种无缝纹理映射方法，可以有效消除高光反射。通过衡量纹理图像的质量，为每个模型表面选择最佳的纹理图像，并利用色度一致性约束优化的相机位姿以消除明显的纹理错位。针对高光问题，提出了一个高光处理模块，使用多视图图像信息并基于双色反射模型定位和处理高光纹理。最后，对纹理图进行纹理颜色一致性调整，以处理纹理之间的颜色差异。实验结果表明，与现有方法相比，提出的算法可以有效消除高光反射的影响，从而获得更好的纹理映射结果。

华北电力大学石敏副教授及其学生王炳祺等针对复杂光照条件下的重建模型的纹理真实感下降问题，提出了一种带高光处理的纹理映射方法。首先构建能量函数为模型面选择最佳纹理，而后利用色度一致性约束对相机位姿进行校正，并利用多视图输入信息定位并处理高光纹理，最后进行纹理颜色一致性调整以消除纹理接缝。通过在多种数据中进行测试，验证了算法能够有效地提升复杂光照条件下重建模型纹理的真实感。

关键词: 三维重建, 纹理映射, 高光处理, 纹理渲染, 位姿校正

Abstract:

As an important step of 3D reconstruction, texture mapping is directly related to the visual effect of the generative model. Classical texture mapping methods usually employ a simple blending method to obtain the surface texture. The corresponding texture regions are obtained by projecting the model surface onto each texture image, and then blending them to obtain the surface texture. However, due to the inaccuracy of the camera pose, the texture mapping results exhibit evident problems such as blurriness and ghosting. In addition, the texture images acquired in the high-light environment tend to contain highlight areas, resulting in texture color loss and diminished texture authenticity. To address these issues, a seamless texture mapping method capable of effectively eliminating highlight reflections was proposed. The method selected the best texture image for each model surface by assessing the quality of the texture image, and chromaticity consistency was utilized to constrain the optimized camera pose, eliminating obvious texture misalignments. To address the highlight problem, a highlight processing module was proposed, utilizing multi-view image information and employing a two-color reflection model to identify and process the highlight texture. Finally, adjustments were made to the texture color consistency to tackle color inconsistency between textures. The experimental results demonstrated that the proposed algorithm can obtain superior texture mapping results compared with state-of-the-art methods and effectively eliminate highlight reflections.

Key words: 3D reconstruction, texture mapping, highlight removal, texture rendering, pose correction

中图分类号:

TP391

石敏, 王炳祺, 李兆歆, 朱登明. 一种带高光处理的无缝纹理映射方法[J]. 图学学报, 2024, 45(1): 148-158.

SHI Min, WANG Bingqi, LI Zhaoxin, ZHU Dengming. A seamless texture mapping method with highlight processing[J]. Journal of Graphics, 2024, 45(1): 148-158.

图/表 14

图1 本文算法总体流程((a)输入模型和标定的图像；(b)纹理选择结果；(c)初始纹理映射结果；(d)相机姿态调整；(e)高光去除；(f)颜色统一)

Fig. 1 Overall flow of the algorithm ((a) Input model and calibrated images; (b) Texture selection results; (c) Initial texture mapping results; (d) Camera attitude adjustment; (e) Highlight removal; (f) Uniform colour)

图2 纹理选择的结果((a)初始纹理选择结果；(b)优化后结果)

Fig. 2 The result of texture selection ((a) The result of initial texture selection; (b) The right picture is the result of optimization)

图3 高光处理结果((a)待处理图像；(b)文献[11]处理后结果)

Fig. 3 Highlight processing results ((a) The image to be processed; (b) The result of the processing of Ref [11])

图4 多角度拍摄的成像原理

Fig. 4 Imaging principle of multi-angle shooting

图5 高光纹理筛选结果

Fig. 5 Results of highlight texture screening

图6 泊松编辑示例图

Fig. 6 Poisson editing example diagram

图7 真实数据的实验结果((a)文献[7]；(b)文献[9]；(c)本文)

Fig. 7 The results of real data ((a) Ref [7]; (b) Ref [9]; (c) This paper)

图8 仿真数据的实验结果((a)文献[7]；(b)文献[9]；(c)本文；(d)真实值)

Fig. 8 The results of simulation data ((a) Ref [7]; (b) Ref [9]; (c) This paper; (d) Ground truth)

表1 PSNR和SSIM的评估结果(A，B代表对应的视角)

Table 1 Evaluation results of PSNR and SSIM (A, B represents the corresponding perspective)

模型	PSNR↑			SSIM↑
模型	文献[7]	文献[9]	本文	文献[7]	文献[9]	本文
BowlA	21.235	21.662	25.127	0.822	0.881	0.893
BowlB	18.571	18.742	22.985	0.761	0.829	0.859
BunnyA	25.153	27.431	28.695	0.840	0.922	0.934
BunnyB	17.665	19.088	23.908	0.763	0.853	0.873

图9 针对颜色处理的消融实验((a)仅进行颜色一致性调整；(b)仅进行泊松编辑；(c) 2步处理相结合)

Fig. 9 Ablation experiment for color processing ((a) Color consistency adjustment only; (b) Poisson editing only; (c) Combination of two step process)

图10 无明显高光纹理数据的实验结果((a)文献[7]；(b)文献[9]；(c)本文

Fig. 10 The results of texture data without significant highlights ((a) Ref [7]; (b) Ref [9]; (c) This paper)

图11 与顶点着色的方法进行对比((a)文献[1]；(b)文献[16]；(c)本文)

Fig. 11 Compare with the method of vertex coloring ((a) Ref [1]; (b) Ref [16]; (c) This paper)

表2 不同方法间的时间消耗评估

Table 2 Time consumption assessment between different methods

模型	场景数据			运行时间/s
模型	顶点数/k	面数/k	纹理图数	文献[7]	文献[9]	本文
Fountain	31.2	59.9	21	8.4	5823.7	31.9
Board	48.1	95.5	6	1.4	1043.1	19.5
Can	82.7	162.5	10	3.7	3525.3	25.2
Floor	25.7	50.0	6	1.9	825.4	21.2
Loin	45.5	58.2	11	3.4	3672.3	17.4
Bowl	76.0	152.1	11	5.8	3824.5	16.8
Bunny	15.3	30.3	14	2.6	3238.7	18.5
Gate	37.3	58.4	16	6.3	5782.4	23.5

图12 本文方法与文献[9]方法在纹理细节方面的比较((a)本文；(b)文献[9])

Fig. 12 Comparison of texture details between the method in this paper and Fu’s method ((a) This paper; (b) Ref [9])

参考文献 29

[1]	NEWCOMBE R A, IZADI S, HILLIGES O, et al. KinectFusion: real-time dense surface mapping and tracking[C]// 2011 10th IEEE International Symposium on Mixed and Augmented Reality. New York: IEEE Press, 2011: 127-136.
[2]	CAO Y P, KOBBELT L, HU S M. Real-time high-accuracy three-dimensional reconstruction with consumer RGB-D cameras[J]. ACM Transactions on Graphics, 37(5): 171:1-171:16.
[3]	DAI A, NIEßNER M, ZOLLHÖFER M, et al. BundleFusion: real-time globally consistent 3D reconstruction using on-the-fly surface reintegration[J]. ACM Transactions on Graphics, 36(3): 24:1-24:18.
[4]	包永堂, 周鹏飞, 齐越. 面向单幅图像的逼真3D人脸重建方法[J]. 计算机辅助设计与图形学学报, 2022, 34(12): 1850-1858.
	BAO Y T, ZHOU P F, QI Y. Realistic 3D face reconstruction method for single image[J]. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(12): 1850-1858 (in Chinese).
[5]	LEMPITSKY V, IVANOV D. Seamless mosaicing of image-based texture maps[C]// 2007 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2007: 1-6.
[6]	GAL R, WEXLER Y, OFEK E, et al. Seamless montage for texturing models[J]. Computer Graphics Forum, 2010, 29(2): 479-486. DOI URL
[7]	WAECHTER M, MOEHRLE N, GOESELE M. Let there be color! large-scale texturing of 3D reconstructions[M]// Computer Vision - ECCV 2014. Cham: Springer International Publishing, 2014: 836-850.
[8]	FU Y P, YAN Q G, YANG L, et al. Texture mapping for 3D reconstruction with RGB-D sensor[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 4645-4653.
[9]	FU Y P, YAN Q G, LIAO J, et al. Seamless texture optimization for RGB-D reconstruction[J]. IEEE Transactions on Visualization and Computer Graphics, 2023, 29(3): 1845-1859. DOI URL
[10]	GUO J, ZHOU Z J, WANG L M. Single image highlight removal with a sparse and low-rank reflection model[C]// Computer Vision - ECCV 2018. Cham: Springer, 2018: 282-298.
[11]	YAMAMOTO T, NAKAZAWA A. General improvement method of specular component separation using high-emphasis filter and similarity function[J]. ITE Transactions on Media Technology and Applications, 2019, 7(2): 92-102. DOI URL
[12]	BERNARDINI F, MARTIN I M, RUSHMEIER H. High-quality texture reconstruction from multiple scans[J]. IEEE Transactions on Visualization and Computer Graphics, 2001, 7(4): 318-332. DOI URL
[13]	CALLIERI M, CIGNONI P, CORSINI M, et al. Masked photo blending: mapping dense photographic data set on high-resolution sampled 3D models[J]. Computers & Graphics, 2008, 32(4): 464-473. DOI URL
[14]	TOTZ J, CHUNG A J, YANG G Z. Patient-specific texture blending on surfaces of arbitrary topology[EB/OL]. [2023-05-24]. https://www.doc.ic.ac.uk/amiarcs/proceedings2009/AMI-ARCS2009_proceedings.pdf.
[15]	ZHOU Q Y, KOLTUN V. Color map optimization for 3D reconstruction with consumer depth cameras[J]. ACM Transactions on Graphics, 33(4): 155:1-155:10.
[16]	BI S, KALANTARI N K, RAMAMOORTHI R. Patch-based optimization for image-based texture mapping[J]. ACM Transactions on Graphics, 36(4): 106:1-106:11.
[17]	MAIER R, KIM K, CREMERS D, et al. Intrinsic3D: high-quality 3D reconstruction by joint appearance and geometry optimization with spatially-varying lighting[C]// 2017 IEEE International Conference on Computer Vision. New York: IEEE Press, 2017: 3133-3141.
[18]	PRADA F, KAZHDAN M, CHUANG M, et al. Gradient- domain processing within a texture atlas[J]. ACM Transactions on Graphics, 37(4): 154:1-154:14.
[19]	KIM J, KIM H, PARK J, et al. Global texture mapping for dynamic objects[J]. Computer Graphics Forum, 2019, 38(7): 697-705. DOI
[20]	COELHO D, DAL'COL L, MADEIRA T, et al. A robust 3D-based color correction approach for texture mapping applications[J]. Sensors, 2022, 22(5): 1730. DOI URL
[21]	DAL'COL L, COELHO D, MADEIRA T, et al. A sequential color correction approach for texture mapping of 3D meshes[J]. Sensors, 2023, 23(2): 607. DOI URL
[22]	LIU X Q, LI J T, LU G D. Generating high-fidelity texture in RGB-D reconstruction using patches density regularization[J]. Computer-Aided Design, 2023, 160: 103516. DOI URL
[23]	HUANG J W, THIES J, DAI A, et al. Adversarial texture optimization from RGB-D scans[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 1556-1565.
[24]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[C]// European Conference on Computer Vision. Cham: Springer, 2020: 405-421.
[25]	MÜLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 2022, 41(4): 102:1-102:15.
[26]	TAN R T, IKEUCHI K. Separating reflection components of textured surfaces using a single image[M]// Digitally Archiving Cultural Objects. Boston: Springer, 2008: 353-384.
[27]	FU Y P, YAN Q G, LIAO J, et al. Joint texture and geometry optimization for RGB-D reconstruction[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 5949-5958.
[28]	SHAFER S A. Using color to separate reflection components[J]. Color Research & Application, 1985, 10(4): 210-218.
[29]	PÉREZ P, GANGNET M, BLAKE A. Poisson image editing[J]. ACM Transactions on Graphics, 22(3): 313-318. DOI URL

一种带高光处理的无缝纹理映射方法

A seamless texture mapping method with highlight processing

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 29

相关文章 15

编辑推荐

Metrics

本文评价

[1]	朱宝旭, 刘漫丹, 张雯婷, 谢立志. 高分辨率人脸纹理图全流程生成方法[J]. 图学学报, 2024, 45(4): 814-826.
[2]	黄家晖, 穆太江. 动态三维场景重建研究综述[J]. 图学学报, 2024, 45(1): 14-25.
[3]	周婧怡, 张栖桐, 冯结青. 基于混合结构的多视图三维场景重建[J]. 图学学报, 2024, 45(1): 199-208.
[4]	王江安, 黄乐, 庞大为, 秦林珍, 梁温茜. 基于自适应聚合循环递归的稠密点云重建网络[J]. 图学学报, 2024, 45(1): 230-239.
[5]	成欢, 王硕, 李孟, 秦伦明, 赵芳. 面向自动驾驶场景的神经辐射场综述[J]. 图学学报, 2023, 44(6): 1091-1103.
[6]	薛皓玮, 王美丽. 融合生物力学约束与多模态数据的手部重建[J]. 图学学报, 2023, 44(4): 794-800.
[7]	葛海明, 张维, 王小龙, 朱晶晶, 贾非, 薛亚东. 基于SfM的城市电缆隧道三维重建方法优化研究[J]. 图学学报, 2023, 44(3): 540-550.
[8]	王江安, 庞大为, 黄乐, 秦林珍. 基于多尺度特征递归卷积的稠密点云重建网络 [J]. 图学学报, 2022, 43(5): 875-883.
[9]	白静, 孟庆亮, 徐昊, 范有福, 杨瞻源. ST-Rec3D：基于结构和目标感知的三维重建[J]. 图学学报, 2022, 43(3): 469-477.
[10]	王浩 1，邵堃 1，霍星 2，杨鹏 1，檀结庆 2 . 一种基于极大特征点的三维椎骨分割方法[J]. 图学学报, 2019, 40(1): 40-45.
[11]	邹晓1，陈正鸣1,2，朱红强1，童晶1,2. 基于移动平台的三维虚拟试发型系统实现及应用[J]. 图学学报, 2018, 39(2): 309-316.
[12]	罗昌平1,2，夏海波1，赵琼3. 基于运动视差的三维重建技术在自导引小车中的应用[J]. 图学学报, 2017, 38(4): 615-622.
[13]	缪永伟1,2，冯小红2，林海斌2，张旭东2，高飞2. 2D-3D 转换:由二维线画图生成三维复杂零部件[J]. 图学学报, 2017, 38(2): 162-169.
[14]	郑顾平，邢玥，张荣华. 一种基于GPU 的弹坑实时绘制方法[J]. 图学学报, 2016, 37(4): 451-456.
[15]	陈培兴，王明泉，李世虎，侯慧玲，王玉. 基于形态学和Otsu 的固体火箭发动机CT 缺陷三维分割[J]. 图学学报, 2015, 36(4): 581-586.