欢迎访问《图学学报》 分享到:

图学学报 ›› 2021, Vol. 42 ›› Issue (4): 629-635.DOI: 10.11996/JG.j.2095-302X.2021040629

• 计算机图形学与虚拟现实 • 上一篇    下一篇

基于投票决策的实时遮挡处理技术

  

  1. 1. 南京理工大学自动化学院,江苏 南京 210094; 2. 上海机电工程研究所,上海 201109; 3. 南京理工大学计算机科学与工程学院,江苏 南京 210094
  • 出版日期:2021-08-31 发布日期:2021-08-05
  • 基金资助:
    “十三五”装备预研项目(61409230104,1017,315100104);中央高校基本科研业务费专项(30918012203);上海航天科技创新基金项目(SAST2019009);上海市青年科技英才杨帆计划项目(18YF1410200)

Real-time virtual and real occlusion processing technology based on voting decision

  1. 1. School of Automation, Nanjing University of Science and Technology, Nanjing Jiangsu 210094, China;
    2. Shanghai Institute of Mechanical and Electrical Engineering, Shanghai 201109, China;
    3. School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing Jiangsu 210094, China
  • Online:2021-08-31 Published:2021-08-05
  • Supported by:
    “13th Five-Year” Equipment Pre-research Project (61409230104, 1017, 315100104); Special Funds for Fundamental Scientific Research of Central Universities (30918012203); Shanghai Aerospace Science and Technology Innovation Fund (SAST2019009); Supported by Yang Fan Program for Young Scientists in Shanghai (18YF1410200)

摘要: 针对当前基于深度信息的虚实遮挡处理技术面临的实时性差和精度低的问题,提出一种基于局
部区域深度估计和基于 patch 相似性噪声点投票融合的实时虚实遮挡处理算法。该算法将真实场景视频序列作
为输入,首先利用局部区域深度估计算法通过稀疏重建估算出稀疏关键点的深度信息,对稀疏深度施加目标区
域的约束限制深度向周围像素的传播,从而快速恢复出目标区域的相对深度图;然后,噪声点投票融合算法利
用深度比较确定虚实物体的前后位置关系,基于 patch 相似性和投票决策的方法对区域内的像素进行投票和融
合绘制;最后输出具有真实遮挡关系的融合效果。实验结果表明,该算法不仅可以提高虚实遮挡的实时性,还
能够获得真实和虚拟场景不同空间关系下的良好融合效果。

关键词: 虚实遮挡, 深度估计, 区域约束, 投票决策, 图像块相似性

Abstract: Aiming at the problems of poor real-time performance and low accuracy faced by the current virtual and
real occlusion processing technology based on depth information, a real-time virtual and real occlusion processing
algorithm was proposed based on local area depth estimation and patch similarity-based fusion of noise points. The
algorithm took the real scene video sequence as the input. Firstly, the depth information of sparse key points was
estimated using the local area depth estimation algorithm through sparse reconstruction, and the sparse depth was
imposed on the target area to limit the depth propagation to the surrounding pixels, so as to quickly restore the relative
depth map of the target area. Then, the noise point voting fusion algorithm employed depth comparison to determine the front and back position relationship of the virtual and real objects, and voted and merged the pixels in the area
based on the patch similarity and voting decision method. Finally, the fusion effect with real occlusion relationship
was output. The experimental results show that the proposed algorithm can not only improve the real-time
performance of virtual and real occlusion, but also obtain a good fusion effect under different spatial relationships
between real and virtual scenes.

Key words: occlusion, depth estimation, regional constraints, voting decision, patch similarity

中图分类号: