欢迎访问《图学学报》 分享到：
图学学报

图学学报

• 计算机图形学与虚拟现实 • 上一篇    下一篇

基于多个关键点对应性的手部动态重建

  

  1. (1. 北京航空航天大学虚拟现实技术与系统国家重点实验室，北京 100191； 2. 北京航空航天大学新媒体艺术与设计学院，北京 100191)
  • 出版日期:2020-10-31 发布日期:2020-11-05
  • 通讯作者: 沈旭昆(1965)，男，云南昆明人，教授，博士。主要研究方向为计算机图形学和虚拟现实等。E-mail：xkshen@buaa.edu.cn
  • 作者简介:范 清(1986?)，男，河南信阳人，博士研究生。主要研究方向为计算机视觉、计算机图形学和虚拟现实。E-mail：fanqing@buaa.edu.cn

Hand dynamic 3D reconstruction using multiple keypoint-to-keypoint correspondences

  1. (1. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China; 2. School of New Media Art and Design, Beihang University, Beijing 100191, China)
  • Online:2020-10-31 Published:2020-11-05
  • Contact: SHEN Xu-kun (1965–), male, professor, Ph.D. His main research interests cover computer graphics and virtual reality, Etc. E-mail：xkshen@buaa.edu.cn
  • About author:FAN Qing (1986–), male, PhD candidate. His main research interests cover computer vision, computer graphics and virtual reality. E-mail：fanqing@buaa.edu.cn

PDF (PC)

90

摘要： 提出了一种新的基于单视图深度序列的手部运动跟踪和表面重建方法。在假设任 意一对关键点的对应性在所有手部姿态上均一致基础上，使用一个平滑的手部网格模板来提供 形状和拓扑先验，引入多个能量函数构造输入扫描与模板之间三维关键点到关键点的对应性， 并将其整合到一个可用的非刚性配准管线中，以实现精确的表面拟合。通过最小化手部模板和 输入深度图像序列之间的误差来捕获非刚性的手部运动。采用迭代求解的方法，通过显式的关 键点到关键点之间的对应性，逐步细化手部关节区域的变形，从而达到快速收敛和合理变形的 目的。在含有噪声的手部深度图像序列上的大量实验表明，该方法能够重建精确的手部运动， 并且对较大的变形和遮挡具有鲁棒性。

关键词: 运动跟踪, 表面重建, 关键点, 非刚性变形, 深度序列

Abstract: We propose a new approach to robustly reconstruct non-rigid hand geometries and motions from single-view depth sequences. On the basis of the assumption that the correspondence between any pair of key points is consistent across all hand poses, a smooth hand mesh template is adopted to provide the shape and topology prior, and several energy functions are introduced to formulate these 3D keypoint-to-keypoint correspondences, which are then incorporated into the available non-rigid registration pipeline to achieve an accurate surface fit. The non-rigid hand motion is captured by minimizing the distortion error between a hand template and the partial input. An iterative solver is employed to gradually refine deformations around joint regions with explicit correspondences, leading to fast convergence and plausible deformations. Extensive experiments on hand depth sequences with noise demonstrate that our approach is capable of reconstructing accurate hand motions, and is robust for large deformations and occlusions.

Key words: motion tracking, surface reconstruction, key-point, non-rigid deformation, depth sequences