欢迎访问《图学学报》 分享到:

图学学报 ›› 2022, Vol. 43 ›› Issue (4): 695-706.DOI: 10.11996/JG.j.2095-302X.2022040695

• 计算机图形学与虚拟现实 • 上一篇    下一篇

面向太极拳学习的人体姿态估计及相似度计算

  

  1. 北方工业大学信息学院,北京 100144
  • 出版日期:2022-08-31 发布日期:2022-08-15
  • 通讯作者: 蔡兴泉(1980),男,教授,博士。主要研究方向为虚拟现实、人机互动、深度学习等
  • 基金资助:
    国家自然科学基金项目(61503005);北京市社会科学基金项目(19YTC043,20YTB011)

Human pose estimation and similarity calculation for Tai Chi learning

  1. School of Information Science and Technology, North China University of Technology, Beijing 100144, China
  • Online:2022-08-31 Published:2022-08-15
  • Contact: CAI Xing-quan (1980), professor, Ph.D. His main research interests cover virtual reality, human-computer interaction, deep learning, etc
  • Supported by:
    National Natural Science Foundation of China (61503005); Social Science Foundation of Beijing (19YTC043, 20YTB011)

摘要:

针对当前线上太极拳学习自然交互性差、缺乏学习反馈等问题,提出一种面向太极拳学习的人体姿态估计及相似度计算方法。首先,输入太极拳视频,利用帧间差分法提取关键帧图像;然后,利用堆叠沙漏网络模型对关键帧图像进行二维关节点检测;接着,使用长短期记忆(LSTM)网络结合 Sequence-to-Sequence网络模型对检测到的二维关节点序列进行二维到三维的映射,预测三维关节点的位置坐标;最后对估计的人体姿态进行二维和三维余弦相似度计算。利用该方法设计并开发了一款相关设备简便、用户体验感强的太极拳学习与反馈应用系统,并在实际中应用。该系统可以检测太极拳学员的整体动作及各肢体段动作是否标准,并给出反馈,学员可以根据反馈结果练习和改善不标准动作,达到提升学习效果的目的。

关键词: 太极拳学习, 人体姿态估计, 帧间差分, 堆叠沙漏网络, 余弦相似度

Abstract:

To address the current problems of poor natural interactivity and lack of learning feedback in the case of online Tai Chi learning, this paper proposed a method of human pose estimation and similarity calculation for Tai Chi learning. First, the proposed method extracted the key-frame images from the Tai Chi video using an inter-frame difference method. Second, our method employed the stacked hourglass network model to perform two-dimensional joint-point detection on the key-frame images. Third, a long short-term memory (LSTM) network combined with the Sequence-to-Sequence network model was used to map the detected two-dimensional joint-point sequence from two-dimensional to three-dimensional, thus predicting the position coordinates of the three-dimensional joint-points. Finally, the two-dimensional and three-dimensional cosine similarities of the estimated human posture were calculated. Using this method, this paper designed and developed a Tai Chi learning and feedback application system with simple equipment and strong user experience, which was applied to real scenarios. This system could detect whether the overall movements of Tai Chi students and the movements of each body segment were standard, with feedback provided. Students could practice and improve non-standard movements based on the feedback, so as to achieve the purpose of improving the learning effect.

Key words: Tai Chi learning, human pose estimation, inter-frame difference, stacked hourglass networks, cosine , similarity

中图分类号: