Welcome to Journal of Graphics share: 

Journal of Graphics ›› 2022, Vol. 43 ›› Issue (4): 695-706.DOI: 10.11996/JG.j.2095-302X.2022040695

• Computer Graphics and Virtual Reality • Previous Articles     Next Articles

Human pose estimation and similarity calculation for Tai Chi learning

  

  1. School of Information Science and Technology, North China University of Technology, Beijing 100144, China
  • Online:2022-08-31 Published:2022-08-15
  • Contact: CAI Xing-quan (1980), professor, Ph.D. His main research interests cover virtual reality, human-computer interaction, deep learning, etc
  • Supported by:
    National Natural Science Foundation of China (61503005); Social Science Foundation of Beijing (19YTC043, 20YTB011)

Abstract:

To address the current problems of poor natural interactivity and lack of learning feedback in the case of online Tai Chi learning, this paper proposed a method of human pose estimation and similarity calculation for Tai Chi learning. First, the proposed method extracted the key-frame images from the Tai Chi video using an inter-frame difference method. Second, our method employed the stacked hourglass network model to perform two-dimensional joint-point detection on the key-frame images. Third, a long short-term memory (LSTM) network combined with the Sequence-to-Sequence network model was used to map the detected two-dimensional joint-point sequence from two-dimensional to three-dimensional, thus predicting the position coordinates of the three-dimensional joint-points. Finally, the two-dimensional and three-dimensional cosine similarities of the estimated human posture were calculated. Using this method, this paper designed and developed a Tai Chi learning and feedback application system with simple equipment and strong user experience, which was applied to real scenarios. This system could detect whether the overall movements of Tai Chi students and the movements of each body segment were standard, with feedback provided. Students could practice and improve non-standard movements based on the feedback, so as to achieve the purpose of improving the learning effect.

Key words: Tai Chi learning, human pose estimation, inter-frame difference, stacked hourglass networks, cosine , similarity

CLC Number: