欢迎访问《图学学报》 分享到：
图学学报

图学学报 ›› 2022, Vol. 43 ›› Issue (3): 504-512.DOI: 10.11996/JG.j.2095-302X.2022030504

• 计算机图形学与虚拟现实 • 上一篇    下一篇

用于手势识别的时空融合网络以及虚拟签名系统

  

  1. 山东大学软件学院，山东 济南 250101
  • 出版日期:2022-06-30 发布日期:2022-06-28
  • 基金资助:
    国家重点研发计划战略性科技创新合作项目(2021YFE0203800)；国家自然科学基金浙江两化融合联合基金(U1909210)；国家自然科学
    基金(62172257，61772312)

Spatiotemporal fusion network for hand gesture recognition and virtual signature system

  1. School of Software, Shandong University, Jinan Shandong 250101, China

  • Online:2022-06-30 Published:2022-06-28
  • Supported by:
    National Key R & D Program Strategic Scientific and Technological Innovation Cooperation Project (2021YFE0203800);
    NSFC-Zhejiang Joint Fund of the Integration of Informatization and Industrialization (U1909210); National Natural Science
    Foundation of China (62172257, 61772312)

PDF (PC)

51

摘要：

由于新型冠状病毒的流行，非接触式个人签名可以在一定程度上降低感染的风险，其将在人们日常的生活中发挥重要作用。因此，提出了一种简单而有效的时空融合网络来实现基于骨架的动态手势识别，并以此为基础开发了一款虚拟签名系统。时空融合网络主要由基于注意力机制的时空融合模块构成，其核心思想是以增量的方式同步实现时空特征的提取与融合。该网络采用不同编码的时空特征作为输入，并在实际应用中采用双滑动窗口机制来进行后处理，从而确保结果更加的稳定与鲁棒。在 2 个基准数据集上的大量对比实验表明，该方法优于最先进的单流网络方法。另外，虚拟签名系统在一个普通的 RGB 相机下表现优异，不仅大大降低了交互系统的复杂性，还提供了一种更为便捷、安全的个人签名方式。

关键词: 手势识别, 特征融合, 骨架表征, 注意力机制, 签名系统

Abstract:

Due to the coronavirus pandemic, the non-touch personal signature can reduce the risk of infection to a certain extent, which is of great significance to our daily life. Therefore, a simple and efficient spatiotemporal fusion network was proposed to realize skeleton-based dynamic hand gesture recognition, based on which a virtual signature system was developed. The spatiotemporal fusion network is mainly composed of spatiotemporal fusion modules based on the attention mechanism, and its key idea is to synchronously realize the extraction and fusion of spatiotemporal features using an incremental method. This network adopts different spatiotemporal coding features as inputs, and employs the double sliding window mechanism for post-processing in practical applications, thus ensuring more stable and robust results. Extensive comparative experiments on two benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art single-stream network. Besides, the virtual signature system performs well with a single normal RGB camera, which not only greatly reduces the complexity of the interaction system, but also provides a more convenient and secure approach to personal signature.

Key words: hand gesture recognition, feature fusion, skeleton representation, attention mechanism, signature system

中图分类号: