基于AI动作捕捉技术的视神经脊髓炎康复训练系统交互性研究

doi:10.11996/JG.j.2095-302X.2025040783

图学学报 ›› 2025, Vol. 46 ›› Issue (4): 783-792.DOI: 10.11996/JG.j.2095-302X.2025040783

• 计算机图形学与虚拟现实 • 上一篇下一篇

基于AI动作捕捉技术的视神经脊髓炎康复训练系统交互性研究

张帅¹(), 洪翱¹, 胡恒瑞², 兰名荥¹(), 郗小超¹

1.北京邮电大学数字媒体与设计艺术学院，北京 100876
2.北京邮电大学国际学院，北京 100876

收稿日期:2024-10-16 修回日期:2025-03-13 出版日期:2025-08-30 发布日期:2025-08-11
通讯作者:兰名荥(1987-)，女，副教授，博士。主要研究方向为智能医学、人工智能测评及训练。E-mail：lanmingying@bupt.edu.cn
第一作者:张帅(2003-)，女，本科生。主要研究方向为交互设计。E-mail：zhangshuai2@bupt.edu.cn
基金资助:
北京市社会科学基金(18YTC034);北京邮电大学“小米青年学者”项目

Study on the interaction of an AI-based motion capture technology in rehabilitation training systems for neuromyelitis optica

ZHANG Shuai¹(), HONG Ao¹, HU Hengrui², LAN Mingying¹(), XI Xiaochao¹

1. School of Digital Media & Design Arts, Beijing University of Posts and Telecommunications, Beijing 100876, China
2. School of Internate, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2024-10-16 Revised:2025-03-13 Published:2025-08-30 Online:2025-08-11
First author：ZHANG Shuai (2003-), undergraduate. Her main research interest covers interactive design. E-mail：zhangshuai2@bupt.edu.cn
Supported by:
Beijing Social Science Foundation(18YTC034);Beijing University of Posts and Telecommunications “Xiaomi Young Scholars” Project

摘要/Abstract

摘要：

近年来，人工智能(AI)视觉模型及其动作捕捉技术在医疗领域备受瞩目，但在视神经脊髓炎(NMO)康复领域中尚未得到广泛应用。针对当前NMO患者康复训练存在的个性化程度低、体验不佳和效果不显著等诸多问题，提出通过Mediapipe姿态跟踪技术及交互反馈设计，对NMO康复训练系统进行研究和实现。主要完成了以下几项核心工作：①康复训练计划制定研究。通过定性和定量相结合的方法，对36名NMO康复患者进行影子观察和深度访谈，并使用中国中枢神经系统炎性脱髓鞘疾病登记研究随访表进行评估，构建了全面的评估体系；②康复训练动作指导反馈分析。设计了康复训练动作库，确定了人体关节点的坐标、关节点形成的角度以及关节点信息与标准信息的差值等关键指标，并利用Mediapipe框架构建人体姿态检测模型，实现康复训练动作的实时识别与反馈；③康复系统交互反馈机制设计。确立了及时性、特殊性、指导性和可追溯性的交互反馈设计原则，设计了包含场景定位、维度分析和交互方式的反馈流程，并引入Fogg行为模型理论，最终搭建了基于微信小程序的轻量化康复训练平台。最终得到了一套NMO康复训练系统及其配套方法，验证了AI动作捕捉技术在该领域应用的可能性，提升了NMO患者在康复训练过程中的体验，并在首都医科大学附属北京天坛医院实际落地推广，论证了其具有广泛的应用价值。

关键词: 视神经脊髓炎, 康复训练, 动作识别, 计算机视觉, 多感官交互

Abstract:

Artificial intelligence (AI) vision models and their motion capture technologies have attracted much attention in the medical field in recent years, but these technologies have not been extensively applied to the field of neuromyelitis optica (NMO) rehabilitation. The purpose of this study was to study and implement a rehabilitation training system for neuromyelitis optica through Mediapipe posture tracking technology and interactive feedback design. Specifically, the following core works were conducted: ① Research on the formulation of rehabilitation training plans. Through a combination of qualitative and quantitative methods, 36 NMO rehabilitation patients were shadowed and interviewed, and the follow-up form of the Chinese Central Nervous System Inflammatory Demyelinating Disease Registry was used to evaluate the comprehensive evaluation system. ② Feedback analysis of rehabilitation training action guidance. The rehabilitation training action library was designed, and the key indicators such as the coordinates of human joint points, the angle of joint point formation, and the difference between joint point information and standard information were determined. The human posture detection model was also constructed using the Mediapipe framework to realize the real-time recognition and feedback of rehabilitation training actions. ③ Design of interactive feedback mechanism of rehabilitation system. The design principles of timeliness, particularity, guidance, and traceability were established. Meanwhile, a feedback process that includes scene positioning, dimensional analysis, and interaction mode was designed. Additionally, the Fogg behavior model theory was introduced..Finally, a lightweight rehabilitation training platform based on WeChat applets was built. In this study, a set of neuromyelitis optica rehabilitation training system and its supporting methods were obtained, which verified the possibility of AI motion capture technology in this field, improved the experience of NMO patients in the process of rehabilitation training, and was actually promoted in Beijing Tiantan Hospital affiliated to Capital Medical University, demonstrating its wide range of application value.

Key words: neuromyelitis optica, rehabilitation training, motion recognition, computer vision, multi-sensory interaction

中图分类号:

张帅, 洪翱, 胡恒瑞, 兰名荥, 郗小超. 基于AI动作捕捉技术的视神经脊髓炎康复训练系统交互性研究[J]. 图学学报, 2025, 46(4): 783-792.

ZHANG Shuai, HONG Ao, HU Hengrui, LAN Mingying, XI Xiaochao. Study on the interaction of an AI-based motion capture technology in rehabilitation training systems for neuromyelitis optica[J]. Journal of Graphics, 2025, 46(4): 783-792.

图/表 13

图1 系统架构

Fig. 1 System architecture

表1 病例组与健康组MoCA，SDMT，PASAT量表结果对比

Table 1 Comparison of results of MoCA, SDMT, PASAT scale between case group and healthy group

分组	例数	MoCA	SDMT	PASAT
病例组	36	24.7±4.2	31.1±13.0	48.1±10.3
健康组	40	27.5±2.9	45.0±7.1	50.3±9.7

表2 运动类型具体数据示例

Table 2 Examples of specific data for the type of motion

图2 4种运动类型示例((a) 肩关节环绕运动；(b) 扩胸运动；(c) 躯干伸展运动；(d) 体侧屈运动)

Fig. 2 Examples of the 4 types of exercises ((a) Shoulder wrap; (b) Chest expansion; (c) Trunk extension; (d) Lateral flexion)

图3 模型关键节点示意图

Fig. 3 Schematic diagram of key nodes in the model

图4 角度平滑效果示意图((a) 腕关节平滑前；(b) 腕关节平滑后；(c) 肘关节平滑前；(d) 肘关节平滑后)

Fig. 4 Schematic diagram of the angular smoothing effect ((a) Wrist joint smooth front; (b) After smoothing the wrist joint; (c) Elbow joint smooth front; (d) After smoothing the elbow joint)

图5 场景定位及维度分析

Fig. 5 Scene positioning and dimensional analysis

图6 Fogg行为模型

Fig. 6 Fogg behavioral model

图7 系统交互反馈方式示意图

Fig. 7 Schematic diagram of the system interaction feedback mode

图8 小程序系统架构图

Fig. 8 Mini Program system architecture diagram

图9 小程序用户界面图

Fig. 9 Diagram of the user interface of the applet

图10 用户测试图

Fig. 10 User test diagram

表3 康复训练动作评价表

Table 3 Rehabilitation training evaluation form

参考文献 14

[1]	CREE B A C, GOODIN D S, HAUSER S L. Neuromyelitis optica[J]. Seminars in Neurology, 2002, 22(2): 105-122. DOI PMID
[2]	BROD S A. Review of approved NMO therapies based on mechanism of action, efficacy and long-term effects[J]. Multiple Sclerosis and Related Disorders, 2020, 46: 102538.
[3]	HOR J Y, ASGARI N, NAKASHIMA I, et al. Epidemiology of neuromyelitis optica spectrum disorder and its prevalence and incidence worldwide[J]. Frontiers in Neurology, 2020, 11: 501. DOI PMID
[4]	CHEN J. Improving detection, recovery of optic neuritis in NMOSD: John Chen, MD, PhD[EB/OL]. Neurology Live, 2023 [2024-05-20]. https://link.gale.com/apps/doc/A776379240/AONE.
[5]	LI Q Y, WANG B, CHENG B Y, et al. Efficacy and safety of rehabilitation exercise in neuromyelitis optica spectrum disorder during the acute phase: a prospective cohort study[J]. Multiple Sclerosis and Related Disorders, 2022, 61: 103726.
[6]	SHABANY M, AYOUBI S, NASER MOGHADASI A, et al. Explaining the individual challenges of women affected by neuromyelitis optica and multiple sclerosis: a comparative content analysis study[J]. Clinical Neurology and Neurosurgery, 2021, 207: 106789.
[7]	BEEKMAN J, KEISLER A, PEDRAZA O, et al. Neuromyelitis optica spectrum disorder[J]. Neurology Neuroimmunology & Neuroinflammation, 2019, 6(4): e580.
[8]	CICCONE I. Promoting various treatment approaches to improve quality of life in NMOSD[EB/OL]. Neurology Live, 2024 [2024-05-20].
[9]	LEI M, LIU P, WANG Q S, et al. Precise identification of rehabilitation actions using AI based strategy[C]// The 29th International Conference on Computer Communications and Networks. New York: IEEE Press, 2020: 1-7.
[10]	ZHAO S, TONG Y, CHEN G, et al. Testing the feasibility of quantizing the progress of stroke patients’ rehabilitation with a computer vision method[C]// The 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications. New York: IEEE Press, 2022: 1-4.
[11]	CÓIAS A R, LEE M H, BERNARDINO A, et al. Skeleton tracking solutions for a low-cost stroke rehabilitation support system[C]// 2023 International Conference on Rehabilitation Robotics. New York: IEEE Press, 2023: 1-6.
[12]	CALISTO F M, FERNANDES J, MORAIS M, et al. Assertiveness-based agent communication for a personalized medicine on medical imaging diagnosis[C]// 2023 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2023: 13.
[13]	TIAN D C, ZHANG C Y, YUAN M, et al. Incidence of multiple sclerosis in China: a nationwide hospital-based study[J]. The Lancet Regional Health - Western Pacific, 2020, 1: 100010.
[14]	China Alliance for Rare Disease. Blue Book on Health Insights of Multiple Sclerosis Patients and 2021 Quality of Life Report for Multiple Sclerosis Patients in China[R]. Beijing: China, 2021.

[1]	孙浩, 谢滔, 何龙, 郭文忠, 虞永方, 吴其军, 王建伟, 东辉. 多模态文本视觉大模型机器人地形感知算法研究[J]. 图学学报, 2025, 46(3): 558-567.
[2]	王雪婷, 郭新, 汪松, 陈恩庆. 基于变分自编码器掩蔽重建的骨骼点动作识别方法[J]. 图学学报, 2025, 46(2): 270-278.
[3]	张天圣, 朱闽峰, 任怡雯, 王琛涵, 张立冬, 张玮, 陈为. BPA-SAM：面向工笔画数据的SAM边界框提示增强方法[J]. 图学学报, 2025, 46(2): 322-331.
[4]	刘冀辰, 李金星, 吴佳, 张威, 齐宇诺, 周国亮. 大模型技术在电力行业的应用展望[J]. 图学学报, 2024, 45(6): 1132-1144.
[5]	李琼, 考月英, 张莹, 徐沛. 面向无人机航拍图像的目标检测研究综述[J]. 图学学报, 2024, 45(6): 1145-1164.
[6]	吴沛宸, 袁立宁, 胡皓, 刘钊, 郭放. 基于注意力特征融合的视频异常行为检测[J]. 图学学报, 2024, 45(5): 922-929.
[7]	李松洋, 王雪婷, 陈相龙, 陈恩庆. 基于骨骼点动态时域滤波的人体动作识别[J]. 图学学报, 2024, 45(4): 760-769.
[8]	郭宗洋, 刘立东, 蒋东华, 刘子翔, 朱熟康, 陈京华. 基于语义引导神经网络的人体动作识别算法[J]. 图学学报, 2024, 45(1): 26-34.
[9]	王江安, 黄乐, 庞大为, 秦林珍, 梁温茜. 基于自适应聚合循环递归的稠密点云重建网络[J]. 图学学报, 2024, 45(1): 230-239.
[10]	李泓萱, 张松洋, 任博. 基于多级可逆神经网络的大容量裁剪稳健型图像隐写技术[J]. 图学学报, 2023, 44(6): 1149-1161.
[11]	毕春艳, 刘越. 基于深度学习的视频人体动作识别综述[J]. 图学学报, 2023, 44(4): 625-639.
[12]	杨柳, 吴晓群. 基于深度学习的三维形状补全研究综述[J]. 图学学报, 2023, 44(2): 201-215.
[13]	王江安, 庞大为, 黄乐, 秦林珍. 基于多尺度特征递归卷积的稠密点云重建网络 [J]. 图学学报, 2022, 43(5): 875-883.
[14]	梁振宇, 华嘉皓, 陈浩龙, 邓逸川. 基于计算机视觉的建筑施工期临时结构损伤识别方法 [J]. 图学学报, 2022, 43(4): 608-615.
[15]	熊琛, 陈立斌, 李林泽, 许镇, 赵杨平. 基于计算机视觉与 BIM 的裂缝可视化管理方法[J]. 图学学报, 2022, 43(4): 721-728.

基于AI动作捕捉技术的视神经脊髓炎康复训练系统交互性研究

Study on the interaction of an AI-based motion capture technology in rehabilitation training systems for neuromyelitis optica

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 14

相关文章 15

编辑推荐

Metrics

本文评价