A multi-indicator fusion digital-twin evaluation method oriented to FMA-UE

doi:10.11996/JG.j.2095-302X.2026030607

Abstract

Abstract:

To address the subjectivity and poor quantifiability of post-stroke upper-limb functional assessment, a digital-twin-based evaluation method targeted at the Fugl-Meyer Assessment of the Upper Extremity (FMA-UE) was proposed and validated in a controlled study. PN3 sensors were used to capture pose data and a Dynamic Fusion Rating Algorithm (DFRA) was constructed to achieve item-level alignment with FMA-UE, outputting individual scores and identifying deficits. In a controlled comparative study with 30 participants, the system improved efficiency by approximately 40% relative to manual assessment. System scores were statistically equivalent to therapist scores within a ±3-point equivalence margin under the Two One-Sided tests (TOST) procedure, with overall errors of MAE=1.97 points and RMSE=2.14 points. Deficit identification achieved Top-1 agreement of 88.0% with therapists across the five FMA-UE domains. The proposed DFRA-plus-digital-twin approach enabled item-level automated scoring and deficit visualization while preserving clinical interpretability, markedly improving efficiency over traditional assessment and demonstrating practical deployability in engineering settings.

Key words: digital twin, Fugl-Meyer assessment for upper extremity, motor function, feature fusion, automated assessment

CLC Number:

TP399
R743.3

WANG Kun, LI Yifan, TIAN Hongliang, ZHU Weiguang, HUANG Yaning. A multi-indicator fusion digital-twin evaluation method oriented to FMA-UE[J]. Journal of Graphics, 2026, 47(3): 607-615.

Figures/Tables 17

References 29

[1]	MARKUS H S, LEUNG T. Stroke in China[J]. International Journal of Stroke, 2023, 18(3): 256-258. DOI PMID
[2]	《中国脑卒中防治报告》编写组. 《中国脑卒中防治报告2020》概要[J]. 中国脑血管病杂志, 2022, 19(2): 136-144.
	Report on Stroke Prevention and Treatment in China Writing Group. Brief report on stroke prevention and treatment in China, 2020[J]. Chinese Journal of Cerebrovascular Diseases, 2022, 19(2): 136-144(in Chinese).
[3]	OÑA SIMBAÑA E D, SÁNCHEZ-HERRERA BAEZA P, JARDÓN HUETE A, et al. Review of automated systems for upper limbs functional assessment in neurorehabilitation[J]. IEEE Access, 2019, 7: 32352-32367. DOI URL
[4]	KAHN M B, MENTIPLAY B F, CLARK R A, et al. Methods of assessing associated reactions of the upper limb in stroke and traumatic brain injury: a systematic review[J]. Brain Injury, 2016, 30(3): 252-266. DOI PMID
[5]	许光旭, 高晓阳, 陈文红. Fugl-Meyer运动功能评分的敏感性及实用性[J]. 中国康复, 2001, 16(1): 18-19.
	XU G X, GAO X Y, CHEN W H. The practicality and sensitivity in Fugl-Meyer motor function assessment[J]. Chinese Journal of Rehabilitation, 2001, 16(1): 18-19(in Chinese).
[6]	黄静, 魏智真, 李国翠, 等. 结合Kinect的上肢运动康复系统的研究进展[J]. 中国康复医学杂志, 2017, 32(9): 1082-1086.
	HUANG J, WEI Z Z, LI G C, et al. Research progress of upper limb exercise rehabilitation system combined with Kinect[J]. Chinese Journal of Rehabilitation Medicine, 2017, 32(9): 1082-1086(in Chinese).
[7]	MA Y, LIU D W, CAI L S. Deep learning-based upper limb functional assessment using a single Kinect v2 sensor[J]. Sensors, 2020, 20(7): 1903. DOI URL
[8]	LATORRE J, COLOMER C, ALCAÑIZ M, et al. Gait analysis with the Kinect v2: normative study with healthy individuals and comprehensive study of its sensitivity, validity, and reliability in individuals with stroke[J]. Journal of NeuroEngineering and Rehabilitation, 2019, 16(1): 97. DOI PMID
[9]	OTTEN P, KIM J, SON S H. A framework to automate assessment of upper-limb motor function impairment: a feasibility study[J]. Sensors, 2015, 15(8): 20097-20114. DOI PMID
[10]	MARTINO CINNERA A, PICERNO P, BISIRRI A, et al. Upper limb assessment with inertial measurement units according to the international classification of functioning in stroke: a systematic review and correlation meta-analysis[J]. Topics in Stroke Rehabilitation, 2024, 31(1): 66-85. DOI URL
[11]	BAI J, LI G C, LU X M, et al. Automatic rehabilitation assessment method of upper limb motor function based on posture and distribution force[J]. Frontiers in Neuroscience, 2024, 18: 1362495.
[12]	SONG X Y, CHEN S G, JIA J, et al. Cellphone-based automated Fugl-Meyer assessment to evaluate upper extremity motor function after stroke[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(10): 2186-2195. DOI PMID
[13]	谢秋蓉, 林婉琪, 张琪, 等. 脑卒中上肢康复智能化评估和虚拟现实训练研究进展[J]. 康复学报, 2023, 33(3): 271-279.
	XIE Q R, LIN W Q, ZHANG Q, et al. Research progress of intelligent evaluation and virtual reality based training in upper limb rehabilitation after stroke[J]. Rehabilitation Medicine, 2023, 33(3): 271-279(in Chinese). DOI URL
[14]	HERNÁNDEZ E D, GALEANO C P, BARBOSA N E, et al. Intra- and inter-rater reliability of Fugl-Meyer Assessment of Upper Extremity in stroke[J]. Journal of Rehabilitation Medicine, 2019, 51(9): 652-659. DOI PMID
[15]	WANG Z Y, ZHANG T, FAN J Y, et al. Clinical validation of automated depth camera-based measurement of the Fugl-Meyer assessment for upper extremity[J]. Clinical Rehabilitation, 2024, 38(8): 1091-1100. DOI URL
[16]	ZHOU W N, FU D Y, DUAN Z Y, et al. Achieving precision assessment of functional clinical scores for upper extremity using IMU-Based wearable devices and deep learning methods[J]. Journal of NeuroEngineering and Rehabilitation, 2025, 22(1): 84. DOI PMID
[17]	ZHOU Y M, RAMAN N, PROIETTI T, et al. Estimating upper extremity Fugl-Meyer assessment scores from reaching motions using wearable sensors[J]. IEEE Journal of Biomedical and Health Informatics, 2025, 29(6): 4134-4146. DOI URL
[18]	HAN J, LIAN S, GUO B, et al. Active rehabilitation training system for upper limb based on virtual reality[J]. Advances in Mechanical Engineering, 2017, 9(12): 1687814017743388.
[19]	ROJO A, CASTRILLO CALVILLO A, LÓPEZ C, et al. Effects of a virtual reality cycling platform on lower limb rehabilitation in patients with ataxia and hemiparesis: pilot randomized controlled trial[J]. JMIR Serious Games, 2024, 12: e39286. DOI URL
[20]	陶飞, 刘蔚然, 张萌, 等. 数字孪生五维模型及十大领域应用[J]. 计算机集成制造系统, 2019, 25(1): 1-18.
	TAO F, LIU W R, ZHANG M, et al. Five-dimension digital twin model and its ten applications[J]. Computer Integrated Manufacturing Systems, 2019, 25(1): 1-18(in Chinese).
[21]	VALLÉE A. Digital twin for healthcare systems[J]. Frontiers in Digital Health, 2023, 5: 1253050.
[22]	CELLUPICA A, CIRELLI M, SAGGIO G, et al. An interactive digital-twin model for virtual reality environments to train in the use of a sensorized upper-limb prosthesis[J]. Algorithms, 2024, 17(1): 35. DOI URL
[23]	BURTON Q, LEJEUNE T, DEHEM S, et al. Performing a shortened version of the Action Research Arm Test in immersive virtual reality to assess post-stroke upper limb activity[J]. Journal of NeuroEngineering and Rehabilitation, 2022, 19(1): 133. DOI PMID
[24]	SADÉE C, TESTA S, BARBA T, et al. Medical digital twins: enabling precision medicine and medical artificial intelligence[J]. The Lancet Digital Health, 2025, 7(7): 100864.
[25]	KATSOULAKIS E, WANG Q, WU H M, et al. Digital twins for health: a scoping review[J]. npj Digital Medicine, 2024, 7(1): 77. DOI PMID
[26]	GARCÍA-RUDOLPH A, WRIGHT M A, TEIXIDÓ-FONT C, et al. Digital twins in stroke rehabilitation: a scoping review of objectives, data sources, mechanisms, outcomes, and desirable properties[EB/OL]. (2025-11-07)[2025-12-20]. http://dio.org/10.1080/10749357.2025.2584031.
[27]	WU G, VAN DER HELM F C T, VEEGER H E J, et al. ISB recommendation on definitions of joint coordinate systems of various joints for the reporting of human joint motion—Part II: shoulder, elbow, wrist and hand[J]. Journal of Biomechanics, 2005, 38(5): 981-992. DOI URL
[28]	GLADSTONE D J, DANELLS C J, BLACK S E. The Fugl-Meyer assessment of motor recovery after stroke: a critical review of its measurement properties[J]. Neurorehabilitation and Neural Repair, 2002, 16(3): 232-240. DOI PMID
[29]	BANGOR A, KORTUM P, MILLER J. Determining what individual SUS scores mean: adding an adjective rating scale[J]. Journal of Usability Studies, 2009, 4(3): 114-123.

指标	符号	数据来源	关键参数	单位/方向	临床语义
ROM	${f}_{1}^{(w)}$	目标关节分量 ${\omega }^{(w)}(t)$	关节w	°；↑	反映幅度是否达标
CI	${f}_{2}^{(w)}$	邻近关节 ${\theta }^{(n)}(t)$与躯干 ${\theta }^{(n)}(t)$	邻近关节n	无量纲；↓	非目标部位代偿现象
PTM	${f}_{3}^{(w)}$	关节 ${\theta }^{(w)}(t)$与标准 ${\theta }^{(w)}(t)$的对齐	标准	无量纲；↓	目标部位是否达到参考位置
jerk	${f}_{4}^{(w)}$	目标关节角加速度 ${a}^{(w)}(t)$	差分阶次	无量纲；↓	动作是否平滑流畅
RS	${f}_{5}^{(w)}$	目标关节角速度稳定性 ${\omega }^{(w)}(t)$	STD	$rad/s$；↓	动作是否波动、停顿
CT	${f}_{6}^{(w)}$	片段起止t_s，t_e	v_th，t_min	s；↓	起始到终止的持续时间

指标	符号	数据来源	关键参数	单位/方向	临床语义
ROM	${f}_{1}^{(w)}$	目标关节分量 ${\omega }^{(w)}(t)$	关节w	°；↑	反映幅度是否达标
CI	${f}_{2}^{(w)}$	邻近关节 ${\theta }^{(n)}(t)$与躯干 ${\theta }^{(n)}(t)$	邻近关节n	无量纲；↓	非目标部位代偿现象
PTM	${f}_{3}^{(w)}$	关节 ${\theta }^{(w)}(t)$与标准 ${\theta }^{(w)}(t)$的对齐	标准	无量纲；↓	目标部位是否达到参考位置
jerk	${f}_{4}^{(w)}$	目标关节角加速度 ${a}^{(w)}(t)$	差分阶次	无量纲；↓	动作是否平滑流畅
RS	${f}_{5}^{(w)}$	目标关节角速度稳定性 ${\omega }^{(w)}(t)$	STD	$rad/s$；↓	动作是否波动、停顿
CT	${f}_{6}^{(w)}$	片段起止t_s，t_e	v_th，t_min	s；↓	起始到终止的持续时间

编号	动作类别	评分项	康复师评分判断逻辑	特征指标	监测传感器
S1	屈肌协同运动	肩后缩	动作幅度是否到位，是否有躯干代偿	ROM，CI	P3，P5，P7
S2		肩上提	是否能顺利提肩，是否伴随肩胛代偿或躯干偏移	ROM，CI	P3，P5，P6，P7
S3		肩外展	外展角度是否达标，是否达到预设空间位置	ROM，PTM	P3，P5，P6，P7
S4		肩外旋	是否能自然外旋，肘部或肩部是否代偿	ROM，CI	P2，P3，P7
S5		肘屈曲	屈肘幅度是否足够，是否过快或不稳定	ROM，CT，Jerk	P2，P3，P7
S6		前臂旋后	是否能完成充分旋后，手部位置是否准确	ROM，PTM	P2，P3，P5，P7

编号	动作类别	评分项	康复师评分判断逻辑	特征指标	监测传感器
S1	屈肌协同运动	肩后缩	动作幅度是否到位，是否有躯干代偿	ROM，CI	P3，P5，P7
S2		肩上提	是否能顺利提肩，是否伴随肩胛代偿或躯干偏移	ROM，CI	P3，P5，P6，P7
S3		肩外展	外展角度是否达标，是否达到预设空间位置	ROM，PTM	P3，P5，P6，P7
S4		肩外旋	是否能自然外旋，肘部或肩部是否代偿	ROM，CI	P2，P3，P7
S5		肘屈曲	屈肘幅度是否足够，是否过快或不稳定	ROM，CT，Jerk	P2，P3，P7
S6		前臂旋后	是否能完成充分旋后，手部位置是否准确	ROM，PTM	P2，P3，P5，P7

编号	动作名称	涉及关节	次数	时间	测试角度
A1	腕背伸	腕关节	20	3 s	0°~60°
A2	肘屈曲	肘关节			0°~90°
A3	肩外展	肩关节			0°~90°
A4	肩前屈	肩关节			0°~90°