图学学报 ›› 2026, Vol. 47 ›› Issue (4): 776-787.DOI: 10.11996/JG.j.2095-302X.2026040776
收稿日期:2026-03-08
接受日期:2026-06-11
出版日期:2026-08-31
发布日期:2026-08-31
通讯作者:吕健,E-mail:jlv@gzu.edu.cn基金资助:
WEN Ruiqi1, LV Jian1(
), SONG Dingan2, SU Le1, LIANG Zhibin1
Received:2026-03-08
Accepted:2026-06-11
Published:2026-08-31
Online:2026-08-31
Contact:
LV Jian,E-mail:jlv@gzu.edu.cnSupported by:摘要:
面向脑卒中下肢康复中动态行走能力评估、训练难度调节和安全过程监测需求,构建具身沉浸式虚拟现实康复训练与评估系统,验证其在健康受试者概念验证阶段对动态稳定性、身体表征和认知负荷的测量可行性。系统以全向跑步机为行走载体,采用全身惯性动作捕捉驱动第一人称全身虚拟化身,结合头显位姿、事件日志与眼动追踪,实现行走运动学、视觉注意和任务行为数据的同步采集。围绕直行、转向、避障和双任务行走构建任务库,设计基于质心加速度均方根、加速度急变均方根、本体感觉定位误差、瞳孔变化量、注视/扫视特征及双任务成本的指标体系,并引入基线归一化阈值与迟滞判定机制,形成“状态采集-规则判定-任务调节-事件记录”的原型级闭环。在下肢可见性实验中(N=27),虚拟化身下肢透明度对本体感觉定位误差和具身体验评分产生显著影响,完全不可见条件的定位误差较高、具身体验较低,而晕动症评分差异不显著;半透明条件在保留身体轮廓线索与减轻环境遮挡之间表现出较好平衡。在双任务行走实验中(N=29),随着认知负荷由低至高提升,瞳孔统计量、瞳孔变化量、注视次数和扫视次数总体升高,平均注视时长缩短,说明眼动特征能够反映动态行走中的负荷分级;高步态难度还引起质心加速度均方根升高及躯干侧向摆动收敛,验证了步态难度操控的有效性。该系统能够在沉浸式动态行走场景下整合运动控制、身体视觉线索和认知负荷证据,为脑卒中康复训练系统的任务设计、参数调节和临床前评估提供方法基础;相关阈值和设计策略仍需在脑卒中患者中进一步验证。
中图分类号:
温瑞祺, 吕健, 宋定安, 苏乐, 梁智斌. 具身视域下康复训练虚拟现实系统设计与评估[J]. 图学学报, 2026, 47(4): 776-787.
WEN Ruiqi, LV Jian, SONG Dingan, SU Le, LIANG Zhibin. Embodied virtual reality system design and evaluation for rehabilitation training[J]. Journal of Graphics, 2026, 47(4): 776-787.
| 任务 | 训练目标 | 难度参数 | 主要输出 | 自适应调节控制规则 |
|---|---|---|---|---|
| 直行 | 基础步态与整体动态稳定 | 行走速度(v);任务时长(T);路径约束强度 | CoM-RMS、急变均方根( | 稳定性超阈值→降低 (v) 或缩短 (T);多次触发→提升约束宽容度 |
| 转向 | 转向过程的姿态控制与再稳定能力 | 转角(θ);转向频率 (f);指令提前量(Δt) | 转向时长;转向阶段角速度特征;转向后稳定性恢复时间 | 若恢复时间显著延长或角速度波动过大→降低 (θ) 或降低 (f);多次“再稳定失败”→增加 (Δt) |
| 避障 | 动态平衡与路径规划能力 | 障碍密度 (d);出现时机;横向偏置幅度 | 避障成功率;避障阶段稳定性表现 | 连续碰撞或RMS超阈值→降低 (d) 或延后出现;若偏置导致频繁失败→减小 (d) |
| 双任务行走 | 认知-运动协同与注意分配能力 | 认知刺激频率 (r);响应窗口 | 步态稳定性DTC、RT/Acc DTC; 瞳孔尺寸、注视/扫视等认知负荷指标 | 眼动提示负荷过高→降低 (r) 或放宽响应窗口; |
表1 任务库定义与参数化难度设置
Table 1 Task library definition and parameterization difficulty setting
| 任务 | 训练目标 | 难度参数 | 主要输出 | 自适应调节控制规则 |
|---|---|---|---|---|
| 直行 | 基础步态与整体动态稳定 | 行走速度(v);任务时长(T);路径约束强度 | CoM-RMS、急变均方根( | 稳定性超阈值→降低 (v) 或缩短 (T);多次触发→提升约束宽容度 |
| 转向 | 转向过程的姿态控制与再稳定能力 | 转角(θ);转向频率 (f);指令提前量(Δt) | 转向时长;转向阶段角速度特征;转向后稳定性恢复时间 | 若恢复时间显著延长或角速度波动过大→降低 (θ) 或降低 (f);多次“再稳定失败”→增加 (Δt) |
| 避障 | 动态平衡与路径规划能力 | 障碍密度 (d);出现时机;横向偏置幅度 | 避障成功率;避障阶段稳定性表现 | 连续碰撞或RMS超阈值→降低 (d) 或延后出现;若偏置导致频繁失败→减小 (d) |
| 双任务行走 | 认知-运动协同与注意分配能力 | 认知刺激频率 (r);响应窗口 | 步态稳定性DTC、RT/Acc DTC; 瞳孔尺寸、注视/扫视等认知负荷指标 | 眼动提示负荷过高→降低 (r) 或放宽响应窗口; |
图3 虚拟化身下肢可见性操控条件((a) 下肢不可见;(b) 下肢半透明;(c) 下肢不透明)
Fig. 3 Virtual avatar lower limb visibility control conditions ((a) Clear lower limbs; (b) Semi-transparent lower limbs; (c) Opaque lower limbs)
| 实验 | 核心目的 | 自变量 | 因变量 | 传感器 |
|---|---|---|---|---|
| 1 (N=27) | 稳定性 定位感知 | OPAQ/ TRANS/ CLR | 定位误差 稳定性RMS AEQ/SSQ | IMU |
| 2 (N=29) | 负荷分级 | LDG/HDG LOW/MED/HIGH | P95/MeanΔ 注视/扫视 躯干指标 | 眼动 + IMU |
表2 实验设计
Table 2 Experimental design
| 实验 | 核心目的 | 自变量 | 因变量 | 传感器 |
|---|---|---|---|---|
| 1 (N=27) | 稳定性 定位感知 | OPAQ/ TRANS/ CLR | 定位误差 稳定性RMS AEQ/SSQ | IMU |
| 2 (N=29) | 负荷分级 | LDG/HDG LOW/MED/HIGH | P95/MeanΔ 注视/扫视 躯干指标 | 眼动 + IMU |
| 指标 | 结果 | 分析 |
|---|---|---|
| 本体定位感知 | F(2,52)=6.14,p=0.004 | CLR > OPAQ;TRANS介于两者之间 |
| 具身体验(AEQ) | F(2,52)=4.91,p=0.011 | CLR < TRANS; CLR < OPAQ |
| 晕动症(SSQ) | F(2,52)=1.12,p=0.330 | 差异不显著;差异主要来自视觉身体线索变化 |
表3 本体定位、具身体验与晕动症统计结果
Table 3 Body positioning, embodied experience, and statistical results of motion sickness
| 指标 | 结果 | 分析 |
|---|---|---|
| 本体定位感知 | F(2,52)=6.14,p=0.004 | CLR > OPAQ;TRANS介于两者之间 |
| 具身体验(AEQ) | F(2,52)=4.91,p=0.011 | CLR < TRANS; CLR < OPAQ |
| 晕动症(SSQ) | F(2,52)=1.12,p=0.330 | 差异不显著;差异主要来自视觉身体线索变化 |
| 指标 | LOW | MED | HIGH | Friedman χ²(2)/p |
|---|---|---|---|---|
| P95/mm | 3.31±0.41 | 3.27±0.42 | 3.46±0.467 | χ²=11.80,p=0.002 6 |
| MeanΔ/mm | 0.23±0.06 | 0.23±0.07 | 0.27±0.083 | χ²=9.17,p=0.010 2 |
| 注视次数/N | 82.40±36.40 | 100.70±41.70 | 129.80±66.900 | χ²=11.00,p=0.004 0 |
| 平均注视时长/s | 0.88±0.59 | 0.78±0.52 | 0.52±0.225 | χ²=10.10,p=0.010 0 |
| 扫视次数/N | 85.70±45.90 | 115.10±68.20 | 125.20±87.700 | χ²=8.07,p=0.0170 0 |
表4 不同认知负荷分级的眼动数据
Table 4 Eye movement data with different levels of cognitive load classification
| 指标 | LOW | MED | HIGH | Friedman χ²(2)/p |
|---|---|---|---|---|
| P95/mm | 3.31±0.41 | 3.27±0.42 | 3.46±0.467 | χ²=11.80,p=0.002 6 |
| MeanΔ/mm | 0.23±0.06 | 0.23±0.07 | 0.27±0.083 | χ²=9.17,p=0.010 2 |
| 注视次数/N | 82.40±36.40 | 100.70±41.70 | 129.80±66.900 | χ²=11.00,p=0.004 0 |
| 平均注视时长/s | 0.88±0.59 | 0.78±0.52 | 0.52±0.225 | χ²=10.10,p=0.010 0 |
| 扫视次数/N | 85.70±45.90 | 115.10±68.20 | 125.20±87.700 | χ²=8.07,p=0.0170 0 |
| 指标 | LDG | HDG | p值 |
|---|---|---|---|
| CoM-RMS/ (m/s²) | 3.090±0.740 | 3.940±0.340 | p=4.30e-06 |
| 躯干侧向角SD/deg | 4.470±2.250 | 2.360±0.840 | p=0.000 180 |
| 躯干侧向角 范围/deg | 23.430±9.550 | 12.950±4.390 | p=1.40e-05 |
| 躯干摆动平 均速度/(deg/s) | 11.770±6.380 | 7.940±2.570 | qFDR=0.004 500 |
| 试次时长/s | 72.770±22.560 | 45.900±4.970 | qFDR=8.09e-06 |
| CoM速度RMS/(m/s) | 0.176±0.041 | 0.215±0.023 | qFDR=0.000 427 |
表5 不同步态难度运动学对比
Table 5 Comparison of different gait difficulty kinematics
| 指标 | LDG | HDG | p值 |
|---|---|---|---|
| CoM-RMS/ (m/s²) | 3.090±0.740 | 3.940±0.340 | p=4.30e-06 |
| 躯干侧向角SD/deg | 4.470±2.250 | 2.360±0.840 | p=0.000 180 |
| 躯干侧向角 范围/deg | 23.430±9.550 | 12.950±4.390 | p=1.40e-05 |
| 躯干摆动平 均速度/(deg/s) | 11.770±6.380 | 7.940±2.570 | qFDR=0.004 500 |
| 试次时长/s | 72.770±22.560 | 45.900±4.970 | qFDR=8.09e-06 |
| CoM速度RMS/(m/s) | 0.176±0.041 | 0.215±0.023 | qFDR=0.000 427 |
| 指标 | 基线条件 | 触发阈值 | 回落阈值 |
|---|---|---|---|
| CoM-RMS | LDG | 4.307 | 4.039 |
| 瞳孔统计量 | LOW | 3.984 | 3.836 |
| 瞳孔波动幅度 | LOW | 0.329 | 0.307 |
| 注视次数 | LOW | 142.278 | 129.065 |
表6 实验系统默认阈值参数
Table 6 Default threshold parameters for system initialization
| 指标 | 基线条件 | 触发阈值 | 回落阈值 |
|---|---|---|---|
| CoM-RMS | LDG | 4.307 | 4.039 |
| 瞳孔统计量 | LOW | 3.984 | 3.836 |
| 瞳孔波动幅度 | LOW | 0.329 | 0.307 |
| 注视次数 | LOW | 142.278 | 129.065 |
| [1] |
MOHLER B J, THOMPSON W B, CREEM-REGEHR S H, et al. Calibration of locomotion resulting from visual motion in a treadmill-based virtual environment[J]. ACM Transactions on Applied Perception, 2007, 4(1): 4-es.
DOI URL |
| [2] |
KETTERER J, GOLLHOFER A, RINGHOF S, et al. Effects of balance training with visual input manipulations on balance performance and sensory integration in healthy young adults: a randomized controlled trial[J]. Scientific Reports, 2024, 14(1): 28589.
DOI |
| [3] |
HORAK F B. Postural orientation and equilibrium: what do we need to know about neural control of balance to prevent falls?[J]. Age and Ageing, 2006, 35(S2): ii7-ii11.
DOI URL |
| [4] |
PETERKA R J. Sensorimotor integration in human postural control[J]. Journal of Neurophysiology, 2002, 88(3): 1097-1118.
DOI PMID |
| [5] |
张帅, 洪翱, 胡恒瑞, 等. 基于AI动作捕捉技术的视神经脊髓炎康复训练系统交互性研究[J]. 图学学报, 2025, 46(4): 783-792.
DOI |
|
ZHANG S, HONG A, HU H R, et al. Study on the interaction of an AI-based motion capture technology in rehabilitation training systems for neuromyelitis optica[J]. Journal of Graphics, 2025, 46(4): 783-79 (in Chinese).
DOI |
|
| [6] | LEI M, LIU P, WANG Q S, et al. Precise identification of rehabilitation actions using AI based strategy[C]// The 29th International Conference on Computer Communications and Networks. New York: IEEE Press, 2020: 1-7. |
| [7] | ZHAO S, TONG Y, CHEN G, et al. Testing the feasibility of quantizing the progress of stroke patients’ rehabilitation with a computer vision method[C]// The 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications. New York: IEEE Press, 2022: 1-4. |
| [8] | CÓIAS A R, LEE M H, BERNARDINO A, et al. Skeleton tracking solutions for a low-cost stroke rehabilitation support system[C]// 2023 International Conference on Rehabilitation Robotics. New York: IEEE Press, 2023: 1-6. |
| [9] | 罗世怀, 吕健, 刘翔. 虚拟现实中多通道信息呈现方式的认知负荷定量研究[J]. 包装工程, 2023, 44(4): 69-76. |
| LUO S H, LYU J, LIU X. Quantitative study on cognitive load of multi-channel information presentation in virtual reality[J]. Packaging Engineering, 2023, 44(4): 69-76 (in Chinese). | |
| [10] |
WICKENS C D. Multiple resources and mental workload[J]. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2008, 50(3): 449-455.
DOI URL |
| [11] |
THORPE A, INNES R, TOWNSEND J, et al. Assessing cross-modal interference in the detection response task[J]. Journal of Mathematical Psychology, 2020, 98: 102390.
DOI URL |
| [12] |
RIGGS S L, SARTER N. Crossmodal matching: the case for developing and employing a valid and feasible approach to equate perceived stimulus intensities in multimodal research[J]. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2019, 61(1): 29-31.
DOI URL |
| [13] |
VAN DER WEL P, VAN STEENBERGEN H. Pupil dilation as an index of effort in cognitive control tasks: a review[J]. Psychonomic Bulletin & Review, 2018, 25(6): 2005-2015.
DOI |
| [14] | HOLMQVIST K, NYSTRÖM M, ANDERSSON R, et al. Eye tracking: a comprehensive guide to methods and measures[M]. Oxford: Oxford University Press, 2011, 295-388. |
| [15] |
WILF M, KORAKIN A, BAHAT Y, et al. Using virtual reality-based neurocognitive testing and eye tracking to study naturalistic cognitive-motor performance[J]. Neuropsychologia, 2024, 194: 108744.
DOI URL |
| [16] |
MÜLLER C, BAUMANN T, EINHÄUSER W, et al. Slipping while counting: gaze-gait interactions during perturbed walking under dual-task conditions[J]. Experimental Brain Research, 2023, 241(3): 765-780.
DOI PMID |
| [17] | HUNT R, BLACKMORE T, MILLS C, et al. Evaluating the integration of eye-tracking and motion capture technologies: quantifying the accuracy and precision of gaze measures[EB/OL]. [2026-01-20]. https://doaj.org/article/33b327a4f63b4b518b997c63bd5dfde1?__cf_chl_f_tk=y7j0AqXxxi6y9OSTjbx3HQA9cw.Oga4ReDQlGiXKKvM-1783050042-1.0.1.1-IHkfQGsbdV_CDkBpaOiSn1BvAOXxxwrq8QfIFC.Q0W8. |
| [18] | CALANDRA D M, CASO A, CUTUGNO F, et al. CoWME: a general framework to evaluate cognitive workload during multimodal interaction[C]// The 15th ACM on International Conference on Multimodal Interaction. New York: ACM, 2013: 111-118. |
| [19] |
郑玲, 乔旭强, 倪涛, 等. 基于多维信息特征分析的驾驶人认知负荷研究[J]. 中国公路学报, 2021, 34(4): 240-250.
DOI |
|
ZHENG L, QIAO X Q, NI T, et al. Driver cognitive loads based on multi-dimensional information feature analysis[J]. China Journal of Highway and Transport, 2021, 34(4): 240-250 (in Chinese).
DOI |
|
| [20] |
柯青, 丁松云, 秦琴. 健康信息可读性对用户认知负荷和信息加工绩效影响眼动实验研究[J]. 数据分析与知识发现, 2021, 5(2): 70-82.
DOI |
| KE Q, DING S Y, QIN Q. Health information readability affects users' cognitive load and information processing: an eye-tracking study[J]. Data Analysis and Knowledge Discovery, 2021, 5(2): 70-82 (in Chinese). | |
| [21] | 徐小萍, 吕健, 金昱潼, 等. 用户认知驱动的VR自然交互认知负荷研究[J]. 计算机应用研究, 2020, 37(7): 1958-1963. |
| XU X P, LYU J, JIN Y T, et al. Research on natural interactive cognitive load of virtual reality driven by user cognition[J]. Application Research of Computers, 2020, 37(7): 1958-1963 (in Chinese). | |
| [22] |
KILTENI K, GROTEN R, SLATER M. The sense of embodiment in virtual reality[J]. Presence: Teleoperators and Virtual Environments, 2012, 21(4): 373-387.
DOI URL |
| [23] |
SLATER M, PEREZ-MARCOS D, EHRSSON H H, et al. Inducing illusory ownership of a virtual body[J]. Frontiers in Neuroscience, 2009, 3(2): 214-220.
DOI PMID |
| [24] |
SCHATZSCHNEIDER C, BRUDER G, STEINICKE F. Who turned the clock? Effects of manipulated zeitgebers, cognitive load and immersion on time estimation[J]. IEEE Transactions on Visualization and Computer Graphics, 2016, 22(4): 1387-1395.
DOI URL |
| [25] |
GONZALEZ-FRANCO M, PECK T C,. Avatar embodiment. Towards a standardized questionnaire[J]. Frontiers in Robotics and AI, 2018, 5: 74.
DOI URL |
| [26] |
ROTH D, LATOSCHIK M E. Construction of the virtual embodiment questionnaire (VEQ)[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(12): 3546-3556.
DOI URL |
| [27] |
PATLA A E. Understanding the roles of vision in the control of human locomotion[J]. Gait & Posture, 1997, 5(1): 54-69.
DOI URL |
| [28] |
WARREN W H JR, KAY B A, ZOSH W D, et al. Optic flow is used to control human walking[J]. Nature Neuroscience, 2001, 4(2): 213-216.
DOI PMID |
| [29] |
KENNEDY R S, LANE N E, BERBAUM K S, et al. Simulator Sickness Questionnaire: an enhanced method for quantifying simulator sickness[J]. The International Journal of Aviation Psychology, 1993, 3(3): 203-220.
DOI URL |
| [1] | 李洁, 任宇航, 郭士杰. 依恋回避调节的情感疗愈机器人交互机制研究[J]. 图学学报, 2026, 47(4): 915-926. |
| [2] | 李和森, 陈影, 么大伟. 基于眼动追踪的塑壳断路器运行状态信息优化设计[J]. 图学学报, 2023, 44(6): 1259-1266. |
| [3] | 裴卉宁, 邵星辰, 谭昭芸, 黄雪芹, 白仲航. 融合DE-GWO与SVR的文化意象预测模型[J]. 图学学报, 2023, 44(1): 184-193. |
| [4] | 闫龙华, 林 丽, 肖华亮, 时 雷. 基于心理与生理测量的苗族服饰纹样美感研究[J]. 图学学报, 2017, 38(5): 683-688. |
| 阅读次数 | ||||||
|
全文 |
|
|||||
|
摘要 |
|
|||||