用于沉浸式物理实验的半实物交互技术

doi:10.11996/JG.j.2095-302X.2026030598

摘要/Abstract

摘要：

现有沉浸式物理实验(IPE)普遍存在力触觉反馈缺失的问题，导致交互真实感不足，进而影响学生实验操作体验和学习效果。为此，提出一种融合主动式与被动式力触觉的半实物交互技术，并通过集成3D打印实体、多类型传感器(拉力、压力、温度等)及执行器的半实物仿真思想，构建了具备视觉-触觉同步反馈的实验系统。设计了集成拉力器、按钮和旋钮的弹簧双振子实验半实物交互装置；基于活塞和温控模块的气体三大定律实验半实物交互装置；以及基于数字舵机和喷射装置的浮力实验半实物交互装置。开展了包含32名大学生的对照实验，实验组采用半实物交互，对照组采用手势交互，结合NASA-TLX主观量表与64通道脑电信号进行评估。主观结果显示，实验组后测总认知负荷为55.27，显著低于对照组的60.13(p=0.041)，特别是在体力需求与受挫程度维度改善明显。神经生理结果表明，实验组在额叶(Fz)和枕叶(Oz)区域的θ，α，β波段平均功率谱密度均呈下降趋势，且显著低于对照组，表明大脑神经资源调配压力较小。研究证实，与传统手势交互相比，半实物交互能有效降低IPE中学生的认知负荷，优化认知资源分配效率，有助于学生更轻松地掌握相关物理知识。

关键词: 沉浸式物理实验, 虚拟现实, 力触觉, 半实物交互, 神经生理评估

Abstract:

Current Immersive Physics Experiments (IPE) generally lack haptic feedback, leading to insufficient interaction realism and negatively affecting students’ experimental operation experience and learning outcomes. To address this issue, a semi-physical interaction technology integrating active and passive haptic feedback was proposed. Drawing from the concept of semi-physical simulation, the proposed technology constructed an experimental system with synchronized visual-haptic feedback by integrating 3D-printed physical entities, multi-type sensors (tension, pressure, temperature, etc.), and actuators. Specifically, a semi-physical interaction device was designed for the coupled spring-mass oscillator experiment integrating a puller, button, and knob; a device for the three gas laws experiment based on a piston and temperature control module; and a device for the buoyancy experiment composed of digital servos and a jet device. A controlled experiment involving 32 university students was conducted, where the experimental group used semi-physical interaction and the control group used gesture interaction. Evaluation was performed using the NASA-TLX (NASA-Task Load Index) subjective scale combined with 64-channel EEG signals. Subjective results indicated that the post-test total cognitive load of the experimental group was 55.27, which was significantly lower than that of the control group (60.13, p=0.041), with marked improvements in physical demand and frustration levels. Neurophysiological results demonstrated that the average power spectral densities of θ, α, and β bands in the frontal (Fz) and occipital (Oz) lobes of the experimental group showed a downward trend and were significantly lower than in the control group, indicating reduced pressure on neural resource allocation. The study confirmed that compared with traditional gesture interaction, semi-physical interaction effectively reduced students’ cognitive load in IPE, optimized the efficiency of cognitive resource allocation, and facilitated easier mastery of relevant physics knowledge.

Key words: immersive physics experiment, virtual reality, haptics, semi-physical interaction, neurophysiological evaluation

中图分类号:

纪海林, 张怡冉, 李亦航, 张鸿文, 骆岩红. 用于沉浸式物理实验的半实物交互技术[J]. 图学学报, 2026, 47(3): 598-606.

JI Hailin, ZHANG Yiran, LI Yihang, ZHANG Hongwen, LUO Yanhong. Semi-physical interaction technology for immersive physics experiments[J]. Journal of Graphics, 2026, 47(3): 598-606.

图/表 11

图1 弹簧双振子实验场景

Fig. 1 Coupled spring-mass oscillator experiment scenario

图2 气体三大定律实验场景

Fig. 2 The three gas laws experiment scenario

图3 浮力实验场景

Fig. 3 Buoyancy experiment scenario

图4 半实物交互技术路线图

Fig. 4 Technology roadmap of semi-physical interaction

图5 弹簧双振子实验中的半实物((a) 拉力器；(b) 按钮；(c) 旋钮)

Fig. 5 Semi-physical objects in the coupled spring-mass oscillator experiment ((a) Puller; (b) Button; (c) Knob)

图6 气体三大定律实验中的半实物((a) 活塞；(b) 温控装置)

Fig. 6 Semi-physical objects in the three gas laws experiment ((a) Piston; (b) Temperature control device)

图7 浮力实验中的半实物((a) 数字舵机；(b) 喷射装置)

Fig. 7 Semi-physical objects in the buoyancy experiment ((a) Digital servo; (b) Fluid delivery device)

图8 脑电评估实验环境

Fig. 8 EEG evaluation experimental environment

表1 不同维度的认知负荷评分比较

Table 1 Comparison of cognitive load scores in different dimensions

维度	总体(前)	实验组(后)	对照组(后)
脑力需求	75.47	72.81	73.75
体力需求	21.25	20.94	27.50
时间需求	50.16	43.75	43.44
绩效水平	45.00	40.31	42.19
努力程度	54.53	54.06	60.31
受挫程度	50.16	42.19	53.44
总体认知负荷	58.73	55.27	60.13

表2 不同电极通道各波段功率比较

Table 2 Comparison of power in each frequency band across different electrode channels

脑区(通道)	波段	总体(前)	实验组(后)	对照组(后)
额叶(Fz)	θ 波	1.78	1.64	2.55
	α 波	0.68	0.64	0.74
	β 波	0.32	0.17	0.26
中央区(Cz)	θ 波	1.62	0.88	1.15
	α 波	0.69	0.49	0.58
	β 波	0.26	0.13	0.19
顶叶(Pz)	θ 波	2.02	1.44	1.53
	α 波	1.25	0.97	0.96
	β 波	0.48	0.37	0.33
枕叶(Oz)	θ 波	3.49	2.70	4.23
	α 波	3.76	2.79	2.84
	β 波	2.13	1.85	1.54

图9 不同通道各波段功率比较

Fig. 9 Power comparison of each band in different channels

参考文献 32

[1]	NAZ Z, AZAM A, KHAN M U G, et al. Development and evaluation of immersive VR laboratories of organic chemistry and physics for students education[J]. Physica Scripta, 2024, 99(5): 056101. DOI
[2]	YU X Z, LI K, LI J Z, et al. Design of an VR-based immersive physics experiment teaching platform[C]// 2021 2nd International Conference on Information Science and Education. New York: IEEE Press, 2021: 1102-1106.
[3]	TSIVITANIDOU O E, GEORGIOU Y, IOANNOU A. A learning experience in inquiry-based physics with immersive virtual reality: student perceptions and an interaction effect between conceptual gains and attitudinal profiles[J]. Journal of Science Education and Technology, 2021, 30(6): 841-861. DOI
[4]	GONÇALVES G, COELHO H, MONTEIRO P, et al. Systematic review of comparative studies of the impact of realism in immersive virtual experiences[J]. ACM Computing Surveys, 2023, 55(6): 115.
[5]	BUCKINGHAM G. Hand tracking for immersive virtual reality: opportunities and challenges[J]. Frontiers in Virtual Reality, 2021, 2: 728461. DOI URL
[6]	LIU Y, SHEN Y, LUO C, et al. Reach out and touch: eliciting the sense of touch through gesture-based interaction[J]. Journal of the Association for Information Systems, 2021, 22(6): 1686-1714. DOI URL
[7]	VENKATRAJ K P, MEIJER W, PERUSQUÍA-HERNÁNDEZ M, et al. ShareYourReality: investigating haptic feedback and agency in virtual avatar co-embodiment[C]// 2024 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2024: 1-15.
[8]	WEE C, YAP K M, LIM W N. Haptic interfaces for virtual reality: Challenges and research directions[J]. IEEE Access, 2021, 9: 112145-112162. DOI URL
[9]	ZHANG Z, XU Z H, EMU L, et al. Active mechanical haptics with high-fidelity perceptions for immersive virtual reality[J]. Nature Machine Intelligence, 2023, 5(6): 643-655. DOI
[10]	ZENNER A, ULLMANN K, KRÜGER A. Combining dynamic passive haptics and haptic retargeting for enhanced haptic feedback in virtual reality[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(5): 2627-2637. DOI URL
[11]	MIHALIČ F, TRUNTIČ M, HREN A. Hardware-in-the-loop simulations: a historical overview of engineering challenges[J]. Electronics, 2022, 11(15): 2462. DOI URL
[12]	ACEVEDO P, MAGANA A J, WALSH Y, et al. Embodied immersive virtual reality to enhance the conceptual understanding of charged particles: a qualitative study[J]. Computers & Education: X Reality, 2024, 5: 100075.
[13]	MUFIT F, HENDRIYANI Y, DHANIL M. Design immersive virtual reality (IVR) with cognitive conflict to support practical learning of quantum physics[J]. Journal of Turkish Science Education, 2024, 21(2): 369-388.
[14]	STĘPNIKOWSKI A W. Virtual reality learning retention in education and trainings[J]. Edukacja Ustawiczna Dorosłych, 2021, 115(4): 55-61.
[15]	BLACKLER A, DESAI S, MCEWAN M, et al. Perspectives on the nature of intuitive interaction[M]//BLACKLER A. Intuitive Interaction: Research and Application. Boca Raton: CRC Press, 2018: 19-39.
[16]	LEE Y. Exploring mixed interaction mode in virtual reality: controller-based and hand gesture integration[D]. Auckland: Auckland University of Technology, 2024.
[17]	GUO L, LU Z X, YAO L G. Human-machine interaction sensing technology based on hand gesture recognition: a review[J]. IEEE Transactions on Human-Machine Systems, 2021, 51(4): 300-309. DOI URL
[18]	JIANG S, KANG P Q, SONG X Y, et al. Emerging wearable interfaces and algorithms for hand gesture recognition: a survey[J]. IEEE Reviews in Biomedical Engineering, 2022, 15: 85-102. DOI URL
[19]	ZHOU H Y, WANG D Y, YU Y, et al. Research progress of human-computer interaction technology based on gesture recognition[J]. Electronics, 2023, 12(13): 2805. DOI URL
[20]	BORZELLI D, BOARINI V, CASILE A. A quantitative assessment of the hand kinematic features estimated by the oculus Quest 2[J]. Scientific Reports, 2025, 15(1): 8842. DOI
[21]	GIBBS J K, GILLIES M, PAN X N. A comparison of the effects of haptic and visual feedback on presence in virtual reality[J]. International Journal of Human-Computer Studies, 2022, 157: 102717. DOI URL
[22]	ADILKHANOV A, RUBAGOTTI M, KAPPASSOV Z. Haptic devices: wearability-based taxonomy and literature review[J]. IEEE Access, 2022, 10: 91923-91947. DOI URL
[23]	TUNCA E, FLEISCHER R, SCHMIDT L, et al. Advantages of active haptics on touch surfaces[C]// The 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. New York: ACM, 2016: 137-144.
[24]	KOMENO N, MATSUBARA T. Tactile perception based on injected vibration in soft sensor[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 5365-5372. DOI URL
[25]	GUTIÉRREZ-FERNÁNDEZ A, FERNÁNDEZ-LLAMAS C, ESTEBAN G, et al. Haptic zoom: an interaction model for desktop haptic devices with limited workspace[J]. International Journal of Human-Computer Interaction, 2023, 39(4): 851-862. DOI URL
[26]	LI P, ZHANG X X, HU X W, et al. Human-computer interaction on virtual reality-based training system for vascular interventional surgery[J]. Computer Methods and Programs in Biomedicine, 2025, 265: 108731. DOI URL
[27]	ZENNER A, KRÜGER A. Shifty: a weight-shifting dynamic passive haptic proxy to enhance object perception in virtual reality[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(4): 1285-1294. DOI URL
[28]	ZHU K N, CHEN T Z, HAN F, et al. HapTwist: creating interactive haptic proxies in virtual reality using low-cost twistable artefacts[C]// 2019 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2019: 693.
[29]	FEICK M, BATEMAN S, TANG A, et al. Tangi: tangible proxies for embodied object exploration and manipulation in virtual reality[C]// 2020 IEEE International Symposium on Mixed and Augmented Reality. New York: IEEE Press, 2020: 195-206.
[30]	Meta for Work. Hand tracking technology and haptic feedback explained[EB/OL]. (2025-05-30) [2025-07-09]. https://forwork.meta.com/blog/hand-tracking-technology-and-haptic-feedback-mr/.
[31]	HART S G. Nasa-task load index (NASA-TLX); 20 years later[J]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006, 50(9): 904-908. DOI URL
[32]	MOLLET G A. Fundamentals of human neuropsychology[J]. Journal of Undergradute Neuroscience Education, 2008, 6(2): R3-R4.