Semi-physical interaction technology for immersive physics experiments

doi:10.11996/JG.j.2095-302X.2026030598

Abstract

Abstract:

Current Immersive Physics Experiments (IPE) generally lack haptic feedback, leading to insufficient interaction realism and negatively affecting students’ experimental operation experience and learning outcomes. To address this issue, a semi-physical interaction technology integrating active and passive haptic feedback was proposed. Drawing from the concept of semi-physical simulation, the proposed technology constructed an experimental system with synchronized visual-haptic feedback by integrating 3D-printed physical entities, multi-type sensors (tension, pressure, temperature, etc.), and actuators. Specifically, a semi-physical interaction device was designed for the coupled spring-mass oscillator experiment integrating a puller, button, and knob; a device for the three gas laws experiment based on a piston and temperature control module; and a device for the buoyancy experiment composed of digital servos and a jet device. A controlled experiment involving 32 university students was conducted, where the experimental group used semi-physical interaction and the control group used gesture interaction. Evaluation was performed using the NASA-TLX (NASA-Task Load Index) subjective scale combined with 64-channel EEG signals. Subjective results indicated that the post-test total cognitive load of the experimental group was 55.27, which was significantly lower than that of the control group (60.13, p=0.041), with marked improvements in physical demand and frustration levels. Neurophysiological results demonstrated that the average power spectral densities of θ, α, and β bands in the frontal (Fz) and occipital (Oz) lobes of the experimental group showed a downward trend and were significantly lower than in the control group, indicating reduced pressure on neural resource allocation. The study confirmed that compared with traditional gesture interaction, semi-physical interaction effectively reduced students’ cognitive load in IPE, optimized the efficiency of cognitive resource allocation, and facilitated easier mastery of relevant physics knowledge.

Key words: immersive physics experiment, virtual reality, haptics, semi-physical interaction, neurophysiological evaluation

CLC Number:

JI Hailin, ZHANG Yiran, LI Yihang, ZHANG Hongwen, LUO Yanhong. Semi-physical interaction technology for immersive physics experiments[J]. Journal of Graphics, 2026, 47(3): 598-606.

Figures/Tables 11

Fig. 1 Coupled spring-mass oscillator experiment scenario

Fig. 2 The three gas laws experiment scenario

Fig. 3 Buoyancy experiment scenario

Fig. 4 Technology roadmap of semi-physical interaction

Fig. 5 Semi-physical objects in the coupled spring-mass oscillator experiment ((a) Puller; (b) Button; (c) Knob)

Fig. 6 Semi-physical objects in the three gas laws experiment ((a) Piston; (b) Temperature control device)

Fig. 7 Semi-physical objects in the buoyancy experiment ((a) Digital servo; (b) Fluid delivery device)

Fig. 8 EEG evaluation experimental environment

Table 1 Comparison of cognitive load scores in different dimensions

维度	总体(前)	实验组(后)	对照组(后)
脑力需求	75.47	72.81	73.75
体力需求	21.25	20.94	27.50
时间需求	50.16	43.75	43.44
绩效水平	45.00	40.31	42.19
努力程度	54.53	54.06	60.31
受挫程度	50.16	42.19	53.44
总体认知负荷	58.73	55.27	60.13

Table 2 Comparison of power in each frequency band across different electrode channels

脑区(通道)	波段	总体(前)	实验组(后)	对照组(后)
额叶(Fz)	θ 波	1.78	1.64	2.55
	α 波	0.68	0.64	0.74
	β 波	0.32	0.17	0.26
中央区(Cz)	θ 波	1.62	0.88	1.15
	α 波	0.69	0.49	0.58
	β 波	0.26	0.13	0.19
顶叶(Pz)	θ 波	2.02	1.44	1.53
	α 波	1.25	0.97	0.96
	β 波	0.48	0.37	0.33
枕叶(Oz)	θ 波	3.49	2.70	4.23
	α 波	3.76	2.79	2.84
	β 波	2.13	1.85	1.54

Fig. 9 Power comparison of each band in different channels

References 32

[1]	NAZ Z, AZAM A, KHAN M U G, et al. Development and evaluation of immersive VR laboratories of organic chemistry and physics for students education[J]. Physica Scripta, 2024, 99(5): 056101. DOI
[2]	YU X Z, LI K, LI J Z, et al. Design of an VR-based immersive physics experiment teaching platform[C]// 2021 2nd International Conference on Information Science and Education. New York: IEEE Press, 2021: 1102-1106.
[3]	TSIVITANIDOU O E, GEORGIOU Y, IOANNOU A. A learning experience in inquiry-based physics with immersive virtual reality: student perceptions and an interaction effect between conceptual gains and attitudinal profiles[J]. Journal of Science Education and Technology, 2021, 30(6): 841-861. DOI
[4]	GONÇALVES G, COELHO H, MONTEIRO P, et al. Systematic review of comparative studies of the impact of realism in immersive virtual experiences[J]. ACM Computing Surveys, 2023, 55(6): 115.
[5]	BUCKINGHAM G. Hand tracking for immersive virtual reality: opportunities and challenges[J]. Frontiers in Virtual Reality, 2021, 2: 728461. DOI URL
[6]	LIU Y, SHEN Y, LUO C, et al. Reach out and touch: eliciting the sense of touch through gesture-based interaction[J]. Journal of the Association for Information Systems, 2021, 22(6): 1686-1714. DOI URL
[7]	VENKATRAJ K P, MEIJER W, PERUSQUÍA-HERNÁNDEZ M, et al. ShareYourReality: investigating haptic feedback and agency in virtual avatar co-embodiment[C]// 2024 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2024: 1-15.
[8]	WEE C, YAP K M, LIM W N. Haptic interfaces for virtual reality: Challenges and research directions[J]. IEEE Access, 2021, 9: 112145-112162. DOI URL
[9]	ZHANG Z, XU Z H, EMU L, et al. Active mechanical haptics with high-fidelity perceptions for immersive virtual reality[J]. Nature Machine Intelligence, 2023, 5(6): 643-655. DOI
[10]	ZENNER A, ULLMANN K, KRÜGER A. Combining dynamic passive haptics and haptic retargeting for enhanced haptic feedback in virtual reality[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(5): 2627-2637. DOI URL
[11]	MIHALIČ F, TRUNTIČ M, HREN A. Hardware-in-the-loop simulations: a historical overview of engineering challenges[J]. Electronics, 2022, 11(15): 2462. DOI URL
[12]	ACEVEDO P, MAGANA A J, WALSH Y, et al. Embodied immersive virtual reality to enhance the conceptual understanding of charged particles: a qualitative study[J]. Computers & Education: X Reality, 2024, 5: 100075.
[13]	MUFIT F, HENDRIYANI Y, DHANIL M. Design immersive virtual reality (IVR) with cognitive conflict to support practical learning of quantum physics[J]. Journal of Turkish Science Education, 2024, 21(2): 369-388.
[14]	STĘPNIKOWSKI A W. Virtual reality learning retention in education and trainings[J]. Edukacja Ustawiczna Dorosłych, 2021, 115(4): 55-61.
[15]	BLACKLER A, DESAI S, MCEWAN M, et al. Perspectives on the nature of intuitive interaction[M]//BLACKLER A. Intuitive Interaction: Research and Application. Boca Raton: CRC Press, 2018: 19-39.
[16]	LEE Y. Exploring mixed interaction mode in virtual reality: controller-based and hand gesture integration[D]. Auckland: Auckland University of Technology, 2024.
[17]	GUO L, LU Z X, YAO L G. Human-machine interaction sensing technology based on hand gesture recognition: a review[J]. IEEE Transactions on Human-Machine Systems, 2021, 51(4): 300-309. DOI URL
[18]	JIANG S, KANG P Q, SONG X Y, et al. Emerging wearable interfaces and algorithms for hand gesture recognition: a survey[J]. IEEE Reviews in Biomedical Engineering, 2022, 15: 85-102. DOI URL
[19]	ZHOU H Y, WANG D Y, YU Y, et al. Research progress of human-computer interaction technology based on gesture recognition[J]. Electronics, 2023, 12(13): 2805. DOI URL
[20]	BORZELLI D, BOARINI V, CASILE A. A quantitative assessment of the hand kinematic features estimated by the oculus Quest 2[J]. Scientific Reports, 2025, 15(1): 8842. DOI
[21]	GIBBS J K, GILLIES M, PAN X N. A comparison of the effects of haptic and visual feedback on presence in virtual reality[J]. International Journal of Human-Computer Studies, 2022, 157: 102717. DOI URL
[22]	ADILKHANOV A, RUBAGOTTI M, KAPPASSOV Z. Haptic devices: wearability-based taxonomy and literature review[J]. IEEE Access, 2022, 10: 91923-91947. DOI URL
[23]	TUNCA E, FLEISCHER R, SCHMIDT L, et al. Advantages of active haptics on touch surfaces[C]// The 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. New York: ACM, 2016: 137-144.
[24]	KOMENO N, MATSUBARA T. Tactile perception based on injected vibration in soft sensor[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 5365-5372. DOI URL
[25]	GUTIÉRREZ-FERNÁNDEZ A, FERNÁNDEZ-LLAMAS C, ESTEBAN G, et al. Haptic zoom: an interaction model for desktop haptic devices with limited workspace[J]. International Journal of Human-Computer Interaction, 2023, 39(4): 851-862. DOI URL
[26]	LI P, ZHANG X X, HU X W, et al. Human-computer interaction on virtual reality-based training system for vascular interventional surgery[J]. Computer Methods and Programs in Biomedicine, 2025, 265: 108731. DOI URL
[27]	ZENNER A, KRÜGER A. Shifty: a weight-shifting dynamic passive haptic proxy to enhance object perception in virtual reality[J]. IEEE Transactions on Visualization and Computer Graphics, 2017, 23(4): 1285-1294. DOI URL
[28]	ZHU K N, CHEN T Z, HAN F, et al. HapTwist: creating interactive haptic proxies in virtual reality using low-cost twistable artefacts[C]// 2019 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2019: 693.
[29]	FEICK M, BATEMAN S, TANG A, et al. Tangi: tangible proxies for embodied object exploration and manipulation in virtual reality[C]// 2020 IEEE International Symposium on Mixed and Augmented Reality. New York: IEEE Press, 2020: 195-206.
[30]	Meta for Work. Hand tracking technology and haptic feedback explained[EB/OL]. (2025-05-30) [2025-07-09]. https://forwork.meta.com/blog/hand-tracking-technology-and-haptic-feedback-mr/.
[31]	HART S G. Nasa-task load index (NASA-TLX); 20 years later[J]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006, 50(9): 904-908. DOI URL
[32]	MOLLET G A. Fundamentals of human neuropsychology[J]. Journal of Undergradute Neuroscience Education, 2008, 6(2): R3-R4.