基于观察质量场的虚拟对象协同操作方法

doi:10.11996/JG.j.2095-302X.2024061338

图学学报 ›› 2024, Vol. 45 ›› Issue (6): 1338-1348.DOI: 10.11996/JG.j.2095-302X.2024061338

• 计算机图形学与虚拟现实 • 上一篇下一篇

基于观察质量场的虚拟对象协同操作方法

栾帅¹(), 吴健¹, 樊润泽¹, 王莉莉¹^,²()

1.北京航空航天大学虚拟现实技术与系统全国重点实验室，北京 100191
2.鹏城实验室，广东深圳 518000

收稿日期:2024-06-05 接受日期:2024-09-01 出版日期:2024-12-31 发布日期:2024-12-24
通讯作者:王莉莉(1977-)，女，教授，博士。主要研究方向为虚拟现实、实时渲染和人机交互等。E-mail：wanglily@buaa.edu.cn
第一作者:栾帅(2000-)，男，硕士研究生。主要研究方向为虚拟现实、增强现实和人机交互。E-mail：luanshuai@buaa.edu.cn
基金资助:
国家自然科学基金(62372026);国家自然科学基金(61932003);北京市科技计划项目(Z221100007722004);国家重点研发计划(2019YFC1521102)

Observation quality field based collaborative object manipulation in VR

LUAN Shuai¹(), WU Jian¹, FAN Runze¹, WANG Lili¹^,²()

1. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China
2. Peng Cheng Laboratory, Shengzhen Guangdong 518000, China

Received:2024-06-05 Accepted:2024-09-01 Published:2024-12-31 Online:2024-12-24
Contact: WANG Lili (1977-), professor, Ph.D. Her main research interests cover virtual reality, real-time rendering and HCI, etc. E-mail：wanglily@buaa.edu.cn
First author：LUAN Shuai (2000-), master student. His main research interests cover virtual reality, augmented reality and HCI. E-mail：luanshuai@buaa.edu.cn
Supported by:
National Natural Science Foundation of China(62372026);National Natural Science Foundation of China(61932003);Beijing Science and Technology Plan Project(Z221100007722004);National Key Research and Development Plan(2019YFC1521102)

摘要/Abstract

摘要：

在虚拟现实(VR)中，对象操作是关键的交互方式。特别是在协作VR应用中，执行高效且准确地操作是非常重要的。然而，传统的协作操作技术未能充分考虑到与操作相关的对象、目标以及环境动态之间的相互作用，并且未提供有效地指导以帮助用户在操作时选择最佳视点。为了解决这一问题，引入了一种基于观察质量场(OQF)的新型协作操作技术，旨在提高操作的准确性和效率。并根据用户的观察质量分数，引导其选择最合适的视角，以实现更加高效和协调的对象操控。首先介绍OQF的概念及其构建方法，并提出2种策略加速OQF更新过程，随后提出了一种利用OQF的指导来操作物体的协同操作方法。通过在3种不同的虚拟环境：客厅、仓库和管道场景中进行的含36名参与者的用户研究，评估了其操作效率和准确性。结果表明，与传统方法相比，OQF技术显著减少了任务完成时间、位置误差、旋转误差和任务负荷。

关键词: 虚拟现实, 协作, 对象操作, 观察质量场, 人机交互

Abstract:

In virtual reality (VR), interacting with objects serves as a crucial form of interaction, especially in collaborative VR applications where efficient and accurate operations are extremely important. However, traditional collaborative operation techniques have failed to adequately consider the interactions among objects, targets, and environmental dynamics, nor have they provided effective guidance to assist users in selecting the best viewpoint during operations. To address this issue, a new collaborative operation technique based on the Observation Quality Field (OQF) was introduced, aimed at enhancing operational accuracy and efficiency. This technique guided users to choose the most appropriate viewpoint based on their observation quality score, facilitating more efficient and coordinated object operation. Initially, the concept and construction method of the Observation Quality Field were introduced, followed by two strategies to accelerate the OQF update process. Subsequently, a collaborative operation method using OQF guidance for object operation was presented. Through a user study involving 36 participants conducted in three different virtual environments—living room, warehouse, and pipeline—the efficiency and accuracy of this technique were evaluated. The results showed that compared to traditional methods, the OQF technique significantly reduced task completion time, positional errors, rotational errors, and overall task load.

Key words: virtual reality, collaboration, object manipulation, observation quality field, human-computer interaction

中图分类号:

TP391.4

栾帅, 吴健, 樊润泽, 王莉莉. 基于观察质量场的虚拟对象协同操作方法[J]. 图学学报, 2024, 45(6): 1338-1348.

LUAN Shuai, WU Jian, FAN Runze, WANG Lili. Observation quality field based collaborative object manipulation in VR[J]. Journal of Graphics, 2024, 45(6): 1338-1348.

图/表 16

图1 基于OQF的协同对象操作方法流程图

Fig. 1 Pipeline of collaborative object operation method based on OQF

图2 OQF初步构建流程图((a)场景俯视图；(b)可行区域WA；(c)场景采样视点俯视图，蓝色球为采样视点)

Fig. 2 Pipeline of the preliminary construction of OQF ((a) The top view of scene; (b) The walkable area WA; (c) The top-down view of the sampling viewpoints in the scene, with blue spheres representing the sampling viewpoints)

图3 场景第三人称视图与特定采样点视图((a)场景的第三视图；(b)显示SV中视点A的视图，其中蓝色球为采样视点)

Fig. 3 Third-person view of the scene and the view from a specific sampling point ((a) A third view of scene; (b) A view of viewpoint A in SV, the blue balls are the sampled viewpoints)

图4 OQF视点优化示意图((a)删除不满足条件的视点(红色球)效果图；(b)删除不满足条件的视点(黑色球)效果图；(c) OQF视点最终效果图)

Fig. 4 Illustration of OQF Viewpoint Optimization ((a) Delete viewpoint that do not meet the conditions (red ball); (b) Delete viewpoint that do not meet the conditions (black ball); (c) The final result of OQF viewpoints)

图5 OQF可视化((a) OQF视点可视化为小球；(b) OQF视点可视化为方块；(c)小地图示意图) 注：箭头代表用户位置，绿色代表目标物体，青色代表被操作物体。

Fig. 5 OQF visualization ((a) Viewpoints in the OQF are visualized as small spheres; (b) Viewpoints in the OQF are visualized as color squares; (c) A mini-map is added to give the global cues of OQF)

图6 视图对比较(第1~3列衡量OQF的T值，第4~5列衡量OQF的RS值)

Fig. 6 Image pair comparison in the pilot user study (Columns 1-3 are for values of the T component of the OQF, columns 4-5 are for values of the RS component of OQF)

图7 2个用户根据OQF的提示，在S1场景协同操作兔子(青色)到目标(绿色)位置((a)和(b)显示地板上T和RS可视化为方块；(c)和(e)显示当用户T站在不同正方形的位置上时的用户视图(绿色圆圈表示传送)；(d)和(f)显示当用户RS站在不同正方形的位置上时的用户视图)

Fig. 7 Two users collaboratively manipulate the bunny (cyan) to the target (green) position according to the cues of the manipulation guidance field ((a) and (b) Show the colors of the squares on the floor visualize the values of the T and RS components of the manipulation guidance field; (c) and (e) Show the user views when user T stands on the locations of the different squares (green circles indicate teleportation); (d) and (f) Show the user views when user RS stands on the locations of the different squares)

图8 S2场景图((a)和(b)显示2名参与者通过OQF的地图操作弥勒佛至各自小球引导的绿色目标位置；(c)和(d)显示从2名参与者的视角看到的视图)

Fig. 8 The second scene S2 of our user study ((a) and (b) Show two participants manipulating Maitreya to a green target position guided by their respective little balls with a map of OQF; (c) and (d) Show views seen from two viewpoints of two participants)

图9 S3场景图((a)和(b)显示2名参与者通过OQF的地图在颜色方块的引导下操作蓝色管道到绿色目标位置；(c)和(d)显示从2名参与者的视角看到的视图)

Fig. 9 The third scene S3 of our user study ((a) and (b) Show two participants manipulating blue pipe to a green target position guided by their respective color squares with map of OQF; (c) and (d) Show views seen from two viewpoints of two participants)

表1 任务完成时间

Table 1 The completion time

任务	条件	时间/s	[(CC_i-EC)/CC_i]/%	p	d	效果
S1	CC	144.25±47.36
	EC₁	84.78±16.82	70.2	<0.001*	1.67	非常大
	EC₂	130.90±31.12	10.2	0.311	0.33	小
	EC₃	72.23±22.16	99.7	<0.001*	1.95	非常大
	EC₄	88.70±22.16	62.6	<0.001*	1.48	非常大
	EC₅	74.05±17.54	94.8	<0.001*	1.97	非常大
S2	CC	114.90±32.51
	EC₁	95.30±24.49	20.6	0.042*	0.68	中等
	EC₂	102.90±23.77	11.7	0.202	0.42	小
	EC₃	84.75±24.82	35.6	0.002*	1.04	大
	EC₄	89.30±36.13	28.7	0.027*	0.74	中等
	EC₅	74.85±27.47	53.5	<0.001*	1.33	非常大
S3	CC	135.75±50.75
	EC₁	106.65±29.51	27.3	0.037*	0.70	中等
	EC₂	154.70±42.13	-12.2	0.218	0.41	小
	EC₃	93.70±23.30	44.9	0.002*	1.06	大
	EC₄	98.15±28.36	38.3	0.007*	0.91	大
	EC₅	86.50±22.66	56.9	0.007*	1.25	非常大

表2 各任务条件位置误差

Table 2 The position error

任务	条件	位置误差/mm	[(CC_i-EC)/CC_i]/%	p	d	效果
S1	CC	5.4±2.0
	EC1	3.0±1.4	80.4	0.001 6*	1.36	非常大
	EC2	4.5±1.9	19.7	0.167 8	0.46	小
	EC3	3.7±1.2	47.2	0.002 7*	1.04	大
	EC4	3.2±0.7	67.5	<0.001 0*	1.45	非常大
	EC5	3.1±0.7	75.9	0.001 6*	1.36	非常大
S2	CC	3.8±2.2
	EC1	2.2±0.9	74.6	0.003 5*	0.98	大
	EC2	3.8±2.7	1.3	0.952 0	0.02	非常小
	EC3	2.5±1.4	52.6	0.031 0*	0.73	中等
	EC4	2.5±1.3	49.6	0.036 7*	0.70	中等
	EC5	2.1±1.1	82.9	0.003 6*	1.01	大
S3	CC	3.6±1.2
	EC1	2.9±1.5	22.8	0.141 0	0.49	小
	EC2	3.2±1.7	11.3	0.460 0	0.24	小
	EC3	2.2±1.0	62.6	0.000 6*	1.21	非常大
	EC4	3.1±1.5	26.9	0.0714	0.60	中等
	EC5	2.3±1.0	57.9	0.0007*	1.19	大

表3 各任务条件旋转误差

Table 3 The rotation error

任务	条件	旋转误差/mm	[(CC_i-EC)/CC_i]/%	p	d	效果
S1	CC	5.52±3.48
	EC1	3.62±0.67	52.6	0.024 6*	0.76	大
	EC2	4.19±0.97	31.9	0.115 0	0.52	中等
	EC3	3.22±1.42	71.4	0.011 1*	0.87	大
	EC4	3.29±1.11	68.0	0.011 2*	0.86	大
	EC5	3.15±1.34	75.3	0.008 0*	0.90	大
S2	CC	6.12±2.56
	EC1	3.79±0.75	61.6	0.000 4*	1.24	非常大
	EC2	3.84±0.90	59.2	0.070 0	0.39	小
	EC3	3.18±1.11	92.3	<0.000 1*	1.49	非常大
	EC4	3.69±1.48	66.0	0.000 9*	1.16	大
	EC5	3.02±1.25	102.8	<0.000 1*	1.54	非常大
S3	CC	4.20±1.92
	EC1	2.50±0.97	68.4	0.001 3*	1.12	大
	EC2	3.26±0.98	29.4	0.062 0	0.62	中等
	EC3	2.32±1.14	80.8	0.000 7*	1.19	大
	EC4	2.54±1.39	80.0	0.003 9*	0.86	大
	EC5	2.12±1.30	97.9	0.000 3*	1.29	非常大

表4 各任务条件尺度误差

Table 4 The scale error

任务	条件	尺度误差/倍	[(CC_i-EC)/CC_i]/%	p	d	效果
S1	CC	0.020±0.010
	EC1	0.014±0.013	49.9	0.275	0.73	中等
	EC2	0.020±0.013	0.5	0.982	0.01	非常小
	EC3	0.013±0.012	54.9	0.138	0.63	中等
	EC4	0.013±0.007	54.9	0.090	0.63	中等
	EC5	0.011±0.007	70.4	0.090	0.74	中等
S2	CC	0.033±0.018
	EC1	0.030±0.018	8.9	0.654	0.15	非常小
	EC2	0.029±0.010	13.8	0.410	0.27	非常小
	EC3	0.032±0.022	1.5	0.948	0.02	非常小
	EC4	0.025±0.006	28.7	0.104	0.63	中等
	EC5	0.030±0.022	7.2	0.743	0.11	非常小
S3	CC	0.012±0.007
	EC1	0.011±0.007	2.9	0.886	0.10	非常小
	EC2	0.012±0.007	0.1	0.956	0.01	非常小
	EC3	0.010±0.005	18.6	0.441	0.29	小
	EC4	0.011±0.004	2.9	0.805	0.05	非常小
	EC5	0.011±0.010	7.0	0.769	0.10	非常小

图10 不同条件的平均传送次数注：T是用户T的传送次数；RS是用户RS的传送次数。误差条表示标准偏差。

Fig. 10 Mean teleportation

图11 所有场景中6种条件的任务负荷得分的箱型图注：星号表示统计显著性。

Fig. 11 Box plots for task load scores of the six conditions in the all scenes

图12 每个条件的排名分数

Fig. 12 Rank score for each condition

参考文献 23

[1]	BOWMAN D A, KRUIJFF E, LAVIOLA J J, et al. 3D user interfaces: theory and practice[M]. Boston: Addison-Wesley, 2005: 117-118.
[2]	BOWMAN D A, HODGES L F. An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments[C]// 1997 Symposium on Interactive 3D Graphics. New York: ACM, 1997: 35-38.
[3]	GLOUMEAU P C, STUERZLINGER W, HAN J. PinNPivot: object manipulation using pins in immersive virtual environments[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 27(4): 2488-2494.
[4]	NGUYEN T T H, DUVAL T, PONTONNIER C. A new direct manipulation technique for immersive 3D virtual environments[C]// The 24th International Conference on Artificial Reality and Telexistence and the 19th Eurographics Symposium on Virtual Environments. Goslar: Eurographics Association, 2014: 67-74.
[5]	WANG R, PARIS S, POPOVIĆ J. 6D hands: markerless hand-tracking for computer aided design[C]// The 24th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2011: 549-558.
[6]	GRUDIN J, RUDDLE R A, SAVAGE J C D, et al. Symmetric and asymmetric action integration during cooperative object manipulation in virtual environments[J]. ACM Transactions on Computer-Human Interaction, 2002, 9(4): 285-308.
[7]	GRANDI J G, DEBARBA H G, MACIEL A. Characterizing asymmetric collaborative interactions in virtual and augmented realities[C]// 2019 IEEE Conference on Virtual Reality and 3D User Interfaces. New York: IEEE Press, 2019: 127-135.
[8]	DUVAL T, LECUYER A, THOMAS S. SkeweR: a 3D interaction technique for 2-user collaborative manipulation of objects in virtual environments[C]// 3D User Interfaces. New York: IEEE Press, 2006: 69-72.
[9]	BARON N. CollaborativeConstraint: UI for collaborative 3D manipulation operations[C]// 2016 IEEE Symposium on 3D User Interfaces. New York: IEEE Press, 2016: 273-274.
[10]	WANG L L, LIU X L, LI X Y. VR collaborative object manipulation based on viewpoint quality[C]// 2021 IEEE International Symposium on Mixed and Augmented Reality. New York: IEEE Press, 2021: 60-68.
[11]	LE CHENECHAL M, LACOCHE J, ROYAN J, et al. When the giant meets the ant an asymmetric approach for collaborative and concurrent object manipulation in a multi-scale environment[C]// 2016 IEEE Third VR International Workshop on Collaborative Virtual Environments. New York: IEEE Press, 2016: 18-22.
[12]	SOARES L P, KOPPER R, PINHO M S. EGO-EXO: a cooperative manipulation technique with automatic viewpoint control[C]// 2018 20th Symposium on Virtual and Augmented Reality. New York: IEEE Press, 2018: 82-88.
[13]	LAGES W. Ray, camera, action! A technique for collaborative 3D manipulation[C]// 2016 IEEE Symposium on 3D User Interfaces. New York: IEEE Press, 2016: 281-282.
[14]	BACHMANN E R, HODGSON E, HOFFBAUER C, et al. Multi-user redirected walking and resetting using artificial potential fields[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(5): 2022-2031.
[15]	DONG T Y, CHEN X W, SONG Y F, et al. Dynamic artificial potential fields for multi-user redirected walking[C]// 2020 IEEE Conference on Virtual Reality and 3D User Interfaces. New York: IEEE Press, 2020: 146-154.
[16]	BOZGEYIKLI E, RAIJ A, KATKOORI S, et al. Point & teleport locomotion technique for virtual reality[C]// 2016 Annual Symposium on Computer-Human Interaction in Play. New York: ACM, 2016: 205-216.
[17]	FREES S, KESSLER G D, KAY E. PRISM interaction for enhancing control in immersive virtual environments[J]. ACM Transactions on Computer-Human Interaction, 2007, 14(1): 2:1-2:31.
[18]	ZAGATA K, GULIJ J, HALIK Ł, et al. Mini-map for gamers who walk and teleport in a virtual stronghold[J]. ISPRS International Journal of Geo-Information, 2021, 10(2): 96:1-96:17.
[19]	HART S G. NASA-task load index (NASA-TLX); 20 years later[J]. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006, 50(9): 904-908.
[20]	HART S G, STAVELAND L E. Development of NASA-TLX (Task Load Index): results of empirical and theoretical research[J]. Advances in Psychology, 1988, 52: 139-183.
[21]	SHAPIRO S S, WILK M B. An analysis of variance test for normality (complete samples)[J]. Biometrika, 1965, 52(3/4): 591-611.
[22]	MAUCHLY J W. Significance test for sphericity of a normal n-variate distribution[J]. The Annals of Mathematical Statistics, 1940, 11(2): 204-209.
[23]	LIU X S. Statistical power analysis for the social and behavioral sciences[M]. New York: Routledge, 2013.

基于观察质量场的虚拟对象协同操作方法

Observation quality field based collaborative object manipulation in VR

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 16

参考文献 23

相关文章 15

编辑推荐

Metrics

本文评价

[1]	杨浩中, 孔笑宇, 辜睿坤, 汪淼. 虚拟现实中的大模型技术研究进展与趋势[J]. 图学学报, 2024, 45(6): 1117-1131.
[2]	任洋甫, 于歌, 傅月瑶, 胥森哲, 何煜, 王巨宏, 张松海. 虚拟现实中场景和时间对用户空间方向认知的影响[J]. 图学学报, 2024, 45(6): 1349-1363.
[3]	厉向东, 夏涵飞, 单逸飞, 阴凯琳, 耿卫东. 图形用户界面自动生成技术的现状与挑战[J]. 图学学报, 2024, 45(3): 409-421.
[4]	严家豪, 吕健, 侯宇康, 莫心祝. 虚拟现实中眼动交互频率对视觉疲劳影响的研究[J]. 图学学报, 2024, 45(3): 528-538.
[5]	王浩淼, 桑胜举, 段晓东, 张伟华, 陶体伟, 马婷. 虚拟现实环境下的协同式三维建模方法[J]. 图学学报, 2024, 45(1): 169-182.
[6]	韩兆阳, 翁冬冬, 郭署山, 贺文杰, 江海燕, 李冬. 一种基于简易标记点编码的光学跟踪系统[J]. 图学学报, 2023, 44(5): 997-1012.
[7]	谢红霞, 胡毓宁, 张赟, 王亚奇, 杜辉, 秦爱红. 全景图像视频的场景分析与内容处理方法综述[J]. 图学学报, 2023, 44(4): 640-657.
[8]	高越, 韩红雷. 基于可发光设备的观众席画面展示系统[J]. 图学学报, 2023, 44(4): 784-793.
[9]	段艳花, 刘子建, 宁铎. 基于眼动技术的 TMA 界面评估及优化[J]. 图学学报, 2022, 43(4): 745-752.
[10]	马宁, 王亚辉. 智能汽车座舱人机交互任务复杂度分析方法[J]. 图学学报, 2022, 43(2): 356-360.
[11]	王秋惠, 王雅馨. 医院消杀机器人作业安全与交互设计策略[J]. 图学学报, 2022, 43(1): 172-180.
[12]	朱永宁 , 葛婷 , 杜盛瑀 , 楼泽如 , 王建民 . 虚拟现实全景流体绘画系统的可用性研究[J]. 图学学报, 2021, 42(5): 833-840.
[13]	黄进, 张浩, 田丰. 面向动态交互场景的计算模型[J]. 图学学报, 2021, 42(3): 359-366.
[14]	柳有权 , 王愿超 , 徐琨 , 刘正雄 , 黄攀峰 . 基于混合现实的远程协同式装配维修引导[J]. 图学学报, 2021, 42(2): 216-221.
[15]	赵建军, 黄竣鹏, 陈俊良. 基于 Leap Motion 的电影前期预演人机交互方法[J]. 图学学报, 2021, 42(1): 71-78.