Collaborative 3D modeling technique in virtual reality

doi:10.11996/JG.j.2095-302X.2024010169

Abstract

Abstract:

3D modeling technology plays a crucial role in various fields, but the predominantly desktop-based interaction in 3D modeling remains complex, abstract, and lacks support for online collaboration. Therefore, utilizing the advantages of immersion, interaction, and imagination of virtual reality (VR) technology, a network collaborative 3D modeling method in a VR environment was proposed. This method enabled users to create 3D models through immersive interaction and supported multi-user real-time online visual collaboration. Firstly, a 3D model drawing interaction method in a VR environment was proposed. Secondly, the 3D models were categorized, and a 3D model mesh generation algorithm based on Layered Build was proposed for building planar models and 3D models. Finally, a 3D modeling network collaboration module in a VR environment was designed, achieving network synchronization through Socket communication. Comparative experiments with the 3D modeling methods of traditional 3D modeling software demonstrated that this method is more concise, intuitive, and efficient, making it easy for ordinary users to master.

Key words: virtual reality, 3D modeling, mesh generation, network collaboration, immersive interaction

CLC Number:

TP391

WANG Haomiao, SANG Shengju, DUAN Xiaodong, ZHANG Weihua, TAO Tiwei, MA Ting. Collaborative 3D modeling technique in virtual reality[J]. Journal of Graphics, 2024, 45(1): 169-182.

Figures/Tables 24

Fig. 1 Functional framework

Fig. 2 Button interaction design ((a) Left handle button interaction; (b) Right handle button interaction)

Fig. 3 Polygonal mesh data description

Fig. 4 Normal direction calculation ((a) Right-handed coordinate system; (b) Left-handed coordinate system)

Fig. 5 Mesh construction process ((a) Triangle mesh construction; (b) Quad mesh construction)

Fig. 6 Layered build

Fig. 7 Column bottom primitive build ((a) Circumferential coordinates calculation; (b) Mesh build)

Fig. 8 Single layer cube mesh flat plane ((a) Cube mesh; (b) Flat plane)

Fig. 9 Straight line mode principle ((a) Plane model line mode; (b) Stereoscopic model line mode)

Fig. 10 Axial control

Fig. 11 Network collaboration

Fig. 12 Network synchronization framework

Fig. 13 Server and client synchronization

Fig. 14 Control point registration and automatic synchronization

Table 1 OBJ keywords

分类	关键字	描述
顶点数据	v	几何体顶点三维坐标
	vt	顶点对应贴图坐标
	vn	顶点法线
元素	p	点
	l	线
	f	面
成组	g	组名
渲染	usemtl	材质名
渲染	mtllib	材质库

Fig. 15 Bone binding and color picker in virtual scene ((a) Virtual avatar bone binding; (b) Color picker)

Fig. 16 3D modeling and editing ((a) Plane model editing; (b) Stereoscopic model)

Fig. 17 Basic model prefab

Fig. 18 Collaborative work in VR ((a) First perspective; (b) Third perspective)

Fig. 19 Display of test works ((a) Test reference diagram; (b) 3D modeling effects; (c) Completion time of the work)

Fig. 20 3D model export ((a) 3D viewer; (b) Maya)

Fig. 21 User experience analysis ((a) Learning difficulty value; (b) Interactive intuitiveness; (c) Operation complexity; (d) DoF of Modeling)

Table 2 Comparison of 3D modeling methods

方法对比	人机交互	建模方法	工作效率	缺陷与不足
本文方法	沉浸式交互，建模过程直观，易于非专业人员学习掌握	跟随用柄轨迹生成模型，通过模型组合表达创意更轻松	直观便捷的交互过程和网络协同的支持，能够提高工作效率	网格编辑、UV与渲染机制不完善，VR设备长时间佩戴易劳累
传统方法	桌面式交互，建模过程抽象，专业性强	具备完善的网格编辑、UV、渲染机制，建模自由度高	交互过程复杂抽象提高了学习成本，工作效率相对不高	建模过程复杂、学习难度大，不支持远程协作

Table 3 System load testing

资源项	CPU 利用率/ %	GPU 利用率/ %	内存占用率/ %	WIFI 吞吐量/ Kbps
系统未运行	3	1	16	0
运行空闲时	20	85	27	10
建模工作时	40	86	28	100

References 33

[1]	SHERIDAN T B. Interaction, imagination and immersion some research needs[C]// The ACM Symposium on Virtual Reality Software and Technology. New York: ACM, 2000: 1-7.
[2]	FUNKHOUSER T, KAZHDAN M, SHILANE P, et al. Modeling by example[J]. ACM Transactions on Graphics, 2004, 23(3): 652-663. DOI URL
[3]	LEE J, FUNKHOUSER T. Sketch-based search and composition of 3D models[C]// The 5th Eurographics Conference on Sketch-Based Interfaces and Modeling. New York: ACM, 2008: 97-104.
[4]	IGARASHI T, HUGHES J F. A suggestive interface for 3D drawing[C]// The 14th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2001: 173-181.
[5]	孙正兴, 冯桂焕, 周若鸿. 基于草图的人机交互技术研究进展[J]. 计算机辅助设计与图形学学报, 2005, 17(9): 1889-1899.
	SUN Z X, FENG G H, ZHOU R H. Techniques for sketch-based user interface: review and research[J]. Journal of Computer Aided Design & Computer Graphics, 2005, 17(9): 1889-1899 (in Chinese).
[6]	胡事民, 杨永亮, 来煜坤. 数字几何处理研究进展[J]. 计算机学报, 2009, 32(8): 1451-1469.
	HU S M, YANG Y L, LAI Y K. Research progress of digital geometry processing[J]. Chinese Journal of Computers, 2009, 32(8): 1451-1469 (in Chinese).
[7]	WANG Y Z, ZHENG J W, WANG H. Fast mesh simplification method for three-dimensional geometric models with feature-preserving efficiency[J]. Scientific Programming, 2019, 2019: 4926190.
[8]	LIU X P, LIN L P, WU J, et al. Generating sparse self-supporting wireframe models for 3D printing using mesh simplification[J]. Graphical Models, 2018, 98: 14-23. DOI URL
[9]	王永志, 李振朝, 刘鹏彧, 等. 基于能量算子的三维空间实体LOD建模方法[J]. 系统仿真学报, 2022, 34(2): 247-257. DOI
	WANG Y Z, LI Z C, LIU P Y, et al. LOD modeling method for three-dimensional objects with energy operator[J]. Journal of System Simulation, 2022, 34(2): 247-257 (in Chinese). DOI
[10]	EBNER T, SCHREER O, FELDMANN I. Fully automated highly accurate 3D reconstruction from multiple views[C]// 2017 IEEE International Conference on Image Processing. New York: IEEE Press, 2018: 2528-2532.
[11]	SHARP N, CRANE K. A Laplacian for nonmanifold triangle meshes[J]. Computer Graphics Forum, 2020, 39(5): 69-80.
[12]	FOTI D, GIORNO S, DURAISAMY K. An adaptive mesh refinement approach based on optimal sparse sensing[J]. Theoretical and Computational Fluid Dynamics, 2020, 34(4): 457-482. DOI
[13]	DOSOVITSKIY A, SPRINGENBERG J T, TATARCHENKO M, et al. Learning to generate chairs, tables and cars with convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 692-705. DOI PMID
[14]	WU Z J, WANG X, LIN D, et al. SAGNet: structure-aware generative network for 3D-shape modeling[J]. ACM Transactions on Graphics, 2019, 38(4): 91:1-91:14.
[15]	GADELHA M, RAI A, MAJI S, et al. Inferring 3D shapes from image collections using adversarial networks[J]. International Journal of Computer Vision, 2020, 128(10-11): 2651-2664. DOI
[16]	ROSALES E, ARAÚJO C, RODRIGUEZ J, et al. AdaptiBrush: adaptive general and predictable VR ribbon brush[J]. ACM Transactions on Graphics, 2021, 40(6): 247:1-247:15.
[17]	WANG J Y, WANG Y Y, ZHANG N, et al. The key to the future development of interactive art - virtual reality technology[J]. J Multim Inf Syst, 2018, 5: 277-282.
[18]	FEEMAN S M, WRIGHT L B, SALMON J L. Exploration and evaluation of CAD modeling in virtual reality[J]. Computer-Aided Design and Applications, 2018, 15(6): 892-904. DOI URL
[19]	BECKER M W, SCHUETZ J W. An introduction to ground-water modeling using virtual reality modeling language (VRML)[J]. Journal of Geoscience Education, 2003, 51(5): 506-511. DOI URL
[20]	SCHNACK A, WRIGHT M J, HOLDERSHAW J L. Does the locomotion technique matter in an immersive virtual store environment? - Comparing motion-tracked walking and instant teleportation[J]. Journal of Retailing and Consumer Services, 2021, 58: 102266. DOI URL
[21]	BOVIM L P, GJESDAL B E, MÆLAND S, et al. The impact of motor task and environmental constraints on gait patterns during treadmill walking in a fully immersive virtual environment[J]. Gait & Posture, 2020, 77: 243-249. DOI URL
[22]	TIAN J, ZHANG W L, ZHANG T, et al. Research status of gesture recognition based on vision: a review[J]. IOP Conference Series: Earth and Environmental Science, 2021, 632(4): 042019. DOI
[23]	SAKAMOTO M, SHINODA T, ISHIZU T, et al. A proposal of interactive projection mapping using kinect[C]// 2018 International Conference on Information and Communication Technology Robotics. New York: IEEE Press, 2018: 1-4.
[24]	张维, 林泽一, 程坚, 等. 动态手势理解与交互综述[J]. 软件学报, 2021, 32(10): 3051-3067.
	ZHANG W, LIN Z Y, CHENG J, et al. Survey of dynamic hand gesture understanding and interaction[J]. Journal of Software, 2021, 32(10): 3051-3067 (in Chinese).
[25]	王珊, 沈旭昆, 赵沁平. 三维人脸表情获取及重建技术综述[J]. 系统仿真学报, 2018, 30(7): 2423-2444. DOI
	WANG S, SHEN X K, ZHAO Q P. Review of 3D facial expression acquisition and modeling technology[J]. Journal of System Simulation, 2018, 30(7): 2423-2444 (in Chinese). DOI
[26]	胡春花, 陈晓梅, 陈仕鸿. 虚拟现实技术在儿童室内火灾逃生教育中的应用研究[J]. 系统仿真学报, 2016, 28(4): 934-939.
	HU C H, CHEN X M, CHEN S H. Application research on children’s indoor fire escape education system based on virtual reality technology[J]. Journal of System Simulation, 2016, 28(4): 934-939 (in Chinese).
[27]	谭力恒, 蒋秉川, 李锋, 等. 面向沉浸式虚拟现实的数字地球交互漫游方法[J/OL]. 计算机辅助设计与图形学学报, 2023: 1-15. (2023-04-23). https://kns.cnki.net/kcms/detail/11.2925.TP.20230423.1604.004.html.
	TAN L H, JIANG B C, LI F, et al. Immersive virtual reality oriented interactive exploration of digital earth[J/OL]. Journal of Computer-Aided Design & Computer Graphics, 2023: 1-15. (2023-04-23). https://kns.cnki.net/kcms/detail/11.2925.TP.202 30423.1604.004.html (in Chinese).
[28]	ZHAO W R, SU L P, DOU F J. Designing virtual reality based 3D modeling and interaction technologies for museums[J]. Heliyon, 2023, 9(6): e16486. DOI URL
[29]	徐森. 基于手势交互的复杂装配系统三维建模软件设计研究[D]. 南京: 南京邮电大学, 2022.
	XU S. Research on 3D modeling software design of complex assembly system based on gesture interaction[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2022 (in Chinese).
[30]	刘喜明, 郑国勤, 孙家广. 基于C/S模式的同步协同设计运行机制和策略[J]. 计算机工程与应用, 2001, 37(15): 64-67.
	LIU X M, ZHENG G Q, SUN J G. Mechanism and strategy of synchronous cooperative design based on client/server mode[J]. Computer Engineering and Applications, 2001, 37(15): 64-67 (in Chinese).
[31]	NAM T J, WRIGHT D. The development and evaluation of Syco3D: a real-time collaborative 3D CAD system[J]. Design Studies, 2001, 22(6): 557-582. DOI URL
[32]	先志宏, 陈小安, 林建德, 等. 基于特征模型的网络实时协同建模技术[J]. 重庆大学学报: 自然科学版, 2003, 26(12): 25-28.
	XIAN Z H, CHEN X A, LIN J D, et al. A realtime collaborative modeling technology based on feature model over Internet[J]. Journal of Chongqing University: Natural Science Edition, 2003, 26(12): 25-28 (in Chinese).
[33]	周余斌, 罗天洪. 网络协同设计下的三维建模技术[J]. 重庆大学学报, 2008, 31(9): 1038-1043.
	ZHOU Y B, LUO T H. 3D modeling based on Internet collaborative design[J]. Journal of Chongqing University, 2008, 31(9): 1038-1043 (in Chinese).