几何特征引导的物体点云模型多层级分割

doi:10.11996/JG.j.2095-302X.2023040755

图学学报 ›› 2023, Vol. 44 ›› Issue (4): 755-763.DOI: 10.11996/JG.j.2095-302X.2023040755

• 计算机图形学与虚拟现实 • 上一篇下一篇

几何特征引导的物体点云模型多层级分割

刘妍(), 熊游依, 韩妙妙, 杨龙()

西北农林科技大学信息工程学院，陕西杨凌 712100

收稿日期:2022-12-05 接受日期:2023-03-01 出版日期:2023-08-31 发布日期:2023-08-16
通讯作者: 杨龙(1982-)，男，副教授，博士。主要研究方向为计算机图形学。E-mail：yl@nwafu.edu.cn
作者简介:
刘妍(1997-)，女，硕士研究生。主要研究方向为计算机图形学。E-mail：yanl@nwafu.edu.cn
基金资助:
国家自然科学基金项目(61702422);陕西省自然科学基础研究计划项目(2022JM-378)

Geometric feature guided multi-level segmentation for object point clouds

LIU Yan(), XIONG You-yi, HAN Miao-miao, YANG Long()

College of Information Engineering, Northwest A&F University, Yangling Shaanxi 712100, China

Received:2022-12-05 Accepted:2023-03-01 Online:2023-08-31 Published:2023-08-16
Contact: YANG Long (1982-), associate professor, Ph.D. His main research interest covers computer graphics. E-mail：yl@nwafu.edu.cn
About author:
LIU Yan (1997-), master student. Her main research interest covers computer graphics. E-mail：yanl@nwafu.edu.cn
Supported by:
National Natural Science Foundation of China(61702422);Natural Science Basic Research Program of Shaanxi(2022JM-378)

摘要/Abstract

摘要：

扫描物体点云的大量出现使得点云表面形状分析成为计算机图形学领域的研究热点。结构提取、形状编辑以及人-物交互等许多任务都需要尽可能精细的点云部件分割。由于浅层几何特征不易提取、形状缺失和噪声干扰等问题，扫描点云模型的部件分割(尤其细小部件实例检测)相对困难。因此，提出一种几何特征引导的物体点云模型多层级部件实例分割方法，根据点云表面局部弯曲程度提取凹、凸和边界特征，首先沿全局最显著凹线分割模型大致结构，再筛选几何特征差异明显的片段根据局部较浅特征进行细分，最后针对部分片段进行凹-凸协同分割，得到多层级分割结果。沿着由深到浅的几何特征，进行由粗到细的三次分割，能够更好地兼顾尺度不一的部件，实现了更细粒度的部件实例分割。实验结果表明，该方法在真实物体扫描点云和CAD物体模型采样点云上达到了较理想的分割效果，为真实物体点云部件分割提供了更精细且有效的方法。

关键词: 点云模型, 三维表面分割, 特征检测, 凹凸性, 区域生长

Abstract:

The proliferation of scanning object point clouds has brought surface shape analysis to the forefront of research in the computer graphics community. Tasks such as structure extraction, shape editing, and human-object interaction necessitate as precise a point cloud part segmentation as possible. However, due to the complexities of shallow geometric feature extraction, shape loss, and noise interference, the part segmentation, especially the small instance part extraction for scanning point clouds, remains relatively difficult. To address this issue, a multi-level part instance segmentation method guided by geometric features was proposed for object point clouds. The concave, convex, and boundary features were extracted based on the local bending extent of the point cloud surface. Firstly, our method segmented the general structure along the most global prominent concave line. Then it subdivided those segments with obvious geometric differences according to the local shallow features. Finally, the method performed concave-convex collaborative segmentation for some segments to obtain the multi-level segmentation results. The three-stage segmentation process from coarse to fine, along the geometric features from deep to shallow, allowed for better consideration of parts with different scales and finer grained part instance segmentation. The experimental results demonstrated that the proposed method could achieve superior segmentation results on both scanning point clouds and sampled point clouds from CAD models. It provided a more precise and effective method for part segmentation of scanning object point clouds.

Key words: point cloud model, 3D surface segmentation, feature detection, convexity-concavity, region growing

中图分类号:

TP391

刘妍, 熊游依, 韩妙妙, 杨龙. 几何特征引导的物体点云模型多层级分割[J]. 图学学报, 2023, 44(4): 755-763.

LIU Yan, XIONG You-yi, HAN Miao-miao, YANG Long. Geometric feature guided multi-level segmentation for object point clouds[J]. Journal of Graphics, 2023, 44(4): 755-763.

图/表 9

表1 点云模型分割任务研究现状

Table 1 Research status of point cloud segmentation

分割程度	室内场景	室内物体(部件)
分割程度	扫描模型	CAD模型	扫描模型
语义级分割	√	√	√
实例级分割	√	√	×

图1 几何特征引导的点云模型多层级分割整体流程图

Fig. 1 The holistic flow chart of geometric feature-guided multi-level segmentation

图2 分割点与特征点示例

Fig. 2 Example of break points and feature points

图3 飞机模型几何特征((a)全局特征；(b)机身局部特征)

Fig. 3 Geometric features of airplane ((a) Global features of airplane; (b) Local features of fuselage)

图4 本文方法在椅子和马桶模型上分割结果((a)，(e)原始模型；(b)，(f)初分割；(c)，(g)局部凹分割；(d)，(h)凹-凸协同分割)

Fig. 4 The segment results of chair model and toilet model by our method ((a), (e) Original model; (b), (f) The initial segmentation result; (c), (g) The local concave segmentation result; (d), (h) The concave-convex collaborative segmentation result)

图5 不同方法对不同室内扫描物体点云的部件分割结果((a) LCCP[27]；(b) CPC[28]；(c) PointNet++[35]；(d)本文；(e) Ground truth) (各部件着以随机颜色)

Fig. 5 Indoor scanning object point cloud part segmentation results of different methods ((a) LCCP[27]; (b) CPC[28]; (c) PointNet++[35]; (d) Ours; (e) Ground truth) (Random colors for each segment)

图6 室内扫描物体点云模型不同方法部件分割片段OBB ((a) LCCP[27]；(b) CPC[28]；(c) PointNet++[35]；(d)本文；(e) 参照片段OBB；(f)参照模型)

Fig. 6 The OBB of every segment of different methods for indoor scanning object point cloud part segmentation ((a) LCCP[27]; (b) CPC[28]; (c) PointNet++[35]; (d) Ours; (e) OBB of ground truth; (f) Model of ground truth)

图7 本文方法在CAD模型采样点云上的分割结果((a)立式钢琴；(b)笔记本电脑；(c)三角钢琴；(d)男人；(e)女人)

Fig. 7 Segmentation results of CAD sampling point cloud by our method ((a) Upright piano; (b) Laptop; (c) Grand piano; (d) Man; (e) Woman)

表2 4种方法在不同扫描物体点云模型上的部件分割量化(recall@boundary指标)对比

Table 2 Quantitative comparison (recall@boundary indicator) of four methods for part segmentation on different scanning object point clouds

方法	Average	Chair1	Chair2	Chair3	Chair4	Chair5	Sofa1	Sofa2	Sofa3	Sofa4	Sofa5	Toilet1	Toilet2	Table
LCCP^[27]	0.509	0.650	0.663	0.713	0.638	0.630	0.495	0.509	0.392	0.376	0.447	0.296	0.370	0.443
CPC^[28]	0.398	0.456	0.521	0.331	0.558	0.561	0.888	0.149	0.377	0.358	0.244	0.434	0.082	0.212
PointNet++^[35]	0.531	0.260	0.381	0.874	0.561	1.000	0.795	0.437	0.360	0.407	0.645	0.014	0.542	0.626
Ours	0.590	0.559	0.707	0.604	0.639	0.782	0.701	0.600	0.561	0.447	0.805	0.443	0.413	0.412

参考文献 41

[1]	HU R Z, VAN KAICK O, WU B J, et al. Learning how objects function via co-analysis of interactions[J]. ACM Transactions on Graphics, 2016, 35(4): 1-13.
[2]	HU R Z, YAN Z H, ZHANG J W, et al. Predictive and generative neural networks for object functionality[EB/OL]. [2022-06-28]. https://arxiv.org/abs/2006.15520.
[3]	KIM V G, CHAUDHURI S, GUIBAS L, et al. Shape2Pose: human-centric shape analysis[J]. ACM Transactions on Graphics, 2014, 33(4): 1-12.
[4]	JAIN A, THORMÄHLEN T, RITSCHEL T, et al. Exploring shape variations by 3D-model decomposition and part-based recombination[J]. Computer Graphics Forum, 2012, 31(2pt3): 631-640. DOI URL
[5]	OVSJANIKOV M, LI W, GUIBAS L, et al. Exploration of continuous variability in collections of 3D shapes[J]. ACM Transactions on Graphics, 2011, 30(4): 1-10.
[6]	MO K C, ZHU S L, CHANG A X, et al. PartNet: a large-scale benchmark for fine-grained and hierarchical part-level 3D object understanding[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 909-918.
[7]	ZHOU Q Y, PARK J, KOLTUN V. Open3D: a modern library for 3D data processing[EB/OL]. [2022-01-30]. https://arxiv.org/abs/1801.09847.
[8]	DAI A, CHANG A X, SAVVA M, et al. ScanNet: richly-annotated 3D reconstructions of indoor scenes[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 2432-2443.
[9]	YANG L, YAN Q A, FU Y P, et al. Surface reconstruction via fusing sparse-sequence of depth images[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(2): 1190-1203. DOI URL
[10]	LIAO Y B, ZHU H Y, ZHANG Y G, et al. Point cloud instance segmentation with semi-supervised bounding-box mining[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 10159-10170. DOI URL
[11]	HUANG Q G, WANG W Y, NEUMANN U. Recurrent slice networks for 3D segmentation of point clouds[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 2626-2635.
[12]	YE X Q, LI J M, HUANG H X, et al. 3D recurrent neural networks with context fusion for point cloud semantic segmentation[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 415-430.
[13]	CHANG A X, FUNKHOUSER T, GUIBAS L, et al. ShapeNet: an information-rich 3D model repository[EB/OL]. [2022-06-28]. https://arxiv.org/abs/1512.03012.
[14]	WU Z R, SONG S R, KHOSLA A, et al. 3D ShapeNets: a deep representation for volumetric shapes[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 1912-1920.
[15]	XU M T, DING R Y, ZHAO H S, et al. PAConv: position adaptive convolution with dynamic kernel assembling on point clouds[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 3172-3181.
[16]	XU Y F, FAN T Q, XU M Y, et al. SpiderCNN: deep learning on point sets with parameterized convolutional filters[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 90-105.
[17]	WANG Y, SUN Y B, LIU Z W, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[18]	ZHANG K E, HAO M, WANG J, et al. Linked dynamic graph CNN: learning on point cloud via linking hierarchical features[EB/OL]. [2022-06-10]. https://arxiv.org/abs/1904.10014.
[19]	YU F G, LIU K, ZHANG Y, et al. PartNet: a recursive part decomposition network for fine-grained and hierarchical shape segmentation[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9483-9492.
[20]	UY M A, PHAM Q H, HUA B S, et al. Revisiting point cloud classification: a new benchmark dataset and classification model on real-world data[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 1588-1597.
[21]	RUSU R B, COUSINS S. 3D is here: point cloud library (PCL)[C]// 2011 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2011: 1-4.
[22]	BHANU B, LEE S, HO C C, et al. Range data processing: representation of surfaces by edges[EB/OL]. [2022-06-28]. https://collections.lib.utah.edu/details?id=706994.
[23]	BESL P J, JAIN R C. Segmentation through variable-order surface fitting[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(2): 167-192. DOI URL
[24]	CHEN J, CHEN B Q. Architectural modeling from sparsely scanned range data[J]. International Journal of Computer Vision, 2008, 78(2): 223-236. DOI URL
[25]	李仁忠, 刘阳阳, 杨曼, 等. 基于改进的区域生长三维点云分割[J]. 激光与光电子学进展, 2018, 55(5): 325-331.
	LI R Z, LIU Y Y, YANG M, et al. Three-dimensional point cloud segmentation algorithm based on improved region growing[J]. Laser & Optoelectronics Progress, 2018, 55(5): 325-331 (in Chinese).
[26]	汪文琪, 李宗春, 付永健, 等. 基于改进多规则区域生长的点云多要素分割[J]. 光学学报, 2021, 41(5): 198-212.
	WANG W Q, LI Z C, FU Y J, et al. Multi-factor segmentation of point cloud based on improved multi-rule region growing[J]. Acta Optica Sinica, 2021, 41(5): 198-212 (in Chinese).
[27]	STEIN S C, SCHOELER M, PAPON J, et al. Object partitioning using local convexity[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 304-311.
[28]	SCHOELER M, PAPON J, WÖRGÖTTER F. Constrained planar cuts - Object partitioning for point clouds[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 5207-5215.
[29]	BORRMANN D, ELSEBERG J, LINGEMANN K, et al. The 3D Hough Transform for plane detection in point clouds: a review and a new accumulator design[J]. 3D Research, 2011, 2(2): 3.
[30]	FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395. DOI URL
[31]	HULIK R, SPANEL M, SMRZ P, et al. Continuous plane detection in point-cloud data based on 3D Hough Transform[J]. Journal of Visual Communication and Image Representation, 2014, 25(1): 86-97. DOI URL
[32]	SCHNABEL R, WAHL R, KLEIN R. Efficient RANSAC for point-cloud shape detection[J]. Computer Graphics Forum, 2007, 26(2): 214-226. DOI URL
[33]	田朋举, 花向红, 康停军, 等. 一种基于点云数据的建筑物平面精细分割方法[J]. 测绘科学, 2021, 46(2): 122-129.
	TIAN P J, HUA X H, KANG T J, et al. Fine segmentation of building planar feature from large scale unorganized point cloud[J]. Science of Surveying and Mapping, 2021, 46(2): 122-129 (in Chinese).
[34]	CHARLES R Q, HAO S, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 77-85.
[35]	QI C R, YI L, SU H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space[C]// The 24th Annual Conference on Neural Information Processing System. Cambridge: MIT Press, 2017: 5205-5114.
[36]	LIU T, YANG Z H, HU S J, et al. Neighbor reweighted local centroid for geometric feature identification[J]. IEEE Transactions on Visualization and Computer Graphics, 2021, 29(2): 1545-1558. DOI URL
[37]	NGUYEN K W L, APRILIA A, KHAIRYANTO A, et al. Edge detection from point cloud of worn parts[EB/OL]. [2022-06-28]. https://dr.ntu.edu.sg/handle/10356/88666.
[38]	GUENNEBAUD G, GROSS M. Algebraic point set surfaces[J]. ACM Transactions on Graphics, 2007, 26(3): 23-es. DOI URL
[39]	SHI S S, WANG X G, LI H S. PointRCNN: 3D object proposal generation and detection from point cloud[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 770-779.
[40]	ZHAO L, TAO W B. JSNet: joint instance and semantic segmentation of 3D point clouds[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12951-12958. DOI URL
[41]	吴军, 崔玥, 赵雪梅, 等. SSA-PointNet++: 空间自注意力机制下的3D点云语义分割网络[J]. 计算机辅助设计与图形学学报, 2022, 34(3): 437-448.
	WU J, CUI Y, ZHAO X M, et al. SSA- PointNet ++: a space self-attention CNN for the semantic segmentation of 3D point cloud[J]. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(3): 437-448 (in Chinese).

几何特征引导的物体点云模型多层级分割

Geometric feature guided multi-level segmentation for object point clouds

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 41

相关文章 6

编辑推荐

Metrics

本文评价

[1]	王佳栋, 曹娟, 陈中贵. 保特征的点云骨架提取算法[J]. 图学学报, 2023, 44(1): 146-157.
[2]	王春香, 刘流, 周国勇, 纪康辉. 面向自动修补的圆柱特征孔洞识别[J]. 图学学报, 2021, 42(3): 511-516.
[3]	袁清洌1，吴学群2. 结合Delaunay 三角面分离法与搜索球策略的三维曲面重建算法[J]. 图学学报, 2018, 39(2): 278-286.
[4]	朱云龙，翁桂荣. 面积项能量加强的距离规则水平集演化模型[J]. 图学学报, 2018, 39(1): 12-20.
[5]	程志全，叶永凯，李宝. 一种基于RANSAC 框架的椭球提取算法[J]. 图学学报, 2012, 33(2): 68-71.
[6]	石爽，曲仕茹，何力. 一种新的边界跟踪算法[J]. 图学学报, 2011, 32(3): 52-56.