Data mining and deep structured semantic model-based process sequence recommendation method

doi:10.11996/JG.j.2095-302X.2025040864

Abstract

Abstract:

To address the challenge of “data overload” encountered with traditional “experience-driven” methods in aerospace manufacturing process designs, we propose a process sequence recommendation method based on data mining and deep semantic models. This method integrates the PrefixSpan algorithm and BERT to extract typical manufacturing process sequences and their associated capabilities from component instance data, thereby constructing a reusable and updatable manufacturing process knowledge base. Building on this foundation, an enhanced spatial channel attention mechanism is introduced to accommodate the characteristics of aerospace manufacturing data, enabling implicit feature extraction form part instance data. Additionally, to mitigate the “cold start” issue caused by the uneven component instance distribution, self-supervised learning is employed to uncover the deep structure of the data, thereby ensuring the model’s generalization ability and improving its capability to learn from small sample instances. By combining a dual-channel attention-based deep semantic model with self-supervised learning, this approach effectively extract features, acquire knowledge, and accurately recommend process sequences suitable for aerospace manufacturing, even in the presence of data imbalance. Finally, a case study involving a specific aerospace component was conducted to validate the proposed method. Experimental results demonstrate that this method consistently outperforms benchmark models across various metrics in manufacturing process sequence recommendation, confirming its effectiveness and fulfilling the intelligent process design requirements of aerospace engineers.

Key words: data mining, self-supervised learning, deep semantic models, aviation complex parts, manufacturing sequence recommendation

CLC Number:

ZHENG Jiahui, GUO Yu, WU Tao, WANG Shengbo, HUANG Shaohua, ZHENG Kaiwen. Data mining and deep structured semantic model-based process sequence recommendation method[J]. Journal of Graphics, 2025, 46(4): 864-873.

Figures/Tables 12

Fig. 1 Framework diagram of process sequence recommendation method based on data mining and deep semantic models

Fig. 2 The flow of manufacturing process knowledge database (MPKD) construction

Fig. 3 The framework diagram of dual-channel attention deep structured semantic model

Fig. 4 The structure of dual-channel attention mode

Table 1 The extraction of record samples

元素	内容
几何属性特征	{“15.0 mm”,“1.5 mm”,“75.0 mm”,“6%”,“0°”}
材料属性特征	{“铝”,“LF2”,“75*1.5”,“M”,“管材”}
制造工艺特征	{“无氧化”,“无焊接”,“无酸洗”,“无扩口”,“除油”,“无喷漆”}
制造工艺序列	[“下料”，“领料”，“标记”，“除油”，“检查”，“标记”，“成品检验”]

Table 2 The experimental comparison results of process sequence recommendation for balanced data

模型	AUC	MRR@K			NDCG@K
模型	AUC	K=1	K=3	K=10	K=1	K=3	K=10
LSTM	0.938 4	0.571 8	0.721 9	0.743 5	0.571 8	0.770 6	0.808 2
Transformer	0.947 2	0.681 5	0.795 7	0.810 1	0.681 5	0.831 8	0.858 0
DSSM	0.974 7	0.746 7	0.864 8	0.866 5	0.746 7	0.897 9	0.901 1
DA-DSSM	0.981 7	0.834 5	0.908 0	0.910 5	0.834 5	0.928 9	0.933 5
DA-SLDSSM	0.988 8	0.887 2	0.940 4	0.940 4	0.887 2	0.954 1	0.955 8

Fig. 5 The iterative results of process sequence recommendation metrics for balanced data ((a) Loss iteration curve; (b) AUC metric iteration curve)

Table 3 The experimental comparison results of process sequence recommendation for long-tail data

模型	AUC	MRR@K			NDCG@K
模型	AUC	K=1	K=3	K=10	K=1	K=3	K=10
LSTM	0.881 8	0.522 0	0.625 6	0.669 4	0.522 0	0.662 5	0.749 1
Transformer	0.935 3	0.634 1	0.754 8	0.773 7	0.634 1	0.793 7	0.829 4
DSSM	0.959 0	0.672 1	0.807 2	0.815 2	0.847 6	0.847 6	0.862 4
DA-DSSM	0.974 5	0.794 5	0.882 2	0.886 3	0.907 3	0.907 3	0.915 3
DA-SLDSSM	0.984 7	0.869 5	0.928 4	0.928 9	0.869 5	0.942 6	0.947 1

Fig. 6 The experimental comparison results of process sequence recommendation for long-tail data ((a) Loss iteration curve; (b) AUC metric iteration curve)

Table 4 The validation results of part instance recommendation

零件ID	零件实例部分特征	推荐的典型工艺序列ID (K=3)
001	[15，1.5，75，6，0，铝，LF2，75 $∗$ 1.5，M，无氧化、无焊接，无扩口······]	[9，7，8]
042	[20，0，0，0，6，0，辅料，ET200，0，无，有氧化，有焊接，无扩口······]	[3，1，4]
091	[15，0，0，0，6，0，辅料，6602，0，无，无氧化，无焊接、有扩口······]	[4，3，7]
165	[20，1.5，75，6，90，铝，LF2，75 $∗$ 1.5，M，无氧化、无焊接，无扩口······]	[7，5，8]
320	[25，1，16，6，90，不锈钢，1Cr18Ni9Ti，16 $∗$ 1，固溶，无氧化、无焊接，无扩口······]	[11，12，8]

Table 4 The validation results of part instance recommendation

零件ID	零件实例部分特征	推荐的典型工艺序列ID (K=3)
001	[15，1.5，75，6，0，铝，LF2，75 $∗$ 1.5，M，无氧化、无焊接，无扩口······]	[9，7，8]
042	[20，0，0，0，6，0，辅料，ET200，0，无，有氧化，有焊接，无扩口······]	[3，1，4]
091	[15，0，0，0，6，0，辅料，6602，0，无，无氧化，无焊接、有扩口······]	[4，3，7]
165	[20，1.5，75，6，90，铝，LF2，75 $∗$ 1.5，M，无氧化、无焊接，无扩口······]	[7，5，8]
320	[25，1，16，6，90，不锈钢，1Cr18Ni9Ti，16 $∗$ 1，固溶，无氧化、无焊接，无扩口······]	[11，12，8]

Table 5 The data of expanded dataset

元素	复杂辅材数据	简单钣金数据
几何属性特征	{纤维厚度：15 mm，树脂层厚度：1.5 mm，面板宽度：75 mm，纤维角度偏差：6%，纤维方向：0°，铺层间距：3 mm，边缘容差：0.1 mm，孔径：5 mm}	{板材厚度：2 mm，孔径：10 mm，折弯半径：5 mm，长度：100 mm，宽度：50 mm，边缘容差：0.2 mm}
材料属性特征	{纤维密度：1.6 g/cm³，孔隙率限制：1.5%，复合材料弹性模量：70 GPa，断裂伸长率：2%，剪切模量：5 GPa}	{材料类型：不锈钢，材料密度：7.85 g/cm³，屈服强度：250 MPa，抗拉强度：450 MPa，硬度：45 HRC}
制造工艺特征	{固化时间：120 min，层数：5层，模具温度：180 ℃，铺层顺序：0°/90°/45°，固化速率：1 ℃/min，固化结束温度：180 ℃，真空度：95%，压实时间：30 min，铺层类型：UD单向，铺层厚度偏差：±0.02 mm}	{切割方式：激光切割，折弯角度：90°，表面处理方式：喷砂，涂层厚度：10 μm，加工精度：±0.1 mm}
制造工艺序列	{材料准备，模具制作与处理，初始铺层，层间检查，真空袋密封，热压罐固化，压力控制，缓冷处理，脱模，后固化，切割修边，钻孔与组装，表面打磨，无损检测，尺寸检测，机械性能测试，表面涂层}	{材料准备，切割，折弯成型，边缘处理，焊接，表面处理，质量检测，包装}

Fig. 7 The results of the model generalization experiments

References 22

[1]	汤文俊, 王裴岩, 蔡东风, 等. 基于编码器—解码器的工艺过程生成方法[J]. 计算机集成制造系统, 2023, 29(11): 3656-3668.
	TANG W J, WANG P Y, CAI D F, et al. Encoder-decoder based process generation method[J]. Computer Integrated Manufacturing System, 2023, 29(11): 3656-3668 (in Chinese).
[2]	张一鸣, 刘金锋, 陈亚杰, 等. 零件加工隐性工艺知识获取方法研究[J]. 图学学报, 2024, 45(2): 399-408. DOI
	ZHANG Y M, LIU J F, CHEN Y J, et al. Tacit process knowledge acquisition methods for the parts machining[J]. Journal of Graphics, 2024, 45(2): 399-408 (in Chinese). DOI
[3]	何柳, 安然, 刘姝妍, 等. 基于知识图谱的航空多模态数据组织与知识发现技术研究[J]. 图学学报, 2024, 45(2): 300-307. DOI
	HE L, AN R, LIU S Y, et al. Research on knowledge graph-based aviation multi-modal data organization and discovery method[J]. Journal of Graphics, 2024, 45(2): 300-307 (in Chinese). DOI
[4]	冯锦丹, 何楠, 刘金山, 等. 基于知识驱动的航天复杂产品装配工艺重构方法[J]. 计算机集成制造系统, 2024, 30(12): 4314-4327.
	FEN J D, HE N, LIU J H, et al. Knowledge-driven assembly process reconfiguration method for aerospace complex products[J]. Computer Integrated Manufacturing Systems, 2024, 30(12): 4314-4327 (in Chinese).
[5]	曹勇. 基于数据挖掘的工艺知识发现与重用研究[D]. 济南: 山东大学, 2019.
	CAO Y. Research on process knowledge discovering and reusing based on data mining[D]. Jinan: Shandong University, 2019 (in Chinese).
[6]	SAXENA A, PRASAD M, GUPTA A, et al. A review of clustering techniques and developments[J]. Neurocomputing, 2017, 267: 664-681.
[7]	ZHAO C X, DINAR M, MELKOTE S N. A data-driven framework for learning the capability of manufacturing process sequences[J]. Journal of Manufacturing Systems, 2022, 64: 68-80.
[8]	CHEN X C, YAO L N, MCAULEY J, et al. Deep reinforcement learning in recommender systems: a survey and new perspectives[J]. Knowledge-Based Systems, 2023, 264: 110335.
[9]	陈治宇, 鲍劲松, 郑小虎, 等. 基于长短期记忆网络的装配工艺语义识别方法[J]. 计算机集成制造系统, 2021, 27(6): 1582-1593. DOI
	CHEN Z Y, BAO J S, ZHENG X H, et al. Semantic recognition method of assembly process based on LSTM[J]. Computer Integrated Manufacturing Systems, 2021, 27(6): 1582-1593 (in Chinese).
[10]	KOREN Y, RENDLE S, BELL R. Advances in collaborative filtering[M]//RICCI F, ROKACH L, SHAPIRA B. Recommender Systems Handbook. 3rd ed. New York: Springer, 2022: 91-142.
[11]	HUANG P S, HE X D, GAO J F, et al. Learning deep structured semantic models for web search using clickthrough data[C]// The 22nd ACM International Conference on Information & Knowledge Management. New York: ACM, 2013: 2333-2338.
[12]	KRICHENE W, MAYORAZ N, RENDLE S, et al. Efficient training on very large corpora via Gramian estimation[EB/OL]. [2024-03-24]. https://dblp.org/db/conf/iclr/iclr2019.html#KricheneMRZYHCA19.
[13]	GUO C, MOUSAVI A, WU X, et al. Breaking the glass ceiling for embedding-based classifiers for large output spaces[C]// The 33rd International Conference on Neural Information Processing Systems. New York: ACM, 2019: 445.
[14]	PANDA D K, RAY S. Approaches and algorithms to mitigate cold start problems in recommender systems: a systematic literature review[J]. Journal of Intelligent Information Systems, 2022, 59(2): 341-366.
[15]	YAO T S, YI X Y, CHENG D Z, et al. Self-supervised learning for large-scale item recommendations[C]// The 30th ACM International Conference on Information & Knowledge Management. New York: ACM, 2021: 4321-4330.
[16]	DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C]// 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Minnesota: Association for Computational Linguistics, 2019: 4171-4186.
[17]	林佳瑞, 程志刚, 韩宇, 等. 基于BERT预训练模型的灾害推文分类方法[J]. 图学学报, 2022, 43(3): 530-536.
	LIN J R, CHENG Z G, HAN Y, et al. Disaster tweets classification method based on pretrained BERT model[J]. Journal of Graphics, 2022, 43(3): 530-536 (in Chinese). DOI
[18]	张晋, 刘检华, 赵文浩, 等. 基于预训练模型与相似度算法的复杂产品装配技术问题风险预警方法[J]. 机械工程学报, 2024, 60(24): 317-329.
	ZHANG J, LIU J H, ZHAO W H, et al. Risk warning method for complex product assembly technology problems based on pre-training model and similarity algorithm[J]. Journal of Mechanical Engineering, 2024, 60(24): 317-329 (in Chinese).
[19]	SUBAKTI A, MURFI H, HARIADI N. The performance of BERT as data representation of text clustering[J]. Journal of Big Data, 2022, 9(1): 15. DOI PMID
[20]	PEI J, HAN J W, MORTAZAVI-ASL B, et al. PrefixSpan: mining sequential patterns efficiently by prefix-projected pattern growth[C]// The 17th International Conference on Data Engineering. New York: IEEE Press, 2001: 215-224.
[21]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// The 15th European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[22]	AKIBA T, SANO S, YANASE T, et al. Optuna: a next-generation hyperparameter optimization framework[C]// The 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 2623-2631.