基于大模型的生成式数字孪生体建模

doi:10.11996/JG.j.2095-302X.2026010173

摘要/Abstract

摘要：

针对数字孪生(DT)技术与大模型在工业设计中融合应用的挑战，提出了一种基于生成式大模型的CAD-LDT数字孪生体建模框架。通过构建物理实体模块、智能生成模块和虚拟实体模块的三元架构，创新性地整合多模态数据融合机制与领域知识约束，实现从物理实体描述到参数化CAD模型的自主生成。采用LLaVA-7B和LLaMA-7B作为基础模型，通过LoRA轻量化适配器实现视觉-文本特征的跨模态对齐，并设计约束编码器将几何公差与物理规则转化为结构化JSON对象。为增强空间变换的数学一致性，引入李群算法优化刚体变换表征；采用几何权重分箱方法离散化复杂装配关系；提出时空解耦生成策略，协同规划空间布局与装配时序。在DeepCAD数据集上的实验结果表明，该框架在几何达到83.6%，约束满足率达91.3%，生成效率提升了26.5%，显著优于主流基线模型。消融实验进一步验证了多模态融合、约束编码机制和李群算法对建模质量的关键贡献。为智能制造领域提供了新的DT建模范式，在参数化设计、装配工艺优化等方面展现出工程应用价值。

关键词: 大模型, 数字孪生, 多模态数据, 智能制造, 参数化设计

Abstract:

To address the challenges in integrating Digital-Twin (DT) technology with large-scale generative models in industrial design, a CAD-LDT digital-twin modeling framework based on generative foundation models was proposed. The framework adopted a triadic architecture consisting of a physical-entity module, an intelligent generation module, and a virtual-entity module, and innovatively incorporated multi-modal data fusion mechanisms and domain-knowledge constraints to enable autonomous generation of parameterized CAD models from physical-entity descriptions. Utilizing LLaVA-7B and LLaMA-7B as backbone models, the framework employed LoRA-based lightweight adapters to achieve cross-modal alignment between visual and textual features, and introduced a constraint encoder that transformed geometric tolerances and physical rules into structured JSON objects. To enhance the mathematical consistency of spatial transformations, Lie-group algorithms were adopted for the optimization of rigid-body transformations, while a geometric-weight binning strategy was proposed to discretize complex assembly relationships. Moreover, a spatiotemporal-decoupled generation strategy was designed to jointly optimize spatial layout and assembly sequencing. Experimental results on the DeepCAD dataset indicated that the proposed framework achieved an Intersection- over-Union (IoU) of 83.6%, a constraint satisfaction rate of 91.3%, and a 26.5% improvement in generation efficiency, significantly outperforming existing baseline models. Further ablation studies confirmed the critical contributions of multi-modal fusion, constraint encoding mechanisms, and Lie-group optimization to modeling performance, providing a novel DT modeling paradigm for intelligent manufacturing with demonstrated value in parametric design and assembly process optimization.

Key words: large models, digital twin, multimodal data, intelligent manufacturing, parametric design

中图分类号:

梁生龙, 范秋霞. 基于大模型的生成式数字孪生体建模[J]. 图学学报, 2026, 47(1): 173-178.

LIANG Shenglong, FAN Qiuxia. Generative digital twin modeling based on large models[J]. Journal of Graphics, 2026, 47(1): 173-178.

图/表 7

图1 生成式数字孪生体CAD-LDT

Fig. 1 Generative digital twin CAD-LDT

表1 数据参数表

Table 1 Data parameter table

参数项	配置值
优化器	AdamW
学习率	5e-5
批量大小	32
训练周期	50
正则化	0.01+0.1

表2 孪生体测试对比表

Table 2 Comparison table of model testing

模型	IoU/%	约束满足率/%	生成时间/s	材料利用率/%	最小壁厚合格率/%
GPT-3.5+ Adapter	75.3	81.2	12.4	79.2	76.5
LLaVA-7B CAD-LDT	79.1 83.6	85.7 91.3	9.8 7.2	86.7 92.4	82.3 94.1

图2 多模态输入场景不同方法的比较

Fig. 2 Comparison of different methods for multimodal input scenarios ((a) Original image; (b) GPT-3.5+Adapter; (c) LLaVA-7B; (d) CAD-LDT)

表3 复杂装配场景测试对比表

Table 3 Comparison table for complex assembly scenarios testing

模型	特征完整性/ 分数	关键尺寸误差率/%	生成时间/s
GPT-3.5+Adapter	75.3	18.8	32.5
LLaVA-7B	79.1	14.3	30.8
CAD-LDT	83.6	8.7	31.2

图3 复杂装配场景不同方法的比较

Fig. 3 Comparison of different methods in complex assembly scenarios ((a) Original image; (b) GPT-3.5+Adapter; (c) LLaVA-7B; (d) CAD-LDT)

表4 不同约束条件下孪生体实验对比/%

Table 4 Comparison of twin experiments under different constraint conditions/%

消融项	IoU	约束满足率	失败率
完整模型	83.6	91.3	2.1
移除多模态数据禁用约束编码器	70.9 76.2	82.5 71.4	15.7 27.3
传统空间表示	78.4	85.1	18.3

参考文献 20

[1]	王剑, 王好臣, 李学伟, 等. 基于OPC UA的数字孪生车间信息物理融合系统[J]. 现代制造工程, 2023(4): 43-50.
	WANG J, WANG H C, LI X W, et al. Digital twin workshop information physical fusion system based on OPC UA[J]. Modern Manufacturing Engineering, 2023(4): 43-50 (in Chinese).
[2]	GRIEVES M, VICKERS J. Digital twin: mitigating unpredictable, undesirable emergent behavior in complex systems[M]// KAHLENF J, FLUMERFELTS, ALVESA. Transdisciplinary Perspectives on Complex Systems:New Findings and Approaches. Cham: Springer, 2017: 85-113.
[3]	王进峰, 问丛川, 花广如. 面向概念、技术与应用的数字孪生综述[J]. 中国工程机械学报, 2023, 21(2): 112-116, 133.
	WANG J F, WEN C C, HUA G R. A survey of digital twins for concept, technology and application[J]. Chinese Journal of Construction Machinery, 2023, 21(2): 112-116, 133 (in Chinese).
[4]	LEE J, LAPIRA E, YANG S H, et al. Predictive manufacturing system-trends of next-generation production systems[J]. IFAC Proceedings Volumes, 2013, 46(7): 150-156.
[5]	BAO J S, GUO D S, LI J, et al. The modelling and operations for the digital twin in the context of manufacturing[J]. Enterprise Information Systems, 2019, 13(4): 534-556. DOI URL
[6]	ZHUANG C B, LIU J H, XIONG H. Digital twin-based smart production management and control framework for the complex product assembly shop-floor[J]. The International Journal of Advanced Manufacturing Technology, 2018, 96(1/4): 1149-1163. DOI
[7]	TAO F, ZHANG M, LIU Y S, et al. Digital twin driven prognostics and health management for complex equipment[J]. CIRP Annals, 2018, 67(1): 169-172. DOI URL
[8]	JIANG Y K, CHEN J H, ZHOU H C, et al. Contour error modeling and compensation of CNC machining based on deep learning and reinforcement learning[J]. The International Journal of Advanced Manufacturing Technology, 2022, 118(1/2): 551-570. DOI
[9]	SAMSONOV V, CHRISMARIE E, KÖPKEN H G, et al. Deep representation learning and reinforcement learning for workpiece setup optimization in CNC milling[J]. Production Engineering, 2023, 17(6): 847-859. DOI
[10]	LI B R, ZHANG H, YE P Q, et al. Trajectory smoothing method using reinforcement learning for computer numerical control machine tools[J]. Robotics and Computer-Integrated Manufacturing, 2020, 61: 101847. DOI URL
[11]	YAO S Y, RAO R H, HAUSKNECHT M, et al. Keep CALM and explore: language models for action generation in text-based games[C]// 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2020: 8736-8754.
[12]	PANG J C, YANG X Y, YANG S H, et al. Natural language-conditioned reinforcement learning with inside-out task language development and translation[EB/OL]. [2024- 04-10]. https://arxiv.org/pdf/2302.09368.
[13]	何援军, 于海燕, 方志刚. 基于形计算重构CAD的计算基础[J]. 计算机集成制造系统, 2025, 31(3): 760-777.
	HE Y J, YU H Y, FANG Z G. Reconstructing computational foundation for CAD based on shape computing[J]. Computer Integrated Manufacturing Systems, 2025, 31(3): 760-777 (in Chinese).
[14]	黄川, 李雅琴, 祁越然, 等. 基于3D-CAE的高光谱解混及小样本分类方法[J]. 自然资源遥感, 2025, 37(1): 8-14.
	HUANG C, LI Y Q, QI Y R, et al. A hyperspectral unmixing and few-shot classification method based on 3DCAE network[J]. Remote Sensing for Natural Resources, 2025, 37(1): 8-14 (in Chinese).
[15]	龙禹辰, 勾智楠, 陈宇欣, 等. 基于大语言模型的多任务生成式重构对话情绪识别[J]. 计算机应用研究, 2025, 42(7): 1964-1971.
	LONG Y C, GOU Z N, CHEN Y X, et al. Multi-task generative emotion recognition in conversation based on large language models[J]. Application Research of Computers, 2025, 42(7): 1964-1971 (in Chinese).
[16]	URBANCZYK A, WRIGHT J. CadQuery: a python parametric CAD scripting framework based on OCCT[EB/OL]. [2025- 04-10]. https://github.com/CadQuery/cadquery.
[17]	刘静文, 刘渊, 袁琮淇. 融合超图卷积和自监督协同训练的组推荐算法[J]. 中文信息学报, 2024, 38(7): 115-126, 136.
	LIU J W, LIU Y, YUAN C Q. Group recommendation algorithms incorporating hypergraph convolution and self-supervised collaborative training[J]. Journal of Chinese Information Processing, 2024, 38(7): 115-126, 136 (in Chinese).
[18]	雷松林, 赵征鹏, 阳秋霞, 等. 基于可解耦扩散模型的零样本风格迁移[EB/OL]. (2025-03-31) [2025-04-02]. http://kns.cnki.net/kcms/detail/10.1034.T.20250331.1207.002.html.
	LEI S L, ZHAO Z P, YANG Q X, et al. Zero-shot style transfer based on decoupled diffusion models[EB/OL]. (2025-03-31) [2025-04-02]. http://kns.cnki.net/kcms/detail/10.1034.T.20250331.1207.002.html (in Chinese).
[19]	孙灏铖, 刘力, 李凡长. 李群模糊C均值聚类图像分割算法[J]. 软件学报, 2024, 35(10): 4806-4825.
	SUN H C, LIU L, LI F Z. Lie group fuzzy C-means clustering algorithm for image segmentation[J]. Journal of Software, 2024, 35(10): 4806-4825 (in Chinese).
[20]	王乃成. 对GB/T 4458.6-2002的分析和讨论[J]. 机械工业标准化与质量, 2006(1): 32-35.
	WANG N C. Analysis and discussion on GB/T 4458.6-2002[J]. Mechanical Industry Standardization & Quality, 2006(1): 32-35 (in Chinese).