适用于不同款式的无监督服装动画预测

doi:10.11996/JG.j.2095-302X.2024030539

图学学报 ›› 2024, Vol. 45 ›› Issue (3): 539-547.DOI: 10.11996/JG.j.2095-302X.2024030539

• 计算机图形学与虚拟现实 • 上一篇下一篇

适用于不同款式的无监督服装动画预测

石敏¹(), 禚心如¹, 孙碧莲¹, 韩国庆¹, 朱登明²()

1.华北电力大学控制与计算机工程学院，北京 102206
2.中国科学院计算技术研究所前瞻研究实验室，北京 100190

收稿日期:2023-08-22 接受日期:2023-12-10 出版日期:2024-06-30 发布日期:2024-06-11
通讯作者:朱登明(1973-)，男，副研究员，博士。主要研究方向为虚拟现实、计算机图形学等。E-mail：mdzhu@ict.ac.cn
第一作者:石敏(1975-)，女，副教授，博士。主要研究方向为计算机图形学、虚拟现实等。E-mail：shi_min@ncepu.edu.cn
基金资助:
国家自然科学基金项目(61972379)

Unsupervised clothing animation prediction for different styles

SHI Min¹(), ZHUO Xinru¹, SUN Bilian¹, HAN Guoqing¹, ZHU Dengming²()

1. School of Control and Computer Engineering, North China Electric Power University, Beijing 102206, China
2. Prospective Research Laboratory, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China

Received:2023-08-22 Accepted:2023-12-10 Published:2024-06-30 Online:2024-06-11
First author：SHI Min (1975-), associate professor, Ph.D. Her main research interests cover graphics, virtual reality, etc. E-mail：shi_min@ncepu.edu.cn
Supported by:
National Natural Science Foundation of China(61972379)

摘要/Abstract

摘要：

虚拟角色着装动画生成是三维动画的关键技术，着装变形作为其核心一直是该领域的研究热点。现有着装变形方法大多基于单一服装款式进行研究，一旦款式变化则需要重新训练，耗费时间和增加计算成本。同时现有方法大多基于有监督的方法进行网络训练，需要大量的数据准备和训练成本。基于此，提出一种适用于不同款式的无监督着装动画生成方法。首先，提出一个可学习的款式特征表示方式，学习款式约束的运动隐空间概率分布模型；其次，基于编解码结构搭建款式约束的无监督服装变形预测网络，进一步引入Transformer编解码层对时序运动特征进行提取；最后，进行多款式动画生成实验，并与现有方法基于视觉效果和定量指标进行对比分析。实验结果表明，相较于现有方法，本文方法可以生成款式可调的、视觉合理的着装动画，在预测精度以及穿透损失等方面具有明显的优越性。

关键词: 着装动画, 无监督, 计算机图形学, 服装变形, Transformer

Abstract:

Dressing animation generation is a key technology in 3D animation, with clothing deformation as its core, which has been a focal point of research in this field. Existing clothing deformation methods mostly focus on a single clothing style for research, requiring retraining for style changes, thus consuming time and increasing computational costs. Additionally, most current methods rely on supervised approaches for network training, necessitating extensive data preparation and training expenses. In light of these challenges, an unsupervised clothing animation generation method applicable to different styles was proposed. Firstly, a learnable style feature representation was introduced, capturing the probabilistic distribution model of style-constrained motion latent space. Secondly, an unsupervised clothing deformation prediction network was established with style constraints and grounded in an encoder-decoder architecture. Furthermore, a Transformer encoder-decoder layer was incorporated to extract temporal motion features. Finally, multiple style animation generation experiments were conducted, comparing the proposed method with existing methods in terms of visual effects and quantitative metrics. Experimental results demonstrated that the proposed method can generate visually plausible clothing animations with adjustable styles, outperforming existing methods in prediction accuracy and reducing penetration loss.

Key words: clothing animation, unsupervised, computer graphics, clothing deformation, Transformer

中图分类号:

TP391

石敏, 禚心如, 孙碧莲, 韩国庆, 朱登明. 适用于不同款式的无监督服装动画预测[J]. 图学学报, 2024, 45(3): 539-547.

SHI Min, ZHUO Xinru, SUN Bilian, HAN Guoqing, ZHU Dengming. Unsupervised clothing animation prediction for different styles[J]. Journal of Graphics, 2024, 45(3): 539-547.

图/表 13

图1 网络结构

Fig. 1 Network architecture

图2 编解码器详细结构

Fig. 2 The details of encoder and decoder

表1 数据集设置细节

Table 1 Setting details of dataset

模型	顶点数	面数	关节
人体	6 890	13 776	24
服装	3 100	6 034	-

图3 服装模型参数对应示意图

Fig. 3 Schematic diagram of corresponding parameters for clothing model

表2 权重设置

Table 2 Setting of loss weights

设置	权重
设置	ω	ω_bend	ω_edge	ω_gra	ω_col
数值	0.000 01	0.000 05	15	1	250

表3 重建穿透损失

Table 3 Collision loss of reconstruction

款式	边损失/ ×10^-2	曲率损失/ ×10^-2	穿透损失/ ×10^-4	穿透率/ %
0	1.84	5.06	1.63	0.37
1	1.76	5.52	1.90	0.38
2	1.63	6.26	1.82	0.37
3	1.52	6.37	1.79	0.37
Mean	1.70	5.80	1.79	0.37

图4 服装变形预测结果((a)款式1；(b)款式2；(c)款式3；(d)款式4)

Fig. 4 Prediction results of clothing deformation ((a) Style1; (b) Style2; (c) Style3; (d) Style4)

图5 插值对比结果((a)人体运动；(b)线性插值；(c)本文方法)

Fig. 5 Comparison of interpolation results ((a) Human movement; (b) Linear interpolation; (c) Ours)

表4 定量对比

Table 4 Quantitative evaluation

方法	边损失/ ×10^-2	曲率损失/ ×10^-2	穿透率/ %
TailorNet	6.11	7.52	13.03
PBNS	3.23	7.12	2.18
LBS	9.51	14.30	26.58
本文方法	2.62	6.98	0.02

图6 定性对比结果

Fig. 6 Comparison of qualitative results ((a) Ours; (b) PBNS; (c) TailorNet; (d) LBS)

表5 消融实验对比

Table 5 Comparison of ablation experiments

方法	边损失/ ×10^-2	曲率损失/ ×10^-2	穿透损失/ ×10^-4	穿透率/ %
消融实验	2.97	6.53	5.66	1.07
本文方法	1.68	5.74	1.80	0.36

图7 消融实验结果((a)款式1；(b)款式2)

Fig. 7 Results of ablation experiments ((a) Style 1; (b) Style 2)

图8 着装动画展示((a)款式1；(b)款式2)

Fig. 8 Dressing animation ((a) Style 1; (b) Style 2)

参考文献 27

[1]	PROVOT X. Deformation constraints in a mass-spring model to describe rigid cloth behavior[EB/OL]. [2023-01-13]. https://xueshu.baidu.com/usercenter/paper/show?paperid=1e550v00em0c0pk0w10u0850mb695262&site=xueshu_se.
[2]	ROHMER D, POPA T, CANI M P, et al. Animation wrinkling: augmenting coarse cloth simulations with realistic-looking wrinkles[J]. ACM Transactions on Graphics, 2010, 29(6): 157: 1-157: 8.
[3]	WU L H, WU B T, YANG Y, et al. A safe and fast repulsion method for GPU-based cloth self collisions[J]. ACM Transactions on Graphics, 2020, 40(1): 1-18.
[4]	CLO Virtual Fashion. Marvelous designer[EB/OL]. [2023-06-30]. https://www.marvelousdesigner.2023.6.30.
[5]	WANG T Y, SHAO T J, FU K, et al. Learning an intrinsic garment space for interactive authoring of garment animation[J]. ACM Transactions on Graphics, 2019, 38(6): 220: 1-220: 12.
[6]	ANGUELOV D, SRINIVASAN P, KOLLER D, et al. SCAPE: shape completion and animation of people[C]// SIGGRAPH ’05: ACM SIGGRAPH 2005 Papers. New York: ACM, 2005: 408-416.
[7]	LOPER M, MAHMOOD N, ROMERO J, et al. SMPL: a skinned multi-person linear model[M]// Seminal Graphics Papers: Pushing the Boundaries, Volume 2. New York: ACM, 2023: 851-866.
[8]	PATEL C, LIAO Z, PONS-MOLL G. TailorNet: predicting clothing in 3D as a function of human pose, shape and garment style[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 7365-7375.
[9]	SAITO S, YANG J L, MA Q L, et al. SCANimate: weakly supervised learning of skinned clothed avatar networks[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 2886-2897.
[10]	SANTESTEBAN I, OTADUY M A, CASAS D. Learning- based animation of clothing for virtual try-on[J]. Computer Graphics Forum, 2019, 38(2): 355-366.
[11]	LI C, TANG M, TONG R F, et al. P-cloth: interactive complex cloth simulation on multi-GPU systems using dynamic matrix assembly and pipelined implicit integrators[J]. ACM Transactions on Graphics, 2020, 39(6): 180: 1-180: 15.
[12]	WANG H M. GPU-based simulation of cloth wrinkles at submillimeter levels[J]. ACM Transactions on Graphics, 2021, 40(4): 169: 1-169: 14.
[13]	ZHU H M, CAO Y, JIN H, et al. Deep Fashion3D: a dataset and benchmark for 3D garment reconstruction from single images[C]// European Conference on Computer Vision. Cham: Springer, 2020: 512-530.
[14]	HE Y, XIE H R, MIYATA K. Sketch2Cloth: sketch-based 3D garment generation with unsigned distance fields[EB/OL]. [2023-02-10]. http://arxiv.org/abs/2303.00167.pdf.
[15]	BERTICHE H, MADADI M, ESCALERA S. PBNS: physically based neural simulator for unsupervised garment pose space deformation[EB/OL]. [2023-01-13]. http://arxiv.org/abs/2012.11310.pdf.
[16]	PAN X Y, MAI J M, JIANG X W, et al. Predicting loose-fitting garment deformations using bone-driven motion networks[C]// ACM SIGGRAPH 2022 Conference Proceedings. New York: ACM, 2022: 1-10.
[17]	LI Y D, TANG M, YANG Y, et al. N-cloth: predicting 3D cloth deformation with mesh-based networks[J]. Computer Graphics Forum, 2022, 41(2): 547-558.
[18]	MA Q L, YANG J L, BLACK M J, et al. Neural point-based shape modeling of humans in challenging clothing[C]// 2022 International Conference on 3D Vision (3DV). New York: IEEE Press, 2022: 679-689.
[19]	TIWARI L, BHOWMICK B. DeepDraper: fast and accurate 3D garment draping over a 3D human body[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops. New York: IEEE Press, 2021: 1416-1426.
[20]	CHONG T, SHEN I C, UMETANI N, et al. Per garment capture and synthesis for real-time virtual try-on[C]// UIST ’21: The 34th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2021: 457-469.
[21]	WU Z H, LIN G S, TAO Q Y, et al. M2E-try on net: fashion from model to everyone[C]// The 27th ACM International Conference on Multimedia. New York: ACM, 2019: 293-301.
[22]	PETROVICH M, BLACK M J, VAROL G. Action-conditioned 3D human motion synthesis with transformer VAE[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 10985-10995.
[23]	TEVET G, RAAB S, GORDON B, et al. Human motion diffusion model[EB/OL]. [2023-01-13]. http://arxiv.org/abs/2209.14916.pdf.
[24]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]// The 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010.
[25]	MAHMOOD N, GHORBANI N, TROJE N F, et al. AMASS: archive of motion capture as surface shapes[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 5441-5450.
[26]	VIDAURRE R, SANTESTEBAN I, GARCES E, et al. Fully convolutional graph neural networks for parametric virtual try-on[J]. Computer Graphics Forum, 2020, 39(8): 145-156.
[27]	ALLEN B, CURLESS B, POPOVIĆ Z. The space of human body shapes: reconstruction and parameterization from range scans[J]. ACM Transactions on Graphics (TOG), 2003, 22(3): 587-594.

适用于不同款式的无监督服装动画预测

Unsupervised clothing animation prediction for different styles

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	罗智徽, 胡海涛, 马潇峰, 程文刚. 基于同质中间模态的跨模态行人再识别方法[J]. 图学学报, 2024, 45(4): 670-682.
[2]	李滔, 胡婷, 武丹丹. 结合金字塔结构和注意力机制的单目深度估计[J]. 图学学报, 2024, 45(3): 454-463.
[3]	刘以, 邱军海, 张嘉星, 张小峰, 王桦, 张彩明. 基于权衡因子和多维空间度量的高鲁棒性图像分割算法[J]. 图学学报, 2024, 45(3): 482-494.
[4]	黄友文, 林志钦, 章劲, 陈俊宽. 结合坐标Transformer的轻量级人体姿态估计算法[J]. 图学学报, 2024, 45(3): 516-527.
[5]	李佳琦, 王辉, 郭宇. 基于Transformer的三角形网格分类分割网络[J]. 图学学报, 2024, 45(1): 78-89.
[6]	吕衡, 杨鸿宇. 一种基于时空运动信息交互建模的三维人体姿态估计方法[J]. 图学学报, 2024, 45(1): 159-168.
[7]	黄少年, 文沛然, 全琪, 陈荣元. 基于多支路聚合的帧预测轻量化视频异常检测[J]. 图学学报, 2023, 44(6): 1173-1182.
[8]	石佳豪, 姚莉. 基于语义引导的视频描述生成[J]. 图学学报, 2023, 44(6): 1191-1201.
[9]	杨陈成, 董秀成, 侯兵, 张党成, 向贤明, 冯琪茗. 基于参考的Transformer纹理迁移深度图像超分辨率重建[J]. 图学学报, 2023, 44(5): 861-867.
[10]	杨红菊, 高敏, 张常有, 薄文, 武文佳, 曹付元. 一种面向图像修复的局部优化生成模型[J]. 图学学报, 2023, 44(5): 955-965.
[11]	宋焕生, 文雅, 孙士杰, 宋翔宇, 张朝阳, 李旭. 基于改进教师学生网络的隧道火灾检测[J]. 图学学报, 2023, 44(5): 978-987.
[12]	郝帅, 赵新生, 马旭, 张旭, 何田, 侯李祥. 基于TR-YOLOv5的输电线路多类缺陷目标检测方法[J]. 图学学报, 2023, 44(4): 667-676.
[13]	李刚, 张运涛, 汪文凯, 张东阳. 采用DETR与先验知识融合的输电线路螺栓缺陷检测方法[J]. 图学学报, 2023, 44(3): 438-447.
[14]	冯尊登, 王洪元, 林龙, 孙博言, 陈海琴. 基于多分支注意网络与相似度学习策略的无监督行人重识别[J]. 图学学报, 2023, 44(2): 280-290.
[15]	闫善武, 肖洪兵, 王瑜, 孙梅. 融合行人时空信息的视频异常检测[J]. 图学学报, 2023, 44(1): 95-103.