Unsupervised clothing animation prediction for different styles

doi:10.11996/JG.j.2095-302X.2024030539

Abstract

Abstract:

Dressing animation generation is a key technology in 3D animation, with clothing deformation as its core, which has been a focal point of research in this field. Existing clothing deformation methods mostly focus on a single clothing style for research, requiring retraining for style changes, thus consuming time and increasing computational costs. Additionally, most current methods rely on supervised approaches for network training, necessitating extensive data preparation and training expenses. In light of these challenges, an unsupervised clothing animation generation method applicable to different styles was proposed. Firstly, a learnable style feature representation was introduced, capturing the probabilistic distribution model of style-constrained motion latent space. Secondly, an unsupervised clothing deformation prediction network was established with style constraints and grounded in an encoder-decoder architecture. Furthermore, a Transformer encoder-decoder layer was incorporated to extract temporal motion features. Finally, multiple style animation generation experiments were conducted, comparing the proposed method with existing methods in terms of visual effects and quantitative metrics. Experimental results demonstrated that the proposed method can generate visually plausible clothing animations with adjustable styles, outperforming existing methods in prediction accuracy and reducing penetration loss.

Key words: clothing animation, unsupervised, computer graphics, clothing deformation, Transformer

CLC Number:

TP391

SHI Min, ZHUO Xinru, SUN Bilian, HAN Guoqing, ZHU Dengming. Unsupervised clothing animation prediction for different styles[J]. Journal of Graphics, 2024, 45(3): 539-547.

Figures/Tables 13

Fig. 1 Network architecture

Fig. 2 The details of encoder and decoder

Table 1 Setting details of dataset

模型	顶点数	面数	关节
人体	6 890	13 776	24
服装	3 100	6 034	-

Fig. 3 Schematic diagram of corresponding parameters for clothing model

Table 2 Setting of loss weights

设置	权重
设置	ω	ω_bend	ω_edge	ω_gra	ω_col
数值	0.000 01	0.000 05	15	1	250

Table 3 Collision loss of reconstruction

款式	边损失/ ×10^-2	曲率损失/ ×10^-2	穿透损失/ ×10^-4	穿透率/ %
0	1.84	5.06	1.63	0.37
1	1.76	5.52	1.90	0.38
2	1.63	6.26	1.82	0.37
3	1.52	6.37	1.79	0.37
Mean	1.70	5.80	1.79	0.37

Fig. 4 Prediction results of clothing deformation ((a) Style1; (b) Style2; (c) Style3; (d) Style4)

Fig. 5 Comparison of interpolation results ((a) Human movement; (b) Linear interpolation; (c) Ours)

Table 4 Quantitative evaluation

方法	边损失/ ×10^-2	曲率损失/ ×10^-2	穿透率/ %
TailorNet	6.11	7.52	13.03
PBNS	3.23	7.12	2.18
LBS	9.51	14.30	26.58
本文方法	2.62	6.98	0.02

Fig. 6 Comparison of qualitative results ((a) Ours; (b) PBNS; (c) TailorNet; (d) LBS)

Table 5 Comparison of ablation experiments

方法	边损失/ ×10^-2	曲率损失/ ×10^-2	穿透损失/ ×10^-4	穿透率/ %
消融实验	2.97	6.53	5.66	1.07
本文方法	1.68	5.74	1.80	0.36

Fig. 7 Results of ablation experiments ((a) Style 1; (b) Style 2)

Fig. 8 Dressing animation ((a) Style 1; (b) Style 2)

References 27

[1]	PROVOT X. Deformation constraints in a mass-spring model to describe rigid cloth behavior[EB/OL]. [2023-01-13]. https://xueshu.baidu.com/usercenter/paper/show?paperid=1e550v00em0c0pk0w10u0850mb695262&site=xueshu_se.
[2]	ROHMER D, POPA T, CANI M P, et al. Animation wrinkling: augmenting coarse cloth simulations with realistic-looking wrinkles[J]. ACM Transactions on Graphics, 2010, 29(6): 157: 1-157: 8.
[3]	WU L H, WU B T, YANG Y, et al. A safe and fast repulsion method for GPU-based cloth self collisions[J]. ACM Transactions on Graphics, 2020, 40(1): 1-18.
[4]	CLO Virtual Fashion. Marvelous designer[EB/OL]. [2023-06-30]. https://www.marvelousdesigner.2023.6.30.
[5]	WANG T Y, SHAO T J, FU K, et al. Learning an intrinsic garment space for interactive authoring of garment animation[J]. ACM Transactions on Graphics, 2019, 38(6): 220: 1-220: 12.
[6]	ANGUELOV D, SRINIVASAN P, KOLLER D, et al. SCAPE: shape completion and animation of people[C]// SIGGRAPH ’05: ACM SIGGRAPH 2005 Papers. New York: ACM, 2005: 408-416.
[7]	LOPER M, MAHMOOD N, ROMERO J, et al. SMPL: a skinned multi-person linear model[M]// Seminal Graphics Papers: Pushing the Boundaries, Volume 2. New York: ACM, 2023: 851-866.
[8]	PATEL C, LIAO Z, PONS-MOLL G. TailorNet: predicting clothing in 3D as a function of human pose, shape and garment style[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 7365-7375.
[9]	SAITO S, YANG J L, MA Q L, et al. SCANimate: weakly supervised learning of skinned clothed avatar networks[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 2886-2897.
[10]	SANTESTEBAN I, OTADUY M A, CASAS D. Learning- based animation of clothing for virtual try-on[J]. Computer Graphics Forum, 2019, 38(2): 355-366.
[11]	LI C, TANG M, TONG R F, et al. P-cloth: interactive complex cloth simulation on multi-GPU systems using dynamic matrix assembly and pipelined implicit integrators[J]. ACM Transactions on Graphics, 2020, 39(6): 180: 1-180: 15.
[12]	WANG H M. GPU-based simulation of cloth wrinkles at submillimeter levels[J]. ACM Transactions on Graphics, 2021, 40(4): 169: 1-169: 14.
[13]	ZHU H M, CAO Y, JIN H, et al. Deep Fashion3D: a dataset and benchmark for 3D garment reconstruction from single images[C]// European Conference on Computer Vision. Cham: Springer, 2020: 512-530.
[14]	HE Y, XIE H R, MIYATA K. Sketch2Cloth: sketch-based 3D garment generation with unsigned distance fields[EB/OL]. [2023-02-10]. http://arxiv.org/abs/2303.00167.pdf.
[15]	BERTICHE H, MADADI M, ESCALERA S. PBNS: physically based neural simulator for unsupervised garment pose space deformation[EB/OL]. [2023-01-13]. http://arxiv.org/abs/2012.11310.pdf.
[16]	PAN X Y, MAI J M, JIANG X W, et al. Predicting loose-fitting garment deformations using bone-driven motion networks[C]// ACM SIGGRAPH 2022 Conference Proceedings. New York: ACM, 2022: 1-10.
[17]	LI Y D, TANG M, YANG Y, et al. N-cloth: predicting 3D cloth deformation with mesh-based networks[J]. Computer Graphics Forum, 2022, 41(2): 547-558.
[18]	MA Q L, YANG J L, BLACK M J, et al. Neural point-based shape modeling of humans in challenging clothing[C]// 2022 International Conference on 3D Vision (3DV). New York: IEEE Press, 2022: 679-689.
[19]	TIWARI L, BHOWMICK B. DeepDraper: fast and accurate 3D garment draping over a 3D human body[C]// 2021 IEEE/CVF International Conference on Computer Vision Workshops. New York: IEEE Press, 2021: 1416-1426.
[20]	CHONG T, SHEN I C, UMETANI N, et al. Per garment capture and synthesis for real-time virtual try-on[C]// UIST ’21: The 34th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2021: 457-469.
[21]	WU Z H, LIN G S, TAO Q Y, et al. M2E-try on net: fashion from model to everyone[C]// The 27th ACM International Conference on Multimedia. New York: ACM, 2019: 293-301.
[22]	PETROVICH M, BLACK M J, VAROL G. Action-conditioned 3D human motion synthesis with transformer VAE[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2022: 10985-10995.
[23]	TEVET G, RAAB S, GORDON B, et al. Human motion diffusion model[EB/OL]. [2023-01-13]. http://arxiv.org/abs/2209.14916.pdf.
[24]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all You need[C]// The 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000-6010.
[25]	MAHMOOD N, GHORBANI N, TROJE N F, et al. AMASS: archive of motion capture as surface shapes[C]// 2019 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2019: 5441-5450.
[26]	VIDAURRE R, SANTESTEBAN I, GARCES E, et al. Fully convolutional graph neural networks for parametric virtual try-on[J]. Computer Graphics Forum, 2020, 39(8): 145-156.
[27]	ALLEN B, CURLESS B, POPOVIĆ Z. The space of human body shapes: reconstruction and parameterization from range scans[J]. ACM Transactions on Graphics (TOG), 2003, 22(3): 587-594.