Welcome to Journal of Graphics share: 

Journal of Graphics ›› 2022, Vol. 43 ›› Issue (2): 316-323.DOI: 10.11996/JG.j.2095-302X.2022020316

• Computer Graphics and Virtual Reality • Previous Articles     Next Articles

Two-stage adjustable perceptual distillation network for virtual try-on

  

  1. School of Mathematical Sciences, Dalian University of Technology, Dalian Liaoning 116024, China
  • Online:2022-04-30 Published:2022-05-07
  • Supported by:
    National Natural Science Foundation of China (61976040)

Abstract: It is known that image-based virtual try-on can fit a target garment image to a person image, and that this
task has gained much attention in recent years for its wide applications in e-commerce and fashion image editing. In
response to the characteristics of the task and the shortcomings of existing approaches, a method of two-stage
adjustable perceptual distillation (TS-APD) was proposed in this paper. This method consisted of 3 steps. Firstly, two
semantic segmentation networks were pre-trained on garment image and person image respectively, thus generating
more accurate garment foreground segmentation and upper garment segmentation. Then, these two semantic
segmentations and other parsing information were employed to train a parser-based “tutor” network. Finally, a
parser-free “student” network was trained through a two-stage adjustable perceptual distillation scheme, taking the
fake image generated by the “tutor” network as input and the original real person images as supervision. It can be
perceived that the “student” model with distillation is able to produce high-quality try-on images without human
parsing. The experimental results on VITON datasets show that this algorithm can achieve 9.10 FID score, 0.015 3 L 1
score, and 0.985 6 PCKh score, outperforming the existing methods. The user survey also shows that compared with
other methods, the images generated by the proposed method are more photo-realistic, with all the preference scores reaching more than 77%.

Key words: virtual try-on, knowledge distillation, image segmentation, image generation, adjustable factor

CLC Number: