Welcome to Journal of Graphics share: 

Journal of Graphics ›› 2022, Vol. 43 ›› Issue (4): 624-631.DOI: 10.11996/JG.j.2095-302X.2022040624

• Image Processing and Computer Vision • Previous Articles     Next Articles

Zero-shot image classification based on random propagation graph convolution model

  

  1. 1. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou Jiangsu 221116, China;
    2. School of Information Science and Technology, University of Science and Technology of China, Hefei Anhui 230027, China
  • Online:2022-08-31 Published:2022-08-15
  • Contact: LU Nan-nan (1985), associate professor, Ph. D. Her main research interests conver pattern recognition, machine learning and image processing
  • Supported by:
    National Natural Science Foundation of China (62006233, 51734009)

Abstract:

Zero-shot image classification aims to recognize new categories, namely, unseen categories that do not appear during training. Therefore, auxiliary information is needed to model the relationship between unseen and seen categories. With the aid of knowledge graph, zero-shot classification models based on graph convolution network (GCN) can explicitly express the relationship between categories, but GCN is susceptible to over-smoothing, resulting in the degradation of model performance. To address this problem, a zero-shot classification model based on random propagation graph convolution was proposed. In this model, the raw features were processed by random propagation mechanism to achieve feature perturbation and data augmentation. The generated knowledge graph based on category hierarchy could model the semantic relationship between categories, where graph nodes stand for categories and graph edges stand for relationships. Then the GCN was constructed to train the processed features, and the classifier parameters containing unseen categories, which were the output of nodes, could achieve zero-shot classification. Experimental results show that the model can significantly decrease time consumption, and improve accuracy and generalization performance.

Key words: zero-shot image classification, knowledge graph, graph convolution network, random propagation, data augmentation

CLC Number: