欢迎访问《图学学报》 分享到：
图学学报

图学学报

• 专论：第16届媒体智能与大数据计算会议（CIDE & DEA 2019 大连） • 上一篇    下一篇

基于深度学习中间层卷积特征的图像标注

  

  1. (1. 大连民族大学计算机科学与工程学院，辽宁 大连 116600； 2. 安迅达盛医疗科技有限公司，北京 100020)
  • 出版日期:2019-10-31 发布日期:2019-11-06
  • 基金资助:
    国家自然科学基金项目(61300089)；辽宁省自然科学基金项目(201602199，2019-ZD-0182)；辽宁省高等学校创新人才支持计划项目 (LR2016071)

Image Annotation Based on Middle-Layer Convolution  Features of Deep Learning

  1. (1. College of Computer Science and Engineering, Dalian Nationalities University, Dalian Liaoning 116600, China; 
    2. Anxundasheng Medical Technology Company, Beijing 100020, China)
  • Online:2019-10-31 Published:2019-11-06

PDF (PC)

126

摘要： 针对基于深度特征的图像标注模型训练复杂、时空开销大的不足，提出一种由深 度学习中间层特征表示图像视觉特征、由正例样本均值向量表示语义概念的图像标注方法。首 先，通过预训练深度学习模型的中间层直接输出卷积结果作为低层视觉特征，并采用稀疏编码 方式表示图像；然后，采用正例均值向量法为每个文本词汇构造视觉特征向量，从而构造出文 本词汇的视觉特征向量库；最后，计算测试图像与所有文本词汇的视觉特征向量相似度，并取 相似度最大的若干词汇作为标注词。多个数据集上的实验证明了所提出方法的有效性，就 F1 值而言，该方法在 IAPR TC-12 数据集上的标注性能比采用端到端深度特征的 2PKNN 和 JEC 分 别提高 32%和 60%。

关键词: 深度学习, 图像标注, 卷积, 正例均值向量, 特征向量

Abstract: Image annotation based on deep features always requires complex model training and huge space-time cost. To overcome these shortcomings, an efficient and effective approach was proposed, whose visual feature was described by middle-level features of deep learning and semantic concept was represented by mean vector of positive samples. Firstly, the convolution result is directly outputted as the low-level visual feature by the middle layer of the pre-training deep learning model, and the sparse coding method was used to represent image. Then, visual feature vector was constructed for each textual word by the mean vector method of positive samples, and the visual feature vector database of the text vocabulary was constructed. Finally, the similarities of visual feature vectors between test image and all textual words were computed, and some words with largest similarities were selected as annotation words. The experimental results on several datasets demonstrate the effectiveness of the proposed method. In terms of F1-measure, the experimental results on IAPR TC-12 dataset show that the performance of the proposed method was improved by 32% and 60% respectively, compared to 2PKNN and JEC with end-to-end deep features.

Key words: deep learning, image annotation, convolution, mean vector of positive sample, feature vector