Abstract: The RGB-D images add the Depth information with the given RGB information of the scene, which can effectively describe the color and three-dimensional geometric information of the scene. With the integration of the characteristics of RGB image and Depth image, this paper proposed a reverse fusion instance segmentation algorithm that reversely merged high-level semantic features to low-level edge detail features. In order to achieve RGB-D reverse fusion instance segmentation, this method extracted RGB and depth image features separately using feature pyramid networks (FPN) of different depths, upsampling high-level features to the same size as the bottom-level features. Then reverse fusion was utilized to fuse the high-level features to the low-level, and at the same time mask optimization structurewas introduced to mask branch. The experimental results show that the proposed reverse fusion feature model can produce more excellent results in the research on RGB-D instance segmentation, effectively fusing two different feature image features of Depth image and color image. On the basis of ResNet-101 serving as the backbone network, compared with mask R-CNN without depth information, the average accuracy was increased by 10.6%, and that of the two features was increased by 4.5% with the direct forward fusion. 

Key words: Depth images, instance segmentation, feature fusion, reverse fusion, mask refinement