Abstract:

The current post-training quantization methods can achieve near lossless quantization at high quantization bit-width, however, for lightweight convolutional neural networks (CNN), the quantization error remains nonnegligible, especially in the case of low bit-width quantization (<4 bits). To address this, a post-training quantization method for lightweight CNN, called the block-level BatchNorm learning (BBL) method, was proposed. Unlike current post-training quantization methods that merge the batch normalization layers, this method retained the weights of the batch normalization layer on a per-block basis, and learned the quantized model parameters and batch normalization layer parameters based on the block-level feature map reconstruction loss. It also updated the mean and variance statistics of the batch normalization layer. This method mitigated the distribution shift problem caused by low-bit quantization of lightweight CNN in a simple and effective manner. Furthermore, to reduce overfitting of the post-training quantization method to the calibration dataset, the method constructed a block-level data augmentation approach by ensuring different model blocks did not learn from the same batch of calibration data. To verify the proposed method, extensive experiments on the ImageNet dataset, demonstrated that compared with current post-training quantization algorithms, the BBL method can improve the accuracy by up to 7.72 percentage points and can effectively reduce the quantization error caused by low-bit post-training quantization of lightweight CNN.

Key words: deep neural networks compression, post-training quantization, low-bit quantization, lightweight convolutional neural networks, lightweight intelligence