The key to achieving point cloud face recognition is discriminative feature extraction and noise robustness for low quality data. To address the problems that the existing lightweight point cloud face recognition algorithms cannot adequately extract discriminative features and that the large amount of noise in the dataset affects model training, we designed a lightweight and efficient network model and proposed a point cloud face recognition algorithm based on multi-scale attention fusion and noise-resistant adaptive loss function. Firstly, the features of receptive fields of different sizes were generalized. Then, the multi-scale attention features were extracted, and high-level attention weights were utilized to guide the generation of low-level attention weights. Finally, channel fusion was performed to obtain multi-scale fusion features, which improved the model’s ability to capture face details. Meanwhile, according to the noise information characteristics of low-quality point cloud face images, a novel anti-noise adaptive loss function was designed to deal with the possible negative impact of the large amount of noise in the dataset on the model training process, thus enhancing the robustness and generalization ability of the model. Experiments on open-source datasets such as Lock3Dface and KinectFaces show that the proposed method yields better performance on low-quality 3D face recognition accuracy.