Images are susceptible to external lighting conditions or camera parameters, resulting in overall darkness and poor visualization, which can degrade the performance of downstream vision tasks and thus lead to security issues. In this paper, a contrastive learning-based unpaired low-light image enhancement method termed Low-FaceNet was proposed for face recognition tasks. The backbone of Low-FaceNet was in the form of an image enhancement network based on the U-Net structure, introducing three sub-networks, i.e., feature retention network, semantic segmentation network, and face recognition network, thereby assisting the training of the image enhancement network. The contrastive learning paradigm enabled a large number of real-world unpaired low-light and normal-light images to be used as negative/positive samples, improving the generalization ability of the proposed model in the wild scenarios. The incorporation of high-level semantic information could guide the low-level image enhancement network to enhance images with higher quality. In addition, the task-driven approach made it possible to enhance images and improve the accuracy of face recognition simultaneously. Validated on several publicly available datasets, both visualization and quantification results show that Low-FaceNet can effectively improve the accuracy of face recognition under low-light conditions by enhancing the brightness of images while maintaining various detailed features of the images.