Various quality problems would arise in the process of vintage cartoons digitization, for example, a mixture of scratches and stains, reduction of resolution, and complex noises. The enhancement of old cartoon videos is a special sub-problem of video enhancement, which is barely researched. Hence, a multi-frame recurrent alignment network for black and white cartoon video reconstruction was proposed to enhance video quality. The recurrent neural network was employed to fully exploit the temporal redundancy among the neighboring frames, and extract historical information to remove scratches and stains, thus solving the difficult problems of continuous scratches and stains. Deformable convolution was applied in a coarse-to-fine manner to frame alignment at the feature level, which improved the capability of extracting the related inter-frame information in large motion scenes. The pyramid network with residual dense connections on multiple scales was introduced as the basic network unit to facilitate information aggregation. Experiments were conducted on multiple real vintage cartoon datasets and degraded datasets, which validated the performance of the proposed method. Meanwhile, such objective evaluation metrics as peak signal-to-noise ratio (PSNR) was adopted to measure the quality of the reconstructed cartoons. The test data confirms that the enhancing network can fully exploit the temporal redundancy among neighboring frames and quickly remove scratches and spots. The comparative experimental results show that our method outperforms several state-of-the-art approaches. The subjective experimental results demonstrate that the reconstructed cartoons can meet the needs of modern visual quality.