Abstract: Audio driven cross-modal visual generation algorithms have been widely employed in many fields, and
have gained attention from industry and academia in recent years. Audio and vision are the most important and
common modalities in people’s daily life. However, it has been a great challenge to creatively generate a visual scene
corresponding to the audio. The existing literature has not systematically and comprehensively studied the topic of
audio driven cross-modal visual generation. This paper summarized the existing algorithms for audio-driven
cross-modal visual generation and divided them into three categories: audio to image, audio to body motion video, and
audio to talking face video. For each category, we first described the fields of its specific applications and processes of
mainstream algorithms, and analyzed the framework technologies involved. Then the core contents, advantages, and
disadvantages of related algorithms were described according to the order of technology advancement, and their generation and performance effects were explained. Finally, the opportunities and challenges in the current field were
discussed and the future research suggestions were provided.

Key words: cross-modal generation, audio, vision, deep learning, review