In recent years, the rapid development of software and hardware technologies for acquiring and interacting with panoramic content has led to a significant increase in the number of panoramic images and videos. Immersive media with 360-degree panoramic images and videos as the main content has been widely used in the field of virtual reality and enhancement implementation. Compared with traditional 2D images and videos, panoramic images and videos can provide users with a new immersive experience. With wearable devices, users can freely watch the content from all perspectives through head movement. At present, the number of panoramic images and videos has soared, but it is usually difficult to obtain satisfactory panoramic images and videos, due to the difficulty in obtaining panoramic content and the lack of effective editing tools. Therefore, analyzing and processing panoramic content with high quality has become an increasingly important research topic in the field of virtual reality. However, both in theory and application, the analysis and processing of panoramic content face significant challenges. Despite this, there is a lack of systematic and comprehensive summaries and research on the key issues in this field in existing literature. In order to better promote research and application in this area, a survey was provided on the recent works of scene analysis and content processing of panoramic images and videos. In terms of panoramic scene analysis, this survey reviewed the research on depth learning networks, depth recovery, importance detection, and target detection for panoramic images and videos. In terms of panoramic content processing, the survey analyzed the research on interactive browsing, stabilization and correction, and content editing of panoramic image video. Finally, the overview was summarized, with an outlook on future research trends in scene analysis and content processing of panoramic images and videos under the stereo view.