Survey of methods for scene analysis and content processing in panoramic images and videos

doi:10.11996/JG.j.2095-302X.2023040640

Abstract

Abstract:

In recent years, the rapid development of software and hardware technologies for acquiring and interacting with panoramic content has led to a significant increase in the number of panoramic images and videos. Immersive media with 360-degree panoramic images and videos as the main content has been widely used in the field of virtual reality and enhancement implementation. Compared with traditional 2D images and videos, panoramic images and videos can provide users with a new immersive experience. With wearable devices, users can freely watch the content from all perspectives through head movement. At present, the number of panoramic images and videos has soared, but it is usually difficult to obtain satisfactory panoramic images and videos, due to the difficulty in obtaining panoramic content and the lack of effective editing tools. Therefore, analyzing and processing panoramic content with high quality has become an increasingly important research topic in the field of virtual reality. However, both in theory and application, the analysis and processing of panoramic content face significant challenges. Despite this, there is a lack of systematic and comprehensive summaries and research on the key issues in this field in existing literature. In order to better promote research and application in this area, a survey was provided on the recent works of scene analysis and content processing of panoramic images and videos. In terms of panoramic scene analysis, this survey reviewed the research on depth learning networks, depth recovery, importance detection, and target detection for panoramic images and videos. In terms of panoramic content processing, the survey analyzed the research on interactive browsing, stabilization and correction, and content editing of panoramic image video. Finally, the overview was summarized, with an outlook on future research trends in scene analysis and content processing of panoramic images and videos under the stereo view.

Key words: virtual reality, panoramic images and videos, scene analysis, content processing, stereoscopic views

CLC Number:

TP391

XIE Hong-xia, HU Yu-ning, ZHANG Yun, WANG Ya-qi, DU Hui, QIN Ai-hong. Survey of methods for scene analysis and content processing in panoramic images and videos[J]. Journal of Graphics, 2023, 44(4): 640-657.

Figures/Tables 18

Fig. 1 User experience renderings of latest wearable devices[1-2] and different planar projections of 360° panoramas

Table 1 Summary of research work on scene analysis of panoramic images and videos

研究大类	研究内容	代表文献	主要特点
1全景图像视频的场景分析	1.1全景图像视频的深度学习网络	文献[14-15]	基于立方体投影表示
		文献[16-17]	基于球面的投影表示
		文献[18]	基于等矩形的投影表示
	1.2场景深度恢复	文献[24-26]	单目深度恢复——基于卷积神经网络
		文献[33-34]	双目深度恢复——基于全局优化的方法
		文献[35-38]	双目深度恢复——基于深度学习的方法
	1.3场景重要性检测	文献[45-46]	全景图像重要性检测——基于平面投影
		文献[48]	全景图像重要性检测——基于球面
		文献[51]	全景视频重要性检测——基于立方体填充
		文献[52-53]	全景视频重要性检测——基于球面卷积
		文献[55]	全景视频重要性检测——基于视点预测
	1.4场景目标检测	文献[60-61]	基于单一投影的方法
		文献[62-63]	基于视野并交集的方法
		文献[64-65]	基于多种投影的方法

Fig. 2 Visualization of distortions in 360° panoramas[9] (a) Spherical 360° image; (b) The equirectangular image with Tissot’s indicatrices)

Fig. 3 CNN based on cube map, sphere and equirectangular representations ((a) Cubmap representation[14]; (b) SpherePHD representation[16]; (c) Equirectangular representation[18])

Fig. 4 The sampling grid is deformed according to the image distortion model, so that the receptive field is rectified[4] ((a) Input feature map; (b) Output feature map)

Fig. 5 The tangent-patch is proposed to remove panoramic distortions, and the token flows force the token positions to fit the structure of the sofa better[26]

Fig. 6 360° stereo depth estimation network[38] ((a) Two-branch feature extractor; (b) The ASPP module to enlarge the receptive field; (c) The learnable cost volume to account for the nonlinear spherical projection)

Fig. 7 Flowchart of saliency detection of 360° images[47]

Fig. 8 Network for the saliency prediction of 360° images[55]

Fig. 9 Two existing strategies for applying CNNs to 360° images[5]

Table 2 Summary of research work on content processing of panoramic images and videos

研究大类	研究内容	代表文献	主要特点
2全景图像视频的内容处理	2.1交互式浏览	文献[72-73]	自动视频导航
		文献[75-77]	交互式和可视化浏览
	2.2去抖和校正	文献[78]	自动全景图像校正
		文献[79-80]	全景视频去抖
		文献[81]	全景视频去抖+校正
	2.3内容编辑	文献[83-85]	全景内容补全
		文献[87-88]	全景克隆与颜色编辑
		文献[89-90]	内容增强——全景图超分辨率

Fig. 10 Interactive and automatic 360° video navigation [73] ((a) 360° video; (b) Normal field-of-view (NFoV) video converted from (a); (c) Camera path for converting (a) to (b))

Fig. 11 Collaborative navigation of 360° videos[77]

Fig. 12 Hand-held 360° camera, panoramic video, and the tracked motion before and after stabilization[79] ((a) Hand-held 360° cameras; (b) Full-spherical 360°×180° video; (c) Original motion tracks; (d) Pure rotation stabilization; (e) Deformed rotation stabilization)

Fig. 13 Viewport-aware inpainting based on the ERP image[85]

Fig. 14 Flowchart of 360° video completion[86]

Fig. 15 360° panorama edit propagation based on the RBF interpolation[88] ((a) Input image; (b) Color quantization; (c) Adaptive sampling; (d) RBF interpolation; (e) Result rendering; (f) Spherical & Zoom-in views)

Fig. 16 Overall framework of the proposed SphereSR[91]

References 94

[1]	HTC. VIVE XR Elite[EB/OL]. (2022-12-10]) [2023-01-02]. https://www.vive.com/uk/product/vive-xr-elite/overview/.
[2]	Meta. Quest Pro[EB/OL]. (2022-11-10) [2023-01-02]. https://www.meta.com/gb/quest/quest-pro/.
[3]	SHEN Z J, LIN C Y, NIE L, et al. Distortion-tolerant monocular depth estimation on omnidirectional images using dual-cubemap[C]// 2021 IEEE International Conference on Multimedia and Expo. New York: IEEE Press, 2021: 1-6.
[4]	TATENO K, NAVAB N, TOMBARI F. Distortion-aware convolutional filters for dense prediction in panoramic images[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 732-750.
[5]	SU Y C, GRAUMAN K. Learning spherical convolution for fast features from 360° imagery[EB/OL]. [2022-12-02]. https://arxiv.org/abs/1708.00919.
[6]	COORS B, CONDURACHE A P, GEIGER A. SphereNet: learning spherical representations for detection and classification in omnidirectional images[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 525-541.
[7]	WANG M, LI Y J, ZHANG W X, et al. Transitioning360: content-aware NFoV virtual camera paths for 360° video playback[C]// 2020 IEEE International Symposium on Mixed and Augmented Reality. New York: IEEE Press, 2020: 185-194.
[8]	LI Y J, SHI J C, ZHANG F L, et al. Bullet comments for 360° video[C]// 2022 IEEE Conference on Virtual Reality and 3D User Interfaces. New York: IEEE Press, 2022: 1-10.
[9]	ZHANG Y, ZHANG F L, LAI Y K, et al. Efficient propagation of sparse edits on 360^∘ panoramas[J]. Computers & Graphics, 2021, 96: 61-70. DOI URL
[10]	WANG W G, LAI Q X, FU H Z, et al. Salient object detection in the deep learning era: an In-depth survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(6): 3239-3259. DOI URL
[11]	BAI S J, GENG Z Y, SAVANI Y, et al. Deep equilibrium optical flow estimation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 610-620.
[12]	GUO M H, LU C Z, HOU Q B, et al. SegNeXt: rethinking convolutional attention design for semantic segmentation[EB/OL]. [2022-12-02]. https://arxiv.org/abs/2209.08575.
[13]	WANG W G, SHEN J B, XIE J W, et al. Revisiting video saliency prediction in the deep learning era[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(1): 220-237. DOI URL
[14]	XIONG B, GRAUMAN K. Snap angle prediction for 360° panoramas[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-20.
[15]	WOO HAN S, YOUNG SUH D. A 360-degree panoramic image inpainting network using a cube map[J]. Computers, Materials & Continua, 2020, 66(1): 213-228.
[16]	LEE Y, JEONG J, YUN J, et al. SpherePHD: applying CNNs on a spherical PolyHeDron representation of 360° images[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9173-9181.
[17]	EDER M, SHVETS M, LIM J, et al. Tangent images for mitigating spherical distortion[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 12423-12431.
[18]	SU Y C, GRAUMAN K. Kernel transformer networks for compact spherical convolution[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 9434-9443.
[19]	BHOI A. Monocular depth estimation: a survey[EB/OL]. (2019-01-27) [2022-12-23]. https://arxiv.org/abs/1901.09402.
[20]	ZHANG Y R, GONG M G, LI J Z, et al. Self-supervised monocular depth estimation with multiscale perception[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2022, 31: 3251-3266. DOI URL
[21]	LEI Z Y, WANG Y, LI Z J, et al. Attention based multilayer feature fusion convolutional neural network for unsupervised monocular depth estimation[J]. Neurocomputing, 2021, 423: 343-352. DOI URL
[22]	MING Y, MENG X Y, FAN C X, et al. Deep learning for monocular depth estimation: a review[J]. Neurocomputing, 2021, 438: 14-33. DOI URL
[23]	WANG F E, YEH Y H, SUN M, et al. BiFuse:monocular 360 depth estimation via Bi-projection fusion[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 459-468.
[24]	JIN L, XU Y Y, ZHENG J, et al. Geometric structure based and regularized depth estimation from 360 indoor imagery[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 886-895.
[25]	PINTORE G, AGUS M, ALMANSA E, et al. SliceNet: deep dense depth estimation from a single indoor panorama using a slice-based representation[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 11531-11540.
[26]	SHEN Z J, LIN C Y, LIAO K, et al. PanoFormer: panorama transformer for indoor 360° depth estimation[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2022: 195-211.
[27]	LI M, WANG S B, YUAN W H, et al. S²Net: accurate panorama depth estimation on spherical surface[J]. IEEE Robotics and Automation Letters, 2023, 8(2): 1053-1060. DOI URL
[28]	REY-AREA M, YUAN M Z, RICHARDT C. 360MonoDepth: high-resolution 360° monocular depth estimation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 3752-3762.
[29]	PENG C H, ZHANG J Y. High-resolution depth estimation for 360° panoramas through perspective and panoramic depth images registration[C]// 2023 IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2023: 3115-3124.
[30]	WANG L, YANG R G. Global stereo matching leveraged by sparse ground control points[C]// 2011 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2011: 3033-3040.
[31]	YUAN W M, MENG C, TONG X Y, et al. Efficient local stereo matching algorithm based on fast gradient domain guided image filtering[J]. Signal Processing: Image Communication, 2021, 95: 116280. DOI URL
[32]	YANG Q X. Hardware-efficient bilateral filtering for stereo matching[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(5): 1026-1032. DOI PMID
[33]	LI S G. Binocular spherical stereo[J]. IEEE Transactions on Intelligent Transportation Systems, 2008, 9(4): 589-600. DOI URL
[34]	KIM H, HILTON A. 3D scene reconstruction from multiple spherical stereo pairs[J]. International Journal of Computer Vision, 2013, 104(1): 94-116. DOI URL
[35]	ŽBONTAR J, LECUN Y. Computing the stereo matching cost with a convolutional neural network[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 1592-1599.
[36]	CHEN B L, JUNG C. Patch-based stereo matching using 3D convolutional neural networks[C]// The 25th IEEE International Conference on Image Processing. New York: IEEE Press, 2018: 3633-3637.
[37]	CHANG J R, CHEN Y S. Pyramid stereo matching network[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 5410-5418.
[38]	WANG N H, SOLARTE B, TSAI Y H, et al. 360SD-net: 360° stereo depth estimation with learnable cost volume[C]// 2020 IEEE International Conference on Robotics and Automation. New York: IEEE Press, 2020: 582-588.
[39]	WEGNER K, STANKIEWICZ O, GRAJEK T, et al. Depth estimation from stereoscopic 360-degree video[C]// The 25th IEEE International Conference on Image Processing. New York: IEEE Press, 2018: 2945-2948.
[40]	BORJI A, CHENG M M, HOU Q B, et al. Salient object detection: a survey[J]. Computational Visual Media, 2019, 5(2): 117-150. DOI
[41]	WU Y H, LIU Y, ZHANG L, et al. EDN: salient object detection via extremely-downsampled network[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2022, 31: 3125-3136. DOI URL
[42]	张璐. 基于深度特征融合的显著目标检测算法研究[D]. 大连: 大连理工大学, 2021.
	ZHANG L. Research on salient object detection algorithms based on deep feature integration[D]. Dalian: Dalian University of Technology, 2021 (in Chinese).
[43]	WANG W G, SHEN J B, XIE J W, et al. Revisiting video saliency prediction in the deep learning era[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(1): 220-237. DOI URL
[44]	LIU J J, LIU Z A, PENG P, et al. Rethinking the U-shape structure for salient object detection[J]. IEEE Transactions on Image Processing: a Publication of the IEEE Signal Processing Society, 2021, 30: 9030-9042. DOI URL
[45]	MONROY R, LUTZ S, CHALASANI T, et al. SalNet360: Saliency maps for omni-directional images with CNN[J]. Signal Processing: Image Communication, 2018, 69: 26-34. DOI URL
[46]	LI J, SU J M, XIA C Q, et al. Distortion-adaptive salient object detection in 360° omnidirectional images[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(1): 38-48. DOI URL
[47]	MA G X, LI S, CHEN C, et al. Stage-wise salient object detection in 360° omnidirectional image via object-level semantical saliency ranking[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(12): 3535-3545. DOI URL
[48]	LV H R, YANG Q, LI C L, et al. SalGCN: saliency prediction for 360-degree images based on spherical graph convolutional networks[C]// The 28th ACM International Conference on Multimedia. New York: ACM, 2020: 682-690.
[49]	ZHANG R P, CHEN C Y, ZHANG J C, et al. 360-degree visual saliency detection based on fast-mapped convolution and adaptive equator-bias perception[J]. The Visual Computer, 2023, 39(3): 1163-1180. DOI
[50]	GAO P, CHEN X L, QUAN R, et al. MRGAN360: multi-stage recurrent generative adversarial network for 360 degree image saliency prediction[EB/OL]. [2023-01-13]. https://arxiv.org/abs/2303.08525.
[51]	CHENG H T, CHAO C H, DONG J D, et al. Cube padding for weakly-supervised saliency prediction in 360° videos[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 1420-1429.
[52]	ZHANG Z H, XU Y Y, YU J Y, et al. Saliency detection in 360° videos[C]// European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 504-520.
[53]	QIAO M L, XU M, WANG Z L, et al. Viewport-dependent saliency prediction in 360° video[J]. IEEE Transactions on Multimedia, 2021, 23: 748-760. DOI URL
[54]	DU R F, VARSHNEY A. Saliency computation for virtual cinematography in 360° videos[J]. IEEE Computer Graphics and Applications, 2021, 41(4): 99-106. DOI PMID
[55]	BERNAL-BERDUN E, MARTIN D, GUTIERREZ D, et al. SST-sal: a spherical spatio-temporal approach for saliency prediction in 360° videos[J]. Computers & Graphics, 2022, 106, C: 200-209.
[56]	YUN H, LEE S H, KIM G. Panoramic vision transformer for saliency detection in 360° videos[M]// Lecture Notes in Computer Science. Cham: Springer Nature Switzerland, 2022: 422-439.
[57]	ZHANG D W, HAN J W, CHENG G, et al. Weakly supervised object localization and detection: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 5866-5885.
[58]	LAI W S, HUANG Y J, JOSHI N, et al. Semantic-driven generation of hyperlapse from 360 degree video[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(9): 2610-2621. DOI URL
[59]	XIAO J X, EHINGER K A, OLIVA A, et al. Recognizing scene viewpoint using panoramic place representation[C]// 2012 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2012: 2695-2702.
[60]	YANG W, QIAN Y, KÄMÄRÄINEN J K, et al. Object detection in equirectangular panorama[C]// The 24th International Conference on Pattern Recognition. New York: IEEE Press, 2018: 2190-2195.
[61]	WANG K H, LAI S H. Object detection in curved space for 360-degree camera[C]// 2019 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE Press, 2019: 3642-3646.
[62]	ZHAO P Y, YOU A S, ZHANG Y X, et al. Spherical criteria for fast and accurate 360° object detection[J]. The AAAI Conference on Artificial Intelligence, 2020, 34(7): 12959-12966.
[63]	CAO M, IKEHATA S, AIZAWA K. Field-of-view IoU for object detection in 360° images[EB/OL]. [2022-12-12]. https://arxiv.org/abs/2202.03176.
[64]	ZHENG Z S, LIN C Y, NIE L, et al. Bi-projection for 360° image object detection bridged by RoI searcher[J]. Journal of Visual Communication and Image Representation, 2022, 89: 103660. DOI URL
[65]	CAO M, IKEHATA S, AIZAWA K. Dual-erp representation for object detection in 360° images[C]// 2022 IEEE International Conference on Image Processing. New York: IEEE Press, 2022: 2016-2020.
[66]	WIEN M, BOYCE J M, STOCKHAMMER T, et al. Standardization status of immersive video coding[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2019, 9(1): 5-17. DOI
[67]	JPEG Requirements Subgroup. JPEG 360 metadata use cases[EB/OL]. (2018-02-02) [2023-01-02]. https://jpeg.org/downloads/jpeg360/JPEG360-use-cases.pdf.
[68]	XU M, LI C, ZHANG S Y, et al. State-of-the-art in 360° video/image processing: perception, assessment and compression[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(1): 5-26. DOI URL
[69]	GUTIÉRREZ J, DAVID E, RAI Y, et al. Toolbox and dataset for the development of saliency and scanpath models for omnidirectional/360° still images[J]. Signal Processing: Image Communication, 2018, 69: 35-42. DOI URL
[70]	CHEN J Y, LUO Z X, WANG Z L, et al. Live360: viewport-aware transmission optimization in live 360-degree video streaming[J]. IEEE Transactions on Broadcasting, 2023, 69(1): 85-96. DOI URL
[71]	HU M, WANG L F, TAN B, et al. Two-tier 360-degree video delivery control in multiuser immersive communications systems[J]. IEEE Transactions on Vehicular Technology, 2023, 72(3): 4119-4123. DOI URL
[72]	HU H N, LIN Y C, LIU M Y, et al. Deep 360 pilot: learning a deep agent for piloting through 360° sports videos[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 1396-1405.
[73]	KANG K, CHO S. Interactive and automatic navigation for 360° video playback[J]. ACM Transactions on Graphics, 2019, 38(4): 108:1-108:11.
[74]	IRFAN M, MUHAMMAD K, SAJJAD M, et al. Deepview: deep-learning-based users field of view selection in 360° videos for industrial environments[J]. IEEE Internet of Things Journal, 2023, 10(4): 2903-2912. DOI URL
[75]	PAVEL A, HARTMANN B, AGRAWALA M. Shot orientation controls for interactive cinematography with 360 video[C]// The 30th Annual ACM Symposium on User Interface Software and Technology. New York: ACM, 2017: 289-297.
[76]	WALLGRÜN J O, BAGHER M M, SAJJADI P, et al. A comparison of visual attention guiding approaches for 360° image-based VR tours[C]// 2020 IEEE Conference on Virtual Reality and 3D User Interfaces. New York: IEEE Press, 2020: 83-91.
[77]	KUMAR K, PORETSKI L, LI J N, et al. Tourgether360: collaborative exploration of 360° videos using pseudo-spatial navigation[J]. Proceedings of the ACM on Human-Computer Interaction, 2022, 6(CSCW2): 1-27.
[78]	JUNG J, KIM B, LEE J Y, et al. Robust upright adjustment of 360 spherical panoramas[J]. The Visual Computer, 2017, 33(6): 737-747. DOI URL
[79]	KOPF J. 360° video stabilization[J]. ACM Transactions on Graphics, 2016, 35(6): 1-9.
[80]	SHEN L C, HUANG T K, CHEN C S, et al. A 2.5D approach to 360 panorama video stabilization[C]// 2018 25th IEEE International Conference on Image Processing. New York: IEEE Press, 2018: 3184-3188.
[81]	TANG C Z, WANG O, LIU F, et al. Joint stabilization and direction of 360° videos[J]. ACM Transactions on Graphics, 2019, 38(2): 1-13.
[82]	CHEN Z Z, LI Y M, ZHANG Y X. Recent advances in omnidirectional video coding for virtual reality: projection and evaluation[J]. Signal Processing, 2018, 146: 66-78. DOI URL
[83]	GAO J H, BROWN M S. An interactive editing tool for correcting panoramas[C]// SA ’12: SIGGRAPH Asia 2012 Technical Briefs. New York: ACM, 2012: 31:1-31:4.
[84]	ZHU Z, MARTIN R R, HU S M. Panorama completion for street views[J]. Computational Visual Media, 2015, 1(1): 49-57. DOI URL
[85]	SHANG Z Y, LIU Y W, LI G Y, et al. Viewport-oriented panoramic image inpainting[C]// 2022 IEEE International Conference on Image Processing. New York: IEEE Press, 2022: 3031-3035.
[86]	XU B B, PATHAK S, FUJII H, et al. Spatio-temporal video completion in spherical image sequences[J]. IEEE Robotics and Automation Letters, 2017, 2(4): 2032-2039. DOI URL
[87]	ZHAO Q, WAN L, FENG W, et al. 360 panorama cloning on sphere[EB/OL]. [2022-12-12]. https://arxiv.org/abs/1709.01638.
[88]	ZHANG Y, ZHANG F L, ZHU Z, et al. Fast edit propagation for 360 degree panoramas using function interpolation[J]. IEEE Access, 2022, 10: 43882-43894. DOI URL
[89]	ZHANG Y P, ZHANG H Z, LI D J, et al. Toward real-world panoramic image enhancement[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2020: 2675-2684.
[90]	LIU H Y, RUAN Z B, FANG C W, et al. A single frame and multi-frame joint network for 360-degree panorama video super-resolution[EB/OL]. [2022-12-01]. https://arxiv.org/abs/2008.10320.
[91]	YOON Y, CHUNG I, WANG L, et al. SphereSR: 360° image super-resolution with arbitrary projection via continuous spherical image representation[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 5667-5676.
[92]	WANG M, LYU X Q, LI Y J, et al. VR content creation and exploration with deep learning: a survey[J]. Computational Visual Media, 2020, 6(1): 3-28. DOI
[93]	ISOLA P, ZHU J Y, ZHOU T H, et al. Image-to-image translation with conditional adversarial networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 5967-5976.
[94]	PARK T, LIU M Y, WANG T C, et al. Semantic image synthesis with spatially-adaptive normalization[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2020: 2332-2341.