Unsupervised image stitching method guided by flow field confidence and anisotropic constraints

doi:10.11996/JG.j.2095-302X.2026040683

Abstract

Abstract:

To address the non-rigid deformation challenges faced by existing unsupervised deep image stitching methods in large-parallax scenes, an unsupervised image stitching approach guided by flow-field confidence and constrained by anisotropic geometric structure was proposed. The goal was to mitigate the conflict between alignment accuracy and structural integrity in deep image stitching and to enable image stitching in large-parallax scenes. First, a Flow Confidence Estimation Module (FCEM) was designed to assign confidence weights to the dense correspondence flow field generated by the context-aware layer. This module guided the network to prioritize reliable displacement signals in texture-rich regions while actively suppressing mismatches in low-confidence areas, thereby enhancing the reliability of mesh deformation. Second, an Anisotropic Geometric Structure Constraint (AGSC) was proposed, which constructed adaptive weights using Sobel edge responses and decoupled the traditional isotropic smoothness constraint into a mesh-smoothing term and a vertical-curvature term. By applying curvature penalties in the vertical direction to preserve straight-line structures while allowing horizontal deformation flexibility, the method achieves a balance between alignment accuracy and structural constraints. Experimental results on the UDIS-D dataset demonstrated that, while maintaining alignment performance comparable to that of mainstream deep stitching methods, the proposed method reduced the Line-fitting Root Mean Square Error (LRMSE) by 28.1% and the collinearity error (Eerr) by 33.4% across the full test set, effectively alleviating vertical structural bending in stitched images. Additional experiments on a 147-pair multi-scene image stitching dataset showed reductions in LRMSE and Eerr of 5.8% and 27.0%, respectively, compared to the baseline model. Both objective metrics and visual stitching results confirm the superiority of the proposed method in preserving vertical structures, verifying its geometric structure-preserving capability and cross-scene generalization performance.

Key words: image stitching, mesh deformation, flow confidence estimation, anisotropic geometric structure constraint, geometric structure preservation

CLC Number:

TIAN Shuo, HUANG Yan, QI Jiawang, SHI Chaojun, QI Yincheng. Unsupervised image stitching method guided by flow field confidence and anisotropic constraints[J]. Journal of Graphics, 2026, 47(4): 683-694.

Figures/Tables 11

References 35

[1]	杨利春, 田彬, 党建武. 基于深度学习的图像拼接算法研究综述[J]. 计算机应用研究, 2024, 41(7): 1930-1939.
	YANG L C, TIAN B, DANG J W. Survey on image stitching algorithm based on deep learning[J]. Application Research of Computers, 2024, 41(7): 1930-1939 (in Chinese).
[2]	HARTLEY R, ZISSERMAN A. Multiple view geometry in computer vision[M]. 2nd ed. Cambridge: Cambridge University Press, 2003: 325-343.
[3]	陈殷齐, 郑慧诚, 严志伟, 等. 基于单应性扩散约束的二步网格优化视差图像对齐[J]. 自动化学报, 2024, 50(6): 1129-1142.
	CHEN Y Q, ZHENG H C, YAN Z W, et al. Parallax image alignment with two-stage mesh optimization based on homography diffusion constraints[J]. Acta Automatica Sinica, 2024, 50(6): 1129-1142 (in Chinese).
[4]	刘海桥, 刘萌, 龚子超, 等. 基于深度学习的图像匹配方法综述[J]. 航空学报, 2024, 45(3): 028796.
	LIU H Q, LIU M, GONG Z C, et al. A review of image matching methods based on deep learning[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(3): 028796 (in Chinese).
[5]	杨志林, 尹勇, 张如凯, 等. 面向无人船视觉感知的几何结构保护视差图像拼接[J]. 系统仿真学报, 2024, 36(8): 1749-1763. DOI
	YANG Z L, YIN Y, ZHANG R K, et al. Parallax-tolerant image stitching with geometric structure protection for unmanned ship visual perception[J]. Journal of System Simulation, 2024, 36(8): 1749-1763 (in Chinese). DOI
[6]	NIE L, LIN C Y, LIAO K, et al. Parallax-tolerant unsupervised deep image stitching[C]// 2023 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2023: 7365-7374.
[7]	蒋敏, 蒋品群, 宋树祥, 等. 基于点线配准和超像素分割的图像拼接算法[J]. 计算机应用与软件, 2023, 40(10): 250-254, 267.
	JIANG M, JIANG P Q, SONG S X, et al. Image stitching algorithm based on point-line registration and superpixel segmentation[J]. Computer Applications and Software, 2023, 40(10): 250-254, 267 (in Chinese).
[8]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[9]	LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110. DOI
[10]	FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395. DOI URL
[11]	ZARAGOZA J, CHIN T J, BROWN M S, et al. As-projective-as-possible image stitching with moving DLT[C]// 2013 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2013: 2339-2346.
[12]	LIN W Y, LIU S Y, MATSUSHITA Y, et al. Smoothly varying affine stitching[C]// 2011 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2011: 345-352.
[13]	LIN C C, PANKANTI S U, RAMAMURTHY K N, et al. Adaptive as-natural-as-possible image stitching[C]// 2015 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2015: 1155-1163.
[14]	LI J, WANG Z M, LAI S M, et al. Parallax-tolerant image stitching based on robust elastic warping[J]. IEEE Transactions on Multimedia, 2018, 20(7): 1672-1687. DOI URL
[15]	LIAO T L, LI N. Single-perspective warps in natural image stitching[J]. IEEE Transactions on Image Processing, 2020, 29: 724-735. DOI URL
[16]	MA S K, LI X H, LIU K W, et al. Research on image stitching algorithm based on point-line consistency and local edge feature constraints[J]. Entropy, 2024, 26(1): 61. DOI URL
[17]	WANG Y, SHENG S, LIANG Z Y, et al. Image stitching strategy based on point-line feature fusion for homography estimation and seam-driven integration[C]// SPIE 13646: International Conference on Pattern Recognition and Image Analysis. Bellingham: SPIE, 2025: 136460M.
[18]	DETONE D, MALISIEWICZ T, RABINOVICH A. Deep image homography estimation[EB/OL]. (2016-06-13) [2026-01-08]. https://arxiv.org/pdf/1606.03798.pdf.
[19]	NIE L, LIN C Y, LIAO K, et al. Unsupervised deep image stitching: reconstructing stitched features to images[J]. IEEE Transactions on Image Processing, 2021, 30: 6184-6197. DOI URL
[20]	KIM M, LEE Y, HAN W K, et al. Learning residual elastic warps for image stitching under dirichlet boundary condition[C]// 2024 IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2024: 4016-4024.
[21]	JIANG Z Y, YAN R H, ZHANG Z X, et al. Depth-supervised fusion network for seamless-free image stitching[EB/OL]. [2026-01-25]. https://proceedings.neurips.cc/paper_files/paper/2025/hash/bcf5b917751e92c941d03512ae75dd03-Abstract-Conference.html.
[22]	QIU L W, LIU C, LI G Z, et al. Unsupervised deep image stitching for remote sensing: aligning features across large-scale geometric distortions[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2025, 18: 25305-25324. DOI URL
[23]	CHANG C H, SATO Y, CHUANG Y Y. Shape-preserving half-projective warps for image stitching[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2014: 3254-3261.
[24]	CHEN Y S, CHUANG Y Y. Natural image stitching with the global similarity prior[C]// The 14th European Conference on Computer Vision. Cham: Springer, 2016: 186-201.
[25]	JIA Q, LI Z J, FAN X, et al. Leveraging line-point consistence to preserve structures for wide parallax image stitching[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 12181-12190.
[26]	DU P, NING J F, CUI J G, et al. Geometric structure preserving warp for natural image stitching[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 3678-3686.
[27]	CAI W, YANG W. Object-level geometric structure preserving for natural image stitching[C]// 2025 AAAI conference on artificial intelligence. Washington DC: AAAI Press, 2025, 39(2): 1926-1934.
[28]	JIN H Z, NIE L, LIN C Y, et al. PixelStitch: structure- preserving pixel-wise bidirectional warps for unsupervised image stitching[C]// 2025 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2025: 28125-28134.
[29]	NIE L, LIN C Y, LIAO K, et al. Depth-aware multi-grid deep homography estimation with contextual correlation[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(7): 4460-4472. DOI URL
[30]	BOOKSTEIN F L. Principal warps: thin-plate splines and the decomposition of deformations[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1989, 11(6): 567-585. DOI URL
[31]	JÄHNE B. Digital image processing[M]. 6th ed. Berlin, Heidelberg: Springer, 2005: 331-357.
[32]	KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2026-01-25]. https://dblp.org/db/conf/iclr/iclr2015.html#conf/iclr/KingmaB14.
[33]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. DOI PMID
[34]	VON GIOI R G, JAKUBOWICZ J, MOREL J M, et al. LSD: a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(4): 722-732. DOI PMID
[35]	NIE L, MEI Y, LIAO K, et al. Robust image stitching with optimal plane[EB/OL]. [2026-01-25]. https://ieeexplore.ieee.org/document/11387056.

方法	PSNR				SSIM
方法	简单	中等	困难	平均值	简单	中等	困难	平均值
SIFT+RANSAC	28.57	24.08	18.55	23.27	0.916	0.833	0.636	0.779
APAP	27.96	24.39	20.21	23.79	0.901	0.837	0.682	0.794
LPC	26.94	22.63	19.31	22.59	0.878	0.764	0.610	0.736
UDIS	25.16	20.96	18.36	21.17	0.834	0.669	0.495	0.648
UDIS++	30.19	25.84	21.57	25.43	0.933	0.875	0.739	0.838
DSFN	30.20	25.85	21.65	25.46	0.933	0.875	0.742	0.838
本文方法	29.95	25.54	21.21	25.12	0.931	0.871	0.732	0.831

方法	PSNR				SSIM
方法	简单	中等	困难	平均值	简单	中等	困难	平均值
SIFT+RANSAC	28.57	24.08	18.55	23.27	0.916	0.833	0.636	0.779
APAP	27.96	24.39	20.21	23.79	0.901	0.837	0.682	0.794
LPC	26.94	22.63	19.31	22.59	0.878	0.764	0.610	0.736
UDIS	25.16	20.96	18.36	21.17	0.834	0.669	0.495	0.648
UDIS++	30.19	25.84	21.57	25.43	0.933	0.875	0.739	0.838
DSFN	30.20	25.85	21.65	25.46	0.933	0.875	0.742	0.838
本文方法	29.95	25.54	21.21	25.12	0.931	0.871	0.732	0.831

方法	PSNR 均值	SSIM 均值	LRMSE
方法	PSNR 均值	SSIM 均值	简单	中等	困难	均值
UDIS++	25.12	0.830	0.25	0.28	0.34	0.30
本文方法	25.12	0.831	0.20	0.24	0.31	0.26

方法	PSNR 均值	SSIM 均值	LRMSE
方法	PSNR 均值	SSIM 均值	简单	中等	困难	均值
UDIS++	25.12	0.830	0.25	0.28	0.34	0.30
本文方法	25.12	0.831	0.20	0.24	0.31	0.26

方法	模块		指标
方法	FCEM	AGSC	PSNR	SSIM	LRMSE
UDIS++	×	×	25.43	0.836	0.359
UDIS++	√	×	25.39	0.837	0.308
UDIS++	×	√	25.06	0.826	0.296
本文方法	√	√	25.12	0.831	0.258