Document image tampering region segmentation method based on multi-feature fusion

doi:10.11996/JG.j.2095-302X.2026050984

Abstract

Abstract:

With the widespread adoption of document images in critical fields such as government administration and finance, tampering with these images not only incurs economic losses but also undermines public trust, making information security essential. Meanwhile, rapid advancement in tampering techniques has escalated the challenge of authenticity verification, thereby fueling growing research interest in document image tampering detection. To enhance the localization accuracy of tampered regions in document images, a multi-feature fusion-based segmentation method for document image tampered regions was proposed, namely the Multi-Feature Fusion Network (MF-Net). Firstly, a multi-feature encoder was designed, comprising three parallel branches. The spatial branch extracted rich spatial-domain information, the frequency branch integrates multi-level wavelet transform and convolution operations to fully capture tampering-related features in the frequency domain; and the noise branch employed the Spatial Rich Model (SRM) filter bank to analyze noise characteristics, providing the model with an additional perspective. These three branches were used jointly enable comprehensive extraction of multi-view features. Secondly, a hybrid attention feature-fusion module was developed, which serially combined channel attention, spatial attention, and parameter-free attention. Through multi-level refinement involving dual-feature-guided channel weighting, spatial focusing, and parameter-free suppression, deep fusion of multi-source features was achieved. Finally, inspired by the human visual system’s mechanism of prioritizing attention to abnormal tampered regions, an attention module was incorporated into a multi-view iterative decoder. This allowed the decoder to adaptively focus on anomalous regions, thereby realizing precise localization of tampered areas. Experimental validation of the proposed method was conducted on the public TextTamper dataset. Compared with state-of-the-art methods, the proposed MF-Net achieved improvements of 2.8% and 5.2% in F1-score and IoU, respectively. The experimental results demonstrated that the proposed method effectively enhanced the overall performance of tampered region localization in document images.

Key words: tampering detection, document image, multi-feature fusion, digital image forensics, deep learning

CLC Number:

TP 391.41

ZHAO Jie, YANG Ruoquan, WU Bin. Document image tampering region segmentation method based on multi-feature fusion[J]. Journal of Graphics, 2026, 47(5): 984-993.

Figures/Tables 13

References 32

[1]	WU H W, ZHOU J T, TIAN J Y, et al. Robust image forgery detection over online social network shared images[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 13430-13439.
[2]	GUILLARO F, COZZOLINO D, SUD A, et al. TruFor: leveraging all-round clues for trustworthy image forgery detection and localization[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 20606-20615.
[3]	GUO X, LIU X H, REN Z Y, et al. Hierarchical fine-grained image forgery detection and localization[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 3155-3165.
[4]	CHEN J X, LIAO X, WANG W, et al. SNIS: a signal noise separation-based network for post-processed image forgery detection[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(2): 935-951. DOI URL
[5]	ZHU J Y, LI D, FU X Y, et al. Learning discriminative noise guidance for image forgery detection and localization[C]// The 38th AAAI Conference on Artificial Intelligence. Washington: AAAI, 2024: 7739-7747.
[6]	王裕鑫, 张博强, 谢洪涛, 等. 基于空域与频域关系建模的篡改文本图像检测[J]. 网络与信息安全学报, 2022, 8(3): 29-40.
	WANG Y X, ZHANG B Q, XIE H T, et al. Tampered text detection via RGB and frequency relationship modeling[J]. Chinese Journal of Network and Information Security, 2022, 8(3): 29-40 (in Chinese).
[7]	QU C F, LIU C Y, LIU Y L, et al. Towards robust tampered text detection in document image: new dataset and new solution[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 5937-5946.
[8]	DONG L, LIANG W P, WANG R D. Robust text image tampering localization via forgery traces enhancement and multiscale attention[J]. IEEE Transactions on Consumer Electronics, 2024, 70(1): 3495-3507. DOI URL
[9]	FERRARA P, BIANCHI T, DE ROSA A, et al. Image forgery localization via fine-grained analysis of CFA artifacts[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(5): 1566-1577. DOI URL
[10]	AMERINI I, URICCHIO T, BALLAN L, et al. Localization of JPEG double compression through multi-domain convolutional neural networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2017: 1865-1871.
[11]	ZHOU P, HAN X T, MORARIU V I, et al. Learning rich features for image manipulation detection[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2018: 1053-1061.
[12]	FRIDRICH J, KODOVSKY J. Rich models for steganalysis of digital images[J]. IEEE Transactions on Information Forensics and Security, 2012, 7(3): 868-882. DOI URL
[13]	WU Y, ABDALMAGEED W, NATARAJAN P. ManTra-Net: manipulation tracing network for detection and localization of image forgeries with anomalous features[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9535-9544.
[14]	KWON M J, YU I J, NAM S H, et al. CAT-Net: compression artifact tracing network for detection and localization of image splicing[C]// 2021 IEEE Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2021: 375-384.
[15]	ZENG N Y, WU P S, ZHANG Y Q, et al. DPMSN: a dual-pathway multiscale network for image forgery detection[J]. IEEE Transactions on Industrial Informatics, 2024, 20(5): 7665-7674. DOI URL
[16]	WU Y B, KONG X W, YOU X G, et al. Printer forensics based on page document's geometric distortion[C]// The 16th IEEE International Conference on Image Processing. New York: IEEE Press, 2009: 2909-2912.
[17]	SCHREYER M, SCHULZE C, STAHL A, et al. Intelligent printing technique recognition and photocopy detection for forensic document examination[EB/OL]. [2025-09-30]. https://dblp.uni-trier.de/db/conf/informatiktage/informatiktage2009.html#conf/informatiktage/Schreyer09.
[18]	VAN BEUSEKOM J, SHAFAIT F, BREUEL T M. Document inspection using text-line alignment[C]// The 9th IAPR International Workshop on Document Analysis Systems. New York: ACM, 2010: 263-270.
[19]	VAN BEUSEKOM J, SHAFAIT F, BREUEL T M. Text-line examination for document forgery detection[J]. International Journal on Document Analysis and Recognition, 2013, 16(2): 189-207. DOI URL
[20]	BERTRAND R, RAMOS TERRADES O, GOMEZ-KRÄMER P, et al. A conditional random field model for font forgery detection[C]// The 13th International Conference on Document Analysis and Recognition. New York: IEEE Press, 2015: 576-580.
[21]	ABRAMOVA S, BÖHME R. Detecting copy-move forgeries in scanned text documents[J]. Electronic Imaging, 2016, 2016(8): 1-9.
[22]	SIRAJUDEEN M, ANITHA R. Forgery document detection in information management system using cognitive techniques[J]. Journal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology, 2020, 39(6): 8057-8068.
[23]	REN R Y, HAO Q X, GU F, et al. EMF-Net: an edge-guided multi-feature fusion network for text manipulation detection[J]. Expert Systems with Applications, 2024, 249: 123548. DOI URL
[24]	DUAN J X, SUN H, JI F, et al. RealDTT: towards a comprehensive real-world dataset for tampered text detection[J]. International Journal of Computer Vision, 2025, 133(10): 6993-7011. DOI
[25]	GUO M H, LU C Z, HOU Q B, et al. SegNeXt: rethinking convolutional attention design for semantic segmentation[C]// The 36th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2022: 84.
[26]	MMSegmentation Contributors. MMSegmentation: OpenMMLab semantic segmentation toolbox and benchmark[EB/OL]. (2024-03-22) [2025-09-26]. https://github.com/open-mmlab/mmsegmentation
[27]	ZHUANG P Y, LI H D, TAN S Q, et al. Image tampering localization using a dense fully convolutional network[J]. IEEE Transactions on Information Forensics and Security, 2021, 16: 2986-2999. DOI URL
[28]	CHEN X R, DONG C B, JI J Q, et al. Image manipulation detection by multi-view multi-scale supervision[C]// 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 14165-14173.
[29]	ZHANG Y L, ZHU G P, WU L G, et al. Multi-task SE-network for image splicing localization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(7): 4828-4840. DOI URL
[30]	BI X L, WEI Y, XIAO B, et al. RRU-Net: the ringed residual U-net for image splicing forgery detection[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. New York: IEEE Press, 2019: 30-39.
[31]	NILOY F F, BHAUMIK K K, WOO S S. CFL-Net: image forgery localization using contrastive learning[C]//2023 IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2023: 4631-4640.
[32]	阿里云天池, 阿里安全. 真实场景篡改图像检测挑战赛[EB/OL]. (2022-02-16) [2025-10-30]. https://tianchi.aliyun.com/competition/entrance/531945/introduction.
	Alibaba Cloud Tianchi, Alibaba Security. Real-world image forgery localization challenge[EB/OL]. (2022-02-16) [2025-10-30]. https://tianchi.aliyun.com/competition/entrance/531945/introduction(in Chinese).

网络结构	评价指标
网络结构	F1	IoU	AUC	FNR
Baseline	0.633	0.619	0.835	0.307
w/o B_fre	0.675	0.674	0.857	0.266
w/o B_noise	0.680	0.676	0.862	0.257
w/o HATF	0.682	0.673	0.864	0.253
w/o MAD	0.664	0.651	0.852	0.276
MF-Net	0.689	0.685	0.869	0.244

网络结构	评价指标
网络结构	F1	IoU	AUC	FNR
Baseline	0.633	0.619	0.835	0.307
w/o B_fre	0.675	0.674	0.857	0.266
w/o B_noise	0.680	0.676	0.862	0.257
w/o HATF	0.682	0.673	0.864	0.253
w/o MAD	0.664	0.651	0.852	0.276
MF-Net	0.689	0.685	0.869	0.244

对比方法	评价指标
对比方法	F1	IoU	AUC	FNR
DFCN^[27]	0.643	0.624	0.858	0.264
MVSS-Net^[28]	0.581	0.524	0.861	0.257
SE-Net^[29]	0.632	0.608	0.857	0.266
RRU-Net^[30]	0.651	0.600	0.866	0.245
CFL-Net^[31]	0.520	0.462	0.842	0.291
FTMA-Net^[8]	0.661	0.633	0.868	0.248
MF-Net (Ours)	0.689	0.685	0.869	0.244

对比方法	评价指标
对比方法	F1	IoU	AUC	FNR
DFCN^[27]	0.643	0.624	0.858	0.264
MVSS-Net^[28]	0.581	0.524	0.861	0.257
SE-Net^[29]	0.632	0.608	0.857	0.266
RRU-Net^[30]	0.651	0.600	0.866	0.245
CFL-Net^[31]	0.520	0.462	0.842	0.291
FTMA-Net^[8]	0.661	0.633	0.868	0.248
MF-Net (Ours)	0.689	0.685	0.869	0.244

方法	参数量/M	FLOPs/G
DFCN^[27]	2.55	35.46
MVSS-Net^[28]	142.78	166.48
SE-Net^[29]	3.94	38.62
RRU-Net^[30]	4.10	69.57
CFL-Net^[31]	111.91	68.05
FTMA-Net^[8]	24.70	48.98
MF-Net (Ours)	9.27	21.53