一种定位分支辅助的多任务协同图像伪造检测模型

doi:10.11996/JG.j.2095-302X.2026030524

摘要/Abstract

摘要：

随着深度学习的快速发展，图像伪造技术(如生成对抗网络和扩散模型)能够生成高度逼真的图像，给传统的图像伪造检测方法带来巨大挑战。现有的图像伪造检测方法通常将伪造检测与伪造区域定位视为独立任务，导致两任务间特征信息共享不足、协同性较差，限制了方法的检测准确率与定位精度。为此，提出一种基于定位分支辅助的图像伪造检测模型，通过多任务协同优化提升图像伪造检测性能。模型核心包含2个创新模块：①双域特征增强模块，结合颜色分布异常与高频失真特征，联合捕捉伪造痕迹，解决单一特征对复杂伪造场景适应性不足的问题；②中值增强交互模块，通过多尺度特征融合与通道-空间注意力机制，增强检测与定位分支的交互学习，解决任务间信息割裂问题。此外，采用端到端优化的联合训练框架，结合多任务损失函数，实现定位协同检测的提升。在Columbia，CASIA和NIST16等基准数据集上的实验表明，该方法在检测与定位任务中均超越现有对比方法，验证了模型在复杂伪造场景下的有效性与泛化能力。

关键词: 深度学习, 图像伪造检测, 模块融合

Abstract:

With the rapid development of artificial intelligence, deepfake technology, such as generative adversarial networks and diffusion models, has advanced to generate highly realistic images, posing substantial challenges to traditional image forgery detection methods. Existing methods often treat forgery detection and forgery localization as separate tasks, which results in insufficient feature information sharing and weak synergy between the two tasks, therefore limiting the detection accuracy and localization precision. To address this issue, a novel localization-branch- assisted image forgery detection framework was proposed, and multi-task collaborative optimization was used to enhance detection performance. The proposed model included two key innovative modules. First, a Dual-Domain Feature Enhancement Module was designed to combine color distribution anomalies with high-frequency distortion features, so that forgery traces could be jointly captured and discriminative capability enhanced, addressing the limited adaptability of a single feature in complex forgery scenarios. Second, a Median-Enhanced Interaction Module was introduced to enhance interaction between the detection and localization branches through multi-scale feature fusion and a combined channel-spatial attention mechanism, effectively mitigating the task-isolation problem. An end-to-end joint training framework with a multi-task loss function was adopted to further reinforce the synergy between detection and localization tasks. Extensive experiments were conducted on datasets such as Columbia, CASIA, and NIST16, and the results showed that the proposed method outperformed existing comparison approaches in both detection and localization tasks, demonstrating the effectiveness and generalization capability in complex forgery scenarios.

Key words: deep learning, image forgery detection, module fusion

中图分类号:

李秀梅, 周正鑫, 孙军梅. 一种定位分支辅助的多任务协同图像伪造检测模型[J]. 图学学报, 2026, 47(3): 524-533.

LI Xiumei, ZHOU Zhengxin, SUN Junmei. A multi-task collaborative image forgery detection framework assisted by localization branch[J]. Journal of Graphics, 2026, 47(3): 524-533.

图/表 14

图1 定位分支和检测分支交互模型结构图

Fig. 1 Structure diagram of the localization branch and detection branch interaction model

图2 DDEM模块细节图

Fig. 2 Detailed diagram of the DDEM module

图3 MEIM实现细节图

Fig. 3 Detailed diagram of the MEIM module

表1 模型的定位性能AUC/%

Table 1 Model localization performance AUC/%

Method	Columbia	Coverage	CASIA	NIST16	IMD20
ManTra-Net	82.4	81.9	81.7	79.5	74.8
PSCC-Net	98.2	84.7	82.9	85.5	80.6
HiFi	97.1	91.2	84.2	83.3	79.8
Ours	98.8	88.9	85.3	86.3	81.2

表2 不同检测方法在CASIA数据集的AUC和F1检测结果/%

Table 2 AUC and F1 detection results of different detection methods on CASIA dataset/%

Model	AUC	F1
CNN-det	91.1	85.7
PSCC	94.7	92.2
SPAN	67.3	63.8
Utkarsh	96.1	95.4
HiFi	95.8	93.4
Ours	98.1	97.6

表3 不同检测方法在IFDL数据集的AUC和F1检测结果/%

Table 3 AUC and F1 detection results of different detection methods on IFDL dataset/%

Model	AUC	F1
CNN-det	90.1	83.7
PSCC	93.2	91.3
Utkarsh	95.6	94.5
HiFi	96.8	94.1
Ours	97.5	95.3

图4 不同分块大小下模型的检测性能

Fig. 4 Detection performance of the model under different block sizes

图5 不同类型图片定位结果

Fig. 5 Localization results on different types of images

图6 真伪检测数据((a) 真实图像；(b) IMD20数据集中的伪造图像；(c) CASIA数据集中的伪造图像)

Fig. 6 Genuine and forged detection data ((a) Genuine images; (b) Forged images from the IMD20 dataset; (c) Forged images from the CASIA dataset)

表4 在IMD20上训练的不同方法跨数据集AUC检测结果比较

Table 4 Comparison of cross-dataset AUC detection results of different methods trained on IMD20

Method	Columbia	Coverage	CASIA	NIST16
PSCC	0.713	0.796	0.757	0.702
SPAN	0.679	0.746	0.721	0.624
HiFi	0.752	0.845	0.752	0.797
Utkarsh	0.779	0.813	0.784	0.722
Ours	0.808	0.864	0.813	0.754

表5 在IFDL上训练的不同数据集AUC的检测结果比较

Table 5 Comparison of detection results of AUC on different datasets trained on IFDL

Method	Columbia	Coverage	CASIA	NIST16	IMD20
PSCC	0.818	0.789	0.821	0.732	0.701
CNN-det	0.752	0.773	0.743	0.711	0.654
HiFi	0.817	0.835	0.802	0.823	0.742
Utkarsh	0.824	0.812	0.774	0.798	0.754
Ours	0.874	0.890	0.831	0.803	0.768

表6 在IFDL上训练的不同数据集的AUC检测结果比较

Table 6 Comparison of detection results of AUC on different datasets trained on IFDL

Method	PS-Battles	MFC2019	DEFACTO
HiFi	0.823	0.787	0.805
Utkarsh	0.802	0.792	0.773
Ours	0.845	0.812	0.791

表7 DDEM模块、MEIM和损失函数消融实验

Table 7 Ablation experiments of DDEM module, MEIM, and loss function

方式	Module	Loss	AUC/%	F1/%
1	M	L_M，L_c	82.2	83.5
2	D	L_M	64.7	71.2
3	D，M	L_M	85.6	84.3
Full	D，M	L_M，L_c	97.5	95.3

图7 不同λ大小时，本文模型的AUC检测结果

Fig. 7 Detection results of our model AUC under different λ values

参考文献 23

[1]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144. DOI URL
[2]	HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[C]//The 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 574.
[3]	OJHA U, LI Y, LEE Y J. Towards universal fake image detectors that generalize across generative models[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 24480-24489.
[4]	LIU X H, LIU Y J, CHEN J, et al. PSCC-Net: progressive spatio-channel correlation network for image manipulation detection and localization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(11): 7505-7517. DOI URL
[5]	GUO X, LIU X H, REN Z Y, et al. Hierarchical fine-grained image forgery detection and localization[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 3155-3165.
[6]	GHIASI G, LIN T Y, LE Q V. NAS-FPN: learning scalable feature pyramid architecture for object detection[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 7029-7038.
[7]	PALERMO R, RHODES G. Change detection in the flicker paradigm: do faces have an advantage?[J]. Visual Cognition, 2003, 10(6): 683-713. DOI URL
[8]	VANDENHENDE S, GEORGOULIS S, GANSBEKE W V, et al. Multi-task learning for dense prediction tasks: a survey[EB/OL]. [2025-03-12]. https://arxiv.org/abs/2004.13379.
[9]	HU X F, ZHANG Z H, JIANG Z Y, et al. SPAN: spatial pyramid attention network for image manipulation localization[C]//The 16th European Conference on Computer Vision. Cham: Springer, 2020: 312-328.
[10]	ZHANG J H, CHEN J B, WANG Q C, et al. Spatiotemporal saliency detection based on maximum consistency superpixels merging for video analysis[J]. IEEE Transactions on Industrial Informatics, 2020, 16(1): 606-614. DOI URL
[11]	YAN J Z, LI Z Q, HE Z W, et al. Generalizable deepfake detection via effective local-global feature extraction[EB/OL]. [2025-07-01]. https://arxiv.org/abs/2501.15253v1.
[12]	WU Y, ABDALMAGEED W, NATARAJAN P. ManTra-Net: manipulation tracing network for detection and localization of image forgeries with anomalous features[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9535-9544.
[13]	NILOY F F, BHAUMIK K K, WOO S S. CFL-Net: image forgery localization using contrastive learning[C]//The IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2023: 4631-4640.
[14]	SHAO R, WU T X, LIU Z W. Detecting and grounding multi-modal media manipulation[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 6904-6913.
[15]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//The IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 9992-10002.
[16]	LIU Z K, ZHOU Y M, XU Y S, et al. SimpleNet: a simple network for image anomaly detection and localization[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 20402-20411.
[17]	ZHUANG P Y, LI H D, YANG R, et al. ReLoc: a restoration-assisted framework for robust image tampering localization[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 5243-5257. DOI URL
[18]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//The IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 1800-1807.
[19]	HSU Y F, CHANG S F. Detecting image splicing using geometry invariants and camera characteristics consistency[C]//2006 IEEE International Conference on Multimedia and Expo. New York: IEEE Press, 2006: 549-552.
[20]	WEN B H, ZHU Y, SUBRAMANIAN R, et al. COVERAGE—A novel database for copy-move forgery detection[C]//2016 IEEE International Conference on Image Processing. New York: IEEE Press, 2016: 161-165.
[21]	DONG J, WANG W, TAN T N. CASIA image tampering detection evaluation database[C]//2013 IEEE China Summit and International Conference on Signal and Information Processing. New York: IEEE Press, 2013: 422-426.
[22]	GUAN H Y, DELGADO A, LEE Y, et al. User guide for NIST media forensic challenge (MFC) datasets[EB/OL]. [2025-03-12]. https://doi.org/10.6028/NIST.IR.8377.
[23]	NOVOZÁMSKÝ A, MAHDIAN B, SAIC S. IMD2020: a large-scale annotated dataset tailored for detecting manipulated images[C]//The IEEE Winter Conference on Applications of Computer Vision Workshops. New York: IEEE Press, 2020: 71-80.