A multi-task collaborative image forgery detection framework assisted by localization branch

doi:10.11996/JG.j.2095-302X.2026030524

Abstract

Abstract:

With the rapid development of artificial intelligence, deepfake technology, such as generative adversarial networks and diffusion models, has advanced to generate highly realistic images, posing substantial challenges to traditional image forgery detection methods. Existing methods often treat forgery detection and forgery localization as separate tasks, which results in insufficient feature information sharing and weak synergy between the two tasks, therefore limiting the detection accuracy and localization precision. To address this issue, a novel localization-branch- assisted image forgery detection framework was proposed, and multi-task collaborative optimization was used to enhance detection performance. The proposed model included two key innovative modules. First, a Dual-Domain Feature Enhancement Module was designed to combine color distribution anomalies with high-frequency distortion features, so that forgery traces could be jointly captured and discriminative capability enhanced, addressing the limited adaptability of a single feature in complex forgery scenarios. Second, a Median-Enhanced Interaction Module was introduced to enhance interaction between the detection and localization branches through multi-scale feature fusion and a combined channel-spatial attention mechanism, effectively mitigating the task-isolation problem. An end-to-end joint training framework with a multi-task loss function was adopted to further reinforce the synergy between detection and localization tasks. Extensive experiments were conducted on datasets such as Columbia, CASIA, and NIST16, and the results showed that the proposed method outperformed existing comparison approaches in both detection and localization tasks, demonstrating the effectiveness and generalization capability in complex forgery scenarios.

Key words: deep learning, image forgery detection, module fusion

CLC Number:

LI Xiumei, ZHOU Zhengxin, SUN Junmei. A multi-task collaborative image forgery detection framework assisted by localization branch[J]. Journal of Graphics, 2026, 47(3): 524-533.

Figures/Tables 14

Fig. 1 Structure diagram of the localization branch and detection branch interaction model

Fig. 2 Detailed diagram of the DDEM module

Fig. 3 Detailed diagram of the MEIM module

Table 1 Model localization performance AUC/%

Method	Columbia	Coverage	CASIA	NIST16	IMD20
ManTra-Net	82.4	81.9	81.7	79.5	74.8
PSCC-Net	98.2	84.7	82.9	85.5	80.6
HiFi	97.1	91.2	84.2	83.3	79.8
Ours	98.8	88.9	85.3	86.3	81.2

Table 2 AUC and F1 detection results of different detection methods on CASIA dataset/%

Model	AUC	F1
CNN-det	91.1	85.7
PSCC	94.7	92.2
SPAN	67.3	63.8
Utkarsh	96.1	95.4
HiFi	95.8	93.4
Ours	98.1	97.6

Table 3 AUC and F1 detection results of different detection methods on IFDL dataset/%

Model	AUC	F1
CNN-det	90.1	83.7
PSCC	93.2	91.3
Utkarsh	95.6	94.5
HiFi	96.8	94.1
Ours	97.5	95.3

Fig. 4 Detection performance of the model under different block sizes

Fig. 5 Localization results on different types of images

Fig. 6 Genuine and forged detection data ((a) Genuine images; (b) Forged images from the IMD20 dataset; (c) Forged images from the CASIA dataset)

Table 4 Comparison of cross-dataset AUC detection results of different methods trained on IMD20

Method	Columbia	Coverage	CASIA	NIST16
PSCC	0.713	0.796	0.757	0.702
SPAN	0.679	0.746	0.721	0.624
HiFi	0.752	0.845	0.752	0.797
Utkarsh	0.779	0.813	0.784	0.722
Ours	0.808	0.864	0.813	0.754

Table 5 Comparison of detection results of AUC on different datasets trained on IFDL

Method	Columbia	Coverage	CASIA	NIST16	IMD20
PSCC	0.818	0.789	0.821	0.732	0.701
CNN-det	0.752	0.773	0.743	0.711	0.654
HiFi	0.817	0.835	0.802	0.823	0.742
Utkarsh	0.824	0.812	0.774	0.798	0.754
Ours	0.874	0.890	0.831	0.803	0.768

Table 6 Comparison of detection results of AUC on different datasets trained on IFDL

Method	PS-Battles	MFC2019	DEFACTO
HiFi	0.823	0.787	0.805
Utkarsh	0.802	0.792	0.773
Ours	0.845	0.812	0.791

Table 7 Ablation experiments of DDEM module, MEIM, and loss function

方式	Module	Loss	AUC/%	F1/%
1	M	L_M，L_c	82.2	83.5
2	D	L_M	64.7	71.2
3	D，M	L_M	85.6	84.3
Full	D，M	L_M，L_c	97.5	95.3

Fig. 7 Detection results of our model AUC under different λ values

References 23

[1]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144. DOI URL
[2]	HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[C]//The 34th International Conference on Neural Information Processing Systems. New York: ACM, 2020: 574.
[3]	OJHA U, LI Y, LEE Y J. Towards universal fake image detectors that generalize across generative models[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 24480-24489.
[4]	LIU X H, LIU Y J, CHEN J, et al. PSCC-Net: progressive spatio-channel correlation network for image manipulation detection and localization[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(11): 7505-7517. DOI URL
[5]	GUO X, LIU X H, REN Z Y, et al. Hierarchical fine-grained image forgery detection and localization[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 3155-3165.
[6]	GHIASI G, LIN T Y, LE Q V. NAS-FPN: learning scalable feature pyramid architecture for object detection[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 7029-7038.
[7]	PALERMO R, RHODES G. Change detection in the flicker paradigm: do faces have an advantage?[J]. Visual Cognition, 2003, 10(6): 683-713. DOI URL
[8]	VANDENHENDE S, GEORGOULIS S, GANSBEKE W V, et al. Multi-task learning for dense prediction tasks: a survey[EB/OL]. [2025-03-12]. https://arxiv.org/abs/2004.13379.
[9]	HU X F, ZHANG Z H, JIANG Z Y, et al. SPAN: spatial pyramid attention network for image manipulation localization[C]//The 16th European Conference on Computer Vision. Cham: Springer, 2020: 312-328.
[10]	ZHANG J H, CHEN J B, WANG Q C, et al. Spatiotemporal saliency detection based on maximum consistency superpixels merging for video analysis[J]. IEEE Transactions on Industrial Informatics, 2020, 16(1): 606-614. DOI URL
[11]	YAN J Z, LI Z Q, HE Z W, et al. Generalizable deepfake detection via effective local-global feature extraction[EB/OL]. [2025-07-01]. https://arxiv.org/abs/2501.15253v1.
[12]	WU Y, ABDALMAGEED W, NATARAJAN P. ManTra-Net: manipulation tracing network for detection and localization of image forgeries with anomalous features[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2019: 9535-9544.
[13]	NILOY F F, BHAUMIK K K, WOO S S. CFL-Net: image forgery localization using contrastive learning[C]//The IEEE/CVF Winter Conference on Applications of Computer Vision. New York: IEEE Press, 2023: 4631-4640.
[14]	SHAO R, WU T X, LIU Z W. Detecting and grounding multi-modal media manipulation[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 6904-6913.
[15]	LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//The IEEE/CVF International Conference on Computer Vision. New York: IEEE Press, 2021: 9992-10002.
[16]	LIU Z K, ZHOU Y M, XU Y S, et al. SimpleNet: a simple network for image anomaly detection and localization[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 20402-20411.
[17]	ZHUANG P Y, LI H D, YANG R, et al. ReLoc: a restoration-assisted framework for robust image tampering localization[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 5243-5257. DOI URL
[18]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//The IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 1800-1807.
[19]	HSU Y F, CHANG S F. Detecting image splicing using geometry invariants and camera characteristics consistency[C]//2006 IEEE International Conference on Multimedia and Expo. New York: IEEE Press, 2006: 549-552.
[20]	WEN B H, ZHU Y, SUBRAMANIAN R, et al. COVERAGE—A novel database for copy-move forgery detection[C]//2016 IEEE International Conference on Image Processing. New York: IEEE Press, 2016: 161-165.
[21]	DONG J, WANG W, TAN T N. CASIA image tampering detection evaluation database[C]//2013 IEEE China Summit and International Conference on Signal and Information Processing. New York: IEEE Press, 2013: 422-426.
[22]	GUAN H Y, DELGADO A, LEE Y, et al. User guide for NIST media forensic challenge (MFC) datasets[EB/OL]. [2025-03-12]. https://doi.org/10.6028/NIST.IR.8377.
[23]	NOVOZÁMSKÝ A, MAHDIAN B, SAIC S. IMD2020: a large-scale annotated dataset tailored for detecting manipulated images[C]//The IEEE Winter Conference on Applications of Computer Vision Workshops. New York: IEEE Press, 2020: 71-80.