基于大模型的皮肤病图像掩膜生成与分割

doi:10.11996/JG.j.2095-302X.2026020322

摘要/Abstract

摘要：

作为恶性肿瘤中发病率较高的病种，皮肤癌的及时检出具有重要临床意义，其中皮肤病灶的准确识别与分割是计算机辅助诊断的重要前提。尽管深度学习技术在医学图像分割领域展现出了卓越性能，但现有模型普遍面临着病灶边缘分割精度不足、训练数据规模与多样性受限等问题。为解决上述问题，提出了一种名为BESA-Diff边界增强模型系统，采用边界增强扩散模型DermoSegDiff作为核心分割架构，通过模型优化训练流程。其核心技术贡献体现在：①基于扩散模型构建了皮肤病理性图像及掩膜自动生成框架；②设计了一个创新的掩膜精细化流程：创新性地整合了分割任意模型(SAM)与DermosegDiff的边缘细化模块，构建了高质量的合成医学影像数据集。通过在ISIC2018标准数据集、PH2数据集、HAM10000数据集与合成数据集上的实验结果表明，本模型在戴斯相似系数(Dice)和交并比(IoU)等关键分割指标上均显著优于基线模型。消融实验证实，引入SAM进行掩膜精细化是性能提升的关键，该模块有效改善了病灶边缘的分割效果，特别是在边界模糊或对比度低的区域。研究证实，融合扩散模型的数据生成能力与通用分割模型的边界优化能力，能够有效地提升皮肤病灶分割的精确性与鲁棒性，为皮肤癌辅助诊断提供了一种高性能的解决方案，展现了合成数据技术在突破医学人工智能数据瓶颈方面的巨大潜力。

关键词: 皮肤癌, 数据增强, 扩散模型, 边缘分割, 图像分割

Abstract:

As a malignant tumor with a relatively high incidence rate, the timely detection of skin cancer carries substantial clinical significance. Accurate identification and segmentation of skin lesions serve as critical prerequisites for computer-aided diagnosis. Despite the remarkable performance of deep learning techniques in medical image segmentation, existing models commonly encounter challenges such as insufficient segmentation accuracy at lesion edges and constraints on the scale and diversity of training data. To address these issues, a boundary-enhanced model system named BESA-Diff was proposed. The system employed the boundary-enhanced diffusion model DermoSegDiff as its core segmentation architecture and optimized the model training workflow. The core technical contributions of this research were twofold: First, a framework for the automatic generation of pathological skin images and masks was constructed based on diffusion models. Second, an innovative mask refinement pipeline was designed by innovatively integrating the Segment Anything Model (SAM) with the edge refinement module of DermoSegDiff, and a high-quality synthetic medical image dataset was established. Experimental evaluations on the ISIC2018 standard dataset, PH2 dataset, HAM10000 dataset, and the synthetic dataset demonstrated that the proposed model significantly outperformed baseline models in key segmentation metrics, including the Dice Similarity Coefficient (Dice) and Intersection over Union (IoU). Ablation experiments confirmed that the introduction of SAM for mask refinement was the pivotal factor driving performance improvement. This module effectively enhanced the segmentation of lesion edges, particularly in regions with blurred boundaries or low contrast. The findings of this study validated that integrating the data generation capability of diffusion models with the boundary optimization capability of general segmentation models can effectively improve the accuracy and robustness of skin lesion segmentation. This work provided a high-performance solution for auxiliary diagnosis of skin cancer and highlighted the immense potential of synthetic data technology in overcoming the data bottleneck in medical artificial intelligence.

Key words: skin cancer, data enhancement, diffusion model, edge segmentation, image segmentation

中图分类号:

陈梦琪, 赵俊莉, 邓晓丹. 基于大模型的皮肤病图像掩膜生成与分割[J]. 图学学报, 2026, 47(2): 322-331.

CHEN Mengqi, ZHAO Junli, DENG Xiaodan. SAM-based mask generation and segmentation for dermatological images[J]. Journal of Graphics, 2026, 47(2): 322-331.

图/表 13

图1 训练增强框架

Fig. 1 Training enhancement framework

图2 扩散模型生成图像整体架构

Fig. 2 Overall architecture of image generation by diffusion model

图3 主干网络结构

Fig. 3 Main trunk network structure

图4 真实图像与生成图像((a), (b) 真实图像；(c), (d) 生成图像)

Fig. 4 Real images and generated images ((a), (b) Real images; (c), (d) Generated images)

图5 真实掩膜与生成掩膜((a), (b) 原始掩膜；(c), (d) 生成掩膜)

Fig. 5 Real masks and generated masks ((a), (b) Original masks; (c), (d) Generated masks)

表1 掩膜性能对比

Table 1 Mask performance comparison

实验组别	Dice	IoU
Baseline (Real)	0.917 3	0.860 1
Syn-DiffMask	0.911 7	0.844 1
Syn-SAMMask	0.922 8	0.887 0

图6 皮肤病灶分割结果对比((a), (e)原始合成图像；(b), (f) Diffusion模型生成的掩膜(红色区域代表Diffusion模型分割结果)；(c), (g) SAM模型生成的掩膜(绿色区域代表SAM分割结果)；(d), (h) 两者的叠加结果(黄色区域代表两种方法重叠部分))

Fig. 6 Comparison of skin lesion segmentation results ((a), (e) Original synthetic image; (b), (f) Mask generated by the Diffusion model (the red area represents the segmentation result of the Diffusion model); (c), (g) Mask generated by the SAM model (the green area represents the segmentation result of SAM); (d), (h) The superimposed result of the two (the yellow area represents the overlapping part of the two methods))

表2 模型分割性能(ISIC2018)

Table 2 Model segmentation performance on ISIC2018

方法	Dice ↑	IoU ↑	BF4 ↑
BASE-Diff(ISIC2018)	0.893 2	0.819 2	0.192 3
BASE-Diff (+ 1 000张合成图像)	0.918 5	0.855 4	0.222 9

表3 不同皮肤病变类型上的分割性能对比

Table 3 Segmentation performance across different skin lesion types

病灶类型	方法	Dice↑	IoU↑
黑色素瘤	Baseline	0.878 3	0.805 0
黑色素瘤	BESA-Diff	0.901 5	0.825 5
基底细胞癌	Baseline	0.885 1	0.812 2
基底细胞癌	BESA-Diff	0.912 0	0.840 1
痣	Baseline	0.915 8	0.848 9
痣	BESA-Diff	0.925 3	0.862 0
光化性角化病	Baseline	0.868 8	0.798 8
光化性角化病	BESA-Diff	0.895 2	0.815 5

图7 Baseline模型(仅使用真实掩膜训练)的训练与验证损失曲线

Fig. 7 Training and validation loss curves of the baseline model (trained solely using ground-truth masks)

图8 BESA-Diff模型(引入SAM生成掩膜并经过数据清洗)的训练与验证损失曲线

Fig. 8 Training and validation loss curves of the BESA-DIFF model (incorporates SAM-generated masks followed by data cleaning)

图9 分割结果对比((a) 原始皮肤病图像；(b)真实标签掩膜；(c) 预测结果(绿色区域))

Fig. 9 Comparison of skin lesion segmentation results ((a) Original dermatological image; (b) Ground truth mask; (c) Predicted results (green regions))

表4 模型分割性能(PH2/HAM10000)

Table 4 Model segmentation performance on PH2 and HAM10000

方法	PH2		HAM10000
方法	Dice↑	IoU↑	Dice↑	IoU↑
Baseline (PH2)	0.865 8	0.770 1	0.848 5	0.752 8
BESA-Diff (生成图像)	0.881 2	0.788 3	0.862 7	0.763 9

参考文献 25

[1]	CAO H, WANG Y Y, CHEN J, et al. Swin-Unet: Unet-like pure transformer for medical image segmentation[C]// European Conference on Computer Vision. Cham: Springer, 2023: 205-218.
[2]	CHEN J N, LU Y Y, YU Q H, et al. TransUNet: Transformers make strong encoders for medical image segmentation[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2102.04306.
[3]	HUANG X H, DENG Z F, LI D D, et al. MISSFormer: an effective transformer for 2D medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2023, 42(5): 1484-1494. DOI PMID
[4]	HO J, JAIN A, ABBEEL P. Denoising diffusion probabilistic models[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2006.11239.
[5]	AZAD R, KAZEROUNI A, HEIDARI M, et al. Advances in medical image analysis with vision transformers: a comprehensive review[J]. Medical Image Analysis, 2024, 91: 103000. DOI URL
[6]	KUMAR A, KANTHEN K R, JOHN J. GS-TransUNet:integrated 2D Gaussian splatting and transformer UNet for accurate skin lesion analysis[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2502.16748.
[7]	BOZORGPOUR A, SADEGHEIH Y, KAZEROUNI A, et al. DermoSegDiff: a boundary-aware segmentation diffusion model for skin lesion delineation[C]// The 6th International Workshop on Predictive intelligence in Medicine. Cham: Springer, 2023: 146-158.
[8]	CHEN L C, ZHU Y K, GEORGE P, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// The 15th European Conference on Computer Vision. Cham: Springer, 2018: 833-851.
[9]	CODELLA N, ROTEMBERG V, TSCHANDL P, et al. Skin lesion analysis toward melanoma detection 2018:a challenge hosted by the international skin imaging collaboration (ISIC)[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.1902.03368.
[10]	KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2304.02643.
[11]	FANG L X, XU Y Y, MA X, et al. Minding fuzzy regions: a data-driven alternating learning paradigm for stable lesion segmentation[C]// 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Computer Vision and Pattern Recognition. New York: IEEE Press, 2025: 10425-10434.
[12]	JIANG L, ZHOU Z Y, LEUNG T, et al. MentorNet: learning data-driven curriculum for very deep neural networks on corrupted labels[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.1712.05055.
[13]	RONG S H, TU B H, WANG Z L, et al. Boundary-enhanced co-training for weakly supervised semantic segmentation[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 19574-19584.
[14]	YANG H R, HUA W B, XU Z B, et al. Domain-generalized discrete diffusion model for cross-domain medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2025, 44(11): 4131-4142. DOI URL
[15]	XIE P T, ZHANG L, JINDAL B, et al. Generative AI enables medical image segmentation in ultra low-data regimes[J]. Nature Communications, 2025, 16(1): 6486. DOI
[16]	KAZEROUNI A, AGHDAM E K, HEIDARI M, et al. Diffusion models in medical imaging: a comprehensive survey[J]. Medical Image Analysis, 2023, 88: 102846. DOI URL
[17]	KIM B, OH Y, YE J C. Diffusion adversarial representation learning for self-supervised vessel segmentation[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2209.14566.
[18]	KHAN T M, LIN D, IQBAL S, et al. TAFM-Net: a novel approach to skin lesion segmentation using transformer attention and focal modulation[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2411.17556.
[19]	LEI B Y, XIA Z M, JIANG F, et al. Skin lesion segmentation via generative adversarial networks with dual discriminators[J]. Medical Image Analysis, 2020, 64: 101716. DOI URL
[20]	LIU X W, YANG L, CHEN J G, et al. Region-to-boundary deep learning model with multi-scale feature fusion for medical image segmentation[J]. Biomedical Signal Processing and Control, 2022, 71: 103165. DOI URL
[21]	BI L, FULHAM M, KIM J. Hyper-fusion network for semi-automatic segmentation of skin lesions[J]. Medical Image Analysis, 2022, 76: 102334. DOI URL
[22]	MOLAEI A, AMINIMEHR A, TAVAKOLI A, et al. Implicit neural representation in medical imaging: a comparative survey[C]// 2023 IEEE/CVF International Conference on Computer Vision Workshops. New York: IEEE Press, 2023: 2373-2383.
[23]	NICHOL A, DHARIWAL P. Improved denoising diffusion probabilistic models[EB/OL]. [2025-07-21]. https://doi.org/10.48550/arXiv.2102.09672.
[24]	WANG H N, CAO P, WANG J Q, et al. UCTransnet: rethinking the skip connections in U-Net from a channel-wise perspective with transformer[C]// The 36th AAAI Conference on Artificial Intelligence. Philadelphia: AAAI, 2022: 2441-2449.
[25]	WANG J C, WEI L, WANG L S, et al. Boundary-aware transformers for skin lesion segmentation[C]// The 24th International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer, 2021: 206-216.