TCPColor：基于文本到图像生成模型的中国画配色方案推荐系统

doi:10.11996/JG.j.2095-302X.2025030520

图学学报 ›› 2025, Vol. 46 ›› Issue (3): 520-531.DOI: 10.11996/JG.j.2095-302X.2025030520

• 图像处理与计算机视觉 • 上一篇下一篇

TCPColor：基于文本到图像生成模型的中国画配色方案推荐系统

张迪¹(), 张文安¹, 姜智德¹, 吴爱霞¹, 孔浩¹, 郭显¹, 陈为²()

1.兰州理工大学计算机与通信学院，甘肃兰州 730070
2.浙江大学计算机辅助设计与图形系统全国重点实验室，浙江杭州 310058

收稿日期:2024-07-05 接受日期:2024-12-13 出版日期:2025-06-30 发布日期:2025-06-13
通讯作者:陈为(1976-)，男，教授，博士。主要研究领域为可视化、可视分析等。E-mail：chenvis@zju.edu.cn
第一作者:张迪(1987-)，男，副教授，博士。主要研究方向为数据可视化、AIGC应用。E-mail：zhangdi.cuc@gmail.com
基金资助:
中央高校基本科研业务费专项资金(226-2022-00235)

TCPColor: a Chinese painting color scheme recommendation system based on text-to-image generation model

ZHANG Di¹(), ZHANG Wenan¹, JIANG Zhide¹, WU Aixia¹, KONG Hao¹, GUO Xian¹, CHEN Wei²()

1. School of Computer and Communication, Lanzhou University of Technology, Lanzhou Gansu 730070, China
2. State Key Laboratory of CAD & CG, Zhejiang University, Hangzhou Zhejiang 310058, China

Received:2024-07-05 Accepted:2024-12-13 Published:2025-06-30 Online:2025-06-13
Contact: CHEN Wei (1976-), professor, Ph.D. His main research interests cover visualization, visual analytics, etc. E-mail：chenvis@zju.edu.cn
First author：ZHANG Di (1987-), associate professor, Ph.D. His main research interests cover data visualization, AIGC application. E-mail：zhangdi.cuc@gmail.com
Supported by:
Special Fund for Fundamental Research Funds for the Central Universities(226-2022-00235)

摘要/Abstract

摘要：

传统中国绘画(国画)是中国特有的绘画形式，在其基础上探索配色方案的使用，对现代设计师探索传统艺术与现代设计理念融合有重要意义，但是基于国画知识的色彩推荐系统研究较少，尚未有基于主题、物象和意境等多维度特征提供有效的配色检索和推荐的方案。为此，提出一个中国画配色方案推荐系统TCPColor，以中文文本到图像生成模型太乙为基础，使用专家标注的宋代国画数据对模型进行了微调，然后对生成图像使用视觉显著性算法、K-Means聚类和基于色彩距离的色表匹配生成了具有国画风格的配色方案，通过消融实验验证了配色方案提取方法的有效性，并通过客观色彩分析评估了生成配色方案的差异性及与国画配色方案的相似度。与国画专家和志愿者合作进行案例分析、专家评估和用户研究，证明了该研究在配色方案推荐方面的实用性。

关键词: 传统中国绘画, 配色方案, 色彩推荐系统, 单词颜色关联, 文本到图像生成

Abstract:

Traditional Chinese painting (TCP) is a unique form of painting specific to China. Exploring the use of color schemes based on TCP holds significant importance for modern designers in integrating traditional art with contemporary design concepts. However, limited research was conducted on color recommendation systems based on TCP knowledge, and no effective solutions have yet been provided for color scheme retrieval and recommendation based on multi-dimensional features such as themes, objects, and artistic conceptions. A traditional Chinese painting color scheme recommendation system named TCPColor was proposed. Based on the Taiyi Chinese text-to-image generation model, this system fine-tuned the model using Song Dynasty TCP data annotated by experts. Then, it employed visual saliency algorithms, K-Means clustering, and color-distance-based palette matching on the generated images to produce color schemes that reflected the style of traditional Chinese painting. The effectiveness of the color extraction method was verified through ablation experiments, while objective color analysis was used to evaluate the distinctiveness of the generated color schemes and their similarity to traditional Chinese painting color schemes. In collaboration with TCP experts and volunteers, case studies, expert evaluations, and user research were conducted, which demonstrated the practicality of the system in recommending color schemes.

Key words: traditional Chinese painting, color scheme, color recommendation system, word-color association, text- to-image generation

中图分类号:

TP391
J212

张迪, 张文安, 姜智德, 吴爱霞, 孔浩, 郭显, 陈为. TCPColor：基于文本到图像生成模型的中国画配色方案推荐系统[J]. 图学学报, 2025, 46(3): 520-531.

ZHANG Di, ZHANG Wenan, JIANG Zhide, WU Aixia, KONG Hao, GUO Xian, CHEN Wei. TCPColor: a Chinese painting color scheme recommendation system based on text-to-image generation model[J]. Journal of Graphics, 2025, 46(3): 520-531.

图/表 14

图1 系统整体框架

Fig. 1 Overall system framework

表1 国画标注系统规则

Table 1 Rules of the Chinese painting annotation system

特征类别	特征描述	标签举例
主题	国画的画科	山水画、人物画、花鸟画
意境	画作在情感、氛围、思想、哲理等方面所展现出的深远意义	韶秀、沉雄、空灵等^[32]
技法	画作中使用的具有代表性的技法	披麻皴、双勾法等
物象	画作中包含的物象	叶、鸟、花等
构图	画作的构图特点	之字形构图，两线构图等
赋彩	画作的用色方案	设色、水墨、白描等

表2 TGM模型微调

Table 2 Fine-tuning of the TGM model

步骤	参数/技术	详细说明
数据集构建	数据来源	故宫博物院开放数据专区
数据集构建	预处理步骤	去除图片背景，图片对应的标签集合并为描述语句
模型微调参数设置	硬件设备	2台Nvidia 3090显卡
	迭代次数	10个epoch
	学习率	32
	精度	fp16 (半精度)

图2 使用国画相关提示词时不同模型生成结果对比((a)花鸟画，鸟，勾染法；(b)人物画，侍女，神妙；(c)人物画，僧侣；(d)平均得分)

Fig. 2 Comparison of generation results from different models using Chinese painting-related prompt words ((a) Flower and bird painting, bird, outline and wash technique; (b) Figure painting, maid, exquisite and divine; (c) Figure painting, monk; (d) Average score)

表3 消融实验结果

Table 3 Results of ablation study

步骤	评价标准	完整处理	消融处理
前景图像提取	CR	0.9	0.7
K-Means颜色聚类	用户满意度	3.9	2.7
国画标准色替代	CH	0.7	0.5

图3 消融实验对比结果

Fig. 3 Ablation study comparison results

图4 各模型配色方案的客观色彩分析((a)色彩差异性评估；(b)与国画配色方案的相似度评估)

Fig. 4 Objective color analysis of color schemes for various models ((a) Assessment of color differences; (b) Evaluation of similarity to traditional Chinese painting color schemes)

图5 推荐配色方案的生成结果((a)生成图配色方案；(b)推荐配色方案)

Fig. 5 Generation results of recommended color schemes((a) Generated image color scheme; (b) Recommended color scheme)

图6 系统可视界面((a)控制面板；(b)画作概览；(c)推荐配色方案；(d)测试配色方案)

Fig. 6 System visual interface ((a) Control panel; (b) Overview of the painting; (c) Recommended color scheme; (d) Test color scheme)

图7 案例1：凝练国色之美——国风商品包装配色方案设计过程((a)案例提示词；(b)画作概览；(c)推荐配色；(d)包装盒配色效果图)

Fig. 7 Case 1: distilling the beauty of national colors-the design process of Chinese-style product packaging color schemes ((a) Case prompt words; (b) Overview of the painting; (c) Recommended color scheme; (d) Color effect diagram for packaging box)

图8 原图配色方案

Fig. 8 Original image color scheme

图9 案例3：为国画绘制推荐单色方案((a)案例3推荐配色；(b) 云纹图案布料配色效果图)

Fig. 9 Case 3: recommending monochromatic schemes for Chinese painting ((a) Recommended color scheme for case 3; (b) Color effect diagram of fabric with cloud pattern)

图10 与其他配色方案的对比

Fig. 10 Comparison with other color schemes

图11 用户实验

Fig. 11 User experiment

参考文献 37

[1]	陈佳舟, 黄可妤, 封颖超杰, 等. 基于古代绘画的古诗自动生成方法[J]. 计算机辅助设计与图形学学报, 2021, 33(7): 1038-1044.
	CHEN J Z, HUANG K Y, FENG Y C J, et al. Automatic poetry generation based on ancient Chinese paintings[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(7): 1038-1044 (in Chinese).
[2]	LIANG K. Color performance of Chinese flower-and-bird fine brushwork[EB/OL]. [2024-05-05]https://www.atlantis-press.com/proceedings/icesame-16/25850996.
[3]	TIAN Z X. Chinese style digital painting[D]. Menomonie: University of Wisconsin-Stout, 2017.
[4]	YU T X, LIN C, ZHANG S J, et al. End-to-end partial convolutions neural networks for Dunhuang grottoes wall-painting restoration[C]// 2019 IEEE/CVF International Conference on Computer Vision Workshop. New York: IEEE Press, 2019: 1447-1455.
[5]	XU L, PARK S J, LEE S. Color2Vec: web-based modeling of word-color association with sociocultural contexts[J]. ACM Transactions on Computer-Human Interaction, 2023, 30(4): 51.
[6]	PERCHOULI AI. 中国色[EB/OL]. [2024-05-05]https://www.zhongguose.com/.
	PERCHOULI AI. Chinese colors[EB/OL]. [2024-05-05]https://www.zhongguose.com/.
[7]	MCCARTHY B, GIACCAI J. Scientific studies of pigments in Chinese paintings[M]. London: Archetype Publications, 2021: 158.
[8]	LIZUN D, KURKIEWICZ T, SZCZUPAK B, et al. A multi-analytical investigation of Liu Kang’s colour palette and painting technique from the Shanghai period (1933-1937)[J]. Applied Sciences, 2023, 13(4): 2414.
[9]	CHEN Y, YANG J, PAN Q Q, et al. A method for exploring word-colour associations[J]. Color Research & Application, 2020, 45(1): 85-94.
[10]	HU J Y, HAN X, YI X Y, et al. Efficient cross-lingual transfer for Chinese stable diffusion with images as pivots[EB/OL]. [2024-05-05]https://arxiv.org/abs/2305.11540.
[11]	O’DONOVAN P, AGARWALA A, HERTZMANN A. Color compatibility from large datasets[C]// ACM SIGGRAPH 2011 Papers. New York: ACM, 2011: 63.
[12]	MUNSELL A H. A color notation[M]. New York: Munsell Color Co., 1919: 1-146.
[13]	GRANVILLE W C. The color harmony manual, a color atlas based on the Ostwald color system[J]. Color Research & Application, 1994, 19(2): 77-98.
[14]	KOBAYASHI S. The aim and method of the color image scale[J]. Color Research & Application, 1981, 6(2): 93-107.
[15]	WANG W N, YU Y L. Image emotional semantic query based on color semantic description[C]// 2005 International Conference on Machine Learning and Cybernetics. New York: IEEE Press, 2005: 4571-4576.
[16]	CHEN T W, CHEN Y L, CHIEN S Y. Fast image segmentation based on K-means clustering with histograms in HSV color space[C]// 2008 IEEE 10th Workshop on Multimedia Signal Processing. New York: IEEE Press, 2008: 322-325.
[17]	DELON J, DESOLNEUX A, LISANI J L, et al. Automatic color palette[C]// IEEE International Conference on Image Processing 2005. New York: IEEE Press, 2005: II-706.
[18]	ACHANTA R, HEMAMI S, ESTRADA F, et al. Frequency-tuned salient region detection[C]// 2009 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2009: 1597-1604.
[19]	刘静, 庄梅玲, 商蕾, 等. 基于色彩聚类的皮影服饰纹样分割[J]. 现代纺织技术, 2021, 29(5): 71-77.
	LIU J, ZHUANG M L, SHANG L, et al. Pattern segmentation of shadow puppetry costumes based on color clustering[J]. Advanced Textile Technology, 2021, 29(5): 71-77 (in Chinese).
[20]	PHAN H Q, FU H B, CHAN A B. Color orchestra: ordering color palettes for interpolation and prediction[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(6): 1942-1955.
[21]	HAN J, LEE Y. Image sentiment considering color palette recommendations based on influence scores for image advertisement[J]. Electronic Commerce Research, 2024: 1-28.
[22]	HONG Y, ZENG X Y, WANG Y Y, et al. CBCRS: an open case-based color recommendation system[J]. Knowledge- Based Systems, 2018, 141: 113-128.
[23]	LEE H, ULLAH U, LEE J S, et al. A brief survey of text driven image generation and maniulation[C]// 2021 IEEE International Conference on Consumer Electronics-Asia. New York: IEEE Press, 2021: 1-4.
[24]	ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2022: 10674-10685.
[25]	RAMESH A, DHARIWAL P, NICHOL A, et al. Hierarchical text-conditional image generation with CLIP latents[EB/OL]. [2024-05-05]https://arxiv.org/abs/2204.06125.
[26]	KO H K, PARK G, JEON H, et al. Large-scale text-to-image generation models for visual artists’ creative works[C]// The 28th International Conference on Intelligent User Interfaces. New York: ACM, 2023: 919-933.
[27]	GAL R, ALALUF Y, ATZMON Y, et al. An image is worth one word: personalizing text-to-image generation using textual inversion[EB/OL]. [2024-05-05]https://arxiv.org/abs/2208.01618.
[28]	RUIZ N, LI Y Z, JAMPANI V, et al. Dreambooth: fine tuning text-to-image diffusion models for subject-driven generation[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2023: 22500-22510.
[29]	ZHANG J X, GAN R Y, WANG J J, et al. Fengshenbang 1.0: being the foundation of Chinese cognitive intelligence[EB/OL]. [2024-05-05]https://arxiv.org/abs/2209.02970#.
[30]	LIU V, CHILTON L B. Design guidelines for prompt engineering text-to-image generative models[C]// 2022 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2022: 384.
[31]	XU H R, CHEN S Y, ZHANG Y. Magical brush: a symbol-based modern Chinese painting system for novices[C]// 2023 CHI Conference on Human Factors in Computing Systems. New York: ACM, 2023: 131.
[32]	National Palace Museum. Open data[EB/OL].[2024-05-05]https://digitalarchive.npm.gov.tw/.opendata.
[33]	吴功正. 我写《六朝美学史》[J]. 中国美术, 1995(5): 5-7.
	WU Q Z. I wrote history of aesthetics in the Six Dynasties[J]. Art in China, 1995(5):5-7.
[34]	LERTRUSDACHAKUL T, RUXPAITOON K, THIPTARAJAN K. Color palette extraction by using modified k-means clustering[C]// 2019 7th International Electrical Engineering Congress. New York: IEEE Press, 2019: 1-4.
[35]	邝智杰. 基于颜色和谐度的可计算图像美学评估关键技术研究与应用[D]. 北京: 北京邮电大学, 2015.
	KUANG Z J. Color harmony based computational a esthetics of image and its application[D]. Beijing: Beijing University of Posts and Telecommunications, 2015 (in Chinese).
[36]	INOUE K, JIANG M Y, HARA K. Hue-preserving saturation improvement in RGB color cube[J]. Journal of Imaging, 2021, 7(8): 150.
[37]	ZHANG L M, LI C Z, SIMO-SERRA E, et al. User-guided line art flat filling with split filling mechanism[C]// 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2021: 9884-9893.

TCPColor：基于文本到图像生成模型的中国画配色方案推荐系统

TCPColor: a Chinese painting color scheme recommendation system based on text-to-image generation model

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 37

相关文章 1

编辑推荐

Metrics

本文评价