Knowledge-aware recommendation based on hypergraph representation learning and Transformer model optimization

doi:10.11996/JG.j.2095-302X.2025051050

Abstract

Abstract:

The recommendation algorithm based on knowledge graphs has emerged as a significant research focus and hotspot in the field of recommender systems in recent years. The introduction of knowledge graphs, enables the acquisition of auxiliary information about items, thereby significantly enhancing the capabilities of recommendation systems and providing users with more precise and personalized recommendation experiences. In response to this trend, a knowledge-aware recommendation model optimized via hypergraph representation learning and the Transformer model was proposed. This model leveraged the unique advantages of hypergraphs in handling high-order relationships to directly model the complex interaction information between users and items, thereby greatly enriching their interaction information. Since local graphs lack global interaction information between users and items, global hypergraphs were constructed within local graphs. On the other hand, nonlocal graphs contain redundant information, so nonlocal hypergraphs were built to capture more comprehensive interaction information between users and items. Additionally, the attention mechanism of the Transformer model was employed to strengthen the collaboration between user nodes and item nodes, mining more valuable preference information from noisy user interaction data such as clicks on uninteresting items. This optimized the embeddings of user and item nodes, mitigating the impact of noise and enhancing the recommendation performance based on user preferences.

Key words: knowledge graph, knowledge-aware recommendation, hypergraph, Transformer, attention mechanism

CLC Number:

TP391.3
TP18

ZUO Yuqi, ZHANG Yunfeng, ZHANG Qiuyue, XU Yingcheng. Knowledge-aware recommendation based on hypergraph representation learning and Transformer model optimization[J]. Journal of Graphics, 2025, 46(5): 1050-1060.

Figures/Tables 6

References 34

[1]	HE X N, LIAO L Z, ZHANG H W, et al. Neural collaborative filtering[C]// The 26th International Conference on World Wide Web. Geneva: International World Wide Web Conferences Steering Committee, 2017: 173-182.
[2]	李振波, 杨晋琪, 岳峻. 基于协同回归模型的矩阵分解推荐[J]. 图学学报, 2019, 40(6): 983-990. DOI
	LI Z B, YANG J Q, YUE J. Matrix factorization recommendation based on collaborative regression model[J]. Journal of Graphics, 2019, 40(6): 983-990 (in Chinese).
[3]	HE X N, ZHANG H W, KAN M Y, et al. Fast matrix factorization for online recommendation with implicit feedback[C]// The 39th International ACM SIGIR conference on Research and Development in Information Retrieval. New York: ACM, 2016: 549-558.
[4]	KOREN Y, BELL R, VOLINSKY C. Matrix factorization techniques for recommender systems[J]. Computer, 2009, 42(8): 30-37.
[5]	PAZZANI M J, BILLSUS D. Content-based recommendation systems[M]// BRUSILOVSKYP, KOBSAA, NEJDLW. The Adaptive Web:Methods and Strategies of Web Personalization. Heidelberg: Springer, 2007: 325-341.
[6]	RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback[C]// The 25th Conference on Uncertainty in Artificial Intelligence. Arlington: AUAI Press, 2009: 452-461.
[7]	WANG H W, ZHAO M, XIE X, et al. Knowledge graph convolutional networks for recommender systems[C]// The World Wide Web Conference. New York: ACM, 2019: 3307-3313.
[8]	王永贵, 陈书铭, 刘义海, 等. 结合超图对比学习和关系聚类的知识感知推荐算法[J]. 计算机科学与探索, 2024, 18(8): 2140-2155. DOI
	WANG Y G, CHEN S M, LIU Y H, et al. Knowledge-aware recommendation algorithm combining hypergraph contrast learning and relational clustering[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(8): 2140-2155 (in Chinese).
[9]	赵超. 采用集成学习的大规模推荐系统优化研究[C]// 第七届创新教育学术会议论文集. 太原: 山西省中大教育研究院, 2023: 105-106.
	ZHAO C. Research on optimization of large-scale recommendation system based on ensemble learning[C]// The Seventh Innovation Education Academic Conference Proceedings. Taiyuan: Shanxi Zhongda Institute of Education, 2023: 105-106 (in Chinese).
[10]	GUO G B, ZHANG J, THALMANN D. A simple but effective method to incorporate trusted neighbors in recommender systems[C]// The 20th International Conference on User Modeling, Adaptation, and Personalization. Cham: Springer, 2012: 114-125.
[11]	何柳, 安然, 刘姝妍, 等. 基于知识图谱的航空多模态数据组织与知识发现技术研究[J]. 图学学报, 2024, 45(2): 300-307. DOI
	HE L, AN R, LIU Z Y, et al. Research on knowledge graph-based aviation multi-modal data organization and discovery method[J]. Journal of Graphics, 2024, 45(2): 300-307 (in Chinese). DOI
[12]	TANG X Y, ZHOU J. Dynamic personalized recommendation on sparse data[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 25(12): 2895-2899.
[13]	HU Z D, XU G Q, ZHENG X, et al. SSL-SVD: semi- supervised learning--based sparse trust recommendation[J]. ACM Transactions on Internet Technology (TOIT), 2020, 20(1): 4.
[14]	WU C, LIU S, ZENG Z Y, et al. Knowledge graph-based multi-context-aware recommendation algorithm[J]. Information Sciences, 2022, 595: 179-194.
[15]	YU J L, YIN H Z, LI J D, et al. Self-supervised multi-channel hypergraph convolutional network for social recommendation[C]// The Web Conference 2021. New York: ACM, 2021: 413-424.
[16]	XIA L H, HUANG C, XU Y, et al. Hypergraph contrastive collaborative filtering[C]// The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 70-79.
[17]	YERA R, CASTRO J, MARTÍNEZ L. A fuzzy model for managing natural noise in recommender systems[J]. Applied Soft Computing, 2016, 40: 187-198.
[18]	LIU Z, FENG X D, WANG Y C, et al. Self-paced learning enhanced neural matrix factorization for noise-aware recommendation[J]. Knowledge-Based Systems, 2021, 213: 106660.
[19]	WANG H W, ZHANG F Z, ZHAO M, et al. Multi-task feature learning for knowledge graph enhanced recommendation[C]// The World Wide Web Conference. New York: ACM, 2019: 2000-2010.
[20]	REN L J, LU J, SUO W. Multi-source knowledge embedding research of knowledge graph[C]// 2019 IEEE 3rd International Conference on Circuits, Systems and Devices. New York: IEEE Press, 2019: 163-166.
[21]	ZHANG F Z, YUAN N J, LIAN D F, et al. Collaborative knowledge base embedding for recommender systems[C]// The 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 353-362.
[22]	OKURA S, TAGAMI Y, ONO S, et al. Embedding-based news recommendation for millions of users[C]// The 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2017: 1933-1942.
[23]	GRAD-GYENGE L, KISS A, FILZMOSER P. Graph embedding based recommendation techniques on the knowledge graph[C]// Adjunct PUBLICATION of the 25th Conference on User Modeling, Adaptation and Personalization. New York: ACM, 2017: 354-359.
[24]	WU G, LI L L, LI X Y, et al. Graph embedding based real-time social event matching for EBSNs recommendation[J]. World Wide Web, 2022, 25(1): 335-356.
[25]	WANG H W, ZHANG F Z, WANG J L, et al. RippleNet: propagating user preferences on the knowledge graph for recommender systems[C]// The 27th ACM International Conference on Information and Knowledge Management. New York: ACM, 2018: 417-426.
[26]	WANG X, HUANG T, WANG D, et al. Learning intents behind interactions with knowledge graph for recommendation[C]// The Web Conference 2021. New York: ACM, 2021: 878-887.
[27]	ZHAO H, DU H, YANG S Q, et al. Rec-RN: user representations learning over the knowledge graph for recommendation systems[C]// The 4th International Conference on Machine Learning, Big Data and Business Intelligence. New York: IEEE Press, 2022: 228-233.
[28]	ZOU D, WEI W, MAO X L, et al. Multi-level cross-view contrastive learning for knowledge-aware recommender system[C]// The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 1358-1368.
[29]	FENG Y F, YOU H X, ZHANG Z Z, et al. Hypergraph neural networks[C]// The 33rd AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2019: 3558-3565.
[30]	XIA X, YIN H Z, YU J L, et al. Self-supervised hypergraph convolutional networks for session-based recommendation[C]// The 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2021, 35(5): 4503-4511.
[31]	刘昺昊. 基于超图神经网络的推荐方法研究[D]. 苏州: 苏州大学, 2022.
	LIU B H. Research on recommendation method with hypergraph neural network[D]. Suzhou: Soochow University, 2022 (in Chinese).
[32]	YANG Z, XU L K, ZHAO L. Efbh: collaborative filtering model based on multi-hypergraph encoder[J]. IEEE Transactions on Consumer Electronics, 2023, 70(1): 2939-2948.
[33]	WEI Y W, LIU W Q, LIU F, et al. LightGT: a light graph transformer for multimedia recommendation[C]// The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2023: 1508-1517.
[34]	LIU J, SONG L Y, WANG G T, et al. Meta-HGT: metapath-aware HyperGraph transformer for heterogeneous information network embedding[J]. Neural Networks, 2023, 157: 65-76.

分类	具体项	Book-Crossing	MovieLens-1M	Last.FM
用户-项目交互	#用户	17860	6036	1872
	#项目	14967	2445	3846
	#交互	139746	753772	42346
知识图谱	#实体	77903	182011	9366
	#关系	25	12	60
	#元组	151500	1241996	15518
超参数设置	#迭代次数	1000	1000	1000
	#嵌入向量维数	64	64	64
	#超边数	128	128	128
	#学习率	0.005 0	0.010 0	0.000 1

分类	具体项	Book-Crossing	MovieLens-1M	Last.FM
用户-项目交互	#用户	17860	6036	1872
	#项目	14967	2445	3846
	#交互	139746	753772	42346
知识图谱	#实体	77903	182011	9366
	#关系	25	12	60
	#元组	151500	1241996	15518
超参数设置	#迭代次数	1000	1000	1000
	#嵌入向量维数	64	64	64
	#超边数	128	128	128
	#学习率	0.005 0	0.010 0	0.000 1

方法	Book-Crossing		MovieLens-1M		Last.FM
方法	AUC	F₁	AUC	F₁	AUC	F₁
BPRFM^[5]	0.649 3	0.610 8	0.883 8	0.789 3	0.713 4	0.640 6
CKE^[21]	0.675 6	0.620 2	0.900 8	0.797 2	0.721 6	0.643 1
RippleNet^[25]	0.719 1	0.645 0	0.911 5	0.827 1	0.770 2	0.694 8
KGCN^[7]	0.682 1	0.629 3	0.907 8	0.831 0	0.757 9	0.701 9
KGIN^[26]	0.721 5	0.661 9	0.912 5	0.841 3	0.808 1	0.731 3
HGNN^[28]	0.708 5	0.672 9	0.933 8	0.852 7	0.838 2	0.753 6
HCCF^[16]	0.722 6	0.680 9	0.932 7	0.847 4	0.854 6	0.765 2
EFBH^[32]	0.733 3	0.683 2	0.934 9	0.848 3	0.860 1	0.786 9
HC-CRKG^[8]	0.752 6	0.682 9	0.945 0	0.872 0	0.886 0	0.798 0
HTKR(本文)	0.759 4	0.694 2	0.957 9	0.895 4	0.862 8	0.794 7

方法	Book-Crossing		MovieLens-1M		Last.FM
方法	AUC	F₁	AUC	F₁	AUC	F₁
BPRFM^[5]	0.649 3	0.610 8	0.883 8	0.789 3	0.713 4	0.640 6
CKE^[21]	0.675 6	0.620 2	0.900 8	0.797 2	0.721 6	0.643 1
RippleNet^[25]	0.719 1	0.645 0	0.911 5	0.827 1	0.770 2	0.694 8
KGCN^[7]	0.682 1	0.629 3	0.907 8	0.831 0	0.757 9	0.701 9
KGIN^[26]	0.721 5	0.661 9	0.912 5	0.841 3	0.808 1	0.731 3
HGNN^[28]	0.708 5	0.672 9	0.933 8	0.852 7	0.838 2	0.753 6
HCCF^[16]	0.722 6	0.680 9	0.932 7	0.847 4	0.854 6	0.765 2
EFBH^[32]	0.733 3	0.683 2	0.934 9	0.848 3	0.860 1	0.786 9
HC-CRKG^[8]	0.752 6	0.682 9	0.945 0	0.872 0	0.886 0	0.798 0
HTKR(本文)	0.759 4	0.694 2	0.957 9	0.895 4	0.862 8	0.794 7

方法	Book-Crossing		MovieLens-1M		Last.FM
方法	AUC	F₁	AUC	F₁	AUC	F₁
HTKR	0.759 4	0.694 2	0.957 9	0.895 4	0.862 8	0.794 7
HTKR-全局超图	0.706 3	0.658 6	0.907 3	0.853 0	0.821 6	0.753 8
HTKR-非全局超图	0.718 6	0.678 2	0.916 0	0.875 3	0.844 5	0.776 5
HTKR-Transformer	0.718 8	0.669 7	0.922 7	0.889 1	0.829 5	0.779 6
HTKR-超图-Transformer	0.647 9	0.628 5	0.803 5	0.740 3	0.731 9	0.700 1
HTKR+10%噪声	0.760 7	0.702 2	0.959 8	0.899 1	0.867 9	0.805 4
HTKR+20%噪声	0.767 3	0.708 5	0.958 2	0.899 3	0.880 1	0.808 6
HTKR+50%噪声	0.624 7	0.640 8	0.914 0	0.811 9	0.754 6	0.701 2