结合Transformer与Kolmogorov Arnold网络的分子扩增时序预测研究

doi:10.11996/JG.j.2095-302X.2024061256

图学学报 ›› 2024, Vol. 45 ›› Issue (6): 1256-1265.DOI: 10.11996/JG.j.2095-302X.2024061256

• “大模型与图学技术及应用”专题 • 上一篇下一篇

结合Transformer与Kolmogorov Arnold网络的分子扩增时序预测研究

刘灿锋¹(), 孙浩¹, 东辉¹^,²()

1.福州大学机械工程及自动化学院，福建福州 350108
2.哈尔滨工业大学机电工程学院，黑龙江哈尔滨 150001

收稿日期:2024-07-30 接受日期:2024-09-23 出版日期:2024-12-31 发布日期:2024-12-24
通讯作者:东辉(1985-)，女，教授，博士。主要研究方向为图像处理及机器学习方法等。E-mail：dongh@hit.edu.cn
第一作者:刘灿锋(1999-)，男，硕士研究生。主要研究方向为深度学习及时间序列预测。E-mail：220227106@fzu.edu.cn
基金资助:
国家自然科学基金(62173093);国家自然科学基金(61604042);福建省杰出青年自然科学基金(2023J01310195)

Molecular amplification time series prediction research combining Transformer with Kolmogorov-Arnold network

LIU Canfeng¹(), SUN Hao¹, DONG Hui¹^,²()

1. School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou Fujian 350108, China
2. School of Mechatronics Engineering, Harbin Institute of Technology, Harbin Heilongjiang 150001, China

Received:2024-07-30 Accepted:2024-09-23 Published:2024-12-31 Online:2024-12-24
Contact: DONG Hui (1985-), professor, Ph.D. Her main research interests cover image processing and machine learning, etc. E-mail：dongh@hit.edu.cn
First author：LIU Canfeng (1999-), master student. His main research interests cover deep learning and time series predict. E-mail：220227106@fzu.edu.cn
Supported by:
National Natural Science Foundation of China(62173093);National Natural Science Foundation of China(61604042);Fujian Province Outstanding Youth Natural Science Foundation(2023J01310195)

摘要/Abstract

摘要：

随着医辽诊断和治疗干预技术的不断进步，医学时间序列数据呈现指数级增长。人工智能(AI)，尤其是深度学习在挖掘医学时间序列数据潜在信息方面展现出巨大潜力。为此，首次提出将Transformer与Kolmogorov arnold网络(KAN)相结合的方法，用于核酸扩增实验数据的预测分析。通过实验数据分析，证实模型在准确预测扩增趋势和终点值方面的有效性，终点值误差仅为1.87，R-square系数为0.98，且模型能准确识别不同样本类型的实验数据。进一步地，通过消融实验和超参数调优，深入探究模型各组成部分及其参数对预测性能的影响。最后，在911条临床数据上对10种深度学习模型进行泛化能力测试的结果表明，Transformer-KAN模型在预测准确性和泛化能力上均优于其他模型，不仅为改进大流行病常规诊断技术提供了新视角，还为进一步研究KAN模型及相应基础理论提供了实验佐证。

关键词: 深度学习, 时间序列预测, 核酸扩增检测技术, Kolmogorov-Arnold网络, Transformer

Abstract:

With the development of medical diagnosis and treatment intervention techniques, there has been an exponential growth in medical data along time series. Artificial intelligence (AI), particularly deep learning (DL), has demonstrated significant potential in uncovering medical data along time series. This study proposed, for the first time, a method that integrates the Transformer architecture with the Kolmogorov-Arnold network (KAN) to enable predictive analysis of nucleic acid amplification experimental data. Through experimental data analysis methods, the effectiveness of the model in accurately predicting amplification trends and endpoint values was validated, achieving an endpoint value error of merely 1.87 and an R-square coefficient as high as 0.98. Moreover, the model was capable of effectively identifying experimental data from different sample types. Furthermore, this research delved into the impact of the model’s components and parameters on predictive performance through ablation experiments and hyperparameter tuning. Finally, a generalization capability test was conducted on 911 clinical data records provided by the Fujian Provincial Hospital across 10 deep learning models. The results demonstrated that the proposed Transformer-KAN network outperformed other models in terms of predictive accuracy and generalization capability. This study not only provided a new perspective for improving routine diagnostic techniques during pandemics but also offered empirical evidence for further research on the KAN model and its corresponding foundational theories.

Key words: deep learning, time series prediction, nucleic acid amplification test, Kolmogorov-Arnold network, Transformer

中图分类号:

刘灿锋, 孙浩, 东辉. 结合Transformer与Kolmogorov Arnold网络的分子扩增时序预测研究[J]. 图学学报, 2024, 45(6): 1256-1265.

LIU Canfeng, SUN Hao, DONG Hui. Molecular amplification time series prediction research combining Transformer with Kolmogorov-Arnold network[J]. Journal of Graphics, 2024, 45(6): 1256-1265.

图/表 11

图1 NAAT实验平台及实验数据((a)荧光检测装置；(b) NAAT实验平台布局；(c) NAAT数据)

Fig. 1 NAAT experimental platform and experimental data ((a) Fluorescent detection device; (b) NAAT experimental platform layout; (c) NAAT data)

图2 NAAT操作流程

Fig. 2 NAAT operation process

图3 Transformer-KAN架构

Fig. 3 Transformer-KAN architecture

图4 KAN网络结构

Fig. 4 KAN Network Structure

图5 Transformer-KAN数据流

Fig. 5 Data Flow of Transformer-KAN Network

图6 实验数据预测结果((a)在阳性数据上输入不同周期的NAAT预测曲线,柱形图表示终点值大小；(b)在阴性数据上输入不同周期的NAAT预测曲线；(c)预测曲线的扩增倍率及相邻点的斜率)

Fig. 6 Experimental data prediction results ((a) NAAT prediction curves for positive data with different cycles, bar graphs indicate endpoint values; (b) NAAT prediction curves for negative data with different cycles; (c) Amplification rates and slopes of adjacent points of prediction curves)

表1 消融实验结果

Table 1 Ablation experiment results

Model	KAN	Decoder	Encoder	Attention	Rmse	Endpoint error	R-squared	Params	Train time/s	Volatility
Model1	×	×	√	×	93.94	104.11	-29.02	5.92×10⁶	8.25	5.76
Model2	√	×	×	×	71.52	78.06	-16.4	2.04×10⁶	6.75	0.10
Model3	×	×	√	√	19.76	21.70	-0.33	1.30×10⁷	22.53	10.32
Transf-ormer	×	√	√	√	5.89	2.19	0.88	2.30×10⁷	47.45	6.24
Ours	√	×	√	√	2.28	1.87	0.98	1.15×10⁷	21.82	1.65

表2 模型性能与超参数配置

Table 2 Model performance and hyperparameter configuration

Model	Nhead	Enc.-layer	Window	Dmodel	Grid_size	Activation	Rmse	Endpoint error	R-squared
Model1	4	2	8	700	5	Silu	7.05	2.39	0.83
Model2	1	2	8	700	5	Silu	5.97	4.47	0.88
Model3	2	1	8	700	5	Silu	15.35	17.53	0.20
Model4	2	3	8	700	5	Silu	5.65	0.18	0.89
Model5	2	2	6	700	5	Silu	10.74	4.76	0.61
Model6	2	2	10	700	5	Silu	10.20	11.54	0.65
Model7	2	2	8	650	5	Silu	18.05	20.61	-0.11
Model8	2	2	8	750	5	Silu	14.95	17.08	0.24
Model9	2	2	8	700	3	Silu	37.22	41.69	-3.71
Model10	2	2	8	700	7	Silu	7.94	8.45	0.78
Model11	2	2	8	700	5	Relu	6.94	6.76	0.84
Model12	2	2	8	700	5	Sig-moid	9.41	9.76	0.70
Ours	2	2	8	700	5	Silu	2.28	1.87	0.98

图7 不同算法实验结果，柱形图表示各周期绝对误差((a) CNN系列；(b) RNN系列；(c)双向RNN系列；(d) Transformer及TS.-KAN)

Fig. 7 Experimental results of different algorithms, bar graphs represent absolute errors for each cycle ((a) CNN series; (b) RNN series; (c) Bidirectional RNN series; (d) Transformer and TS.-KAN)

图8 不同算法泛化实验结果

Fig. 8 Generalization experimental results of different algorithms

图9 泛化实验具体数值

Fig. 9 Specific values of generalization experiments

参考文献 27

[1]	YANG X, QI X J, ZHOU X B. Deep learning technologies for time series anomaly detection in healthcare: a review[J]. IEEE Access, 2023, 11: 117788-117799.
[2]	MORID M A, SHENG O R L, DUNBAR J. Time series prediction using deep learning methods in healthcare[J]. ACM Transactions on Management Information Systems, 2023, 14(1): 2.
[3]	KODE H, ELLEITHY K, ALMAZAYDEH L. Epileptic seizure detection in EEG signals using machine learning and deep learning techniques[J]. IEEE Access, 2024, 12: 80657-80668.
[4]	XIE F, YUAN H, NING Y L, et al. Deep learning for temporal data representation in electronic health records: a systematic review of challenges and methodologies[J]. Journal of Biomedical Informatics, 2022, 126: 103980.
[5]	THUWAJIT P, RANGPONG P, SAWANGJAI P, et al. EEGWaveNet: multiscale CNN-based spatiotemporal feature extraction for EEG seizure detection[J]. IEEE Transactions on Industrial Informatics, 2022, 18(8): 5547-5557.
[6]	TANG Y X, WU Q Y, MAO H F, et al. Epileptic seizure detection based on path signature and Bi-LSTM network with attention mechanism[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024, 32: 304-313. DOI PMID
[7]	LU W J, LI J Z, WANG J Y, et al. A CNN-BiLSTM-AM method for stock price prediction[J]. Neural Computing and Applications, 2021, 33(10): 4741-4753.
[8]	RAFI S H, NAHID-AL-MASOOD, DEEBA S R, et al. A short-term load forecasting method using integrated CNN and LSTM network[J]. IEEE Access, 2021, 9: 32436-32448.
[9]	JIA C Y, TIAN Y K, SHI Y H, et al. State of health prediction of lithium-ion batteries based on bidirectional gated recurrent unit and transformer[J]. Energy, 2023, 285: 129401.
[10]	王玉萍, 曾毅, 李胜辉, 等. 一种基于Transformer的三维人体姿态估计方法[J]. 图学学报, 2023, 44(1): 139-145. DOI
	WANG Y P, ZENG Y, LI S H, et al. A Transformer-based 3D human pose estimation method[J]. Journal of Graphics, 2023, 44(1): 139-145. (in Chinese)
[11]	KIM J, KANG H, KANG P. Time-series anomaly detection with stacked Transformer representations and 1D convolutional network[J]. Engineering Applications of Artificial Intelligence, 2023, 120: 105964.
[12]	LI C, ZHANG Z Z, ZHANG X D, et al. EEG-based emotion recognition via transformer neural architecture search[J]. IEEE Transactions on Industrial Informatics, 2023, 19(4): 6016-6025.
[13]	JIANG Y R, MALLIARAS P, CHEN B, et al. Real-time forecasting of exercise-induced fatigue from wearable sensors[J]. Computers in Biology and Medicine, 2022, 148: 105905.
[14]	LIU Z M, WANG Y X, VAIDYA S, et al. KAN: Kolmogorov-Arnold networks[EB/OL]. [2024-09-21]. https://arxiv.org/abs/2404.19756.pdf.
[15]	KANG T J, LU J M, YU T, et al. Advances in nucleic acid amplification techniques (NAATs): COVID-19 point-of-care diagnostics as an example[J]. Biosensors and Bioelectronics, 2022, 206: 114109.
[16]	QIAN J J, ZHANG Q M, LU M. Integration of on-chip lysis and paper-based sensor for rapid detection of viral and exosomal RNAs[J]. Biosensors and Bioelectronics, 2023, 226: 115114.
[17]	SUN H, JIANG Q H, HUANG Y, et al. Integrated smart analytics of nucleic acid amplification tests via paper microfluidics and deep learning in cloud computing[J]. Biomedical Signal Processing and Control, 2023, 83: 104721.
[18]	SUN H, XIE W T, HUANG Y, et al. Paper microfluidics with deep learning for portable intelligent nucleic acid amplification tests[J]. Talanta, 2023, 258: 124470.
[19]	WANG C J, CHEN Y Y, ZHANG S Q, et al. Stock market index prediction using deep Transformer model[J]. Expert Systems with Applications, 2022, 208: 118128.
[20]	罗智徽, 胡海涛, 马潇峰, 等. 基于同质中间模态的跨模态行人再识别方法[J]. 图学学报, 2024, 45(4): 670-682. DOI
	LUO Z H, HU H T, MA X F, et al. A network based on the homogeneous middle modality for cross-modality person re-identification[J]. Journal of Graphics, 2024, 45(4): 670-682. (in Chinese) DOI
[21]	SHAO L, FAN Z Q, LI J, et al. Research on road vehicle detection based on improved Yolov5s[C]// 2022 IEEE International Conference on Mechatronics and Automation. New York: IEEE Press, 2022: 428-433.
[22]	SHAIN E B, CLEMENS J M. A new method for robust quantitative and qualitative analysis of real-time PCR[J]. Nucleic Acids Research, 2008, 36(14): e91.
[23]	CHICCO D, WARRENS M J, JURMAN G. The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation[J]. Peer J Computer Science, 2021, 7: e623.
[24]	PYO J C, PACHEPSKY Y, KIM S, et al. Long short-term memory models of water quality in inland water environments[J]. Water Research X, 2023, 21: 100207.
[25]	ROSTAMIAN A, O’HARA J G. Event prediction within directional change framework using a CNN-LSTM model[J]. Neural Computing and Applications, 2022, 34(20): 17193-17205.
[26]	LI X, MA Z Q, YUAN Z H, et al. A review on convolutional neural network in rolling bearing fault diagnosis[J]. Measurement Science and Technology, 2024, 35(7): 072002.
[27]	RUIZ-VILLALBA A, RUIJTER J M, VAN DEN HOFF M J B. Use and misuse of C_q in qPCR data analysis and reporting[J]. Life, 2021, 11(6): 496.

结合Transformer与Kolmogorov Arnold网络的分子扩增时序预测研究

Molecular amplification time series prediction research combining Transformer with Kolmogorov-Arnold network

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李琼, 考月英, 张莹, 徐沛. 面向无人机航拍图像的目标检测研究综述[J]. 图学学报, 2024, 45(6): 1145-1164.
[2]	宋思程, 陈辰, 李晨辉, 王长波. 基于密度图多目标追踪的时空数据可视化[J]. 图学学报, 2024, 45(6): 1289-1300.
[3]	王宗继, 刘云飞, 陆峰. Cloud Sphere: 一种基于渐进式变形自编码的三维模型表征方法[J]. 图学学报, 2024, 45(6): 1375-1388.
[4]	许丹丹, 崔勇, 张世倩, 刘雨聪, 林予松. 优化医学影像三维渲染可视化效果：技术综述[J]. 图学学报, 2024, 45(5): 879-891.
[5]	胡凤阔, 叶兰, 谭显峰, 张钦展, 胡志新, 方清, 王磊, 满孝锋. 一种基于改进YOLOv8的轻量化路面病害检测算法[J]. 图学学报, 2024, 45(5): 892-900.
[6]	刘义艳, 郝婷楠, 贺晨, 常英杰. 基于DBBR-YOLO的光伏电池表面缺陷检测[J]. 图学学报, 2024, 45(5): 913-921.
[7]	翟永杰, 李佳蔚, 陈年昊, 王乾铭, 王新颖. 融合改进Transformer的车辆部件检测方法[J]. 图学学报, 2024, 45(5): 930-940.
[8]	刘丽, 张起凡, 白宇昂, 黄凯烨. 结合Swin Transformer的多尺度遥感图像变化检测研究[J]. 图学学报, 2024, 45(5): 941-956.
[9]	姜晓恒, 段金忠, 卢洋, 崔丽莎, 徐明亮. 融合先验知识推理的表面缺陷检测[J]. 图学学报, 2024, 45(5): 957-967.
[10]	章东平, 魏杨悦, 何数技, 徐云超, 胡海苗, 黄文君. 特征融合与层间传递：一种基于Anchor DETR改进的目标检测方法[J]. 图学学报, 2024, 45(5): 968-978.
[11]	熊超, 王云艳, 罗雨浩. 特征对齐与上下文引导的多视图三维重建[J]. 图学学报, 2024, 45(5): 1008-1016.
[12]	孙己龙, 刘勇, 周黎伟, 路鑫, 侯小龙, 王亚琼, 王志丰. 基于DCNv2和Transformer Decoder的隧道衬砌裂缝高效检测模型研究[J]. 图学学报, 2024, 45(5): 1050-1061.
[13]	罗智徽, 胡海涛, 马潇峰, 程文刚. 基于同质中间模态的跨模态行人再识别方法[J]. 图学学报, 2024, 45(4): 670-682.
[14]	胡欣, 常娅姝, 秦皓, 肖剑, 程鸿亮. 基于改进YOLOv8和GMM图像点集匹配的双目测距方法[J]. 图学学报, 2024, 45(4): 714-725.
[15]	牛为华, 郭迅. 基于改进YOLOv8的船舰遥感图像旋转目标检测算法[J]. 图学学报, 2024, 45(4): 726-735.