Molecular amplification time series prediction research combining Transformer with Kolmogorov-Arnold network

doi:10.11996/JG.j.2095-302X.2024061256

Abstract

Abstract:

With the development of medical diagnosis and treatment intervention techniques, there has been an exponential growth in medical data along time series. Artificial intelligence (AI), particularly deep learning (DL), has demonstrated significant potential in uncovering medical data along time series. This study proposed, for the first time, a method that integrates the Transformer architecture with the Kolmogorov-Arnold network (KAN) to enable predictive analysis of nucleic acid amplification experimental data. Through experimental data analysis methods, the effectiveness of the model in accurately predicting amplification trends and endpoint values was validated, achieving an endpoint value error of merely 1.87 and an R-square coefficient as high as 0.98. Moreover, the model was capable of effectively identifying experimental data from different sample types. Furthermore, this research delved into the impact of the model’s components and parameters on predictive performance through ablation experiments and hyperparameter tuning. Finally, a generalization capability test was conducted on 911 clinical data records provided by the Fujian Provincial Hospital across 10 deep learning models. The results demonstrated that the proposed Transformer-KAN network outperformed other models in terms of predictive accuracy and generalization capability. This study not only provided a new perspective for improving routine diagnostic techniques during pandemics but also offered empirical evidence for further research on the KAN model and its corresponding foundational theories.

Key words: deep learning, time series prediction, nucleic acid amplification test, Kolmogorov-Arnold network, Transformer

CLC Number:

LIU Canfeng, SUN Hao, DONG Hui. Molecular amplification time series prediction research combining Transformer with Kolmogorov-Arnold network[J]. Journal of Graphics, 2024, 45(6): 1256-1265.

Figures/Tables 11

References 27

[1]	YANG X, QI X J, ZHOU X B. Deep learning technologies for time series anomaly detection in healthcare: a review[J]. IEEE Access, 2023, 11: 117788-117799.
[2]	MORID M A, SHENG O R L, DUNBAR J. Time series prediction using deep learning methods in healthcare[J]. ACM Transactions on Management Information Systems, 2023, 14(1): 2.
[3]	KODE H, ELLEITHY K, ALMAZAYDEH L. Epileptic seizure detection in EEG signals using machine learning and deep learning techniques[J]. IEEE Access, 2024, 12: 80657-80668.
[4]	XIE F, YUAN H, NING Y L, et al. Deep learning for temporal data representation in electronic health records: a systematic review of challenges and methodologies[J]. Journal of Biomedical Informatics, 2022, 126: 103980.
[5]	THUWAJIT P, RANGPONG P, SAWANGJAI P, et al. EEGWaveNet: multiscale CNN-based spatiotemporal feature extraction for EEG seizure detection[J]. IEEE Transactions on Industrial Informatics, 2022, 18(8): 5547-5557.
[6]	TANG Y X, WU Q Y, MAO H F, et al. Epileptic seizure detection based on path signature and Bi-LSTM network with attention mechanism[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2024, 32: 304-313. DOI PMID
[7]	LU W J, LI J Z, WANG J Y, et al. A CNN-BiLSTM-AM method for stock price prediction[J]. Neural Computing and Applications, 2021, 33(10): 4741-4753.
[8]	RAFI S H, NAHID-AL-MASOOD, DEEBA S R, et al. A short-term load forecasting method using integrated CNN and LSTM network[J]. IEEE Access, 2021, 9: 32436-32448.
[9]	JIA C Y, TIAN Y K, SHI Y H, et al. State of health prediction of lithium-ion batteries based on bidirectional gated recurrent unit and transformer[J]. Energy, 2023, 285: 129401.
[10]	王玉萍, 曾毅, 李胜辉, 等. 一种基于Transformer的三维人体姿态估计方法[J]. 图学学报, 2023, 44(1): 139-145. DOI
	WANG Y P, ZENG Y, LI S H, et al. A Transformer-based 3D human pose estimation method[J]. Journal of Graphics, 2023, 44(1): 139-145. (in Chinese)
[11]	KIM J, KANG H, KANG P. Time-series anomaly detection with stacked Transformer representations and 1D convolutional network[J]. Engineering Applications of Artificial Intelligence, 2023, 120: 105964.
[12]	LI C, ZHANG Z Z, ZHANG X D, et al. EEG-based emotion recognition via transformer neural architecture search[J]. IEEE Transactions on Industrial Informatics, 2023, 19(4): 6016-6025.
[13]	JIANG Y R, MALLIARAS P, CHEN B, et al. Real-time forecasting of exercise-induced fatigue from wearable sensors[J]. Computers in Biology and Medicine, 2022, 148: 105905.
[14]	LIU Z M, WANG Y X, VAIDYA S, et al. KAN: Kolmogorov-Arnold networks[EB/OL]. [2024-09-21]. https://arxiv.org/abs/2404.19756.pdf.
[15]	KANG T J, LU J M, YU T, et al. Advances in nucleic acid amplification techniques (NAATs): COVID-19 point-of-care diagnostics as an example[J]. Biosensors and Bioelectronics, 2022, 206: 114109.
[16]	QIAN J J, ZHANG Q M, LU M. Integration of on-chip lysis and paper-based sensor for rapid detection of viral and exosomal RNAs[J]. Biosensors and Bioelectronics, 2023, 226: 115114.
[17]	SUN H, JIANG Q H, HUANG Y, et al. Integrated smart analytics of nucleic acid amplification tests via paper microfluidics and deep learning in cloud computing[J]. Biomedical Signal Processing and Control, 2023, 83: 104721.
[18]	SUN H, XIE W T, HUANG Y, et al. Paper microfluidics with deep learning for portable intelligent nucleic acid amplification tests[J]. Talanta, 2023, 258: 124470.
[19]	WANG C J, CHEN Y Y, ZHANG S Q, et al. Stock market index prediction using deep Transformer model[J]. Expert Systems with Applications, 2022, 208: 118128.
[20]	罗智徽, 胡海涛, 马潇峰, 等. 基于同质中间模态的跨模态行人再识别方法[J]. 图学学报, 2024, 45(4): 670-682. DOI
	LUO Z H, HU H T, MA X F, et al. A network based on the homogeneous middle modality for cross-modality person re-identification[J]. Journal of Graphics, 2024, 45(4): 670-682. (in Chinese) DOI
[21]	SHAO L, FAN Z Q, LI J, et al. Research on road vehicle detection based on improved Yolov5s[C]// 2022 IEEE International Conference on Mechatronics and Automation. New York: IEEE Press, 2022: 428-433.
[22]	SHAIN E B, CLEMENS J M. A new method for robust quantitative and qualitative analysis of real-time PCR[J]. Nucleic Acids Research, 2008, 36(14): e91.
[23]	CHICCO D, WARRENS M J, JURMAN G. The coefficient of determination R-squared is more informative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation[J]. Peer J Computer Science, 2021, 7: e623.
[24]	PYO J C, PACHEPSKY Y, KIM S, et al. Long short-term memory models of water quality in inland water environments[J]. Water Research X, 2023, 21: 100207.
[25]	ROSTAMIAN A, O’HARA J G. Event prediction within directional change framework using a CNN-LSTM model[J]. Neural Computing and Applications, 2022, 34(20): 17193-17205.
[26]	LI X, MA Z Q, YUAN Z H, et al. A review on convolutional neural network in rolling bearing fault diagnosis[J]. Measurement Science and Technology, 2024, 35(7): 072002.
[27]	RUIZ-VILLALBA A, RUIJTER J M, VAN DEN HOFF M J B. Use and misuse of C_q in qPCR data analysis and reporting[J]. Life, 2021, 11(6): 496.

Model	KAN	Decoder	Encoder	Attention	Rmse	Endpoint error	R-squared	Params	Train time/s	Volatility
Model1	×	×	√	×	93.94	104.11	-29.02	5.92×10⁶	8.25	5.76
Model2	√	×	×	×	71.52	78.06	-16.4	2.04×10⁶	6.75	0.10
Model3	×	×	√	√	19.76	21.70	-0.33	1.30×10⁷	22.53	10.32
Transf-ormer	×	√	√	√	5.89	2.19	0.88	2.30×10⁷	47.45	6.24
Ours	√	×	√	√	2.28	1.87	0.98	1.15×10⁷	21.82	1.65

Model	KAN	Decoder	Encoder	Attention	Rmse	Endpoint error	R-squared	Params	Train time/s	Volatility
Model1	×	×	√	×	93.94	104.11	-29.02	5.92×10⁶	8.25	5.76
Model2	√	×	×	×	71.52	78.06	-16.4	2.04×10⁶	6.75	0.10
Model3	×	×	√	√	19.76	21.70	-0.33	1.30×10⁷	22.53	10.32
Transf-ormer	×	√	√	√	5.89	2.19	0.88	2.30×10⁷	47.45	6.24
Ours	√	×	√	√	2.28	1.87	0.98	1.15×10⁷	21.82	1.65

Model	Nhead	Enc.-layer	Window	Dmodel	Grid_size	Activation	Rmse	Endpoint error	R-squared
Model1	4	2	8	700	5	Silu	7.05	2.39	0.83
Model2	1	2	8	700	5	Silu	5.97	4.47	0.88
Model3	2	1	8	700	5	Silu	15.35	17.53	0.20
Model4	2	3	8	700	5	Silu	5.65	0.18	0.89
Model5	2	2	6	700	5	Silu	10.74	4.76	0.61
Model6	2	2	10	700	5	Silu	10.20	11.54	0.65
Model7	2	2	8	650	5	Silu	18.05	20.61	-0.11
Model8	2	2	8	750	5	Silu	14.95	17.08	0.24
Model9	2	2	8	700	3	Silu	37.22	41.69	-3.71
Model10	2	2	8	700	7	Silu	7.94	8.45	0.78
Model11	2	2	8	700	5	Relu	6.94	6.76	0.84
Model12	2	2	8	700	5	Sig-moid	9.41	9.76	0.70
Ours	2	2	8	700	5	Silu	2.28	1.87	0.98

Model	Nhead	Enc.-layer	Window	Dmodel	Grid_size	Activation	Rmse	Endpoint error	R-squared
Model1	4	2	8	700	5	Silu	7.05	2.39	0.83
Model2	1	2	8	700	5	Silu	5.97	4.47	0.88
Model3	2	1	8	700	5	Silu	15.35	17.53	0.20
Model4	2	3	8	700	5	Silu	5.65	0.18	0.89
Model5	2	2	6	700	5	Silu	10.74	4.76	0.61
Model6	2	2	10	700	5	Silu	10.20	11.54	0.65
Model7	2	2	8	650	5	Silu	18.05	20.61	-0.11
Model8	2	2	8	750	5	Silu	14.95	17.08	0.24
Model9	2	2	8	700	3	Silu	37.22	41.69	-3.71
Model10	2	2	8	700	7	Silu	7.94	8.45	0.78
Model11	2	2	8	700	5	Relu	6.94	6.76	0.84
Model12	2	2	8	700	5	Sig-moid	9.41	9.76	0.70
Ours	2	2	8	700	5	Silu	2.28	1.87	0.98