Motion capture data recovery by combining zero-norm sparsity and temporal difference low rank

doi:10.11996/JG.j.2095-302X.2026030616

Abstract

Abstract:

To address the prevalent noise interference and missing-marker problem during the collection and transmission of Motion Capture (MoCap) data, a recovery model combining the Zero-norm sparsity and Temporal Differential Low-rank minimization (ZTDL) was proposed. Firstly, a temporal difference low-rank regularization term was introduced to capture the global low-rank property and temporal smoothness of MoCap data. Besides, the l₀ norm and the Frobenius norm were employed to characterize sparse missing noise and additive Gaussian noise. Secondly, the non-convex recovery model was transformed into the optimization problem involving a binary mask matrix by exploiting the properties of the l₀ norm. This model enabled the simultaneous estimation of missing regions and the restoration of MoCap data. The optimization problem was efficiently solved using the Alternating Direction Method of Multipliers (ADMM) framework sand the (inverse) Discrete Cosine Transform (DCT/IDCT). The algorithm was rigorously proven to converge to a local minimum in a coordinate-wise manner in theory. Finally, extensive comparative experiments were conducted on the benchmark CMU and HDM05 datasets. The ZTDL algorithm was evaluated against a range of classical methods such as TSMC, TRNN, IRNN-Lp, and TSPN, as well as deep learning approaches. The restoration results, including recovery error and visual effect, demonstrated the significant superiority of ZTDL in both missing-region estimation and corrupted-data restoration.

Key words: motion capture data, temporal difference, sparse and low-rank, non-convex optimization, zero-norm

CLC Number:

TP391.41

HU Wenyu, XU Hao, QIU Xiwen, YI Yun. Motion capture data recovery by combining zero-norm sparsity and temporal difference low rank[J]. Journal of Graphics, 2026, 47(3): 616-628.

Figures/Tables 16

Fig. 1 The comparison of singular value curves before and after temporal difference for two motion capture data ((a) Run; (b) Gymnastics)

Fig. 2 RMSE curve of random noisy data ((a) Walk; (b) Run; (c) Gymnastics; (d) Dance)

Fig. 3 RMSE curve of randomly missing data ((a) Walk; (b) Run; (c) Gymnastics; (d) Dance)

Fig. 4 RMSE curve of continuously missing data ((a) Walk; (b) Run; (c) Gymnastics; (d) Dance)

Fig. 5 RMSE curve of mixed polluted data ((a) Walk; (b) Run; (c) Gymnastics; (d) Dance)

Table 1 MRMSE comparisons of various algorithms in recovering the Walk sequence

算法	随机噪声	随机缺失	连续缺失	混合污染
TSMC	0.198 4	0.180 2	0.057 1	0.234 2
TrNN	0.165 6	0.085 9	0.049 7	0.140 4
IRNN-LP	0.163 7	0.066 7	0.049 8	0.136 1
TSPN	0.138 1	0.079 5	0.062 4	0.135 5
LNM-QR	0.137 7	0.075 7		0.136 9
MFF-N	0.119 9	0.066 3	0.035 3	0.135 6
ZTDL	0.119 1	0.066 3	0.034 5	0.123 7

Table 2 MRMSE comparisons of various algorithms in recovering the Run sequence

算法	随机噪声	随机缺失	连续缺失	混合污染
TSMC	0.585 0	0.469 8	0.177 6	0.427 9
TrNN	0.453 6	0.352 6	0.168 1	0.415 0
IRNN-LP	0.453 5	0.232 5	0.171 2	0.330 5
TSPN	0.343 5	0.277 6	0.178 1	0.367 6
LNM-QR	0.340 1	0.227 4		0.274 9
MFF-N	0.249 3	0.219 9	0.085 7	0.190 2
ZTDL	0.248 7	0.219 1	0.075 1	0.185 5

Table 3 MRMSE comparisons of various algorithms in recovering the Gymnastics sequence

算法	随机噪声	随机缺失	连续缺失	混合污染
TSMC	0.377 0	0.322 6	0.162 7	0.320 3
TrNN	0.298 6	0.328 2	0.147 9	0.355 0
IRNN-LP	0.298 7	0.157 6	0.145 3	0.230 8
TSPN	0.231 5	0.154 2	0.166 9	0.231 4
LNM-QR	0.234 1	0.156 8		0.197 4
MFF-N	0.181 5	0.141 3	0.081 6	0.148 4
ZTDL	0.181 5	0.141 2	0.071 1	0.147 6

Table 4 MRMSE comparisons of various algorithms in recovering the Dance sequence

算法	随机噪声	随机缺失	连续缺失	混合污染
TSMC	0.218 4	0.178 2	0.088 7	0.214 7
TrNN	0.183 2	0.236 5	0.082 8	0.256 5
IRNN-LP	0.183 3	0.118 8	0.080 6	0.188 4
TSPN	0.153 6	0.083 1	0.099 3	0.147 8
LNM-QR	0.158 1	0.091 8		0.144 9
MFF-N	0.132 3	0.075 7	0.050 9	0.134 6
ZTDL	0.132 1	0.075 6	0.041 4	0.123 6

Fig. 6 Visual comparisons of seven Algorithms in recovering the 4 sequences with 30% missing data ((a) Walk; (b) Run; (c) Gymnastics; (d) Dance)

Fig. 7 Visual comparisons of seven Algorithms in recovering four sequences with 30% noisy data ((a) Walk; (b) Run; (c) Gymnastics; (d) Dance)

Fig. 8 MRMSE bar chart comparisons for this paper method and four other classical algorithms under different pollution conditions ((a) Random noise; (b) Randomly missing; (c) Continuously missing; (d) Mixed noise)

Fig. 9 RMSE curves comparisons of the LSTM[25] and ZTDL methods ((a)Random noise; (b) Randomly missing)

Table 5 MRMSE comparisons between the Transformer [25] and ZTDL methods

算法	缺失比例/%	Basketball	Boxing	Jump
Transformer^[25]	10	0.44	2.14	0.52
ZTDL	10	0.73	0.59	0.41
Transformer^[25]	20	0.57	2.63	0.56
ZTDL	20	0.76	0.63	0.45
Transformer^[25]	30	0.59	2.56	0.59
ZTDL	30	0.82	1.35	0.48

Fig. 10 ACC bar chart of TS and ZTDL in the missing region estimation in the case of randomly missing pollution

Table 6 MRMSE comparisons of various algorithms with different ACC values for recovering the randomly missing Gymnastics sequence

算法	ACC
算法	0.889 0	0.997 8	1
TSMC	10.701 7	0.032 6	0.032 5
TRNN	9.603 0	0.028 2	0.027 9
TSPN	10.205 9	0.025 8	0.025 8
IRNN-Lp	6.285 2	0.026 2	0.026 1
ZTDL	10.935 7	0.015 0	0.014 8

References 29

[1]	CHEN K, WANG Y P, ZHANG S H, et al. MoCap-solver: a neural solver for optical motion capture data[J]. ACM Transactions on Graphics, 2021, 40(4): 84.
[2]	DONG J T, SHUI Q, ZHANG Y Q, et al. Motion capture from internet videos[C]//The 16th European Conference on Computer Vision-ECCV 2020. Cham: Springer, 2020: 210-227.
[3]	XIA G Y, SUN H J, ZHANG G Q, et al. Human motion recovery jointly utilizing statistical and kinematic information[J]. Information Sciences, 2016, 339: 189-205. DOI URL
[4]	FENG Y F, JI M M, XIAO J, et al. Mining spatial-temporal patterns and structural sparsity for human motion data denoising[J]. IEEE Transactions on Cybernetics, 2015, 45(12): 2693-2706. DOI PMID
[5]	MARTINI E, CALANCA A, BOMBIERI N. Denoising and completion filters for human motion software: a survey with code[J]. Computer Science Review, 2025, 58: 100780. DOI URL
[6]	MOHAOUI S, DMYTRYSHYN A. Low-rank completion for motion capture data recovery: approaches, constraints, and algorithms[J]. Computer Science Review, 2026, 60: 100878. DOI URL
[7]	CANDÈS E, RECHT B. Exact matrix completion via convex optimization[J]. Communications of the ACM, 2012, 55(6): 111-119.
[8]	CANDES E J, TAO T. The power of convex relaxation: near-optimal matrix completion[J]. IEEE Transactions on Information Theory, 2010, 56(5): 2053-2080. DOI URL
[9]	LAI R Y Q, YUEN P C, LEE K K W. Motion capture data completion and denoising by singular value thresholding[EB/OL]. [2025-07-02]. https://dblp.uni-trier.de/db/conf/eurographics/eg-short2011.html#conf/eurographics/LaiYL11.
[10]	FENG Y F, XIAO J, ZHUANG Y T, et al. Exploiting temporal stability and low-rank structure for motion capture data refinement[J]. Information Sciences, 2014, 277: 777-793. DOI URL
[11]	赫高峰, 彭淑娟, 柳欣. 结合模糊聚类和投影近似点算法的缺失人体运动捕捉数据重构[J]. 计算机辅助设计与图形学学报, 2015, 27(8): 1416-1425.
	HE G F, PENG S J, LIU X, et al. Missing human motion capture data recovery via fuzzy clustering and projected proximal point algorithm[J]. Journal of Computer-Aided Design & Computer Graphics, 2015, 27(8): 1416-1425(in Chinese).
[12]	胡文玉, 朱雪芳, 易云. 利用Capped核范数正则化的人体运动捕获数据恢复[J]. 计算机辅助设计与图形学学报, 2023, 35(8): 1184-1196.
	HU W Y, ZHU X F, YI Y. Human motion capture data recovery using capped nuclear norm regularization[J]. Journal of Computer-Aided Design & Computer Graphics, 2023, 35(8): 1184-1196(in Chinese).
[13]	LU C Y, TANG J H, YAN S C, et al. Nonconvex nonsmooth low rank minimization via iteratively reweighted nuclear norm[J]. IEEE Transactions on Image Processing, 2016, 25(2): 829-839. DOI PMID
[14]	HU W Y, WANG Z, LIU S, et al. Motion capture data completion via truncated nuclear norm regularization[J]. IEEE Signal Processing Letters, 2018, 25(2): 258-262. DOI URL
[15]	CHEN B J, SUN H J, XIA G Y, et al. Human motion recovery utilizing truncated schatten p-norm and kinematic constraints[J]. Information Sciences, 2018, 450: 89-108. DOI URL
[16]	RAJ M S S, GEORGE S N. A fast and efficient approach for human action recovery from corrupted 3-D motion capture data using QR decomposition-based approximate SVD[J]. IEEE Transactions on Human-Machine Systems, 2024, 54(4): 395-405. DOI URL
[17]	HUANG H Y, LI Q M, GAO Y, et al. Multi-level fine-grained fusion based robust low-rank approximation for human motion data restoration[J]. Neurocomputing, 2025, 658: 131587. DOI URL
[18]	YAN M. Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting[J]. SIAM Journal on Imaging Sciences, 2013, 6(3): 1227-1245. DOI URL
[19]	SUN L, JEON B, ZHENG Y H, et al. Hyperspectral image restoration using low-rank representation on spectral difference image[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(7): 1151-1155. DOI URL
[20]	BECK A. First-order methods in optimization[M]. Philadelphia: SIAM, 2017: 129-178.
[21]	BOYD S, PARIKH N, CHU E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends^® in Machine Learning, 2011, 3(1): 1-122.
[22]	CAI J F, CANDÈS E J, SHEN Z W. A singular value thresholding algorithm for matrix completion[J]. SIAM Journal on Optimization, 2010, 20(4): 1956-1982. DOI URL
[23]	Carnegie Mellon University. Motion capture database[EB/OL]. [2025-03-16]. http://mocap.cs.cmu.edu.
[24]	Max Planck Institute for Informatics. Motion capture database hdm05[EB/OL]. [2025-03-16]. http://resources.Mpi-inf.mgp.de/HDM05/.
[25]	ZHANG J J, PENG J L, LV N. Spatial-temporal transformer network for human Mocap data recovery[C]// 2023 IEEE International Conference on Image Processing. New York: IEEE Press, 2023: 2305-2309.
[26]	HOLDEN D. Robust solving of optical motion capture data by denoising[J]. ACM Transactions on Graphics (TOG), 2018, 37(4): 165.
[27]	PAN X Y, ZHENG B W, JIANG X W, et al. A locality-based neural solver for optical motion capture[C]// SIGGRAPH Asia 2023 Conference Papers. New York: ACM, 2023: 117.
[28]	YOU Y X, LIU H, LI X, et al. Gator: graph-aware transformer with motion-disentangled regression for human mesh recovery from a 2D pose[C]// ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE Press, 2023: 1-5.
[29]	MOHAOUI S, DMYTRYSHYN A. CP decomposition-based algorithms for completion problem of motion capture data[J]. Pattern Analysis and Applications, 2024, 27(4): 133. DOI