基于密度图多目标追踪的时空数据可视化

doi:10.11996/JG.j.2095-302X.2024061289

图学学报 ›› 2024, Vol. 45 ›› Issue (6): 1289-1300.DOI: 10.11996/JG.j.2095-302X.2024061289

• 图像处理与计算机视觉 • 上一篇下一篇

基于密度图多目标追踪的时空数据可视化

宋思程¹^,²(), 陈辰³, 李晨辉¹, 王长波¹()

1.华东师范大学计算机科学与技术学院，上海 200062
2.华东师范大学上海智能教育研究院，上海 200062
3.华东师范大学软件工程学院，上海 200062

收稿日期:2024-07-08 接受日期:2024-09-06 出版日期:2024-12-31 发布日期:2024-12-24
通讯作者:王长波(1976-)，男，教授，博士。主要研究方向为计算机图形学、数据可视分析、数字媒体与虚拟现实。E-mail：cbwang@cs.ecnu.edu.cn
第一作者:宋思程(1997-)，男，博士研究生。主要研究方向为数据可视化、深度学习。E-mail：scsong@stu.ecnu.edu.cn
基金资助:
国家自然科学基金(62072183);上海自然科学基金面上项目(24ZR1418300);上海市长三角科技创新共同体领域项目(23002400400)

Spatiotemporal data visualization based on density map multi-target tracking

SONG Sicheng¹^,²(), CHEN Chen³, LI Chenhui¹, WANG Changbo¹()

1. School of Computer Science and Technology, East China Normal University, Shanghai 200062, China
2. Shanghai Institute of Artificial Intelligence for Education, East China Normal University, Shanghai 200062, China
3. Software Engineering Institute, East China Normal University, Shanghai 200062, China

Received:2024-07-08 Accepted:2024-09-06 Published:2024-12-31 Online:2024-12-24
Contact: WANG Changbo (1976-), professor, Ph.D. His main research directions cover computer graphics, data visualization analysis, digital media, and virtual reality. E-mail：cbwang@cs.ecnu.edu.cn
First author：SONG Sicheng (1997-), PhD candidate. His main research interests cover data visualization and deep learning. E-mail：scsong@stu.ecnu.edu.cn
Supported by:
National Natural Science Foundation of China(62072183);Shanghai Natural Science Foundation(24ZR1418300);Shanghai Yangtze River Delta Science and Technology Innovation Community Project(23002400400)

摘要/Abstract

摘要：

时空数据追踪的可视化问题已经受到了广泛的关注，其研究重点在于展示数据的动态细节，并确保轨迹与观测结果的一致性。为此，提出了一种融合深度学习与传统追踪技术的模型，用于执行追踪任务，从而提高可视化的速度和准确度。首先，生成一个高质量的柏林噪声数据集并在该数据集上训练了一个多目标追踪模型。其次，提出了双阶段、多模型的深度学习框架来增强对动态场景的分析深度。最后，为了能够连续地展现详尽的追踪信息，提出了一种可以增强追踪信息的视觉效果结合轨迹和矢量场的可视化解决方案。在不同的案例中展示了该方法的有用性和鲁棒性，并从多个方面进行了量化评估和比较。结果表明该方法可以帮助用户在不同场景中理解多目标追踪信息。

关键词: 照数据可视化, 深度学习, 时空数据, 多目标跟踪

Abstract:

The spatiotemporal data tracking visualization has received widespread attention. The focus of this research is on depicting the dynamic details of the data and ensuring trajectory consistency with the observation results. In this paper, a model that combined deep learning with traditional tracking techniques was proposed to perform tracking tasks, thereby improving the speed and accuracy of visualization. First, a high-quality Perlin noise dataset was generated, on which a multi-target tracking model was trained. Second, a two-stage, multi-model deep learning framework was proposed to enhance the analysis depth of dynamic scenes. Finally, in order to continuously display detailed tracking information, a visualization solution that combined trajectories and vector fields was introduced to enhance the visual effect of tracking information. Different cases in this study demonstrated the usefulness and robustness of the proposed method, quantitatively evaluating and omparing the method from multiple aspects. The results showed that the method proposed in this study can help users in understanding multi-target tracking information in different scenarios.

Key words: data visualization, deep learning, spatial-temporal data, multiple-object tracking

中图分类号:

TP391

宋思程, 陈辰, 李晨辉, 王长波. 基于密度图多目标追踪的时空数据可视化[J]. 图学学报, 2024, 45(6): 1289-1300.

SONG Sicheng, CHEN Chen, LI Chenhui, WANG Changbo. Spatiotemporal data visualization based on density map multi-target tracking[J]. Journal of Graphics, 2024, 45(6): 1289-1300.

图/表 14

图1 多目标追踪可视化的挑战((a)模型的泛化性问题；(b)目标形态变化的信息缺失；(c)不同阈值下的目标划分)

Fig. 1 Challenges of multi-target tracking visualization ((a) Generalization problem of the model; (b) Missing information about target morphological changes; (c) Target segmentation under different thresholds)

图2 总体内容示意图

Fig. 2 Overview of the paper structure

图3 多目标追踪模型

Fig. 3 Multi-target tracking model

图4 训练PG-Flow的过程示意图

Fig. 4 Schematic diagram of the training process of PG-Flow

图5 目标追踪中的可视化符号((a)定位标识；(b)时间渐变指示；(c)微动增强；(d)流场可视化)

Fig. 5 Visual symbols in target tracking ((a) Positioning mark; (b) Time gradient indication; (c) Micromotion enhancement; (d) Flow field visualization)

图6 向量场的动态可视化((a)初始噪声图像；(b)光流可视化；(c)所有场的可视化；(d)关键场的可视化)

Fig. 6 Dynamic visualization of vector fields ((a) Initial noise image; (b) Optical flow visualization; (c) Visualization of all fields; (d) Visualization of key fields)

图7 随机多目标追踪可视化((a)输入图像；(b)追踪结果)

Fig. 7 Visualization of random multi-target tracking ((a) Input image; (b) Tracking result)

图8 购物中心数据集的人群追踪可视化((a)购物中心人群数据集；(b)密度图转换结果；(c)人群追踪可视化结果)

Fig. 8 Crowd tracking visualization for the shopping mall dataset ((a) Shopping mall crowd dataset; (b) Density map conversion result; (c) Crowd tracking visualization result)

图9 二氧化碳排放量的可视化 ((a) 2006年10月5日南半球CO2变化的可视化；(b) 2006年10月25日南半球CO2变化的可视化)

Fig. 9 Visualization of CO2 emissions ((a) Visualization of CO2 changes in the Southern Hemisphere on Oct. 5, 2006; (b) Visualization of CO2 changes in the Southern Hemisphere on Oct. 25, 2006)

图10 随机多目标柏林噪声的追踪评估结果

Fig. 10 Evaluation results of random multi-target tracking with Perlin noise

图11 本方法与传统方法的追踪量化比较((a)轨迹长度的方差；(b)长度阈值为10的轨迹长度)

Fig. 11 Comparison of tracking quantification between this method and traditional methods ((a) Variance of trajectory length; (b) Trajectory length with a length threshold of 10)

图12 本方法与传统方法追踪轨迹比较((a)传统方法的追踪结果；(b)本方法的追踪结果)

Fig. 12 Comparison of tracking trajectories between this method and traditional methods ((a) Tracking results of the traditional method; (b) Tracking results of this method)

图13 FlowNet2, PG-Flow与PG-Flow-1的比较((a)连续柏林噪声数据；(b)购物中心人群数据)

Fig. 13 Comparison between FlowNet2, PG-Flow and PG-Flow-1 ((a) Continuous Perlin noise data; (b) Crowd in a shopping mall data)

图14 不同模型追踪结果与真值相似度的对比

Fig. 14 Comparison of the similarity between different model tracking results and the ground truth

参考文献 47

[1]	DORAISWAMY H, NATARAJAN V, NANJUNDIAH R S. An exploration framework to identify and track movement of cloud systems[J]. IEEE Transactions on Visualization and Computer Graphics, 2013, 19(12): 2896-2905.
[2]	PARK Y, LEPETIT V, WOO W. Extended keyframe detection with stable tracking for multiple 3D object tracking[J]. IEEE Transactions on Visualization and Computer Graphics, 2011, 17(11): 1728-1735.
[3]	VALSANGKAR A A, MONTEIRO J M, NARAYANAN V, et al. An exploratory framework for cyclone identification and tracking[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(3): 1460-1473.
[4]	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1134-1149.
[5]	ILG E, MAYER N, SAIKIA T, et al. FlowNet 2.0: evolution of optical flow estimation with deep networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 1647-1655.
[6]	CABRAL B, LEEDOM L C. Imaging vector fields using line integral convolution[C]// The 20th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM, 1993: 263-270.
[7]	SMEULDERS A W M, CHU D M, CUCCHIARA R, et al. Visual tracking: an experimental survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(7): 1442-1468. DOI PMID
[8]	WANG N Y, YEUNG D Y. Ensemble-based tracking: aggregating crowdsourced structured time series data[C]// The 31st International Conference on International Conference on Machine Learning. Cambridge: JMLR.Org, 2014: II-1107-II-1115.
[9]	OESTERLING P, HEINE C, WEBER G H, et al. Computing and visualizing time-varying merge trees for high-dimensional data[M]//CARR H, GARTH C, WEINKAUF T. Topological Methods in Data Analysis and Visualization: Theory, Algorithms, and Applications. Cham: Springer, 2017: 87-101.
[10]	MUTHUMANICKAM P K, VROTSOU K, NORDMAN A, et al. Identification of temporally varying areas of interest in long-duration eye-tracking data sets[J]. IEEE Transactions on Visualization and Computer Graphics, 2019, 25(1): 87-97.
[11]	RAUTENHAUS M, BÖTTINGER M, SIEMEN S, et al. Visualization in meteorology-a survey of techniques and tools for data analysis tasks[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(12): 3268-3296.
[12]	SBALZARINI I F, KOUMOUTSAKOS P. Feature point tracking and trajectory analysis for video imaging in cell biology[J]. Journal of Structural Biology, 2005, 151(2): 182-195. PMID
[13]	COMANICIU D, RAMESH V, MEER P. Kernel-based object tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(5): 564-577.
[14]	WANG L, TAN T N, NING H Z, et al. Silhouette analysis-based gait recognition for human identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(12): 1505-1518.
[15]	CIAPARRONE G, SÁNCHEZ F L, TABIK S, et al. Deep learning in video multi-object tracking: a survey[J]. Neurocomputing, 2020, 381: 61-88.
[16]	DANELLJAN M, HÄGER G, KHAN F S, et al. Convolutional features for correlation filter based visual tracking[C]// 2015 IEEE International Conference on Computer Vision Workshop. New York: IEEE Press, 2015: 621-629.
[17]	BEWLEY A, GE Z Y, OTT L, et al. Simple online and realtime tracking[C]// 2016 IEEE International Conference on Image Processing. New York: IEEE Press, 2016: 3464-3468.
[18]	REDMON J, FARHADI A. Yolov3: an incremental improvement[EB/OL]. [2024-05-08]. https://arxiv.org/abs/1804.02767.
[19]	LIN T Y, DOLLÁR P, GIRSHICK R B, et al. Feature pyramid networks for object detection[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 936-944.
[20]	POPPE R. Vision-based human motion analysis: an overview[J]. Computer Vision and Image Understanding, 2007, 108(1/2): 4-18.
[21]	WANG L, HU W M, TAN T J. Recent developments in human motion analysis[J]. Pattern Recognition, 2003, 36(3): 585-601.
[22]	ZHOU T H, BROWN M, SNAVELY N, et al. Unsupervised learning of depth and ego-motion from video[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2017: 6612-6619.
[23]	SAQIB M, KHAN S D, SHARMA N, et al. Extracting descriptive motion information from crowd scenes[C]// 2017 International Conference on Image and Vision Computing New Zealand. New York: IEEE Press, 2017: 1-6.
[24]	BROX T, MALIK J. Large displacement optical flow: descriptor matching in variational motion estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(3): 500-513. DOI PMID
[25]	JOBARD B, RAY N, SOKOLOV D. Visualizing 2D flows with animated arrow plots[EB/OL]. [2024-05-08]. https://arxiv.org/abs/1205.5204.
[26]	LEFER W, JOBARD B, LEDUC C. High-quality animation of 2D steady vector fields[J]. IEEE Transactions on Visualization and Computer Graphics, 2004, 10(1): 2-14. PMID
[27]	CHEN S, LI S H, CHEN S M, et al. R-Map: a map metaphor for visualizing information reposting process in social media[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(1): 1204-1214.
[28]	WU Y C, LIU S X, YAN K, et al. OpinionFlow: visual analysis of opinion diffusion on social media[J]. IEEE Transactions on Visualization and Computer Graphics, 2014, 20(12): 1763-1772.
[29]	LI C H, BACIU G, WANG Y Z, et al. DDLVis: real-time visual query of spatiotemporal data distribution via density dictionary learning[J]. IEEE Transactions on Visualization and Computer Graphics, 2022, 28(1): 1062-1072.
[30]	KIM S, JEONG S, WOO I, et al. Data flow analysis and visualization for spatiotemporal statistical data without trajectory information[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(3): 1287-1300.
[31]	LI C H, BACIU G, HAN Y. StreamMap: smooth dynamic visualization of high-density streaming points[J]. IEEE Transactions on Visualization and Computer Graphics, 2018, 24(3): 1381-1393.
[32]	CHEN C, WANG C B, BAI X, et al. GenerativeMap: visualization and exploration of dynamic density maps via generative learning model[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(1): 216-226.
[33]	刘灿, 赖楚凡, 蒋瑞珂, 等. 深度学习驱动的可视化[J]. 计算机辅助设计与图形学学报, 2020, 32(10): 1537-1548.
	LIU C, LAI C F, JIANG R K, et al. Visualization driven by deep learning[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(10): 1537-1548. (in Chinese)
[34]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE Press, 2016: 770-778.
[35]	SZEGEDY C, IOFFE S, VANHOUCKE V, et al. Inception-v4, inception-ResNet and the impact of residual connections on learning[C]// The 31st AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2017: 4278-4284.
[36]	RIBEIRO M I. Kalman and extended Kalman filters: concept, derivation and properties[EB/OL]. [2024-05-08]. https://people.duke.edu/-hpgavin/SystemID/References/Ribeiro-KalmanFilter-2004.pdf.
[37]	LUETTEKE F, ZHANG X, FRANKE J. Implementation of the Hungarian Method for object tracking on a camera monitored transportation system[C]// ROBOTIK 2012; 7th German Conference on Robotics. New York: IEEE Press, 2012: 1-6.
[38]	BOTEV Z, GROTOWSKI J, KROESE D P. Kernel density estimation via diffusion[J]. The Annals of Statistics, 2010, 38(5): 2916-2957.
[39]	PERLIN K. Improving noise[J]. ACM Transactions on Graphics (TOG), 2002, 21(3): 681-682.
[40]	BAKER S, SCHARSTEIN D, LEWIS J, et al. A database and evaluation methodology for optical flow[J]. International Journal of Computer Vision, 2011, 92(1): 1-31.
[41]	FARNEBÄCK G. Two-frame motion estimation based on polynomial expansion[C]// The 13th Scandinavian Conference on Image Analysis. Cham: Springer, 2003: 363-370.
[42]	HERSHBERGER J, SNOEYINK J. Speeding up the douglas-peucker line-simplification algorithm[EB/OL]. [2024-05-08]. https://dl.acm.org/doi/book/10.5555/902273.
[43]	BAO P, ZHANG L, WU X L. Canny edge detection enhancement by scale multiplication[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(9): 1485-1490. PMID
[44]	CHEN K, LOY C C, GONG S, et al. Feature mining for localised crowd counting[EB/OL]. [2024-05-08]. https://personal.ie.cuhk.edu.hk/-ccloy/project_feat_mine_count/index.html.
[45]	VENKATESAN R, KOON S M, JAKUBOWSKI M H, et al. Robust image hashing[C]// 2000 International Conference on Image Processing. New York: IEEE Press, 2000: 664-666.
[46]	VEGA F, MEDINA J, MENDOZA D, et al. A robust video identification framework using perceptual image hashing[C]// 2017 XLIII Latin American Computer Conference. New York: IEEE Press, 2017: 1-10.
[47]	陈辰. 基于空间特征挖掘的时空数据可视化研究[D]. 上海: 华东师范大学, 2020.
	CHEN C. Research on spatio-temporal data visualization based on spatial features mining[D]. Shanghai: East China Normal University, 2020.. (in Chinese)

基于密度图多目标追踪的时空数据可视化

Spatiotemporal data visualization based on density map multi-target tracking

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 14

参考文献 47

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李琼, 考月英, 张莹, 徐沛. 面向无人机航拍图像的目标检测研究综述[J]. 图学学报, 2024, 45(6): 1145-1164.
[2]	刘灿锋, 孙浩, 东辉. 结合Transformer与Kolmogorov Arnold网络的分子扩增时序预测研究[J]. 图学学报, 2024, 45(6): 1256-1265.
[3]	王宗继, 刘云飞, 陆峰. Cloud Sphere: 一种基于渐进式变形自编码的三维模型表征方法[J]. 图学学报, 2024, 45(6): 1375-1388.
[4]	许丹丹, 崔勇, 张世倩, 刘雨聪, 林予松. 优化医学影像三维渲染可视化效果：技术综述[J]. 图学学报, 2024, 45(5): 879-891.
[5]	胡凤阔, 叶兰, 谭显峰, 张钦展, 胡志新, 方清, 王磊, 满孝锋. 一种基于改进YOLOv8的轻量化路面病害检测算法[J]. 图学学报, 2024, 45(5): 892-900.
[6]	刘义艳, 郝婷楠, 贺晨, 常英杰. 基于DBBR-YOLO的光伏电池表面缺陷检测[J]. 图学学报, 2024, 45(5): 913-921.
[7]	翟永杰, 李佳蔚, 陈年昊, 王乾铭, 王新颖. 融合改进Transformer的车辆部件检测方法[J]. 图学学报, 2024, 45(5): 930-940.
[8]	姜晓恒, 段金忠, 卢洋, 崔丽莎, 徐明亮. 融合先验知识推理的表面缺陷检测[J]. 图学学报, 2024, 45(5): 957-967.
[9]	熊超, 王云艳, 罗雨浩. 特征对齐与上下文引导的多视图三维重建[J]. 图学学报, 2024, 45(5): 1008-1016.
[10]	胡欣, 常娅姝, 秦皓, 肖剑, 程鸿亮. 基于改进YOLOv8和GMM图像点集匹配的双目测距方法[J]. 图学学报, 2024, 45(4): 714-725.
[11]	牛为华, 郭迅. 基于改进YOLOv8的船舰遥感图像旋转目标检测算法[J]. 图学学报, 2024, 45(4): 726-735.
[12]	李滔, 胡婷, 武丹丹. 结合金字塔结构和注意力机制的单目深度估计[J]. 图学学报, 2024, 45(3): 454-463.
[13]	朱光辉, 缪君, 胡宏利, 申基, 杜荣华. 基于自增强注意力机制的室内单图像分段平面三维重建[J]. 图学学报, 2024, 45(3): 464-471.
[14]	王稚儒, 常远, 鲁鹏, 潘成伟. 神经辐射场加速算法综述[J]. 图学学报, 2024, 45(1): 1-13.
[15]	王欣雨, 刘慧, 朱积成, 盛玉瑞, 张彩明. 基于高低频特征分解的深度多模态医学图像融合网络[J]. 图学学报, 2024, 45(1): 65-77.