Abstract
Anomaly detection of multivariate time series has become a critical task in the medical domain. With the quick development of medical devices, a large amount of multi-source heterogeneous multivariate time series data is produced. Traditional data platforms have difficulties to organize and explore these data. In addition, the high dimensionality of multivariate time series also makes it difficult for the detection process to capture the correlation between different features. In this paper, we propose an anomaly detection framework based on data lake for medical multivariate time series. Firstly, heterogeneous data are fused to provide a big wide table. Then we help the doctors to explore data, filter out related features and get a filtered dataset. To reduce redundant or noisy data in the filtered dataset, we refine it with Relief algorithm. Finally, we identify anomalies through a multi-scale convolutional recurrent encoder-decoder. Experiments on a synthetic dataset and a use case of heart sound recordings confirm the validity of our framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. (CSUR) 41(3), 1–58 (2009)
Qiao, Z., He, J., Cao, J., Huang, G., Zhang, P.: Multiple time series anomaly detection based on compression and correlation analysis: a medical surveillance case study. In: Sheng, Q.Z., Wang, G., Jensen, C.S., Xu, G. (eds.) APWeb 2012. LNCS, vol. 7235, pp. 294–305. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29253-8_25
Dixon, J.: Pentaho, Hadoop, and Data Lakes, October 2010. https://jamesdixon.wordpress.com/2010/10/14/pentaho-hadoop-and-data-lakes/
Yamada, M., Kimura, A., Naya, F., et al.: Change-point detection with feature selection in high-dimensional time-series data. In: Twenty-Third International Joint Conference on Artificial Intelligence (2013)
He, W., Wang, Z., Jiang, H.: Model optimizing and feature selecting for support vector regression in time series forecasting. Neurocomputing 72(1–3), 600–611 (2008)
Yong, Z., Ming, S., Rui, Z., et al.: HKGB: an inclusive, extensible, intelligent, semi-auto-constructed knowledge graph framework for healthcare with clinicians’ expertise incorporated. Inf. Process. Manag. 57(6), 102324 (2020)
He, J., Rong, J., Sun, L., Wang, H., Zhang, Y., Ma, J.: A framework for cardiac arrhythmia detection from IoT-based ECGs. World Wide Web 23(5), 2835–2850 (2020). https://doi.org/10.1007/s11280-019-00776-9
Hill, D.J., Minsker, B.S.: Anomaly detection in streaming environmental sensor data: a data-driven modeling approach. Environ. Model. Softw. 25(9), 1014–1022 (2010)
Haque, S.A., Rahman, M., Aziz, S.M.: Sensor anomaly detection in wireless sensor networks for healthcare. Sensors 15(4), 8764–8786 (2015)
Ren, P., Lin, W., Liang, Y., Wang, R., Liu, X., Zuo, B., Chen, T., Li, X., Sheng, M., Zhang, Y.: HMDFF: a heterogeneous medical data fusion framework supporting multimodal query. In: Siuly, S., Wang, H., Chen, Lu., Guo, Y., Xing, C. (eds.) HIS 2021. LNCS, vol. 13079, pp. 254–266. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90885-0_23
Laptev, N., Amizadeh, S., Flint, I.: Generic and scalable framework for automated time-series anomaly detection. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2015)
Venkataraman, S., Caballero, J., Song, D., et al.: Black box anomaly detection: is it utopian? Irvine Is Burning 127 (2006)
Siuly, S., Alçin, Ö.F., Kabir, E., et al.: A new framework for automatic detection of patients with mild cognitive impairment using resting-state EEG signals. IEEE Trans. Neural Syst. Rehabil. Eng. 28(9), 1966–1976 (2020)
Zhang, C., Song, D., Chen, Y., et al.: A deep neural network for unsupervised anomaly detection and diagnosis in multivariate time series data. In: The Thirty-Third AAAI Conference on Artificial Intelligence (2019)
Kieu, T., Yang, B., Guo, C., et al.: Outlier detection for time series with recurrent autoencoder ensembles. In IJCAI (2019)
Goodge, A., Hooi, B., Ng, S.-K., et al.: Robustness of autoencoders for anomaly detection under adversarial impact. In: IJCAI (2020)
Lykourentzou, I., Papadaki, K., Kalliakmanis, A., et al.: Ontology-based operational risk management. In: IEEE 13th Conference on Commerce and Enterprise Computing. IEEE (2011)
Glimm, B., Horrocks, I., Motik, B., et al.: HermiT: an OWL 2 reasoner. J. Autom. Reason. 53(3), 245–269 (2014). https://doi.org/10.1007/s10817-014-9305-1
Liu, C., Springer, D., Li, Q., et al.: An open access database for the evaluation of heart sound algorithms. Physiol. Meas. 37(12), 2181 (2016)
Acknowledgement
This work was supported by National Key R&D Program of China (2020AAA0109603).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Ren, P. et al. (2022). An Anomaly Detection Framework Based on Data Lake for Medical Multivariate Time Series. In: Traina, A., Wang, H., Zhang, Y., Siuly, S., Zhou, R., Chen, L. (eds) Health Information Science. HIS 2022. Lecture Notes in Computer Science, vol 13705. Springer, Cham. https://doi.org/10.1007/978-3-031-20627-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-20627-6_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20626-9
Online ISBN: 978-3-031-20627-6
eBook Packages: Computer ScienceComputer Science (R0)