Abstract
Nowadays, the electricity load profiles of customers (consumers and prosumers) are changing as new technologies are being developed, and therefore it is necessary to correctly identify new trends, changes and anomalies in data. Anomalies in load consumption can be caused by abnormal behavior of customers or a failure of smart meters in the grid. Accurate identification of such anomalies is crucial for maintaining stability in the grid and reduce electricity loss of distribution companies. Smart meters produce huge amounts of load consumption measurements every day and analyzing all the measurements is computationally expensive and very inefficient. Therefore, the aim of this work is to propose an anomaly detection method, that addresses this issue. Our proposed method firstly narrows down potential anomalous customers in large datasets by clustering discretized time series, and then analyses selected profiles using statistical method S-H-ESD to calculate final anomaly score. We evaluated and compared our method to four state-of-the-art anomaly detection methods on created synthetic dataset of load consumption time series containing collective anomalies. Our method outperformed other evaluated methods in terms of accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Carter, K.M., Streilein, W.W.: Probabilistic reasoning for streaming anomaly detection. In: 2012 IEEE Statistical Signal Processing Workshop (SSP). IEEE, August 2012. https://doi.org/10.1109/ssp.2012.6319708
Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: a survey. ACM Comput. Surv. 41(3), 15:1–15:58 (2009). https://doi.org/10.1145/1541880.1541882
Cleveland, W.S.: Robust locally weighted regression and smoothing scatterplots. J. Am. Stat. Assoc. 74(368), 829–836 (1979)
Fernandes Jr., G., Rodrigues, J.J., Carvalho, L.F., Al-Muhtadi, J.F., Proença Jr., M.L.: A comprehensive survey on network anomaly detection. Telecommun. Syst. 70(3), 447–489 (2019). https://doi.org/10.1007/s11235-018-0475-8
Hasani, Z.: Robust anomaly detection algorithms for real-time big data: comparison of algorithms. In: 2017 6th Mediterranean Conference on Embedded Computing (MECO), pp. 1–6, June 2017. https://doi.org/10.1109/MECO.2017.7977130
Hochenbaum, J., Vallis, O.S., Kejariwal, A.: Automatic anomaly detection in the cloud via statistical learning. CoRR abs/1704.07706 (2017)
Ishimtsev, V., Nazarov, I., Bernstein, A., Burnaev, E.: Conformal k-NN anomaly detector for univariate data streams (2017)
Laurinec, P., Lucká, M.: Interpretable multiple data streams clustering with clipped streams representation for the improvement of electricity consumption forecasting. Data Min. Knowl. Disc. 33(2), 413–445 (2019)
Lima, M.F., Zarpelão, B.B., Sampaio, L.D.H., Rodrigues, J.J.P.C., Abrão, T., Proença, M.L.: Anomaly detection using baseline and k-means clustering. In: SoftCOM 2010, 18th International Conference on Software, Telecommunications and Computer Networks, pp. 305–309, September 2010
Maaten, V.D.L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Qi, J., Chu, Y., He, L.: Iterative anomaly detection algorithm based on time series analysis. In: 2018 IEEE 15th International Conference on Mobile Ad Hoc and Sensor Systems (MASS), pp. 548–552, October 2018
Raza, H., Prasad, G., Li, Y.: EWMA model based shift-detection methods for detecting covariate shifts in non-stationary environments. Pattern Recogn. 48(3), 659–669 (2015). https://doi.org/10.1016/j.patcog.2014.07.028
Salehi, M., Rashidi, L.: A survey on anomaly detection in evolving data: [with application to forest fire risk prediction]. SIGKDD Explor. Newsl. 20(1), 13–23 (2018). https://doi.org/10.1145/3229329.3229332
Salvador, S., Chan, P.: Learning states and rules for detecting anomalies in time series. Appl. Intell. 23(3), 241–255 (2005)
Shao, X., Zhang, M., Meng, J.: Data stream clustering and outlier detection algorithm based on shared nearest neighbor density. In: 2018 International Conference on Intelligent Transportation, Big Data Smart City (ICITBS), pp. 279–282 (2018)
Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison Wesley, Boston (2005)
Vallis, O., Hochenbaum, J., Kejariwal, A.: A novel technique for long-term anomaly detection in the cloud. In: Proceedings of the 6th USENIX Conference on Hot Topics in Cloud Computing, p. 15. USENIX Association, Berkeley (2014)
Wu, H.: A survey of research on anomaly detection for time series. In: 2016 13th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), pp. 426–431, December 2016
Acknowledgment
This work was partially supported by the Slovak Research and Development Agency under the contract APVV-16-0213, and by the Scientific Grant Agency of the Slovak Republic VEGA, grant No. VG 1/0759/19. The authors would also like to thank for financial assistance from the STU Grant scheme for Support of Excellent Teams of Young Researchers (Grant No. 1391).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cuper, M., Lóderer, M., Rozinajová, V. (2019). Detection of Abnormal Load Consumption in the Power Grid Using Clustering and Statistical Analysis. In: Yin, H., Camacho, D., Tino, P., Tallón-Ballesteros, A., Menezes, R., Allmendinger, R. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2019. IDEAL 2019. Lecture Notes in Computer Science(), vol 11871. Springer, Cham. https://doi.org/10.1007/978-3-030-33607-3_50
Download citation
DOI: https://doi.org/10.1007/978-3-030-33607-3_50
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33606-6
Online ISBN: 978-3-030-33607-3
eBook Packages: Computer ScienceComputer Science (R0)