Abstract
When linearity can be rigorously assumed for stochastic processes, the linear Kalman filter can be used as a powerful tool for anomaly detection in communication networks. However, this assumption done with a strong evidence is not generally proved in a rigorous way. So it is important to develop other methodology, for the scope of anomaly detection, which are not obliged to be based on that assumption. This paper is focused on the use of particle filtering to build a normal behavioral model for an anomaly detector. The particle filter is calibrated for entropy reduction for the scope of noise reduction in the measurements. With the help of a mixture of normal distributions, we can reuse the filtered observations to identify anomalous events in a few number of classes. Generally anomalies might be rare and thus they might happen on a few clusters. So, using a new decision process based on a hidden markov model, we can track and identify the potential abnormal clusters. We study the performances of this system by analyzing the false alarm rate vs detection rate trade-off by means of Receiver Operating Characteristic curve, and compare the results with the Kalman filter. We validate the approach to track volume anomalies over real network traffic.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bergman N., Recursive Bayesian estimation: Navigation and tracking applications, Ph.D. dissertation, Linkoping University, Linkoping, Sweden (1999)
Carpenter, J., Clifford, P., Fearnhead, P.: Improved particle filter for nonlinear problems. Proc. Inst. Elect. Eng, Radar Sonar Navig. (1999)
Crisan, D., Del Moral, P., Lyons, T.J.: Non-linear filtering using branching and interacting particle systems. Markov Process. Relat. Fields 5(3), 293–319 (1999)
Del Moral, P.: Non-linear filtering: interacting particle solution. Markov Process. Relat. Fields 2(4), 555–580
Doucet A., de Freitas J.F.G., Gordon N.J.: An introduction to sequential Monte Carlo methods, in sequential Monte Carlo methods in practice. In: Doucet, A., de Freitas, J.F.G., Gordon, N.J. (eds.) New York, Springer (2001)
Doucet, A.: On sequential Monte Carlo methods for Bayesian filtering, Department of Engineering, University of Cambridge, UK, Technical report (1998)
Gordon, N., Salmond, D., Smith, A.F.M.: Novel approach to nonlinear and non-Gaussian Bayesian state estimation. Proc. Inst. Elect. Eng. F 140, 107–113 (1993)
Lakhina, A., Crovella, M., Diot, C.: Characterization of network-wide traffic anomalies. In: ACM Sigmetrics (2004)
Lakhina, A., Crovella, M., Diot, C.: Diagnosing network-wide traffic anomalies. In: SIGCOMM 2004: Proceedings of the 2004 conference on Applications. technologies, architectures, and protocols for computer communications, pp. 219–230. ACM Press, New York, NY, USA (2004)
Liu, J.S., Chen, R.: Sequential Monte Carlo methods for dynamical systems. J. Amer. Stat. Assoc. 93, 1032–1044 (1998)
MacCormick J., Blake A.: A probabilistic exclusion principle for tracking multiple objects. In: Proceedings of the International Conference on Computer Vision, pp. 572–578 (1999)
Maybeck, P.: Stochastic Models, Estimation and Control, vol. 2. Academic Press (1982). Using MATLAB. Wiley Interscience (2001)
Ndong, J., Salamatian, K.: A robust anomaly detection technique using combined statistical methods. In: CNSR 2011, IEEE Xplore, pp. 101–108, May 2011. ISBN: 978-1-4577-0040-8
Ndong, J., Salamatian, K.: Signal Processing-based Anomaly Detection Techniques: A Comparative Analysis. In: The Third International Conference on Evolving Internet. INTERNET 2011. ISBN: 978-1-61208-141-0
Ndong, J.: Anomaly Detection: A Technique Using Kalman Filtering and Principal Component Analysis. ATAI NTC 2012 GSTF 2012
Ripley, B.: Stochastic Simulation. Wiley, New York (1987)
Sanjeev Arulampalam, M., Maskell, S., Gordon, N., Clapp, T.A.: Tutorial on particle filters for online nonlinear/non-gaussian Bayesian tracking. IEEE Trans. Signal Process. 50(2) (2002)
Shumway, R.H., Stoffer, D.S.: An approach to time series smoothing and forecasting using the EM algorithm. J. Time Ser. Anal. 3(4)
Sumway, R.H., Stoffer, D.S.: Dynamic linear model with switching. J. Am. Stat. Assoc. 86 (1991)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Gueye, I., Ndong, J. (2016). Particle Filtering as a Modeling Tool for Anomaly Detection in Networks. In: Cherifi, H., Gonçalves, B., Menezes, R., Sinatra, R. (eds) Complex Networks VII. Studies in Computational Intelligence, vol 644. Springer, Cham. https://doi.org/10.1007/978-3-319-30569-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-30569-1_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-30568-4
Online ISBN: 978-3-319-30569-1
eBook Packages: EngineeringEngineering (R0)