Abstract
Moving object monitoring is becoming essential for companies and organizations that need to manage thousands or even millions of commercial vehicles or vessels, detect dangerous situations (e.g., collisions or malfunctions) and optimize their behavior. It is a task that must be executed in real-time, reporting any such situations or opportunities as soon as they appear. Given the growing sizes of fleets worldwide, a monitoring system must be highly efficient and scalable. It is becoming an increasingly common requirement that such monitoring systems should be able to automatically detect complex situations, possibly involving multiple moving objects and requiring extensive background knowledge. Building a monitoring system that is both expressive and scalable is a significant challenge. Typically, the more expressive a system is, the less flexible it becomes in terms of its parallelization potential. We present a system that strikes a balance between expressiveness and scalability. Going beyond event detection, we also present an approach towards event forecasting. We show how event patterns may be given a probabilistic description so that our system can forecast when a complex event is expected to occur. Our proposed system employs a formalism that allows analysts to define complex patterns in a user-friendly manner while maintaining unambiguous semantics and avoiding ad hoc constructs. At the same time, depending on the problem at hand, it can employ different parallelization strategies in order to address the issue of scalability. It can also employ different training strategies in order to fine-tune the probabilistic models constructed for event forecasting. Our experimental results show that our system can detect complex patterns over moving entities with minimal latency, even when the load on our system surpasses what is to be realistically expected in real-world scenarios.
Similar content being viewed by others
Notes
[1]: R2C3: motivating the forecasting part of the paper.
[2]: R2C1: clarifying the relationship of this paper to its workshop version.
[3]: R1C2,R2C7: commenting on the generic nature of our engine.
[4] R2C6, R2C11: providing a more concrete example in the context of moving object monitoring.
[5] R2C6, R2C11 continued.
[6] R2C6, R2C11 continued.
[7] R1C1: explaining why we focus on CLASSIFICATION-NETXW.
[8] R2C10: explaining that Wayeb does not necessarily require preprocessing of the input stream.
R2C9: explaining that noise patterns are treated as complex events.
[10] R3C2: showing results in a table.
[11] R1C3: explaining why we currently use only classification metrics.
[12] R1C4: adding a note about order optimization.
References
Apache flink - stateful computations over data streams. https://flink.apache.org/
Apache kafka. https://kafka.apache.org/
Esperonstorm. https://github.com/tomdz/storm-esper
Flinkcep - complex event processing for flink. https://ci.apache.org/projects/flink/flink-docs-stable/dev/libs/cep.html
Monitoring Back Pressure. https://ci.apache.org/projects/flink/flink-docs-release-1.9/monitoring/back∖_pressure.html
Siddhi cep. https://github.com/wso2/siddhi
Task chaining and resource groups. https://ci.apache.org/projects/flink/flink-docs-release-1.9/dev/stream/operators/∖#task-chaining-and-resource-groups
Wso2. creating a storm based distributed execu-tionplan. https://docs.wso2.com/display/CEP410/Creating+a+Storm+Based+Distributed+Execution+Plan
Abe N, Warmuth MK (1992) On the computational complexity of approximating distributions by probabilistic automata. Mach Learn 9:205–260
Alevizos E, Artikis A, Paliouras G (2017) Event forecasting with pattern markov chains. In: DEBS
Alevizos E, Artikis A, Paliouras G (2018) Wayeb: a tool for complex event forecasting. In: LPAR
Alevizos E, Artikis A, Paliouras G (2021) Complex event forecasting with prediction suffix trees. VLDB J
Alevizos E, Skarlatidis A, Artikis A, Paliouras G (2017) Probabilistic complex event recognition: A survey. ACM Comput Surv 50(5):71:1–71:31
Artikis A, Sergot M, Paliouras G (2015) An event calculus for event recognition. IEEE Trans Knowl Data Eng
Balkesen C, Dindar N, Wetter M, Tatbul N (2013) RIP: run-based intra-query parallelism for scalable complex event processing. In: DEBS
Begleiter R, El-Yaniv R, Yona G (2004) On prediction using variable order markov models. J Artif Intell Res 22:385–421
Carbone P, Katsifodimos A, Ewen S, Markl V, Haridi S, Tzoumas K (2015) Apache flink™: Stream and batch processing in a single engine. IEEE Data Eng Bull
Christ M, Krumeich J, Kempa-Liehr AW (2016) Integrating predictive analytics into complex event processing by using conditional density estimations. In: EDOC Workshops. IEEE Computer Society, pp 1–8
Cleary JG, Witten IH (1984) Data compression using adaptive coding and partial string matching. IEEE Trans Commun 32(4):396–402
Cugola G, Margara A (2012) Complex event processing with T-REX. J Syst Softw
Cugola G, Margara A (2012) Processing flows of information: From data stream to complex event processing. ACM Comput Surv
D’Antoni L, Veanes M (2017) The power of symbolic automata and transducers. In: CAV (1)
Demers A, Gehrke J, Panda B, Riedewald M, Sharma V, White W (2007) Cayuga: A general purpose event monitoring system. In: CIDR
Engel Y, Etzion O (2011) Towards proactive event-driven computing. In: DEBS. ACM, pp 125–136
Fülöp LJ, Beszėdes Á, Toth G, Demeter H, Vidȧcs L, Farkas L (2012) Predictive complex event processing: a conceptual framework for combining complex event processing and predictive analytics. In: BCI. ACM, pp 26–31
Giatrakos N, Alevizos E, Artikis A, Deligiannakis A, Garofalakis M (2020) Complex event recognition in the big data era: a survey. VLDB J
Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction, and estimation. J Am Stat Assoc 102(477):359–378
Hirzel M (2012) Partition and compose: parallel complex event processing. In: DEBS
Koutroumanis N, Santipantakis G, Glenis A, Doulkeridis C, Vouros G (2019) Integration of mobility data with weather information. In: EDBT/ICDT Workshops
Li Y, Ge T, Chen CX (2020) Data stream event prediction based on timing knowledge and state transitions. Proc VLDB Endow 13(10):1779–1792
Liu M, Rundensteiner E, Greenfield K, Gupta C, Wang S, Ari I, Mehta A (2011) E-cube: multi-dimensional event sequence analysis using hierarchical pattern query sharing. In: SIGMOD
Mei Y, Madden S (2009) Zstream: a cost-based query processor for adaptively detecting composite events. In: SIGMOD
Muthusamy V, Liu H, Jacobsen H (2010) Predictive publish/subscribe matching. In: DEBS. ACM, pp 14–25
Ntoulias E, Alevizos E, Artikis A, Koumparos A (2021) Online trajectory analysis with scalable event recognition. In: EDBT/ICDT Workshops, CEUR Workshop Proceedings. CEUR-WS.org, vol 2841
Pandey S, Nepal S, Chen S (2011) A test-bed for the evaluation of business process prediction techniques. In: CollaborateCom. ICST / IEEE, pp 382–391
Patroumpas K, Alevizos E, Artikis A, Vodas M, Pelekis N, Theodoridis Y (2017) Online event recognition from moving vessel trajectories. GeoInformatica
Patroumpas K, Spirelis D, Chondrodima E, Georgiou H, P P P (2018) Final dataset of Trajectory Synopses over AIS kinematic messages in Brest area (ver 0.8) [Data set]. https://doi.org/10.5281/zenodo.2563256
Pitsikalis M, Artikis A, Dreo R, Ray C, Camossi E, Jousselme A (2019) Composite event recognition for maritime monitoring. In: DEBS
Ray C, Dreo R, Camossi E, Jousselme A (2018) Heterogeneous integrated dataset for maritime intelligence, surveillance and reconnaissance. https://doi.org/10.5281/zenodo.1167595
Ron D, Singer Y, Tishby N (1993) The power of amnesia. In: NIPS. Morgan Kaufmann, pp 176–183
Ron D, Singer Y, Tishby N (1996) The power of amnesia: Learning probabilistic automata with variable memory length. Mach Learn 25(2-3):117–149
Sakr MA, Gu̇ting RH (2011) Spatiotemporal pattern queries. GeoInformatica 15(3):497–540
Sakr MA, Gu̇ting RH (2014) Group spatiotemporal pattern queries. GeoInformatica 18(4):699–746
Schultz-Møller N, Migliavacca M, Pietzuch P (2009) Distributed complex event processing with query rewriting. In: DEBS
Snidaro L, Visentini I, Bryan K (2015) Fusing uncertain knowledge and evidence for maritime situational awareness via markov logic networks. Inf Fusion
Terroso-Saenz F, Valdės-Vela M, Skarmeta-Gȯmez A (2016) A complex event processing approach to detect abnormal behaviours in the marine environment. Inf. Syst Frontiers
Tsilionis E, Koutroumanis N, Nikitopoulos P, Doulkeridis C, Artikis A (2019) Online event recognition from moving vehicles: Application paper TPLP
Willems FMJ, Shtarkov YM, Tjalkens TJ (1995) The context-tree weighting method: basic properties. IEEE Trans Inf Theory 41(3):653–664
Zhang H, Diao Y, Immerman N (2014) On complexity and optimization of expensive queries in complex event processing. In: SIGMOD
Acknowledgements
This work was funded by European Union’s Horizon 2020 research and innovation programme Track & Know “Big Data for Mobility Tracking Knowledge Extraction in Urban Areas”, under grant agreement No 780754. It is also supported by the European Commission under the INFORE project (H2020-ICT- 825070).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Ntoulias, E., Alevizos, E., Artikis, A. et al. Online fleet monitoring with scalable event recognition and forecasting. Geoinformatica 26, 613–644 (2022). https://doi.org/10.1007/s10707-022-00465-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10707-022-00465-2