Abstract
Per-flow size measurement, which is to count the number of packets for each active flow during a certain measurement period, has many applications in usage accounting, traffic engineering, service provision, and anomaly detection. In order to maintain the high throughput of routers or switchers, the per-flow size measurement module should use high-bandwidth SRAM that allows fast memory accesses. Due to the limited SRAM space, exact counting, which requires to keep a counter for each flow, does not scale to measure big network data consisting of numerous flows. Some recent work takes a different design path to accurately estimate the flow sizes using counter architectures that can fit into tight SRAM. However, existing counter architectures have some limitations, either still requiring considerable SRAM space, or having a very small estimation range. This chapter presents a scalable counter architecture, called Counter Tree, which leverages a two-dimensional counter sharing technique to achieve far better memory efficiency and significantly extend estimation range. Furthermore, we improve the performance of Counter Tree by adding a status bit to each counter. The extensive experiments with real network trace demonstrate that the new architecture can produce accurate estimates for flows of all sizes even under a very tight memory space, e.g., 1 bit per flow.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Since \([\ln (f)]^{{\prime}} = \frac{f^{{\prime}}} {f}\), we know f ′ = f[ln(f)]′.
References
Chen, M., Chen, S.: Counter tree: a scalable counter architecture for per-flow traffic measurement. In: Proceedings of IEEE ICNP, pp. 111–122 (2015)
Cormode, G., Muthukrishnan, S.: An improved data stream summary: the Count-Min sketch and its applications. In: Proceedings of LATIN (2004)
Duffield, N., Lund, C., Thorup, M.: Estimating flow distributions from sampled flow statistics. In: Proceedings of ACM SIGCOMM (2003)
Einziger, G., Friedman, R.: Counting with tinytable: every bit counts! In: Proceedings of IEEE INFOCOM Workshops, pp. 77–78 (2015)
Estan, C., Varghese, G.: New directions in traffic measurement and accounting. In: Proceedings of ACM SIGCOMM (2002)
Kamiyama, N., Mori, T.: Simple and accurate identification of high-rate flows by packet sampling. In: Proceedings of IEEE INFOCOM (2006)
Kumar, A., Sung, M., Xu, J., Wang, J.: Data streaming algorithms for efficient and accurate estimation of flow size distribution. In: Proceedings of ACM SIGMETRICS (2004)
Kumar, A., Xu, J., Wang, J.: Space-code bloom filter for efficient per-flow traffic measurement. IEEE J. Sel. Areas Commun. 24 (12), 2327–2339 (2006)
Lehmann, E., Casella, G.: Theory of Point Estimation. Springer, New York (1998)
Li, T., Chen, S., Ling, Y.: Fast and compact per-flow traffic measurement through randomized counter sharing. In: Proceedings of IEEE INFOCOM, pp. 1799–1807 (2011)
Li, T., Chen, S., Ling, Y.: Per-flow traffic measurement through randomized counter sharing. IEEE/ACM Trans. Netw. 20 (5), 1622–1634 (2012)
Lu, Y., Prabhakar, B.: Robust counting via Counter Braids: an error-resilient network measurement architecture. In: Proceedings of IEEE INFOCOM (2009)
Lu, Y., Montanari, A., Prabhakar, B., Dharmapurikar, S., Kabbani, A.: Counter Braids: a novel counter architecture for per-flow measurement. In: Proceedings of ACM SIGMETRICS (2008)
Mikians, J., Dhamdhere, A., Dovrolis, C., Barlet-Ros, P., Solé-Pareta, J.: Towards a statistical characterization of the interdomain traffic matrix, pp. In: Networking 2012, pp. 111–123. Springer, Berlin (2012)
Ramabhadran, S., Varghese, G.: Efficient implementation of a statistics counter architecture. In: ACM SIGMETRICS Performance Evaluation Review, vol. 31, pp. 261–271 (2003)
Shah, D., Iyer, S., Prabhakar, B., McKeown, N.: Analysis of a statistics counter architecture. In: Hot Interconnects 9, 2001, pp. 107–111. IEEE, New York (2001)
Zhao, Q., Xu, J., Liu, Z.: Design of a novel statistics counter architecture with optimal space and time efficiency. In: ACM SIGMETRICS Performance Evaluation Review, vol. 34(1), pp. 323–334 (2006)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this chapter
Cite this chapter
Chen, S., Chen, M., Xiao, Q. (2017). Per-Flow Size Measurement. In: Traffic Measurement for Big Network Data. Wireless Networks. Springer, Cham. https://doi.org/10.1007/978-3-319-47340-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-47340-6_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47339-0
Online ISBN: 978-3-319-47340-6
eBook Packages: EngineeringEngineering (R0)