Skip to main content
SpringerLink
Log in

Internal Clustering Evaluation of Data Streams

  • Conference paper
  • First Online:
Trends and Applications in Knowledge Discovery and Data Mining

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9441))

Abstract

Clustering validation is a crucial part of choosing a clustering algorithm which performs best for an input data. Internal clustering validation is efficient and realistic, whereas external validation requires a ground truth which is not provided in most applications. In this paper, we analyze the properties and performances of eleven internal clustering measures. In particular, as the importance of streaming data grows, we apply these measures to carefully synthesized stream scenarios to reveal how they react to clusterings on evolving data streams. A series of experimental results show that different from the case with static data, the Calinski-Harabasz index performs the best in coping with common aspects and errors of stream clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: A framework for clustering evolving data streams. In: VLDB, pp. 81–92 (2003)

    Google Scholar 

  2. Bifet, A., Holmes, G., Pfahringer, B., Kranen, P., Kremer, H., Jansen, T., Seidl, T.: MOA: massive online analysis, a framework for stream classification and clustering. JMLR 11, 44–50 (2010)

    Google Scholar 

  3. Calinski, T., Harabasz, J.: A dendrite method for cluster analysis. Commun. Stat. 3(1), 1–27 (1974)

    MathSciNet  MATH  Google Scholar 

  4. Cao, F., Ester, M., Qian, W., Zhou, A.: Density-based clustering over an evolving data stream with noise. In: SIAM SDM, pp. 328–339 (2006)

    Google Scholar 

  5. Davies, D., Bouldin, D.: A cluster separation measure. IEEE PAMI 1(2), 224–227 (1979)

    Article  Google Scholar 

  6. Dunn, J.: Well separated clusters and optimal fuzzy partitions. J. Cybern. 4(1), 95–104 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  7. Halkidi, M., Batistakis, Y., Vazirgiannis, M.: On clustering validation techniques. J. Intell. Inf. Syst. 17(2), 107–145 (2001)

    Article  MATH  Google Scholar 

  8. Halkidi, M., Vazirgiannis, M.: Clustering validity assessment: finding the optimal partitioning of a data set. In IEEE ICDM, pp. 187–194 (2001)

    Google Scholar 

  9. Vazirgiannis, M., Halkidi, M., Batistakis, Y.: Quality scheme assessment in the clustering process. In: Żytkow, J.M., Zighed, D.A., Komorowski, J. (eds.) PKDD 2000. LNCS (LNAI), vol. 1910, pp. 265–276. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  10. Hassani, M., Kim, Y., Seidl, T.: Subspace MOA: subspace stream clustering evaluation using the MOA framework. In: DASFAA, pp. 446–449 (2013)

    Google Scholar 

  11. Hassani, M., Kranen, P., Saini, R., Seidl, T.: Subspace anytime stream clustering. In: SSDBM, p. 37 (2014)

    Google Scholar 

  12. Hassani, M., Spaus, P., Gaber, M.M., Seidl, T.: Density-based projected clustering of data streams. In: Link, S., Fober, T., Seeger, B., Hüllermeier, E. (eds.) SUM 2012. LNCS, vol. 7520, pp. 311–324. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  13. Hassani, M., Spaus, P., Seidl, T.: Adaptive multiple-resolution stream clustering. In: Perner, P. (ed.) MLDM 2014. LNCS, vol. 8556, pp. 134–148. Springer, Heidelberg (2014)

    Google Scholar 

  14. Hubert, L., Arabie, P.: Comparing partitions. J. Intell. Inf. Syst. 2(1), 193–218 (1985)

    MATH  Google Scholar 

  15. Kremer, H., Kranen, P., Jansen, T., Seidl, T., Bifet, A., Holmes, G., Pfahringer, B.: An effective evaluation measure for clustering on evolving data streams. In: ACM SIGKDD, pp. 868–876 (2011)

    Google Scholar 

  16. Liu, Y., Li, Z., Xiong, H., Gao, X., Wu, J.: Understanding of internal clustering validation measures. In: ICDM, pp. 911–916 (2010)

    Google Scholar 

  17. Liu, Y., Li, Z., Xiong, H., Gao, X., Wu, J., Wu, S.: Understanding and enhancement of internal clustering validation measures. IEEE Trans. Cybern. 43(3), 982–994 (2013)

    Article  Google Scholar 

  18. Maulik, U., Bandyopadhyay, S.: Performance evaluation of some clustering algorithms and validity indices. IEEE PAMI 24, 1650–1654 (2002)

    Article  Google Scholar 

  19. Rendón, E., Abundez, I., Arizmendi, A., Quiroz, E.M.: Internal versus external cluster validation indexes. Int. J. Comput. Commun. 5(1), 27–34 (2011)

    Google Scholar 

  20. Rezaee, M.R., Lelieveldt, B.B.F., Reiber, J.H.C.: A new cluster validity index for the fuzzy c-mean. Pattern Recogn. Lett. 19(3–4), 237–246 (1998)

    Article  MATH  Google Scholar 

  21. Rousseeuw, P.: Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Comput. Appl. Math. 20(1), 53–65 (1987)

    Article  MATH  Google Scholar 

  22. Tan, P.-N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley Longman Inc., Boston (2005)

    Google Scholar 

  23. Xie, X.L., Beni, G.: A validity measure for fuzzy clustering. IEEE PAMI 13(8), 841–847 (1991)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Data Management and Data Exploration Group, RWTH Aachen University, Aachen, Germany

    Marwan Hassani & Thomas Seidl

Authors
  1. Marwan Hassani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Seidl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marwan Hassani .

Editor information

Editors and Affiliations

  1. Institute of Infocomm Research, Singapore, Singapore

    Xiao-Li Li

  2. Ho Chi Minh City University of Tech, Ho Chi Minh City, Vietnam

    Tru Cao

  3. School of Information Systems, Singapore Management University, Singapore, Singapore

    Ee-Peng Lim

  4. Nanjing University, Nanjing, China

    Zhi-Hua Zhou

  5. Science & Technology, Japan Advanced Institute of, Nomi-shi, Ishikawa, Japan

    Tu-Bao Ho

  6. The University of Hong Kong, Hong Kong, China

    David Cheung

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Hassani, M., Seidl, T. (2015). Internal Clustering Evaluation of Data Streams. In: Li, XL., Cao, T., Lim, EP., Zhou, ZH., Ho, TB., Cheung, D. (eds) Trends and Applications in Knowledge Discovery and Data Mining. Lecture Notes in Computer Science(), vol 9441. Springer, Cham. https://doi.org/10.1007/978-3-319-25660-3_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25660-3_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25659-7

  • Online ISBN: 978-3-319-25660-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation