Skip to main content

Advertisement

SpringerLink
Log in

Smart Handheld Based Human Activity Recognition Using Multiple Instance Multiple Label Learning

  • Published:
Wireless Personal Communications Aims and scope Submit manuscript

Abstract

Human activity recognition (HAR) and monitoring is beneficial for many medical applications, such as eldercare and post-trauma rehabilitation after surgery. HAR models based on smartphone’s accelerometer data could provide a convenient and ubiquitous solution to this problem. However, such models are mostly concerned with identifying basic activities such as ‘stand’/‘walk’ and thus the high-level context such as ‘walk in a queue’ for which a set of specific activities is performed remain unnoticed. Consequently, in this paper, we design a HAR framework that can identify a group of activities (rather than a single basic activity) being performed in a time window, thus, enables us to extract more meaningful information about the subject’s overall context. An algorithm is designed to formulate HAR as a multi-instance multi-label (MIML) learning problem. The procedure of generating feature bags of consecutive activity traces having multiple labels is formulated. In this work, the temporal relationship among activities is exploited to obtain a more comprehensive HAR model. Interestingly, the framework is found to completely/partially identify activity sequences that may not even be present in the training dataset. The framework is implemented and found to be working adequately when tested with real dataset collected from 8 users for 12 different activity combinations. MIML-kNN is found to provide maximum average precision (around 90%) even for an unseen test data-set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. GSensorLogger: https://play.google.com/store/apps/details?id=com.peterhohsy.gsensor_debug&hl=en.

  2. Matlab 2013. Avaliable online: https://www.mathworks.com.

References

  1. Shany, T., Redmond, S. J., Narayanan, M. R., & Lovell, N. H. (2012). Sensors-based wearable systems for monitoring of human movement and falls. IEEE Sensors Journal, 12(3), 658–670.

    Article  Google Scholar 

  2. Ren, L., & Shi, W. (2015). Chameleon: Personalised and adaptive fall detection of elderly people in home-based environments. International Journal of Sensor Networks, 20, 163–176.

    Article  Google Scholar 

  3. Wannenburg, J., & Malekian, R. (2017). Physical activity recognition from smartphone accelerometer data for user context awareness sensing. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 47(12), 3142–3149.

    Article  Google Scholar 

  4. Guo, M., Wang, Z., Yang, N., Li, Z., & An, T. (2019). A multisensor multiclassifier hierarchical fusion model based on entropy weight for human activity recognition using wearable inertial sensors. IEEE Transactions on Human–Machine Systems, 49(1), 105–111.

    Article  Google Scholar 

  5. Saha, J., Chowdhury, C., & Biswas, S. (2018). Two phase ensemble classifier for smartphone based human activity recognition independent of hardware configuration and usage behaviour. Springer Journal of Microsystem Technologies, 24(6), 2737–2752.

    Article  Google Scholar 

  6. Saha, J., Chowdhury, C., Roy Chowdury, I., Biswas, S., & Aslam, N. (2018). An ensemble of condition based classifiers for device independent detailed human activity recognition using smartphones. MDPI Information, 9(94), 1–22.

    Google Scholar 

  7. Stikic, M., & Schiele, B. (2009). Activity recognition from sparsely labeled data using multi-instance learning. In T. Choudhury, A. Quigley, T. Strang, & K. Suginuma (Eds.), Location and Context Awareness. LoCA. LNCS 2009 (pp. 156–173). London: Springer.

    Google Scholar 

  8. Toda, T., Inoue, S., Tanaka, S., & Ueda, N. (2014). Training human activity recognition for labels with inaccurate time stamps. In Proceedings of the 2014 ACM international joint conference on pervasive and ubiquitous computing: Adjunct publication, UbiComp’14 Adjunct (pp. 863–872).

  9. Gupta, P., & Dallas, T. (2014). Feature selection and activity recognition system using a single triaxial accelerometer. IEEE Transactions on Biomedical Engineering, 61(6), 1780–1786.

    Article  Google Scholar 

  10. Yuan, Y., Wang, C., Zhang, J., Xu, J., & Li, M. (2014). An ensemble approach for activity recognition with accelerometer in mobile-phone. In 2014 IEEE 17th international conference on computational science and engineering (pp. 1469–1474).

  11. Rezaie, H., & Ghassemian, M. (2018). Comparison analysis of radio based and sensor based wearable human activity recognition systems. Wireless Personal Communications, 2018, 1–23.

    Google Scholar 

  12. Choi, S., & Yi, G. (2016). Energy consumption and efficiency issues in human activity monitoring system. Wireless Personal Communications, 91, 1799–1815.

    Article  Google Scholar 

  13. Bao, L., & Intille, S. (2004). Activity recognition from user-annotated acceleration data. Pervasive Computing, 3001, 1–17.

    Article  Google Scholar 

  14. Saguna, S., Zaslavsky, A., & Chakraborty, D. (2011). Complex activity recognition using context driven activity theory in home environments. In S. Balandin, Y. Koucheryavy, & H. Hu (Eds.), Smart spaces and next generation wired/wireless networking. ruSMART 2011, NEW2AN 2011. LNCS (pp. 38–50). London: Springer.

    Google Scholar 

  15. Dernbach, S., Das, B., Krishnan, N. C., Thomas, B. L., & Cook, D. J. (2012). Simple and complex activity recognition through smart phones. In 2012 8th international conference on intelligent environments (pp. 214–221).

  16. Shoaib, M., Bosch, S., Incel, O., Scholten, H., & Havinga, P. (2016). Complex human activity recognition using smartphone and wrist-worn motion sensors. Sensors, 16, 426.

    Article  Google Scholar 

  17. Roy, N., Mishra, A., & Cook, D. (2016). Ambient and Smartphone sensor assisted ADL recognition in multi-inhabitant smart environments. Journal of Ambient Intelligence and Humanized Computing, 7(1), 1–9.

    Article  Google Scholar 

  18. Gani, M. O., Saha, A. K., Ahsan, G. M. T., & Ahamed, S. I. (2017). A novel framework to recognize complex human activity. In IEEE 41st annual computer software and applications conference (COMPSAC) (pp. 948–956).

  19. Achanta, S. D. M., Karthikeyan, T., & Vinothkanna, R. (2019). A novel hidden Markov model-based adaptive dynamic time warping (HMDTW) gait analysis for identifying physically challenged persons. Soft Computing, 23, 8359–8366.

    Article  Google Scholar 

  20. Peng, L., Chen, L., Wu, X., Guo, H., & Chen, G. (2017). Hierarchical complex activity representation and recognition using topic model and classifier level fusion. IEEE Transactions on Biomedical Engineering, 64(6), 1369–1379.

    Article  Google Scholar 

  21. Nandy, A., Saha, J., & Chowdhury, C. (2020). Novel features for intensive human activity recognition based on wearable and Smartphone sensors. Microsystem Technologies, 26, 1889–1903.

    Article  Google Scholar 

  22. Guan, X., Raich, R., & Wong, W. K. (2016). Efficient multi-instance learning for activity recognition from time series data using an auto-regressive hidden Markov model. In Proceedings of the 33rd international conference on international conference on machine learning—Volume 48, ICML’16 (pp. 2330–2339). JMLR.org.

  23. Stikic, M., Larlus, D., & Schiele, B. (2009). Multi-graph based semi-supervised learning for activity recognition. In 2009 international symposium on wearable computers (pp. 85–92).

  24. Liu, Y., Nie, L., Liu, L., Zhang, L., & Rosenblum, D. (2016). From action to activity: Sensor-based activity recognition. Neurocomputing, 181, 108–115.

    Article  Google Scholar 

  25. Zhang, M. (2010). A k-nearest neighbor based multi-instance multi-label learning algorithm. In 2010 22nd IEEE international conference on tools with artificial intelligence (Vol. 2, pp. 207–212).

  26. Wang, J., & Zucker, J. D. (2000). Solving the multiple-instance problem: A lazy learning approach. In Proceedings of the 17th international conference on machine learning, ICML’00 (pp. 1119–1126). San Francisco, CA.

  27. Wu, J., Huang, S., & Zhou, Z. (2014). Genome-wide protein function prediction through multi-instance multi-label learning. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 11(5), 891–902.

    Article  Google Scholar 

  28. Wang, W., Guo, Y., Huang, B., Zhao, G., Liu, B., & Wang, L. (2011). Analysis of filtering methods for 3D acceleration signals in body sensor network. International Symposium on Bioelectronics and Bioinformations, 2011, 263–266.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Artificial Intelligence and Data Science, Koneru Lakshmaiah Education Foundation, Deemed to be University, Hyderabad, India

    Jayita Saha

  2. Machine Intelligence Unit, Indian Statistical Institute, Kolkata, India

    Dip Ghosh & Sanghamitra Bandyopadhyay

  3. Department of CSE, Jadavpur University, Kolkata, India

    Chandreyee Chowdhury

Authors
  1. Jayita Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dip Ghosh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chandreyee Chowdhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sanghamitra Bandyopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jayita Saha.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Saha, J., Ghosh, D., Chowdhury, C. et al. Smart Handheld Based Human Activity Recognition Using Multiple Instance Multiple Label Learning. Wireless Pers Commun 117, 923–943 (2021). https://doi.org/10.1007/s11277-020-07903-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-020-07903-0

Keywords

Search

Navigation