Abstract
Most existing wearable sensor-based human activity recognition (HAR) models are trained on substantial labeled data. It is difficult for HAR to learn new-class activities unseen during training from a few samples. Very few researches of few-shot learning (FSL) have been done in HAR to address the above problem, though FSL has been widely used in computer vision tasks. Besides, it is impractical to annotate sensor data with accurate activity labels in real-life applications. The noisy labels have great negative effects on FSL due to the limited samples. The weakly supervised few-shot learning in HAR is challenging, significant but rarely researched in existing literature. In this paper, we propose an end-to-end Weakly supervised Prototypical Networks (WPN) to learn more latent information from noisy data with multiple instance learning (MIL). In MIL, the noisy instances (subsequences of segmentation) have different labels from the bag’s (segmentation’s) label. The prototype is the center of the instances in WPN rather than less discriminative bags, which determines the bag-level classification accuracy. To get the most representative instance-level prototype, we propose two strategies to refine the prototype by selecting high-probability instances same as their bag’s label iteratively based on the distance-metric. The model is trained by minimizing the instance-level loss function and infers the final bag-level labels from instance-level labels. In the experiments, our proposals outperform existing approaches and achieve higher average ranks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arifoglu, D., Bouchachia, A.: Activity recognition and abnormal behaviour detection with recurrent neural networks. Proc. Comput. Sci. 110, 86–93 (2017)
Boney, R., Ilin, A.: Semi-supervised few-shot learning with prototypical networks. CoRR (2017)
Cao, J., Li, W., Ma, C., Tao, Z.: Optimizing multi-sensor deployment via ensemble pruning for wearable activity recognition. Inf. Fusion 41, 68–79 (2018)
Deng, S., Wang, B., Yang, C., Wang, G.: Convolutional neural networks for human activity recognition using multi-location wearable sensors. Ruan Jian Xue Bao/J. Softw. 30(3), 718–737 (2019)
Dennis, D., Pabbaraju, C., Simhadri, H.V., Jain, P.: Multiple instance learning for efficient sequential data classification on resource-constrained devices. In: Advances in Neural Information Processing Systems (Nips), vol. 31, pp. 10976–10987 (2018)
Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89(1–2), 31–71 (1997)
Fe-Fei, L., et al.: A bayesian approach to unsupervised one-shot learning of object categories. In: Proceedings Ninth IEEE International Conference on Computer Vision, pp. 1134–1141. IEEE (2003)
Feng, S., Duarte, M.F.: Few-shot learning-based human activity recognition. arXiv preprint arXiv:1903.10416 (2019)
Gao, T., Han, X., Liu, Z., Sun, M.: Hybrid attention-based prototypical networks for noisy few-shot relation classification. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-19), New York, USA (2019)
Guan, X., Raich, R., Wong, W.K.: Efficient multi-instance learning for activity recognition from time series data using an auto-regressive hidden markov model. In: International Conference on Machine Learning, pp. 2330–2339 (2016)
Hoffer, E., Ailon, N.: Deep metric learning using triplet network. In: Feragen, A., Pelillo, M., Loog, M. (eds.) SIMBAD 2015. LNCS, vol. 9370, pp. 84–92. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24261-3_7
Kautz, T., Groh, B.H., Hannink, J., Jensen, U., Strubberg, H., Eskofier, B.M.: Activity recognition in beach volleyball using a deep convolutional neural network. Data Mining Knowl. Disc. 31(6), 1678–1705 (2017)
Koch, G., Zemel, R., Salakhutdinov, R.: Siamese neural networks for one-shot image recognition. In: ICML deep learning workshop, vol. 2 (2015)
Liu, L., Zhou, T., Long, G., Jiang, J., Yao, L., Zhang, C.: Prototype propagation networks (ppn) for weakly-supervised few-shot learning on category graph. arXiv preprint arXiv:1905.04042 (2019)
Liu, Y., et al.: Learning to propagate labels: transductive propagation network for few-shot learning. arXiv preprint arXiv:1805.10002 (2018)
Martin, K., Wijekoon, A., Wiratunga, N.: Human activity recognition with deep metric learners. In: CEUR Workshop Proceedings (2019)
Mishra, A., Verma, V.K., Reddy, M.S.K., Arulkumar, S., Rai, P., Mittal, A.: A generative approach to zero-shot and few-shot action recognition. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 372–380. IEEE (2018)
Munkhdalai, T., Yu, H.: Meta networks. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 2554–2563. JMLR. org (2017)
Nguyen-Dinh, L.V., Calatroni, A., Tröster, G.: Supporting one-time point annotations for gesture recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39(11), 2270–2283 (2016)
Ohashi, H., Al-Naser, M., Ahmed, S., Nakamura, K., Sato, T., Dengel, A.: Attributes’ importance for zero-shot pose-classification based on wearable sensors. Sensors 18(8), 2485 (2018)
Pan, W., Li, Z., Zhang, Y., Weng, C.: The new hardware development trend and the challenges in data management and analysis. Data Sci. Eng. 3(3), 263–276 (2018). https://doi.org/10.1007/s41019-018-0072-6
Ren, M., et al.: Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676 (2018)
Snell, J., Swersky, K., Zemel, R.: Prototypical networks for few-shot learning. In: Advances in Neural Information Processing Systems, pp. 4077–4087 (2017)
Stikic, M., Larlus, D., Ebert, S., Schiele, B.: Weakly supervised recognition of daily life activities with wearable sensors. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2521–2537 (2011)
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al.: Matching networks for one shot learning. In: Advances in Neural Information Processing Systems, pp. 3630–3638 (2016)
Wang, S., Cao, J., Yu, P.S.: Deep learning for spatio-temporal data mining: A survey. arXiv preprint arXiv:1906.04928 (2019)
Wang, Y., Yuan, Y., Ma, Y., Wang, G.: Time-dependent graphs: definitions, applications, and algorithms. Data Sci. Eng. 4(4), 352–366 (2019)
Xu, B., Ye, H., Zheng, Y., Wang, H., Luwang, T., Jiang, Y.G.: Dense dilated network for few shot action recognition. In: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval, pp. 379–387. ACM (2018)
Acknowledgements
This work is partially supported by the National Natural Science Foundation of China (Grant No. 61732003 61729201, 61932004 and N181605012), the Australian Queensland Government (Grant No. AQRF12516), and the Australian Research Council (DP170101172).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Deng, S., Hua, W., Wang, B., Wang, G., Zhou, X. (2020). Few-Shot Human Activity Recognition on Noisy Wearable Sensor Data. In: Nah, Y., Cui, B., Lee, SW., Yu, J.X., Moon, YS., Whang, S.E. (eds) Database Systems for Advanced Applications. DASFAA 2020. Lecture Notes in Computer Science(), vol 12113. Springer, Cham. https://doi.org/10.1007/978-3-030-59416-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-030-59416-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59415-2
Online ISBN: 978-3-030-59416-9
eBook Packages: Computer ScienceComputer Science (R0)