Abstract
Federated Learning (FL) lately has shown much promise in improving sharing model and preserving data privacy. However, these existing methods are only of limited utility in the Internet of Things (IoT) scenarios, as they either heavily depend on high-quality labeled data or only perform well under idealized distribution conditions, which typically cannot be found in practical applications. In this work, we propose FedGAN, a Generative Adversarial Network (GAN) based federated learning method for semi-supervised image classification. In IoT scenarios, a big challenge is that decentralized data among multiple clients are normally non-independent and identically distributed (non-IID), leading to performance degradation. To address this issue, we further propose a dynamic aggregation mechanism that can adaptively adjust client weights in aggregation. Extensive experiments on three benchmarks demonstrate that FedGAN outperforms related federated semi-supervised learning methods, including a 55.36% accuracy on CIFAR-10 with 2k labels and 70.65% accuracy on SVHN with 1k labels - just 100 labels per class. Moreover, we carry out an extensive ablation and robust study to tease apart the experimental factors that are important to FedGAN’s improvement.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. Proc. Mach. Learn. Res. 54, 1273–1282 (2017)
Hard, A., et al.: Federated learning for mobile keyboard prediction. arXiv preprint arXiv:1811.03604 (2018)
Tran, N.H., Bao, W., Zomaya, A., Nguyen, M.N.H., Hong, C.S.: Federated learning over wireless networks: Optimization model design and analysis. In: IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, pp. 1387–1395 (2019)
Zhu, Y., Liu, Y., Yu, J.J.Q., Yuan, X.: Semi-supervised federated learning for travel mode identification from GPS trajectories. IEEE Trans. Intell. Transp. Syst. 233, 1–12 (2021)
Itahara, S., Nishio, T., Koda, Y., Morikura, M., Yamamoto, K.: Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. arXiv preprint arXiv:2008.06180 (2020)
Jeong, W., Yoon, J., Yang, E., Hwang, S.J.: Federated semi-supervised learning with inter-client consistency and disjoint learning. arXiv preprint arXiv:2006.12097 (2020)
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X.: Improved techniques for training gans. Adv. Neural. Inf. Process. Syst. 29, 2234–2242 (2016)
Li, C., Xu, K., Zhu, J., Zhang, B.: Triple generative adversarial nets. CoRR abs/1703.02291 (2017)
Dai, Z., Yang, Z., Yang, F., Cohen, W.W., Salakhutdinov, R.: Good semi-supervised learning that requires a bad GAN. CoRR abs/1705.09783 (2017)
Li, W., et al.: Semi-supervised learning using adversarial training with good and bad samples. Mach. Vis. Appl. 31(6), 1–11 (2020). https://doi.org/10.1007/s00138-020-01096-z
Guo, M., Haque, A., Huang, D.-A., Yeung, S., Fei-Fei, L.: Dynamic task prioritization for multitask learning. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11220, pp. 282–299. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01270-0_17
Lin, T., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2999–3007 (2017)
Oliver, A., Odena, A., Raffel, C., Cubuk, E.D., Goodfellow, I.J.: Realistic evaluation of deep semi-supervised learning algorithms. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 3239–3250. NIPS (2018)
Sattler, F., Wiedemann, S., Müller, K.R., Samek, W.: Robust and communication-efficient federated learning from non-iid data. IEEE Trans. Neural Netw. Learn. Syst. 31(9), 3400–3413 (2019)
Xie, Q., Dai, Z., Hovy, E., Luong, M.T., Le, Q.V.: Unsupervised data augmentation for consistency training. arXiv preprint arXiv:1904.12848 (2019)
Lee, D.H.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML, vol. 3, pp. 896 (2013)
Chang, Q., et al.: Synthetic learning: Learn from distributed asynchronized discriminator gan without sharing medical image data. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13856–13866 (2020)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei, L.F.: ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)
Rahman, T., et al.: Exploring the effect of image enhancement techniques on covid-19 detection using chest x-ray images. Comput. Biol. Med. 132, 104319 (2021)
Acknowledgemnt
This work is supported by the National Natural Science Foundation of China (62072049).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zhao, C., Gao, Z., Wang, Q., Mo, Z., Yu, X. (2022). FedGAN: A Federated Semi-supervised Learning from Non-IID Data. In: Wang, L., Segal, M., Chen, J., Qiu, T. (eds) Wireless Algorithms, Systems, and Applications. WASA 2022. Lecture Notes in Computer Science, vol 13472. Springer, Cham. https://doi.org/10.1007/978-3-031-19214-2_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-19214-2_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19213-5
Online ISBN: 978-3-031-19214-2
eBook Packages: Computer ScienceComputer Science (R0)