Abstract
The main challenge in deep learning related to the identification of grape leaf diseases is how to achieve good performance in the case of available sparse datasets or limited number of annotated samples, small lesions, redundant information and blurred background information in grape leaf disease images. This paper proposes a three-stage deep learning-based pipeline, including a convolutional neural netword (Faster R-CNN) for detection of lesions, a generative adversarial network (DCGAN) for data augmentation and a residual neural network (ResNet) for identification of lesions, to solve these problems. Firstly, Faster R-CNN was used to mark the location of grape leaf lesions to obtain lesions dataset for data augmentation and ResNet for identification of lesions. Secondly, leaf lesion images were fed into DCGAN to generate synthetic grape lesion images for identification of lesions. Finally, ResNet trained in the training dataset consisting of real grape leaf lesions and synthetic grape lesion images obtained by DCGAN, was used to identify the grape leaf lesions according to the majority voting principle. The first experimental results showed that Faster R-CNN, DCGAN and ResNet had good performance in each stage. Secondly, the second experimental results showed that the proposed three-stage method is superior to single-stage and two-stage methods in the case of sparse datasets and small lesions. Finally, a generalization experiment was also carried out using 100 images on the internet to verify the generalization of the proposed three-stage method. Experimental results showed that the proposed three-stage method had good generalization ability.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., & Kudlur, M. (2016). Tensorflow: A system for large-scale machine learning. Software available from tensorflow.org
Agarwal, M., Gupta, S. K., & Biswas, K. K. (2019). Grape Disease Identification Using Convolution Neural Network. In 2019 23rd International Computer Science and Engineering Conference (ICSEC) (pp. 224–229). Piscataway, USA: IEEE. https://doi.org/10.1109/ICSEC47112.2019.8974752
Asghari, M., Ahadi, L., & Riaie, S. (2013). Effect of salicylic acid and edible coating based aloe vera gel treatment on storage life and postharvest quality of grape (Vitis vinifera L. cv. Gizel Uzum). International Journal of Agriculture and Crop Sciences (IJACS), 5(23), 2890–2898
Berthelot, D., Schumm, T., & Metz, L. (2017). Began: Boundary equilibrium generative adversarial networks. Non-peer reviewed preprint at arXiv:1703.10717
Bock, C. H., Poole, G. H., Parker, P. E., & Gottwald, T. R. (2010). Plant disease severity estimated visually, by digital photography and image analysis, and by hyperspectral imaging. Critical reviews in plant sciences, 29(2), 59–107. https://doi.org/10.1080/07352681003617285
Brahimi, M., Boukhalfa, K., & Moussaoui, A. (2017). Deep learning for tomato diseases: Classification and symptoms visualization. Applied Artificial Intelligence, 31(4), 299–315. https://doi.org/10.1080/08839514.2017.1315516
Compant, S., Duffy, B., Nowak, J., Clément, C., & Barka, E. A. (2005). Use of plant growth-promoting bacteria for biocontrol of plant diseases: principles, mechanisms of action, and future prospects. Applied Environmental Microbiology, 71(9), 4951–4959. https://doi.org/10.1128/aem.71.9.4951-4959.2005
Cruz, A. C., Luvisi, A., De Bellis, L., & Ampatzidis, Y. (2017). X-FIDO: An effective application for detecting olive quick decline syndrome with deep learning and data fusion. Frontiers in Plant Science, 8, 1741. https://doi.org/10.3389/fpls.2017.01741
Du, S. S., Wang, Y., Zhai, X., Balakrishnan, S., Salakhutdinov, R., & Singh, A. (2018). How many samples are needed to estimate a convolutional or recurrent neural network?. Non-peer reviewed preprint at arXiv:1805.07883
Fuentes, A., Yoon, S., Kim, S. C., & Park, D. S. (2017). A robust deep-learning-based detector for real-time tomato plant diseases and pests recognition. Sensors, 17(9), 2022. https://doi.org/10.3390/s17092022
Geetharamani, G., & Pandian, A. (2019). Identification of plant leaf diseases using a nine-layer deep convolutional neural network. Computers and Electrical Engineering, 76, 323–338. https://doi.org/10.1016/j.compeleceng.2019.04.011
Ghoury, S., Sungur, C., & Durdu, A. (2019). Real-time diseases detection of grape and grape leaves using faster r-cnn and ssd mobilenet architectures. In International conference on advanced technologies, Alanya, Turkey: Computer Engineering and Science (ICATCES 2019)
Girshick, R. (2015). Fast r-cnn. In Proceedings of the IEEE international conference on computer vision (ICCV), (pp. 1440–1448). Piscataway, USA: IEEE
Han, C., Hayashi, H., Rundo, L., Araki, R., Shimoda, W., Muramatsu, S., Furukawa, Y., Mauri, G., & Nakayama, H., (2018). GAN-based synthetic brain MR image generation. In 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018) (pp. 734–738). Piscataway, USA: IEEE. https://doi.org/10.1186/s40708-020-00104-2
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778). Piscataway, USA: IEEE. https://doi.org/10.1109/CVPR.2016.90
Hu, G., Wu, H., Zhang, Y., & Wan, M. (2019). A low shot learning method for tea leaf’s disease identification. Computers and Electronics in Agriculture, 163, 104852. https://doi.org/10.1016/j.compag.2019.104852
Hughes, D., & Salathé, M. (2015). An open access repository of images on plant health to enable the development of mobile disease diagnostics. Non-peer reviewed preprint at arXiv:1511.08060
Ji, M., Zhang, L., & Wu, Q. (2020). Automatic grape leaf diseases identification via UnitedModel based on multiple convolutional neural networks. Information Processing in Agriculture, 7(3), 418–426. https://doi.org/10.1016/j.inpa.2019.10.003
Jiang, P., Chen, Y., Liu, B., He, D., & Liang, C. (2019). Real-time detection of apple leaf diseases using deep learning approach based on improved convolutional neural networks. IEEE Access : Practical Innovations, Open Solutions, 7, 59069–59080. https://doi.org/10.1109/ACCESS.2019.2914929
Kamal, K., Yin, Z., Wu, M., & Wu, Z. (2019). Depthwise separable convolution architectures for plant disease classification. Computers and Electronics in Agriculture, 165, 104948. https://doi.org/10.1016/j.compag.2019.104948
Khan, M. A., Akram, T., Sharif, M., Awais, M., Javed, K., Ali, H., & Saba, T. (2018). CCDF: Automatic system for segmentation and recognition of fruit crops diseases based on correlation coefficient and deep CNN features. Computers and Electronics in Agriculture, 155, 220–236. https://doi.org/10.1016/j.compag.2018.10.013
Liu, B., Tan, C., Li, S., He, J., & Wang, H. (2020). A data augmentation method based on generative adversarial networks for grape leaf disease identification. IEEE Access : Practical Innovations, Open Solutions, 8, 102188–102198. https://doi.org/10.1109/ACCESS.2020.2998839
Mahlein, A. K. (2016). Plant disease detection by imaging sensors–parallels and specific demands for precision agriculture and plant phenotyping. Plant Disease, 100(2), 241–251. https://doi.org/10.1094/PDIS-03-15-0340-FE
Mirza, M., & Osindero, S. (2014). Conditional generative adversarial nets. Non-peer reviewed preprint at arXiv:1411.1784
Polder, G., Blok, P. M., de Villiers, H. A., van der Wolf, J. M., & Kamp, J. (2019). Potato virus Y detection in seed potatoes using deep learning on hyperspectral images. Frontiers in Plant Science, 10, 209. https://doi.org/10.3389/fpls.2019.00209
Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. Non-peer reviewed preprint at arXiv:1511.06434
Ren, S., He, K., Girshick, R., & Sun, J. (2015). Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems, 28, 91–99
Ren, S., He, K., Girshick, R., Zhang, X., & Sun, J. (2016). Object detection networks on convolutional feature maps. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(7), 1476–1481. https://doi.org/10.1109/TPAMI.2016.2601099
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. Non-peer reviewed preprint at arXiv:1409.1556
Sharma, P., Berwal, Y. P. S., & Ghai, W. (2020). Performance analysis of deep learning CNN models for disease detection in plants using image segmentation. Information Processing in Agriculture, 7(4), 566–574. https://doi.org/10.1016/j.inpa.2019.11.001
Shmelkov, K., Schmid, C., & Alahari, K. (2018). How good is my GAN? In Proceedings of the European conference on computer vision (ECCV), pp. 213–229. Munich, Germany: Springer
Tassis, L. M., de Souza, J. E. T., & Krohling, R. A. (2021). A deep learning approach combining instance and semantic segmentation to identify diseases and pests of coffee leaves from in-field images. Computers and Electronics in Agriculture, 186, 106191. https://doi.org/10.1016/j.compag.2021.106191
Weyler, J., Milioto, A., Falck, T., Behley, J., & Stachniss, C. (2021). Joint plant instance detection and leaf count estimation for in-field plant phenotyping. IEEE Robotics and Automation Letters, 6(2), 3599–3606. https://doi.org/10.1109/LRA.2021.3060712
Wu, Q., Chen, Y., & Meng, J. (2020). DCGAN-based data augmentation for tomato leaf disease identification. IEEE Access : Practical Innovations, Open Solutions, 8, 98716–98728. https://doi.org/10.1109/ACCESS.2020.2997001
Zeng, M., Gao, H., & Wan, L. (2021). Few-shot grape leaf diseases classification based on generative adversarial network. Journal of Physics: Conference Series, 1883(1), 012093
Zhu, J., Wu, A., Wang, X., & Zhang, H. (2019). Identification of grape diseases using image analysis and BP neural networks. Multimedia Tools Applications, 79(21), 14539–14551. https://doi.org/10.1007/s11042-018-7092-0
Acknowledgements
This work was supported by the Public Welfare Industry (Agriculture) Research Projects Level-2 under Grant 201503116-04-06; Postdoctoral Foundation of Heilongjiang Province under Grant LBHZ15020; Harbin Applied Technology Research and Development Program under Grant 2017RAQXJ096 and National Key Application Research and Development Program in China under Grant 2018YFD0300105-2.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, Y., Wu, Q. Grape leaf disease identification with sparse data via generative adversarial networks and convolutional neural networks. Precision Agric 24, 235–253 (2023). https://doi.org/10.1007/s11119-022-09941-z
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11119-022-09941-z