Skip to main content
SpringerLink
Log in

Multimodal Deep Learning for Crop Yield Prediction

  • Conference paper
  • First Online:
Doctoral Symposium on Information and Communication Technologies (DSICT 2022)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1647))

Included in the following conference series:

  • 328 Accesses

Abstract

Precision agriculture is a vital practice for improving the production of crops. The present work is aimed to develop a deep learning multimodal model that can predict the crop yield in Ecuadorian corn farms. The model takes multispectral images and field sensor data (humidity, temperature, or soil status) to obtain the yield of a crop. The use of multimodal data is aimed to extract hidden patterns in the status of crops and in this way obtain better results than the use of vegetation indices or other state-of-the-art methods. For the experiments, we utilized multi-spectral satellite images obtained from the google earth engine platform and monthly precipitation and temperature data of the 24 Ecuadorian provinces collected from the Ecuadorian Ministry of agriculture and livestock; likewise, we obtained the area of corn plantation in each province and their corn production for the years 2016 to 2020. Results indicate that the use of multimodal deep learning models (pre-trained CNN for images and LSTM for time series sensor data) gives better prediction accuracy than monomodal prediction models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ramanath, A., Muthusrinivasan, S., Xie, Y., Shekhar, S., Ramachandra, B.: NDVI versus CNN features in deep learning for land cover classification of aerial images. In: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 6483–6486. IEEE (2019)

    Google Scholar 

  2. Tran, T., Choi, J., Le, T., Kim, J.: A comparative study of deep CNN in forecasting and classifying the macronutrient deficiencies on development of tomato plant. Appl. Sci. 9(8), 1601 (2019)

    Article  Google Scholar 

  3. Chlingaryan, A., Sukkarieh, S., Whelan, B.: Machine learning approaches for crop yield prediction and nitrogen status estimation in precision agriculture: a review. Comput. Electron. Agric. 151, 61–69 (2018)

    Article  Google Scholar 

  4. Wiegand, C., Richardson, A., Escobar, D., Gerbermann, A.: Vegetation indices in crop assessments. Remote Sens. Environ. 35(2–3), 105–119 (1991)

    Article  Google Scholar 

  5. Basso, B., Cammarano, D., Carfagna, E.: Review of crop yield forecasting methods and early warning systems. In: Proceedings of the First Meeting of the Scientific Advisory Committee of the Global Strategy to Improve Agricultural and Rural Statistics, FAO Headquarters, Rome, Italy, pp. 18–19 (2013)

    Google Scholar 

  6. Mahdavinejad, M., Rezvan, M., Barekatain, M., Adibi, P., Barnaghi, P., Sheth, A.: Machine learning for Internet of Things data analysis: a survey. Digit. Commun. Netw. 4(3), 161–175 (2018)

    Article  Google Scholar 

  7. Gondchawar, N., Kawitkar, R.: IoT based smart agriculture. Int. J. Adv. Res. Comput. Commun. Eng. 5(6), 838–842 (2016)

    Google Scholar 

  8. Muangprathub, J., Boonnam, N., Kajornkasirat, S., Lekbangpong, N., Wanichsombat, A., Nillaor, P.: IoT and agriculture data analysis for smart farm. Comput. Electron. Agric. 156, 467–474 (2019)

    Article  Google Scholar 

  9. Kim, T., Ramos, C., Mohammed, S.: Smart city and IoT (2017)

    Google Scholar 

  10. Samuel, S.: A review of connectivity challenges in IoT-smart home. In: 2016 3rd MEC International Conference on Big Data and Smart City (ICBDSC), pp. 1–4. IEEE (2016)

    Google Scholar 

  11. Kim, Y., Park, Y., Choi, J.: A study on the adoption of IoT smart home service: using value-based adoption model. Total Qual. Manag. Bus. Excell. 28(9–10), 1149–1165 (2017)

    Article  Google Scholar 

  12. Ukil, A., Bandyoapdhyay, S., Puri, C., Pal, A.: IoT healthcare analytics: the importance of anomaly detection. In: 2016 IEEE 30th International Conference on Advanced Information Networking and Applications (AINA), pp. 994–997. IEEE (2016)

    Google Scholar 

  13. Tyagi, S., Agarwal, A., Maheshwari, P.: A conceptual framework for IoT-based healthcare system using cloud computing. In: 2016 6th International Conference-Cloud System and Big Data Engineering (Confluence), pp. 503–507. IEEE (2016)

    Google Scholar 

  14. Rghioui, A., Sendra, S., Lloret, J., Oumnad, A.: Internet of Things for measuring human activities in ambient assisted living and e-health. Netw. Protoc. Algorithms 8(3), 15–28 (2016)

    Article  Google Scholar 

  15. Shi, C., Liu, J., Liu, H., Chen, Y.: Smart user authentication through actuation of daily activities leveraging WiFi-enabled IoT. In: Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, pp. 1–10 (2017)

    Google Scholar 

  16. Al-Douri, Y.K., Hamodi, H., Lundberg, J.: Time series forecasting using a two-level multi-objective genetic algorithm: a case study of maintenance cost data for tunnel fans. Algorithms 11(8), 123 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  17. Baptista, M., Sankararaman, S., de Medeiros, I., Nascimento, C., Jr., Prendinger, H., Henriques, E.: Forecasting fault events for predictive maintenance using data-driven techniques and ARMA modeling. Comput. Ind. Eng. 115, 41–53 (2018)

    Article  Google Scholar 

  18. Kamir, E., Waldner, F., Hochman, Z.: Estimating wheat yields in Australia using climate records, satellite image time series and machine learning methods. ISPRS J. Photogramm. Remote. Sens. 160, 124–135 (2020)

    Article  Google Scholar 

  19. Adeniyi, O.D., Szabo, A., Tamás, J., Nagy, A.: Wheat Yield Forecasting Based on Landsat NDVI and SAVI Time Series (2020)

    Google Scholar 

  20. Kadri, F., Harrou, F., Chaabane, S., Tahon, C.: Time series modelling and forecasting of emergency department overcrowding. J. Med. Syst. 38(9), 1–20 (2014). https://doi.org/10.1007/s10916-014-0107-0

    Article  Google Scholar 

  21. Demir, E., Dincer, S.: Place and solution proposals of data mining in production planning and control processes: a business application. Press Academia Procedia 11(1), 189–193 (2020)

    Google Scholar 

  22. Srivastava, N., Salakhutdinov, R.: Multimodal learning with deep Boltzmann machines. In: Advances in Neural Information Processing Systems, vol. 25 (2012)

    Google Scholar 

  23. Ramachandram, D., Taylor, G.: Deep multimodal learning: a survey on recent advances and trends. IEEE Sig. Process. Mag. 34(6), 96–108 (2017)

    Article  Google Scholar 

  24. Maimaitijiang, M., Sagan, V., Sidike, P., Hartling, S., Esposito, F., Fritschi, F.: Soybean yield prediction from UAV using multimodal data fusion and deep learning. Remote Sens. Environ. 237, 111599 (2020)

    Article  Google Scholar 

  25. Yalcin, H.: Plant phenology recognition using deep learning: Deep-Pheno. In: 2017 6th International Conference on Agro-Geoinformatics, pp. 1–5. IEEE (2017)

    Google Scholar 

  26. Zheng, Y.Y., Kong, J.L., Jin, X.B., Wang, X.Y., Su, T.L., Zuo, M.: CropDeep: the crop vision dataset for deep-learning-based classification and detection in precision agriculture. Sensors 19(5), 1058 (2019)

    Article  Google Scholar 

  27. Nilsback, M., Zisserman, A.: A visual vocabulary for flower classification. In: Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), New York, NY, USA, pp. 1447–1454 (2006)

    Google Scholar 

  28. Kumar, N., et al.: Leafsnap: a computer vision system for automatic plant species identification. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) Computer Vision, vol. 7573, pp. 502–516. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33709-3_36

    Chapter  Google Scholar 

  29. Wegner, J., Branson, S., Hall, D., Schindler, K., Perona, P.: Cataloging public objects using aerial and street-level images-urban trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas Valley, NV, USA, pp. 6014–6023 (2016)

    Google Scholar 

  30. Kamilaris, A., Prenafeta-Boldú, F.: Deep learning in agriculture: a survey. Comput. Electron. Agric. 147, 70–90 (2018)

    Article  Google Scholar 

  31. Bender, A., Whelan, B., Sukkarieh, S.: Ladybird Cobbitty 2017 Brassica dataset (2019)

    Google Scholar 

  32. Gandhi, A., Sharma, A., Biswas, A., Deshmukh, O.: GeThR-Net: a generalized temporally hybrid recurrent neural network for multimodal information fusion. In: Hua, G., Jégou, H. (eds.) Computer Vision, vol. 9914, pp. 883–899. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_58

    Chapter  Google Scholar 

  33. Gao, J., Li, P., Chen, Z., Zhang, J.: A survey on deep learning for multimodal data fusion. Neural Comput. 32(5), 829–864 (2020)

    Article  MathSciNet  MATH  Google Scholar 

  34. Zhao, X., et al.: Use of unmanned aerial vehicle imagery and deep learning UNet to extract rice lodging. Sensors 19(18), 3859 (2019)

    Article  Google Scholar 

  35. Chen, W., Wang, W., Liu, L., Lew, M.: New ideas and trends in deep multimodal content understanding: a review. arXiv preprint https://arxiv.org/abs/2010.08189 (2020)

  36. Iniap. http://www.iniap.gob.ec/pruebav3/wp-content/uploads/2018/03/281-iniap-OK-baja.pdf

  37. Sistema de Información Pública Agropecuaria. http://sipa.agricultura.gob.ec/index.php/maiz

  38. Google Earth Engine data catalog, Sentinel-2 MSI. https://developers.google.com/earth-engine/datasets/catalog/COPERNICUS_S2

Download references

Author information

Authors and Affiliations

  1. Escuela Superior Politécnica del Litoral, ESPOL, Facultad de Ingeniería en Electricidad y Computación, CIDIS, Gustavo Galindo Km. 30.5 Vía Perimetral, Guayaquil, Ecuador

    Luis-Roberto Jácome-Galarza

Authors
  1. Luis-Roberto Jácome-Galarza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Luis-Roberto Jácome-Galarza .

Editor information

Editors and Affiliations

  1. CEDIA, Cuenca, Ecuador

    Karina Abad

  2. CEDIA, Cuenca, Ecuador

    Santiago Berrezueta

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jácome-Galarza, LR. (2022). Multimodal Deep Learning for Crop Yield Prediction. In: Abad, K., Berrezueta, S. (eds) Doctoral Symposium on Information and Communication Technologies. DSICT 2022. Communications in Computer and Information Science, vol 1647. Springer, Cham. https://doi.org/10.1007/978-3-031-18347-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-18347-8_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-18346-1

  • Online ISBN: 978-3-031-18347-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation