Skip to main content
SpringerLink
Log in

CookingQA: Answering Questions and Recommending Recipes Based on Ingredients

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

In today’s world where an individual is becoming more and more busy and independent, the use of recommendation-based systems is steadily increasing. Thus, making available professional knowledge to the common man in a short-span quite necessary. The aim of our recipe recommendation system is to recommend recipes to users based on their questions. To make the recommendation model important as well as meaningful, it is pertinent to display only those recommendations that have a greater probability to be fit for the asked question. We have addressed this challenge by working on a threshold parameter generated from the recommendation engine. Apart from this, we have also included a question classification (QC) task together with the question answering (QA) module. The QA module is used to extract the requisite answers from the recommended recipe based on the class label obtained from QC. The main contribution of this work is the proposal of a robust recommendation approach by enabling analysis of threshold estimation and proposal of a suitable dataset. The final output of the recommendation system obtains benchmark results on the human evaluation (HE) metric. Our code, pretrained models and the dataset will be made publicly available.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://cookpad.com/.

  2. https://dialogflow.com/.

  3. https://www.ibm.com/watson.

  4. https://www.allrecipes.com/.

  5. https://spacy.io/.

  6. http://yknzhu.wixsite.com/mbweb.

References

  1. Ahn, Y.Y.; Ahnert, S.E.; Bagrow, J.P.; Barabási, A.L.: Flavor network and the principles of food pairing. Sci. Rep. 1, 196 (2011)

    Article  Google Scholar 

  2. Allam, Z.; Dhunny, Z.A.: On big data, artificial intelligence and smart cities. Cities 89, 80–91 (2019)

    Article  Google Scholar 

  3. Argal, A.; Gupta, S.; Modi, A.; Pandey, P.; Shim, S.; Choo, C.: Intelligent travel chatbot for predictive recommendation in echo platform. In: IEEE 8th Annual Computing and Communication Workshop and Conference, CCWC 2018, Las Vegas, NV, USA, January 8-10, 2018, pp. 176–183. IEEE (2018). https://doi.org/10.1109/CCWC.2018.8301732

  4. Battineni, G.; Chintalapudi, N.; Amenta, F.: Ai chatbot design during an epidemic like the novel coronavirus. Healthcare 8(2), 154 (2020)

    Article  Google Scholar 

  5. Bhaskar, P.; Pakray, P.; Banerjee, S.; Banerjee, S.; Bandyopadhyay, S.; Gelbukh, A.: Question answering system for qa4mre@clef 2012. In: . (2012). CEUR Workshop Proceedings ; Conference date: 01-01-2017

  6. Bhawiyuga, A.; Fauzi, M.A.; Pramukantoro, E.S.; Yahya, W.: Design of e-commerce chat robot for automatically answering customer question. In: 2017 International Conference on Sustainable Information Engineering and Technology (SIET), pp. 159–162. IEEE (2017)

  7. Breja, M.; Jain, S.K.: Why-type question classification in question answering system. In: Working notes of FIRE 2017 - Forum for Information Retrieval Evaluation, Bangalore, India, December 8-10, 2017, CEUR Workshop Proceedings, vol. 2036, pp. 149–153. CEUR-WS.org (2017). http://ceur-ws.org/Vol-2036/T6-1.pdf

  8. Chung, K.; Park, R.C.: Chatbot-based heathcare service with a knowledge base for cloud computing. Clust. Comput. 22(Suppl 1), 1925–1937 (2019). https://doi.org/10.1007/s10586-018-2334-5

    Article  Google Scholar 

  9. Davenport, T.H.; Ronanki, V.: Artificial intelligence for the real world. Arvard Business Rev. 96(1), 108–116 (2018)

    Google Scholar 

  10. Devlin, J.; Chang, M.; Lee, K.; Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. CoRR arXiv:abs/1810.04805 (2018)

  11. Forbes, P.; Zhu, M.: Content-boosted matrix factorization for recommender systems: experiments with recipe recommendation. In: Proceedings of the 2011 ACM Conference on Recommender Systems, RecSys 2011, Chicago, IL, USA, October 23-27, 2011, pp. 261–264. ACM (2011). https://doi.org/10.1145/2043932.2043979

  12. Freyne, J.; Berkovsky, S.: Intelligent food planning: personalized recipe recommendation. In: Proceedings of the 15th International Conference on Intelligent User Interfaces, IUI 2010, Hong Kong, China, February 7-10, 2010, pp. 321–324. ACM (2010). https://doi.org/10.1145/1719970.1720021

  13. García-Crespo, Á.; López-Cuadrado, J.L.; Colomo-Palacios, R.; González-Carrasco, I.; Ruiz-Mezcua, B.: Sem-fit: A semantic based expert system to provide recommendations in the tourism domain. Expert Syst. Appl. 38(10), 13310–13319 (2011)

    Article  Google Scholar 

  14. Geleijnse, G.; Nachtigall, P.; van Kaam, P.; Wijgergangs, L.: A personalized recipe advice system to promote healthful choices. In: Proceedings of the 16th International Conference on Intelligent User Interfaces, IUI 2011, Palo Alto, CA, USA, February 13-16, 2011, pp. 437–438. ACM (2011). https://doi.org/10.1145/1943403.1943487

  15. Goh, O.S.; Fung, C.C.; Wong, K.W.; Depickere, A.: Embodied conversational agents for H5N1 pandemic crisis. J. Adv. Comput. Intell. Intell. Inform. 11(3), 282–288 (2007). https://doi.org/10.20965/jaciii.2007.p0282

    Article  Google Scholar 

  16. Hammond, K.J.: CHEF: A model of case-based planning. In: Proceedings of the 5th National Conference on Artificial Intelligence. Philadelphia, PA, USA, August 11-15, 1986. Volume 1: Science, pp. 267–271. Morgan Kaufmann (1986). http://www.aaai.org/Library/AAAI/1986/aaai86-044.php

  17. Hosny, A.; Parmar, C.; Quackenbush, J.; Schwartz, L.H.; Aerts, H.J.: Artificial intelligence in radiology. Nature Rev. Cancer 18(8), 500–510 (2018)

    Article  Google Scholar 

  18. Huang, Z.; Thint, M.; Qin, Z.: Question classification using head words and their hypernyms. In: 2008 Conference on Empirical Methods in Natural Language Processing, EMNLP 2008, Proceedings of the Conference, 25-27 October 2008, Honolulu, Hawaii, USA, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 927–936. ACL (2008). https://www.aclweb.org/anthology/D08-1097/

  19. Inkster, B.; Sarda, S.; Subramanian, V.: An empathy-driven, conversational artificial intelligence agent (wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR mHealth uHealth 6(11), e12106 (2018)

    Article  Google Scholar 

  20. Janarthanam, S.: Hands-on Chatbots and Conversational UI Development: Build Chatbots and Voice user Interfaces with Chatfuel, Dialogflow, Twilio, and Alexa Skills. Packt Publishing Ltd, Microsoft Bot Framework, UK (2017)

    Google Scholar 

  21. Kamieth, F.; Braun, A.; Schlehuber, C.: Adaptive implicit interaction for healthy nutrition and food intake supervision. In: Human-Computer Interaction. Towards Mobile and Intelligent Interaction Environments - 14th International Conference, HCI International 2011, Orlando, FL, USA, July 9-14, 2011, Proceedings, Part III, Lecture Notes in Computer Science, vol. 6763, pp. 205–212. Springer (2011). https://doi.org/10.1007/978-3-642-21616-9_23

  22. Khalil, K.M; Abdel-Aziz, M.; Nazmy, T.T.; Salem, A.M.: The role of artificial intelligence technologies in crisis response. CoRR arXiv:abs/0806.1280 (2008)

  23. Khilji, A.F.U.R.; Laskar, S.R.; Pakray, P.; Kadir, R.A.; Lydia, M.S.; Bandyopadhyay, S.: Healfavor: A Chatbot Application in Healthcare. Analysis of Medical Modalities for Improved Diagnosis in Modern Healthcare, (2020). (in press)

  24. Khilji, A.F.U.R.; Laskar, S.R.; Pakray, P.; Kadir, R.A.; Lydia, M.S.; Bandyopadhyay, S.: Healfavor: Dataset and a prototype system for healthcare chatbot. In: 2020 International Conference on Data Science, Artificial Intelligence, and Business Analytics (DATABIA), pp. 1–4 (2020). https://doi.org/10.1109/DATABIA50434.2020.9190281

  25. Khilji, A.F.U.R.; Manna, R.; Laskar, S.R.; Pakray, P.; Das, D.; Bandyopadhyay, S.; Gelbukh, A.: Question classification and answer extraction for developing a cooking QA system. Computación y Sistemas, 24(2), (2020). https://doi.org/10.13053/cys-24-2-3445

  26. Kinouchi, O.; Diez-Garcia, R.W.; Holanda, A.J.; Zambianchi, P.; Roque, A.C.: The non-equilibrium nature of culinary evolution. New J. Phys. 10(7), 073020 (2008). https://doi.org/10.1088/1367-2630/10/7/073020

    Article  Google Scholar 

  27. Kovásznai, G.: Developing an expert system for diet recommendation. In: 6th IEEE International Symposium on Applied Computational Intelligence and Informatics, SACI, 2011, Timisoara, Romania, May 19-21, 2011, pp. 505–509. IEEE (2011). https://doi.org/10.1109/SACI.2011.5873056

  28. Laskar, S.R.; Khilji, A.F.U.R.; Pakray, P.; Bandyopadhyay, S.: Hindi-marathi cross lingual model. In: Proceedings of the Fifth Conference on Machine Translation, pp. 394–399. Association for Computational Linguistics, Online (2020)

  29. Lee, S.; Jung, H.; Ko, S.; Kim, S.; Kim, H.; Doh, K.; Park, H.; Yeo, J.; Ok, S.; Lee, J., Choi, S.; Hwang, S.; Park, E.; Ma, G.; Han, S.; Cha, K.; Sung, N.; Ha, J.: Carecall: a call-based active monitoring dialog agent for managing COVID-19 pandemic. CoRR arXiv:abs/2007.02642 (2020)

  30. Letheren, K.; Dootson, P.: Banking with a chatbot: a battle between convenience and security. The Conversation, In: The Conversation Media Group Ltd. August (10) (2017). https://eprints.qut.edu.au/114516/

  31. Li, C.; Chen, K.; Chang, Y.: When there is no progress with a task-oriented chatbot: A conversation analysis. In: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI 2019, Taipei, Taiwan, October 1-4, 2019, pp. 59:1–59:6. ACM (2019). https://doi.org/10.1145/3338286.3344407

  32. Li, C.H.; Chen, K.; Chang, Y.J.: When there is no progress with a task-oriented chatbot: A conversation analysis. In: Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 1–6 (2019)

  33. Lin, C.; Xie, R.; Guan, X.; Li, L.; Li, T.: Personalized news recommendation via implicit social experts. Inform. Sci. 254, 1–18 (2014)

    Article  Google Scholar 

  34. Lino, P.M.D.B.: Travel Booking Chatbot. Faculdade de Engenharia da Universidade do Porto. June (25) (2018). https://repositorioaberto.up.pt/bitstream/10216/114378/2/278925.pdf

  35. Loni, B.: A survey of State-of-the-Art Methods on Question Classification. TU Delft Repository (2011)

  36. Manna, R.; Pakray, P.; Banerjee, S.; Das, D.; Gelbukh, A.F.: Cookingqa: A question answering system based on cooking ontology. In: Advances in Computational Intelligence - 15th Mexican International Conference on Artificial Intelligence, MICAI 2016, Cancún, Mexico, October 23-28, 2016, Proceedings, Part I, Lecture Notes in Computer Science, vol. 10061, pp. 67–78. Springer (2016). https://doi.org/10.1007/978-3-319-62434-1_6

  37. Maruyama, T.; Kawano, Y.; Yanai, K.: Real-time mobile recipe recommendation system using food ingredient recognition. In: Proceedings of the 2nd ACM international workshop on Interactive multimedia on mobile and portable devices, IMMPD@ACM Multimedia 2012, Nara, Japan, November 2, 2012, pp. 27–34. ACM (2012). https://doi.org/10.1145/2390821.2390830

  38. McArthur, D.; Lewis, M.; Bishary, M.: The roles of artificial intelligence in education: current progress and future prospects. J. Educational Technol. 1(4), 42–80 (2005)

    Google Scholar 

  39. Mino, Y.; Kobayashi, I.: Recipe recommendation for a diet considering a user’s schedule and the balance of nourishment. In: 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems, vol. 3, pp. 383–387. IEEE (2009)

  40. Moschitti, A.; Quarteroni, S.; Basili, R.; Manandhar, S.: Exploiting syntactic and shallow semantic kernels for question answer classification. In: ACL 2007, Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, June 23-30, 2007, Prague, Czech Republic. The Association for Computational Linguistics (2007). https://www.aclweb.org/anthology/P07-1098/

  41. Müller, M.; Harvey, M.; Elsweiler, D.; Mika, S.: Ingredient matching to determine the nutritional properties of internet-sourced recipes. In: 6th International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth 2012 and Workshops, San Diego, CA, USA, May 21-24, 2012, pp. 73–80. IEEE (2012). https://doi.org/10.4108/icst.pervasivehealth.2012.248681

  42. Nyati, U.; Rawat, S.; Gupta, D.; Aggrawal, N.; Arora, A.: Characterize ingredient network for recipe suggestion. Int. J. Inform. Technol. 1–8 (2019). https://doi.org/10.1007/s41870-019-00277-y

  43. Oh, K.J.; Lee, D.; Ko, B.; Choi, H.J.: A chatbot for psychiatric counseling in mental healthcare service based on emotional dialogue analysis and sentence generation. In: 2017 18th IEEE International Conference on Mobile Data Management (MDM), pp. 371–375. IEEE (2017)

  44. Pakray, P.; Bhaskar, P.; Banerjee, S.; Pal, B.C.; Bandyopadhyay, S.; Gelbukh, A.F.: A hybrid question answering system based on information retrieval and answer validation. In: CLEF 2011 Labs and Workshop, Notebook Papers, 19-22 September 2011, Amsterdam, The Netherlands, CEUR Workshop Proceedings, vol. 1177. CEUR-WS.org (2011). http://ceur-ws.org/Vol-1177/CLEF2011wn-QA4MRE-PakrayEt2011.pdf

  45. Peñas, A.; Forner, P.; Sutcliffe, R.; Rodrigo, A.; Forundefinedscu, C.; Alegria, I.n.; Giampiccolo, D.; Moreau, N.; Osenova, P.: Overview of respubliqa 2009: Question answering evaluation over european legislation. In: Proceedings of the 10th Cross-Language Evaluation Forum Conference on Multilingual Information Access Evaluation: Text Retrieval Experiments, CLEF’09, p. 174-196. Springer-Verlag, Berlin, Heidelberg (2009)

  46. Prager, J.M.; Radev, D.R.; Brown, E.W.; Coden, A.; Samn, V.: The use of predictive annotation for question answering in TREC8. In: Proceedings of The Eighth Text REtrieval Conference, TREC 1999, Gaithersburg, Maryland, USA, November 17-19, 1999, NIST Special Publication, vol. 500-246. National Institute of Standards and Technology (NIST) (1999). http://trec.nist.gov/pubs/trec8/papers/IBMTrec8QA.ps

  47. Saravi, S.; Kalawsky, R.; Joannou, D.; Rivas Casado, M.; Fu, G.; Meng, F.: Use of artificial intelligence to improve resilience and preparedness against adverse flood events. Water 11(5), 973 (2019)

    Article  Google Scholar 

  48. Sezgin, E.; Huang, Y.; Ramtekkar, U.; Lin, S.: Readiness for voice assistants to support healthcare delivery during a health crisis and pandemic. NPJ Digital Med. 3(1), 1–4 (2020)

    Article  Google Scholar 

  49. Shidochi, Y.; Takahashi, T.; Ide, I.; Murase, H.: Finding replaceable materials in cooking recipe texts considering characteristic cooking actions. In: Proceedings of the ACM multimedia 2009 workshop on Multimedia for cooking and eating activities, CEA@ACM Multimedia 2009, Beijing, China, October 23, 2009, pp. 9–14. ACM (2009). https://doi.org/10.1145/1630995.1630998

  50. da Silva, J.P.C.G.; Coheur, L.; Mendes, A.C.; Wichert, A.: From symbolic to sub-symbolic information in question classification. Artif. Intell. Rev. 35(2), 137–154 (2011). https://doi.org/10.1007/s10462-010-9188-4

    Article  Google Scholar 

  51. Simas, T.; Ficek, M.; Diaz-Guilera, A.; Obrador, P.; Rodriguez, P.R.: Food-bridging: a new network construction to unveil the principles of cooking. Frontiers in ICT 4, 14 (2017)

    Article  Google Scholar 

  52. Svensson, M.; Höök, K.; Cöster, R.: Designing and evaluating kalas: A social navigation system for food recipes. ACM Trans. Comput. Hum. Interact. 12(3), 374–400 (2005). https://doi.org/10.1145/1096737.1096739

    Article  Google Scholar 

  53. Taylor, W.L.: “Cloze procedure”: A new tool for measuring readability. Journal. Q. 30(4), 415–433 (1953)

    Article  Google Scholar 

  54. Teng, C.Y.; Lin, Y.R.; Adamic, L.A.: Recipe recommendation using ingredient networks. In: Proceedings of the 4th Annual ACM Web Science Conference, WebSci ’12, p. 298-307. Association for Computing Machinery, New York, NY, USA (2012). https://doi.org/10.1145/2380718.2380757

  55. Ueda, M.; Takahata, M.; Nakajima, S.: User’s food preference extraction for personalized cooking recipe recommendation. In: Proceedings of the Second International Conference on Semantic Personalized Information Management: Retrieval and Recommendation - Volume 781, SPIM’11, p. 98-105. CEUR-WS.org, Aachen, DEU (2011)

  56. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, u.; Polosukhin, I.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, p. 6000-6010. Curran Associates Inc., Red Hook, NY, USA (2017)

  57. Wallace, R.: The Elements of Aiml Style. Alice AI Foundation, USA (2003)

    Google Scholar 

  58. Wang, L.; Li, Q.; Li, N.; Dong, G.; Yang, Y.: Substructure similarity measurement in chinese recipes. In: Proceedings of the 17th International Conference on World Wide Web, WWW 2008, Beijing, China, April 21-25, 2008, pp. 979–988. ACM (2008). https://doi.org/10.1145/1367497.1367629

  59. Weizenbaum, J.: ELIZA - A computer program for the study of natural language communication between man and machine (reprint). Commun. ACM 26(1), 23–28 (1983). https://doi.org/10.1145/357980.357991

    Article  Google Scholar 

  60. Winter, J.S.; Davidson, E.J.: Big data governance of personal health information and challenges to contextual integrity. Inf. Soc. 35(1), 36–51 (2019). https://doi.org/10.1080/01972243.2018.1542648

    Article  Google Scholar 

  61. Woo, W.L.: Future trends in i&m: Human-machine co-creation in the rise of ai. IEEE Instrum. Meas. Mag. 23(2), 71–73 (2020)

    Article  Google Scholar 

  62. Woo, W.L.; Gao, B.; Al-Nima, R.R.O.; Ling, W.K.: Development of conversational artificial intelligence for pandemic healthcare query support. Int. J. Autom. Artif. Intell. Machine Learning 1(1), 54–79 (2020)

    Google Scholar 

  63. Xia, L.; Teng, Z.; Ren, F.: An integrated approach for question classification in chinese cuisine question answering system. In: ISUC 2008, Second International Symposium on Universal Communication, Osaka, Japan, 15-16 December 2008, pp. 317–321. IEEE Computer Society (2008). https://doi.org/10.1109/ISUC.2008.18

  64. Yagcioglu, S.; Erdem, A.; Erdem, E.; Ikizler-Cinbis, N.: Recipeqa: A challenge dataset for multimodal comprehension of cooking recipes. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pp. 1358–1368. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/d18-1166

  65. Yu, J.; Qiu, M.; Jiang, J.; Huang, J.; Song, S.; Chu, W.; Chen, H.: Modelling domain relationships for transfer learning on retrieval-based question answering systems in e-commerce. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, WSDM 2018, Marina Del Rey, CA, USA, February 5-9, 2018, pp. 682–690. ACM (2018). https://doi.org/10.1145/3159652.3159685

  66. Zhang, Q.; Hu, R.; Namee, B.M.; Delany, S.J.: Back to the future: Knowledge light case base cookery. In: ECCBR 2008, The 9th European Conference on Case-Based Reasoning, Trier, Germany, September 1-4, 2008, Workshop Proceedings, pp. 239–248 (2008)

Download references

Acknowledgements

We would like to thank Department of Computer Science and Engineering and Center for Natural Language Processing (CNLP) at National Institute of Technology Silchar and Department of Computer Science and Engineering at Jadavpur University, Kolkata for providing the requisite support and infrastructure to execute this work.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Silchar, Assam, India

    Abdullah Faiz Ur Rahman Khilji, Sahinur Rahman Laskar, Partha Pakray & Sivaji Bandyopadhyay

  2. Natural Language and Text Processing Laboratory Center for Computing Research, National Polytechnic Institute, Mexico City, Mexico

    Alexander Gelbukh

  3. Department of Computer Science and Engineering, Jadavpur University, Kolkata, India

    Riyanka Manna & Dipankar Das

Authors
  1. Abdullah Faiz Ur Rahman Khilji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riyanka Manna
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sahinur Rahman Laskar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Partha Pakray
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dipankar Das
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sivaji Bandyopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexander Gelbukh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Partha Pakray.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khilji, A.F.U.R., Manna, R., Laskar, S.R. et al. CookingQA: Answering Questions and Recommending Recipes Based on Ingredients. Arab J Sci Eng 46, 3701–3712 (2021). https://doi.org/10.1007/s13369-020-05236-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-020-05236-5

Keywords

Search

Navigation