Skip to main content
SpringerLink
Log in

The segmented UEC Food-100 dataset with benchmark experiment on food detection

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Automatic food classification systems have several interesting applications ranging from detecting eating habits, to waste food management and advertisement. When a food image has multiple food items, the food detection step is necessary before classification. This work challenges the food detection issue and it introduces to the research community the Segmented UEC Food-100 dataset, which expands the original UEC Food-100 database with segmentation masks. In the semantic segmentation experiment, the performance of YOLAC and DeeplabV3+ has been compared and YOLAC reached the best accuracy of 64.63% mIoU. In the instance segmentation experiment, YOLACT has been used due to its speed and high accuracy. The benchmark performance on the newly released Segmented UEC Food-100 dataset is 68.83% mAP. For comparison purpose, experiments have been run also on the UEC FoodPix Complete dataset of Okamoto et al. The database and the code will be available after publication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The dataset generated during the current study is available from http://www.okanbatur.com/segmenteduecfood100.html.

References

  1. Dai, J., Li, Y., He, K., Sun, J.: R-fcn: object detection via region-based fully convolutional networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. NIPS’16, pp. 379–387. Curran Associates Inc., Red Hook (2016)

  2. Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017)

  3. Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path aggregation network for instance segmentation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8759–8768 (2018)

  4. Pang, J., Chen, K., Shi, J., Feng, H., Ouyang, W., Lin, D.: Libra r-cnn: towards balanced learning for object detection. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 821–830. IEEE Computer Society, Los Alamitos (2019)

  5. Wang, X., Zhang, S., Yu, Z., Feng, L., Zhang, W.: Scale-equalizing pyramid convolution for object detection. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 13–19, 2020, pp. 13356–13365 (2020)

  6. Dai, X., Chen, Y., Xiao, B., Chen, D., Liu, M., Yuan, L., Zhang, L.: Dynamic head: unifying object detection heads with attentions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2021)

  7. Singh, B., Najibi, M., Sharma, A., Davis, L.S.: Scale normalized image pyramids with autofocus for object detection. IEEE Trans. Pattern Anal. Mach. Intell. 44, 3749–3766 (2022)

    Google Scholar 

  8. Matsuda, Y., Hoashi, H., Yanai, K.: Recognition of multiple-food images by detecting candidate regions. In: 2012 IEEE International Conference on Multimedia and Expo, pp. 25–30 (2012)

  9. Okamoto, K., Yanai, K.: UEC-FoodPIX Complete: A large-scale food image segmentation dataset. In: Proceedings of ICPR Workshop on Multimedia Assisted Dietary Management(MADiMa) (2021)

  10. Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32(9), 1627–1645 (2010)

    Article  Google Scholar 

  11. Deng, Y., Manjunath, B.S.: Unsupervised segmentation of color-texture regions in images and video. IEEE Trans. Pattern Anal. Mach. Intell. 23(8), 800–810 (2001)

    Article  Google Scholar 

  12. Ege, T., Yanai, K.: A new large-scale food image segmentation dataset and its application to food calorie estimation based on grains of rice. In: Proceedings of ACMMM Workshop on Multimedia Assisted Dietary Management (MADiMa) (2019)

  13. Rother, C., Kolmogorov, V., Blake, A.: “grabcut’’: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23(3), 309–314 (2004)

    Article  Google Scholar 

  14. Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation (2018)

  15. Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6517–6525 (2017). https://doi.org/10.1109/CVPR.2017.690

  16. Chen, M.-Y., Yang, Y.-H., Ho, C.-J., Wang, S.-H., Liu, S.-M., Chang, E., Yeh, C.-H., Ouhyoung, M.: Automatic Chinese food identification and quantity estimation. In: SIGGRAPH Asia 2012 Technical Briefs. SA ’12. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2407746.2407775

  17. Wu, X., Fu, X., Liu, Y., Lim, E., Hoi, S.C.H., Sun, Q.: A large-scale benchmark for food image segmentation (2021). CoRR arXiv:2105.05409

  18. Salvador, A., Hynes, N., Aytar, Y., Marin, J., Ofli, F., Weber, I., Torralba, A.: Learning cross-modal embeddings for cooking recipes and food images. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3068–3076 (2017). https://doi.org/10.1109/CVPR.2017.327

  19. Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 88(2), 303–338 (2010)

    Article  Google Scholar 

  20. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. (IJCV) 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-015-0816-y

    Article  MathSciNet  Google Scholar 

  21. Lin, T.-Y., Maire, M., Belongie, S., Bourdev, L., Girshick, R., Hays, J., Perona, P., Ramanan, D., Zitnick, C.L., Dollár, P.: Microsoft coco: common objects in context. In: Proceedings of European Conference on Computer Vision (2014)

  22. Everingham, M., Eslami, S.M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015)

    Article  Google Scholar 

  23. Russakovsky, O., Deng, J., Huang, Z., Berg, A.C., Fei-Fei, L.: Detecting avocados to zucchinis: what have we done, and where are we going? In: International Conference on Computer Vision (ICCV) (2013)

  24. Deng, J., Socher, R., Fei-Fei, L., Dong, W., Li, K., Li, L.-J.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248–255 (2009)

  25. Gao, J., Tan, W., Ma, L., Wang, Y., Tang, W.: Musefood: multi-sensor-based food volume estimation on smartphones. In: 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), pp. 899–906 (2019)

  26. Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: towards real-time object detection with region proposal networks. In: Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems. Curran Associates, Inc. (2015)

  27. Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(4), 640–651 (2017)

    Article  Google Scholar 

  28. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015 (2015)

  29. He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask r-cnn. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017)

  30. Bolya, D., Zhou, C., Xiao, F., Lee, Y.J.: Yolact: real-time instance segmentation. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9156–9165 (2019). https://doi.org/10.1109/ICCV.2019.00925

  31. Bolya, D., Zhou, C., Xiao, F., Lee, Y.J.: Yolact++: better real-time instance segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 44(2), 1108–1121 (2020). https://doi.org/10.1109/TPAMI.2020.3014297. Accessed 1 Feb 2022

  32. Chollet, F.: Xception: deep learning with depthwise separable convolutions (2017)

  33. Jiang, S., Min, W., Liu, L., Luo, Z.: Multi-scale multi-view deep feature aggregation for food recognition. IEEE Trans. Image Process. 29, 265–276 (2020)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Engineering, İstanbul Bilgi University, 34060, Eyüp, İstanbul, Turkey

    Elena Battini Sönmez & Berker Arslan

  2. Electrical and Electronics Engineering, İstanbul Bilgi University, 34060, Eyüp, İstanbul, Turkey

    Sefer Memiş & Okan Zafer Batur

Authors
  1. Elena Battini Sönmez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sefer Memiş
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Berker Arslan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Okan Zafer Batur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elena Battini Sönmez.

Ethics declarations

Conflict of interest

The authors have no conflict of interest to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Battini Sönmez, E., Memiş, S., Arslan, B. et al. The segmented UEC Food-100 dataset with benchmark experiment on food detection. Multimedia Systems 29, 2049–2057 (2023). https://doi.org/10.1007/s00530-023-01088-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-023-01088-9

Keywords

Search

Navigation