Skip to main content
SpringerLink
Log in

Deep understanding of 3-D multimedia information retrieval on social media: implications and challenges

  • Original Article
  • Published:
Iran Journal of Computer Science Aims and scope Submit manuscript

Abstract

With the recent penetration and proliferation of social networks into our lives, human choices and preferences have become more socially accessible. This easy accessibility of private data in different formats has opened many new initiatives. The big explosion of multimedia data on the web has enabled social networks to gauge user likes, dislikes, and needs. This has imposed high demands on multimedia information retrieval (MIR) techniques. This manuscript illustrates the MIR concept in terms of its application to social media. It further positions the current research in the field of 3D MIR. Further it highlights the challenges in 3-D MIR on social media and finally translates them into significant research directions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kou, F., Du, J., He, Y., Ye, L.: Social network search based on semantic analysis and learning. CAAI Trans. Intell. Technol. 1(4), 293–302 (2016)

    Article  Google Scholar 

  2. Tandera, T., Hendro, Suhartono, D., Wongso, R., Prasetio, Y.L.: Personality prediction system from Facebook users. Proc. Comput. Sci. 116, 604–611 (2017)

    Article  Google Scholar 

  3. Araque, O., Corcuera-Platas, I., Sánchez-Rada, J.F., Iglesias, C.A.: Enhancing deep learning sentiment analysis with ensemble techniques in social applications. Expert Syst. Appl. 77, 236–246 (2017)

    Article  Google Scholar 

  4. Guimaraes, R.G., Rosa, R.L., De Gaetano, D., Rodriguez, D.Z., Bressan, G.: Age groups classification in social network using deep learning. IEEE Access 5(c), 10805–10816 (2017)

    Article  Google Scholar 

  5. Najafabadi, M.M., Villanustre, F., Khoshgoftaar, T.M., Seliya, N., Wald, R., Muharemagic, E.: Deep learning applications and challenges in big data analytics. J. Big Data 2(1), 1–21 (2015)

    Article  Google Scholar 

  6. Phan, N., Dou, D., Wang, H., Kil, D., Piniewski, B.: Ontology-based deep learning for human behavior prediction with explanations in health social networks. Inf. Sci. (NY) 384, 298–313 (2017)

    Article  Google Scholar 

  7. Zheng, X., Zeng, Z., Chen, Z., Yu, Y., Rong, C.: Detecting spammers on social networks. Neurocomputing 159(1), 27–34 (2015)

    Article  Google Scholar 

  8. Vinay, A., Shekhar, V.S., Rituparna, J., Aggrawal, T., Murthy, K.N.B., Natarajan, S.: Cloud based big data analytics framework for face recognition in social networks using machine learning. Proc. Comput. Sci. 50, 623–630 (2015)

    Article  Google Scholar 

  9. Arel, I., Rose, D.C., Karnowski, T.P.: Deep machine learning—a new frontier in artificial intelligence research. IEEE Comput. Intell. Mag. 5(4), 13–18 (2010)

    Article  Google Scholar 

  10. Bengio, Y.: Learning deep architectures for AI. Found. Trends® Mach. Learn. 2(1), 1–127 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  11. Seraj, R.M.: Multi-task learning (2014)

  12. Chen, X.-W., Lin, X.: Big data deep learning: challenges and perspectives. IEEE Access 2, 514–525 (2014)

    Article  Google Scholar 

  13. Mesnil, G., Dauphin, Y., Glorot, X., Rifai, S., Bengio, Y., Goodfellow, I.J., Lavoie, E., Muller, X., Desjardins, G., Warde-Farley, D., Vincent, P., Courville, A., Bergstra, J.: Unsupervised and transfer learning challenge: a deep learning approach. JMLR W& CP Proc. Unsuperv. Transf. Learn. Chall. Work. 27, 97–110 (2012)

    Google Scholar 

  14. Phan, N., Dou, D., Wang, H., Kil, D., Piniewski, B.: Ontology-based deep learning for human behavior prediction in health social networks. In: Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology Health Informatics, pp. 433–442 (2015)

  15. Ghodrati, H., Ben Hamza, A.: Deep shape-aware descriptor for nonrigid 3D object retrieval. Int. J. Multimed. Inf. Retr. 5(3), 151–164 (2016)

    Article  Google Scholar 

  16. Ain, Q.T., Ali, M., Riaz, A., Noureen, A., Kamran, M., Hayat, B., Rehman, A.: Sentiment analysis using deep learning techniques: a review, IJACSA). Int. J. Adv. Comput. Sci. Appl. 8(6), 424–433 (2017)

    Google Scholar 

  17. Chen, T., Xu, R., He, Y., Wang, X.: Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Syst. Appl. 72, 221–230 (2017)

    Article  Google Scholar 

  18. Vateekul, P., Koomsubha, T.: A study of sentiment analysis using deep learning techniques on Thai Twitter data. In: 2016 13th International Joint Conference on Computer Science Software Engineering, pp. 1–6 (2016)

  19. Abbes, M., Kechaou, Z., Alimi, A.M.: Enhanced deep learning models for sentiment analysis in arab social media. In: Lecture Notes in Computer Science (including subseries Lecture Notes Artificial Intelligence, Lecture Notes Bioinformatics), vol. 10638, pp. 667–676 (2017)

  20. Liao, L., He, X., Zhang, H., Chua, T.-S.: Attributed Social Network Embedding, vol. 14(8), pp. 1–12 (2017)

  21. Pal, S., Dong, Y., Thapa, B., Chawla, N.V., Swami, A., Ramanathan, R.: Deep learning for network analysis: problems, approaches and challenges. In: MILCOM 2016—2016 IEEE Military Communications Conference, pp. 588–593 (2016)

  22. Jia, Y., Song, X., Zhou, J., Liu, L., Nie, L., Rosenblum, D.S.: Fusing social networks with deep learning for volunteerism tendency prediction. In: Proceedings of the 13th AAAI Conference on Artificial Intelligence, pp. 165–171 (2016)

  23. Nguyen, D.T., Joty, S., Imran, M., Sajjad, H., Mitra, P.: Applications of Online Deep Learning for Crisis Response Using Social Media Information (2016)

  24. Yao, L., Wang, L., Pan, L., Yao, K.: Link prediction based on common-neighbors for dynamic social network. Proc. Comput. Sci. 83, 82–89 (2016)

    Article  Google Scholar 

  25. Zhou Zhonghua, X.J., Huiran, Z.: Data crawler for Sina Weibo based on Python. J. Comput. Appl. 34(11), 3131–3134 (2014)

    Google Scholar 

  26. Yang, C., Wang, Y.: Online social network image classification and application based o n deep learning. ICETA 2016, 41–46 (2016)

    Google Scholar 

  27. Hanafiah, N., Kevin, A., Sutanto Fiona, C., Arifin, Y., Hartanto, J.: Text normalization algorithm on Twitter in complaint category. Proc. Comput. Sci. 116, 20–26 (2017)

    Article  Google Scholar 

  28. Lu, Y., Sakamoto, K., Shibuki, H., Mori, T.: Are deep learning methods better for Twitter sentiment analysis ? 言語処理学会 第23回年次大会 発表論文集 C, 787–790 (2017)

  29. Atrey, P.K., Kankanhalli, M.S., Jain, R.: Information assimilation framework for event detection in multimedia surveillance systems. Multimed. Syst. 12(3), 239–253 (2006)

    Article  Google Scholar 

  30. Chen, Y., Rui, Y.: Real-time speaker tracking using particle filter sensor fusion. Proc. IEEE 92(3), 485–494 (2004)

    Article  Google Scholar 

  31. Datta, R., Li, J., Wang, J.Z.: Content-based image retrieval—approaches and trends of the new age. In: Proceedings of the 7th ACM SIGMM International Workshop on Multimedia Information Retrieval—MIR’05, p. 253 (2005)

  32. Schreck, T.: 3D object. IEEE Comput. Graph. Appl. 27(4), 22–27 (2007)

    Article  Google Scholar 

  33. Murnane, K.: Thirteen Companies that Use Deep Learning to Produce Actionable Results (online). https://www.forbes.com/sites/kevinmurnane/2016/04/01/thirteen-companies-that-use-deep-learning-to-produce-actionable-results/#127cea8033b8. Accessed: 16 Jan 2018 (2016)

  34. Trovati, M., Hill, R., Anjum, A., Zhu, S.Y., Liu, L.: Big-data analytics and cloud computing: theory, algorithms and applications. Big-Data Anal. Cloud Comput. Theory, Algorithms Appl. 2016, i–xvi (2016)

    Google Scholar 

  35. Hanjalic, A.: New grand challenge for multimedia information retrieval: bridging the utility gap. Int. J. Multimed. Inf. Retr. 1(3), 139–152 (2012)

    Article  Google Scholar 

  36. Rüger, S.: Multimedia Information Retrieval, vol. 1(1) (2010)

  37. Raieli, R.: Multimedia Information Retrieval : Theory and Techniques. Chandos Publishing, Oxford (2013)

  38. Zhu, W.: Visions and views multimedia big data computing. IEEE Multimed. 22(3), 96–105 (2015)

    Article  Google Scholar 

  39. Abraham, A.: Computational social networks: mining and visualization. Comput. Soc. Netw. Min. Vis. 9781447140, 1–385 (2012)

    Google Scholar 

  40. Lee, H., Largman, Y., Pham, P., Ng, A.: Unsupervised feature learning for audio classification using convolutional deep belief networks. Adv. Neural Inf. Process. Syst. 22, 1096–1104 (2009)

    Google Scholar 

  41. Kereliuk, C., Sturm, B.L., Larsen, J.: Deep learning and music adversaries. IEEE Trans. Multimed. 17(11), 2059–2071 (2015)

    Article  Google Scholar 

  42. Lao, J., Chen, Y., Li, Z.C., Li, Q., Zhang, J., Liu, J., Zhai, G.: A deep learning-based radiomics model for prediction of survival in glioblastoma multiforme. Sci. Rep. 7(1), 1–8 (2017)

    Article  Google Scholar 

  43. Li, Z., Zou, D., Xu, S., Ou, X., Jin, H., Wang, S., Deng, Z., Zhong, Y.: VulDeePecker: A Deep Learning-Based System for Vulnerability Detection. NDSS (2018)

  44. Korzeniowski, F., Widmer, G.: Feature Learning for Chord Recognition: The Deep Chroma Extractor (2016)

  45. Sun, J.-Q., Lee, S.-P.: Query by singing/humming system based on deep learning. Int. J. Appl. Eng. Res. 12(13), 973–4562 (2017)

    Google Scholar 

  46. Isin, A., Ozdalili, S.: ScienceDirect cardiac arrhythmia detection using deep learning. Proc. Comput. Sci. 120, 268–275 (2017)

    Article  Google Scholar 

  47. Lee, C.S., Tyring, A.J., Deruyter, N.P., Wu, Y., Rokem, A., Lee, A.Y.: Deep-learning based, automated segmentation of macular edema in optical coherence tomography. Biomed. Opt. Express 8(7), 3440 (2017)

    Article  Google Scholar 

  48. Zhang, S., Yao, L., Sun, A.: Deep learning based recommender system: a survey and new perspectives. ACM Comput. Surv. 1(1), 1–35 (2017)

    Article  Google Scholar 

  49. Paltz, N.: Metric learning based data augmentation for environmental sound classification. Department of Electronics, State Key Lab of Intelligent Technologies and Systems Tsinghua National Laboratory for Information Science and Technology (TNList), Beijing, pp. 1–5 (2017)

  50. Monti, F., Boscaini, D., Masci, J., Rodolà, E., Svoboda, J., Bronstein, M.M.: Geometric deep learning on graphs and manifolds using mixture model CNNs (2016)

  51. Khasanova, R., Frossard, P.: Graph-Based Classification of Omnidirectional Images 1 Introduction 2 Related work, pp. 869–878

  52. Raffel, C.: Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching, p. 222 (2016)

  53. Luo, S., Li, X., Li, J.: Automatic Alzheimer’s disease recognition from MRI data using deep learning method. J. Appl. Math. Phys. 5(9), 1892–1898 (2017)

    Article  Google Scholar 

  54. Saiyeda, A.: Cloud computing for deep learning analytics : a survey of current trends and challenges. Int. J. Adv. Res. Comput. Sci. 8(2), 68–72 (2017)

    Google Scholar 

  55. Nakamura, N., Takano, S., Okada, Y.: 3D multimedia data search system based on stochastic ARG matching method. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence, Lecture Notes in Bioinformatics), vol. 5371, pp. 379–389 (2009)

  56. Tangelder, J.W.H., Veltkamp, R.C.: A survey of content based 3D shape retrieval methods. Multimed. Tools Appl. 39(3), 441–471 (2008)

    Article  Google Scholar 

  57. Iyer, N., Jayanti, S., Lou, K., Kalyanaraman, Y., Ramani, K.: Three-dimensional shape searching: state-of-the-art review and future trends. Comput. Des. 37(5), 509–530 (2005)

    Google Scholar 

  58. Havemann, S., Fellner, D.W.: Seven research challenges of generalized 3D documents. IEEE Comput. Graph. Appl. 27(3), 70–76 (2007)

    Article  Google Scholar 

  59. Bustos, B., Keim, D., Saupe, D., Schreck, T., Vrani, D.: An experimental effectiveness comparison of methods for 3D similarity search. Int. J. Digit. Libr. 6(1), 39–54 (2006)

    Article  Google Scholar 

  60. Böhm, C., Berchtold, S., Keim, D.A.: Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput. Surv. 33(3), 322–373 (2001)

    Article  Google Scholar 

  61. Bustos, B., Keim, D.A., Saupe, D., Schreck, T., Vranić, D.V.: Feature-based similarity search in 3D object databases. ACM Comput. Surv. 37(4), 345–387 (2005)

    Article  Google Scholar 

  62. Rea, H.J., Corney, J.R., Clark, D.E.R., Taylor, N.K.: A surface partitioning spectrum (SPS) for retrieval and indexing of 3D CAD models. In: Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004, pp. 167–174

  63. Saito, S., Li, T., Li, H.: Real-time facial segmentation and performance capture from RGB input. In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence, Lecture Notes in Bioinformatics), vol. 9912(1), pp. 244–261 (2016)

  64. Santana, J.M., Wendel, J., Trujillo, A., Suárez, J.P., Simons, A., Koch, A.: Progress in Location-Based Services 2016, pp. 329–353 (2017)

  65. Castellani, U., Cortelazzo, G.M., Cristani, M., Delponte, E., Fusiello, A., Giachetti, A., Mizzaro, S., Odone, F., Puppo, E., Scateni, R., Zanuttigh, P.: 3-SHIRT : three-dimensional shape indexing and retrieval techniques. In: Eurographics Italian Chapter Conference, pp. 113–120 (2008)

  66. Chatfield, K., Arandjelović, R., Parkhi, O., Zisserman, A.: On-the-fly learning for visual search of large-scale image and video datasets. Int. J. Multimed. Inf. Retr. 4(2), 75–93 (2015)

    Article  Google Scholar 

  67. Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., Jacobs, D.: A search engine for 3D models. ACM Trans. Graph. 22(1), 83–105 (2003)

    Article  Google Scholar 

  68. Min, P., Funkhouser, T.: A 3D model search engine. Comput. Sci. 2004, 139 (2004)

    Google Scholar 

  69. Shilane, P., Min, P., Kazhdan, M., Funkhouser, T.: The Princeton shape benchmark. Proc. SMI 8540, 167–178 (2004)

    Google Scholar 

  70. Biasotti, S., Giorgi, D., Marini, S., Spagnuolo, M., Falcidieno, B.: A Comparison Framework for 3D Object Classification Methods

  71. Nehab, D., Shilane, P.: Stratified point sampling of 3D models. In: Proceedings of the First Eurographics Conference on Point-Based Graphics, pp. 49–56 (2004)

  72. Besl, P.J., Jain, R.C.: Three-dimensional object recognition. ACM Comput. Surv. 17(1), 75–145 (1985)

    Article  Google Scholar 

  73. Guo, Y., Liu, Y., Georgiou, T., Lew, M.S.: A review of semantic segmentation using deep neural networks. Int. J. Multimed. Inf. Retr. 7, 1–7 (2017)

    Google Scholar 

  74. Kankanhalli, M.S., Rui, Y.: Application potential of multimedia information retrieval. Proc. IEEE 96(4), 712–720 (2008)

    Article  Google Scholar 

  75. Lew, M.S., Sebe, N., Djeraba, C., Jain, R.: Content-based multimedia information retrieval: state of the art and challenges. ACM Trans. Multimed. Comput. Commun. Appl. 2(1), 1–19 (2006)

    Article  Google Scholar 

  76. Moise, D., Shestakov, D., Gudmundsson, G., Amsaleg, L.: Indexing and searching 100 M images with map-reduce. In: Proceedings of the 3rd ACM International Conference on Multimedia Retrieval—ICMR’13, p. 17 (2013)

  77. Krishna, M., Kannan, B., Ramani, A., CloudCom, 2010 IEEE, and undefined 2010: Implementation and performance evaluation of a hybrid distributed system for storing and processing images from the web. ieeexplore.ieee.org

  78. He, B., Fang, W., Luo, Q., Govindaraju, N.K., Wang, T.: Mars: a MapReduce framework on graphics processors. In: Proceedings of the 17th International Conference on Parallel Architecture and Compilation Techniques—PACT’08, p. 260 (2008)

  79. Chen, S.Y., Lai, C.F., Hwang, R.H., Chao, H.C., Huang, Y.M.: A multimedia parallel processing approach on GPU MapReduce framework. In: Proceedings of the 2014 7th International Conference on Ubi-Media Computing and Workshops, U-MEDIA 2014, pp. 154–159 (2014)

  80. Wang, H., Shen, Y., Wang, L., Zhufeng, K., Wang, W., Cheng, C.: /12/$31.00 ©2012 IEEE large-scale multimedia data mining using MapReduce framework. In: 2012 IEEE 4th International Conference on Cloud Computing Technology Science, pp. 978–1 (2012)

  81. Mera, D., Batko, M., Zezula, P.: Towards fast multimedia feature extraction: Hadoop or storm. In: Proceedings of the 2014 IEEE International Symposium Multimedia, ISM 2014, pp. 106–109 (2015)

  82. Zhang, C., Chen, T.: Indexing and retrieval of 3D models aided by active learning. In: Proceedings of the Ninth ACM International Conference on Multimedia—MULTIMEDIA’01, p. 615 (2001)

  83. Gao, Y., Dai, Q.: View-Based 3-D Object Retrieval. Elsevier, New York (2014)

  84. Vranic, D.V., Saupe, D., Richter, J.: Tools for 3D-object retrieval: Karhunen–Loeve transform and spherical harmonics. In: 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No. 01TH8564), pp. 293–298 (2001)

  85. Hilaga, M., Shinagawa, Y., Kohmura, T., Kunii, T.L.: Topology matching for fully automatic similarity estimation of 3D shapes. In: Proceedings of the 28th Annual Conference on Computing Graphics Interactive Techniques—SIGGRAPH’01, pp. 203–212 (2001)

  86. Laga, H., Takahashi, H., Nakajima, M.: Geometry image based similarity estimation for 3D model retrieval. In: Nicograph International 2004, Taiwan, pp. 133–138 (2004)

  87. Choi, S.-M., Kim, Y.-G.: Similarity Estimation of 3D Shapes Using Modal Strain Energy, pp. 206–212. Springer, Berlin (2005)

    Google Scholar 

  88. Sánchez-Cruz, H., Bribiesca, E.: A method of optimum transformation of 3D objects used as a measure of shape dissimilarity. Image Vis. Comput. 21(12), 1027–1036 (2003)

    Article  Google Scholar 

  89. Shum, H.-Y., Hebert, M., Ikeuchi, K.: On 3D Shape Similarity (1995)

  90. Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  91. Cass, T.A.: Robust affine structure matching for 3D object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1265–1274 (1998)

    Article  Google Scholar 

  92. Flynn, P.J., Jain, A.K.: BONSAI: 3D object recognition using constrained search. IEEE Trans. Pattern Anal. Mach. Intell. 13(10), 1066–1075 (1991)

    Article  Google Scholar 

  93. Wang, W., Iyengar, S.S.: Efficient data structures for model-based 3-D object recognition and localization from range images. IEEE Trans. Pattern Anal. Mach. Intell. 14(10), 1035–1045 (1992)

    Article  Google Scholar 

  94. Zhou, Y., Kaufman, A., Toga, A.W.: Three-dimensional skeleton and centerline generation based on an approximate minimum distance field. Vis. Comput. 14(7), 303–314 (1998)

    Article  Google Scholar 

  95. Modemlis, A., Daras, P., Tzovaras, D., Strintzis, M.G. :On 3D partial matching of meaningful parts. In: Proceedings of the International Conference on Image Processing ICIP, vol. 2, pp. 1–4 (2007)

  96. Funkhouser, T., Shilane, P.: Partial matching of 3D shapes with priority-driven search. In: Proceedings of the fourth Eurographics Symposium on Geometry Processing, pp. 131–142 (2006)

  97. Ioannides, M., Quak, E.: 3D research challenges in cultural heritage : a roadmap in digital heritage preservation

  98. Münster, S., Pfarr-Harfst, M., Kuroczyński, P., Ioannides, M.: 3D research challenges in cultural heritage II : how to manage data and knowledge related to interpretative digital 3D reconstructions of cultural heritage

  99. Dugelay, J.-L., Baskurt, A., Daoudi, M.: 3D Object Processing : Compression, Indexing and Watermarking. Wiley, New York (2008)

  100. Kuroczyński, P., Hauck, O., Dworak, D.: 3D Models on Triple Paths—New Pathways for Documenting and Visualizing Virtual Reconstructions, pp. 149–172 (2016)

  101. von Schwerin, J., Lyons, M., Loos, L., Billen, N., Auer, M., Zipf, A.: Show Me the Data!: Structuring Archaeological Data to Deliver Interactive, Transparent 3D Reconstructions in a 3D WebGIS, pp. 198–230 (2016)

Download references

Author information

Authors and Affiliations

  1. Bharati Vidyapeeth’s Institute of Computer Applications and Management (BVICAM), New Delhi, India

    Ritika Wason, Vishal Jain & Anupam Balyan

  2. CSIR-NISTADS, New Delhi, India

    Gagandeep Singh Narula

Authors
  1. Ritika Wason
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vishal Jain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gagandeep Singh Narula
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anupam Balyan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ritika Wason.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wason, R., Jain, V., Narula, G.S. et al. Deep understanding of 3-D multimedia information retrieval on social media: implications and challenges. Iran J Comput Sci 2, 101–111 (2019). https://doi.org/10.1007/s42044-019-00030-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42044-019-00030-5

Keywords

Search

Navigation