Skip to main content
SpringerLink
Log in

‘Expected Most of the Results, but Some Others...Surprised Me’: Personality Inference in Image Tagging Services

  • Conference paper
  • First Online:
End-User Development (IS-EUD 2021)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 12724))

Included in the following conference series:

  • 846 Accesses

Abstract

Image tagging APIs, offered as Cognitive Services in the movement to democratize AI, have become popular in applications that need to provide a personalized user experience. Developers can easily incorporate these services into their applications; however, little is known concerning their behavior under specific circumstances. We consider how two such services behave when predicting elements of the Big-Five personality traits from users’ profile images. We found that personality traits are not equally represented in the APIs’ output tags, with tags focusing mostly on Extraversion. The inaccurate personality prediction and the lack of vocabulary for the equal representation of all personality traits, could result in unreliable implicit user modeling, resulting in sub-optimal – or even undesirable – user experience in the application.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://azure.microsoft.com/en-us/services/cognitive-services/face/.

  2. 2.

    https://ipip.ori.org/.

  3. 3.

    https://wordnet.princeton.edu/.

  4. 4.

    WordNet is the most widely used English lexical database which includes nouns, verbs, adjectives, and adverbs. The words are organized and linked based on their lexical concept (set of synonyms).

  5. 5.

    It is important to remind the reader that these services are effectively “black boxes,” thus, the complete list of their tags is not publicly available, not even to the developers who are incorporating them in the software they are developing.

  6. 6.

    https://stat.ethz.ch/R-manual/R-devel/library/stats/html/factanal.html.

References

  1. Barlas, P., Kleanthous, S., Kyriakou, K., Otterbacher, J.: Social b(eye)as in image tagging algorithms: Human and machine descriptions of people images. Proc. AAAI ICWSM 13(01), 583–591 (2019)

    Google Scholar 

  2. Buolamwini, J., Gebru, T.: Gender shades: intersectional accuracy disparities in commercial gender classification. In: Conference on Fairness. Accountability and Transparency, pp. 77–91. PMLR, New York (2018)

    Google Scholar 

  3. Celli, F., Bruni, E., Lepri, B.: Automatic personality and interaction style recognition from Facebook profile pictures. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 1101–1104. ACM, New York (2014)

    Google Scholar 

  4. Everitt, B., Hothorn, T.: An Introduction to Applied Multivariate Analysis with R (UseR!), January 2011. https://doi.org/10.1007/978-1-4419-9650-3

  5. Goldberg, L.: An alternative “description of personality’’: the big-five factor structure. J. Pers. Soc. Psychol. 59, 1216–29 (1991)

    Article  Google Scholar 

  6. Guntuku, S.C., Lin, W., Carpenter, J., Ng, W.K., Ungar, L.H., Preoţiuc-Pietro, D.: Studying personality through the content of posted and liked images on twitter. In: Proceedings of the 2017 ACM on Web Science Conference, WebSci 2017, pp. 223–227 ACM, New York (2017). http://doi.acm.org/10.1145/3091478.3091522

  7. Herring, S.C.: Web content analysis: expanding the paradigm. In: Hunsinger, J., Allen, M. (eds.) International Handbook of Internet Research, pp. 233–249. Springer, Dordrecht (2009). https://doi.org/10.1007/978-1-4020-9789-8_14

    Chapter  Google Scholar 

  8. Kyriakou, K., Barlas, P., Kleanthous, S., Otterbacher, J.: Fairness in proprietary image tagging algorithms: a cross-platform audit on people images. In: Proceedings of AAAI ICWSM, vol. 13, pp. 313–322. AAAI, California (2019)

    Google Scholar 

  9. Kyriakou, K., Kleanthous, S., Otterbacher, J., Papadopoulos, G.A.: Emotion-based stereotypes in image analysis services. In: Adjunct Publication of the 28th ACM UMAP Conference, pp. 252–259 (2020)

    Google Scholar 

  10. Norman, W.T.: 2800 personality trait descriptors: normative operating characteristics for a university population. University of Michigan, Ann Arbor, Michigan (1967)

    Google Scholar 

  11. Rhue, L.: Racial influence on automated perceptions of emotions. SSRN 3281765 (2018)

    Google Scholar 

  12. Silveira Jacques Junior, J.C., et al.: First impressions: a survey on vision-based apparent personality trait analysis. IEEE Trans. Affect. Comput. 1 (2019)

    Google Scholar 

Download references

Acknowledgments

This project is partially funded by the Cyprus Research and Innovation Foundation under grant EXCELLENCE/0918/0086 (DESCANT) and by the European Union’s Horizon 2020 Research and Innovation Programme under agreements No. 739578 (RISE) and 810105 (CyCAT).

Author information

Authors and Affiliations

  1. Cyprus Center for Algorithmic Transparency, Open University of Cyprus, Latsia, Cyprus

    Maria Kasinidou, Styliani Kleanthous & Jahna Otterbacher

  2. CYENS Centre of Excellence, Nicosia, Cyprus

    Styliani Kleanthous & Jahna Otterbacher

Authors
  1. Maria Kasinidou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Styliani Kleanthous
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jahna Otterbacher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Styliani Kleanthous .

Editor information

Editors and Affiliations

  1. Ingegneria dell'Informazione, Univ degli Studi di Brescia, Brescia, Italy

    Daniela Fogli

  2. Eindhoven University of Technology, Eindhoven, The Netherlands

    Daniel Tetteroo

  3. University of Brescia, Brescia, Italy

    Barbara Rita Barricelli

  4. University of Twente, Enschede, The Netherlands

    Simone Borsci

  5. Department of Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands

    Panos Markopoulos

  6. University of Cyprus, Nicosia, Cyprus

    George A. Papadopoulos

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kasinidou, M., Kleanthous, S., Otterbacher, J. (2021). ‘Expected Most of the Results, but Some Others...Surprised Me’: Personality Inference in Image Tagging Services. In: Fogli, D., Tetteroo, D., Barricelli, B.R., Borsci, S., Markopoulos, P., Papadopoulos, G.A. (eds) End-User Development. IS-EUD 2021. Lecture Notes in Computer Science(), vol 12724. Springer, Cham. https://doi.org/10.1007/978-3-030-79840-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-79840-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-79839-0

  • Online ISBN: 978-3-030-79840-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation