Skip to main content
SpringerLink
Log in

Assessing the Quality of Online Reviews Using Formal Argumentation Theory

  • Conference paper
  • First Online:
Web Engineering (ICWE 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12706))

Included in the following conference series:

Abstract

Review scores collect users’ opinions in a simple and intuitive manner. However, review scores are also easily manipulable, hence they are often accompanied by explanations. A substantial amount of research has been devoted to ascertaining the quality of reviews, to identify the most useful and authentic scores through explanation analysis. In this paper, we advance the state of the art in review quality analysis. We introduce a rating system to identify review arguments and to define an appropriate weighted semantics through formal argumentation theory. We introduce an algorithm to construct a corresponding graph, based on a selection of weighted arguments, their semantic similarity, and the supported ratings. We provide an algorithm to identify the model of such an argumentation graph, maximizing the overall weight of the admitted nodes and edges. We evaluate these contributions on the Amazon review dataset by McAuley et al. [15], by comparing the results of our argumentation assessment with the upvotes received by the reviews. Also, we deepen the evaluation by crowdsourcing a multidimensional assessment of reviews and comparing it to the argumentation assessment. Lastly, we perform a user study to evaluate the explainability of our method. Our method achieves two goals: (1) it identifies reviews that are considered useful, comprehensible, truthful by online users and does so in an unsupervised manner, and (2) it provides an explanation of quality assessments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Source code available at: https://github.com/davideceolin/FAReviews.

  2. 2.

    https://swish.swi-prolog.org/p/argue.swinb.

  3. 3.

    http://mturk.com.

  4. 4.

    The questionnaire is available at https://forms.gle/srGJpGyYBzWd9RTaA.

References

  1. Amgoud, L., Cayrol, C.: A reasoning model based on the production of acceptable arguments. Ann. Math. Artif. Intell. 34, 197–215 (2002)

    Article  MathSciNet  Google Scholar 

  2. Amgoud, L., Vesic, S.: Two roles of preferences in argumentation frameworks. In: Liu, W. (ed.) ECSQARU 2011. LNCS (LNAI), vol. 6717, pp. 86–97. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22152-1_8

    Chapter  Google Scholar 

  3. Baroni, P., Caminada, M., Giacomin, M.: Abstract argumentation frameworks and their semantics. In: Baroni, P., Gabbay, D., Giacomin, M. (eds.) Handbook of Formal Argumentation, Chap. 4. College Publications, London (2018)

    Google Scholar 

  4. Bench-Capon, T.J.M.: Value-based argumentation frameworks. In: Proceedings of NMR Workshop, pp. 443–454 (2002)

    Google Scholar 

  5. Bench-Capon, T.J.M.: Persuasion in practical argument using value-based argumentation frameworks. J. Logic Comput. 13(3), 429–448 (2003)

    Article  MathSciNet  Google Scholar 

  6. Ceolin, D., Noordegraaf, J., Aroyo, L.: Capturing the ineffable: collecting, analysing, and automating web document quality assessments. In: Blomqvist, E., Ciancarini, P., Poggi, F., Vitali, F. (eds.) EKAW 2016. LNCS (LNAI), vol. 10024, pp. 83–97. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-49004-5_6

    Chapter  Google Scholar 

  7. Coste-Marquis, S., Konieczny, S., Marquis, P., Ouali, M.A.: Selecting extensions in weighted argumentation frameworks. In: Proceedings of COMMA. IOS Press (2012)

    Google Scholar 

  8. Coste-Marquis, S., Konieczny, S., Marquis, P., Ouali, M.A.: Weighted attacks in argumentation frameworks. In: Proceedings of KR, pp. 593–597. AAAI Press (2012)

    Google Scholar 

  9. Dunne, P.E., Hunter, A., McBurney, P., Parsons, S., Wooldridge, M.: Weighted argument systems: basic definitions, algorithms, and complexity results. Artif. Intell. 175(2), 457–486 (2011)

    Article  MathSciNet  Google Scholar 

  10. Ghose, A., Ipeirotis, P.G.: Estimating the helpfulness and economic impact of product reviews: mining text and reviewer characteristics. IEEE Trans. Knowl. Data Eng. 23(10), 1498–1512 (2011)

    Article  Google Scholar 

  11. Kincaid, J., Fishburne, R., Rogers, R., Chissom, B.: Derivation of new readability formulas for navy enlisted personnel. Research branch report 8–75. Technical report, Chief of Naval Technical Training: Naval Air Station Memphis (1975)

    Google Scholar 

  12. Korfiatis, N., García-Bariocanal, E., Sánchez-Alonso, S.: Evaluating content quality and helpfulness of online product reviews: the interplay of review helpfulness vs. review content. Electron. Commer. Res. Appl. 11(3), 205–217 (2012)

    Article  Google Scholar 

  13. Kusner, M.J., Sun, Y., Kolkin, N.I., Weinberger, K.Q.: From word embeddings to document distances. In: Proceedings of ICML, pp. 957–966. JMLR.org (2015)

    Google Scholar 

  14. Martínez, D.C., García, A.J., Simari, G.R.: An abstract argumentation framework with varied-strength attacks. In: Proceedings of KR, pp. 135–144. AAAI Press (2008)

    Google Scholar 

  15. McAuley, J.J., Targett, C., Shi, Q., van den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of SIGIR, pp. 43–52. ACM (2015)

    Google Scholar 

  16. Mihalcea, R., Tarau, P.: TextRank: bringing order into text. In: Proceedings of EMNLP, pp. 404–411. ACL (2004)

    Google Scholar 

  17. Modgil, S.: Reasoning about preferences in argumentation frameworks. Artif. Intell. 173(9), 901–934 (2009)

    Article  MathSciNet  Google Scholar 

  18. Ocampo Diaz, G., Ng, V.: Modeling and prediction of online product review helpfulness: a survey. In: Proceedings of ACL, vol. 1, pp. 698–708. ACL (2018)

    Google Scholar 

  19. Řehůřek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of NLP Frameworks Workshop, pp. 45–50. ELRA (2010)

    Google Scholar 

  20. Roitero, K., Soprano, M., Fan, S., Spina, D., Mizzaro, S., Demartini, G.: Can the crowd identify misinformation objectively? The effects of judgment scale and assessor’s background. In: Proceedings of SIGIR, pp. 439–448. ACM (2020)

    Google Scholar 

  21. Wathen, C.N., Burkell, J.: Believe it or not: factors influencing credibility on the web. J. Am. Soc. Inf. Sci. Technol. 53(2), 134–144 (2002)

    Article  Google Scholar 

  22. Wu, P., Van Der, Heijden, H., Korfiatis, N.: The influences of negativity and review quality on the helpfulness of online reviews. In: Proceedings of ICIS, pp. 3710–3719 (2011)

    Google Scholar 

  23. Wyner, A., Schneider, J., Atkinson, K., Bench-Capon, T.: Semi-automated argumentative analysis of online product reviews. In: Proceedings of COMMA, pp. 43–50. IOS Press (2012)

    Google Scholar 

Download references

Acknowledgements

This work is partially supported by The Credibility Coalition.

Author information

Authors and Affiliations

  1. Centrum Wiskunde & Informatica, Amsterdam, The Netherlands

    Davide Ceolin & Jan Wielemaker

  2. University of Milan, Milan, Italy

    Giuseppe Primiero

  3. University of Udine, Udine, Italy

    Michael Soprano

Authors
  1. Davide Ceolin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giuseppe Primiero
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Wielemaker
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Soprano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Davide Ceolin .

Editor information

Editors and Affiliations

  1. Dipartimento di Elettronica, Politecnico di Milano, Milan, Italy

    Marco Brambilla

  2. E2S UPPA, LIUPPA, Université de Pau et des Pays de l’Adour, Anglet, France

    Richard Chbeir

  3. Econometric Institute, Erasmus University Rotterdam, Rotterdam, The Netherlands

    Flavius Frasincar

  4. Inria Saclay-Île-de-France, Institut Polytechnique de Paris, Palaiseau, France

    Ioana Manolescu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ceolin, D., Primiero, G., Wielemaker, J., Soprano, M. (2021). Assessing the Quality of Online Reviews Using Formal Argumentation Theory. In: Brambilla, M., Chbeir, R., Frasincar, F., Manolescu, I. (eds) Web Engineering. ICWE 2021. Lecture Notes in Computer Science(), vol 12706. Springer, Cham. https://doi.org/10.1007/978-3-030-74296-6_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-74296-6_6

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-74295-9

  • Online ISBN: 978-3-030-74296-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation