Abstract
This paper investigates the significance of numeric user ratings in recommender systems by considering their inclusion / exclusion in both the generation and evaluation of recommendations. When standard evaluation metrics are used, experimental results show that inclusion of numeric rating values in the recommendation process does not enhance the results. However, evaluating the accuracy of a recommender algorithm requires identifying the aim of the system. Evaluation metrics such as precision and recall evaluate how well a system performs at recommending items that have been previously rated by the user. By contrast, a new metric, known as Approval Rate, is intended to evaluate how well a system performs at recommending items that would be rated highly by the user. Experimental results demonstrate that these two aims are not synonymous and that for an algorithm to attempt both obscures the investigation. The results also show that appropriate use of numeric rating valuesin the process of calculating user similarity can enhance the performance when Approval Rate is used.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ahn, H.J.: A New Similarity Measure for Collaborative Filtering to Alleviate the New User Cold-starting Problem. Inf. Sci. 178(1), 37–51 (2008)
Breese, J., Heckerman, D., Kadie, C.: Empirical Analysis of Predictive Algorithms for Collaborative Filtering. In: UAI 1998, pp. 43–52 (1998)
Goldberg, D., Nichols, D., Oki, B.M., Terry, D.: Using Collaborative Filtering to Weave an Information Tapestry. Commun. ACM 35(12), 61–70 (1992)
Hayes, C., Massa, P., Avesani, P., Cunningham, P.: An On-line Evaluation Framework for Recommender Systems. In: Workshop on Personalization and Recommendation in E-Commerce (2002)
Herlocker, J., Konstan, J., Terveen, L., Riedl, J.: Evaluating Collaborative Filtering Recommender Systems. ACM TOIS 22(1), 5–53 (2004)
Huang, Z., Zeng, D., Chen, H.: A Comparison of Collaborative-Filtering Recommendation Algorithms for E-commerce. IEEE IS 22(5), 68–78 (2007)
Konstan, J.A., Riedl, J.: Research Resources for Recommender Systems. In: CHI 1999 (1999)
McLaughlin, M.R., Herlocker, J.L.: A Collaborative Filtering Algorithm and Evaluation Metric that Accurately Model the User Experience. In: SIGIR 2004, pp. 329–336. ACM, New York (2004)
McNee, S.M., Riedl, J., Konstan, J.A.: Being Accurate is Not Enough: How Accuracy Metrics have hurt Recommender Systems. In: CHI 2006, pp. 1097–1101. ACM, New York (2006)
McNee, S.M., Albert, I., Cosley, D., Gopalkrishnan, P., Lam, S.K., Rashid, A.M., Konstan, J.A., Riedl, J.: On the Recommending of Citations for Research Papers. In: ACM CSCW 2002, pp. 116–125. ACM Press, New York (2002)
MovieLens Dataset (2006), http://www.grouplens.org/
Resnick, P., Varian, H.: Recommender Systems. CACM 40(3), 56–58 (1997)
Resnick, P., Iacovou, N., Sushak, M., Bergstrom, P., Riedl, J.: GroupLens: An Open Architecture for Collaborative Filtering of Netnews. In: ACM CSCW 1994, pp. 175–186. ACM Press, New York (1994)
Riedl, J., Jameson, A.: Advanced topics in recommendation. In: The 12th IUI 2007, pp. 11–11. ACM, New York (2007)
Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Analysis of Recommendation Algorithms for E-commerce. In: EC 2000, pp. 158–167. ACM, New York (2000)
Shardanand, U., Maes, P.: Social Information Filtering: Algorithms for Automating ‘Word of Mouth’. In: SIGCHI 1995 Conference on Human Factors in Computing Systems, pp. 210–217. ACM Press, New York (1995)
Swearingen, K., Sinha, R.: Interaction Design for Recommender Systems. In: Designing Interactive Systems. ACM Press, New York (2002)
Symeonidis, P., Nanopoulos, A., Papadopoulos, A., Manolopoulos, Y.: Collaborative Filtering Process in a Whole New Light. In: IDEAS 2006, pp. 29–36. IEEE Computer Society, Washington (2006)
Ziegler, C., McNee, S.M., Konstan, J.A., Lausen, G.: Improving recommendation lists through topic diversification. In: WWW 2005, pp. 22–32. ACM, New York (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Redpath, J., Glass, D.H., McClean, S., Chen, L. (2010). Collaborative Filtering: The Aim of Recommender Systems and the Significance of User Ratings. In: Gurrin, C., et al. Advances in Information Retrieval. ECIR 2010. Lecture Notes in Computer Science, vol 5993. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12275-0_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-12275-0_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12274-3
Online ISBN: 978-3-642-12275-0
eBook Packages: Computer ScienceComputer Science (R0)