Skip to main content

Graph Contrastive Learning with Positional Representation for Recommendation

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13981))

Included in the following conference series:

Abstract

Recently, graph neural networks have become the state-of-the-art in collaborative filtering, since the interactions between users and items essentially have a graph structure. However, a major issue with the user-item interaction graph in recommendation is the absence of the positional information of users/items, which limits the expressive power of graph recommenders in distinguishing the users/items with the same neighbours after propagating several graph convolution layers. Such a phenomenon further induces the well-known over-smoothing problem. We hypothesise that we can obtain a more expressive graph recommender through graph positional encoding (e.g., Laplacian eigenvector) thereby also alleviating the over-smoothing problem. Hence, we propose a novel model named Positional Graph Contrastive Learning (PGCL) for top-K recommendation, which aims to explicitly enhance graph representation learning with graph positional encoding in a contrastive learning manner. We show that concatenating the learned graph positional encoding and the pre-existing users/items’ features in each feature propagation layer can achieve significant effectiveness gains. To further have sufficient representation learning from the graph positional encoding, we use contrastive learning to jointly learn the correlation between the pre-exiting users/items’ features and the positional information. Our extensive experiments conducted on three benchmark datasets demonstrate the superiority of our proposed PGCL model over existing state-of-the-art graph-based recommendation approaches in terms of both effectiveness and alleviating the over-smoothing problem.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.yelp.com/dataset.

  2. 2.

    https://snap.stanford.edu/data/loc-gowalla.html.

  3. 3.

    https://jmcauley.ucsd.edu/data/amazon/.

  4. 4.

    Source code is available at: https://github.com/zxy-ml84/PGCL.

References

  1. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)

    Article  MATH  Google Scholar 

  2. Chen, M., Wei, Z., Huang, Z., Ding, B., Li, Y.: Simple and deep graph convolutional networks. In: Proceedings of the 37th International Conference on Machine Learning (2020)

    Google Scholar 

  3. Dwivedi, V.P., Bresson, X.: A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 (2020)

  4. Dwivedi, V.P., Joshi, C.K., Laurent, T., Bengio, Y., Bresson, X.: Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982 (2020)

  5. Dwivedi, V.P., Luu, A.T., Laurent, T., Bengio, Y., Bresson, X.: Graph neural networks with learnable structural and positional representations. In: Proceedings of the 10th International Conference on Learning Representations (2022)

    Google Scholar 

  6. Gao, C., Wang, X., He, X., Li, Y.: Graph neural networks for recommender system. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining (2022)

    Google Scholar 

  7. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: Proceedings of the 4th International Conference on Learning Representations (2015)

    Google Scholar 

  8. Grégoire, M., Dexiong, C., Margot, S., Julien, M.: GraphiT: encoding graph structure in transformers. arXiv preprint arXiv:2106.05667 (2021)

  9. Haveliwala, T.H.: Topic-sensitive PageRank. In: Proceedings of the 11th International Conference on World Wide Web (2002)

    Google Scholar 

  10. He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., Wang, M.: LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)

    Google Scholar 

  11. KG, S., Sadasivam, G.S.: A survey on personalized recommendation techniques. Int. J. Recent Innov. Trends Comput. Commun. 2(6), 1385–1395 (2014)

    Google Scholar 

  12. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (2014)

    Google Scholar 

  13. Klicpera, J., Bojchevski, A., Günnemann, S.: Predict then propagate: graph neural networks meet personalized PageRank. arXiv preprint arXiv:1810.05997 (2018)

  14. Kreuzer, D., Beaini, D., Hamilton, W., Létourneau, V., Tossou, P.: Rethinking graph transformers with spectral attention. In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  15. Lee, D., Kang, S., Ju, H., Park, C., Yu, H.: Bootstrapping user and item representations for one-class collaborative filtering. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021)

    Google Scholar 

  16. Li, P., Wang, Y., Wang, H., Leskovec, J.: Distance encoding-design provably more powerful GNNs for structural representation learning. arXiv preprint arXiv:2009.00142 (2020)

  17. Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32th AAAI Conference on Artificial Intelligence (2018)

    Google Scholar 

  18. Lim, D., et al.: Sign and basis invariant networks for spectral graph representation learning. arXiv preprint arXiv:2202.13013 (2022)

  19. Liu, S., Ounis, I., Macdonald, C.: An MLP-based algorithm for efficient contrastive graph recommendations. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)

    Google Scholar 

  20. Liu, S., Ounis, I., Macdonald, C., Meng, Z.: A heterogeneous graph neural model for cold-start recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)

    Google Scholar 

  21. Loukas, A.: What graph neural networks cannot learn: depth vs width. arXiv preprint arXiv:1907.03199 (2019)

  22. Morris, C., et al.: Weisfeiler and Leman go neural: Higher-order graph neural networks. In: Proceedings of the 33th AAAI Conference on Artificial Intelligence (2019)

    Google Scholar 

  23. Murphy, R., Srinivasan, B., Rao, V., Ribeiro, B.: Relational pooling for graph representations. In: Proceedings of the 36th International Conference on Machine Learning (2019)

    Google Scholar 

  24. van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)

  25. Rampášek, L., Galkin, M., Dwivedi, V.P., Luu, A.T., Wolf, G., Beaini, D.: Recipe for a general, powerful, scalable graph transformer. In: Proceedings of the 36th Conference on Neural Information Processing Systems (2022)

    Google Scholar 

  26. Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (2009)

    Google Scholar 

  27. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)

    Google Scholar 

  28. Wang, X., Ounis, I., Macdonald, C.: Leveraging review properties for effective recommendation. In: Proceedings of the 30th International Conference on World Wide Web (2021)

    Google Scholar 

  29. Wang, X., He, X., Wang, M., Feng, F., Chua, T.S.: Neural graph collaborative filtering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (2019)

    Google Scholar 

  30. Weisfeiler, B., Leman, A.: The reduction of a graph to canonical form and the algebra which appears therein. NTI, Series 2(9), 12–16 (1968)

    Google Scholar 

  31. Welling, M., Kipf, T.N.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations (2016)

    Google Scholar 

  32. Wu, J., et al.: Self-supervised graph learning for recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021)

    Google Scholar 

  33. Xie, X., et al.: Contrastive learning for sequential recommendation. In: Proceedings of the 38th IEEE International Conference on Data Engineering (2022)

    Google Scholar 

  34. Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: Proceedings of the 6th International Conference on Learning Representations (2018)

    Google Scholar 

  35. Yi, Z., Wang, X., Ounis, I., Macdonald, C.: Multi-modal graph contrastive learning for micro-video recommendation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)

    Google Scholar 

  36. Ying, C., et al.: Do transformers really perform badly for graph representation? In: Advances in Neural Information Processing Systems, vol. 34 (2021)

    Google Scholar 

  37. You, J., Ying, R., Leskovec, J.: Position-aware graph neural networks. In: Proceedings of the 36th International Conference on Machine Learning (2019)

    Google Scholar 

  38. Yu, J., Yin, H., Li, J., Gao, M., Huang, Z., Cui, L.: Enhance social recommendation with adversarial graph convolutional networks. IEEE Trans. Knowl. Data Eng. 34, 3727–3739 (2020)

    Article  Google Scholar 

  39. Yu, J., Yin, H., Li, J., Wang, Q., Hung, N.Q.V., Zhang, X.: Self-supervised multi-channel hypergraph convolutional network for social recommendation. In: Proceedings of the 30th International Conference on World Wide Web (2021)

    Google Scholar 

  40. Yu, J., Yin, H., Xia, X., Chen, T., Cui, L., Nguyen, Q.V.H.: Are graph augmentations necessary? Simple graph contrastive learning for recommendation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)

    Google Scholar 

  41. Yu, J., Yin, H., Xia, X., Chen, T., Li, J., Huang, Z.: Self-supervised learning for recommender systems: a survey. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zixuan Yi .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yi, Z., Ounis, I., Macdonald, C. (2023). Graph Contrastive Learning with Positional Representation for Recommendation. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28238-6_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28237-9

  • Online ISBN: 978-3-031-28238-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics