Abstract
Recently, graph neural networks have become the state-of-the-art in collaborative filtering, since the interactions between users and items essentially have a graph structure. However, a major issue with the user-item interaction graph in recommendation is the absence of the positional information of users/items, which limits the expressive power of graph recommenders in distinguishing the users/items with the same neighbours after propagating several graph convolution layers. Such a phenomenon further induces the well-known over-smoothing problem. We hypothesise that we can obtain a more expressive graph recommender through graph positional encoding (e.g., Laplacian eigenvector) thereby also alleviating the over-smoothing problem. Hence, we propose a novel model named Positional Graph Contrastive Learning (PGCL) for top-K recommendation, which aims to explicitly enhance graph representation learning with graph positional encoding in a contrastive learning manner. We show that concatenating the learned graph positional encoding and the pre-existing users/items’ features in each feature propagation layer can achieve significant effectiveness gains. To further have sufficient representation learning from the graph positional encoding, we use contrastive learning to jointly learn the correlation between the pre-exiting users/items’ features and the positional information. Our extensive experiments conducted on three benchmark datasets demonstrate the superiority of our proposed PGCL model over existing state-of-the-art graph-based recommendation approaches in terms of both effectiveness and alleviating the over-smoothing problem.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
Source code is available at: https://github.com/zxy-ml84/PGCL.
References
Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15(6), 1373–1396 (2003)
Chen, M., Wei, Z., Huang, Z., Ding, B., Li, Y.: Simple and deep graph convolutional networks. In: Proceedings of the 37th International Conference on Machine Learning (2020)
Dwivedi, V.P., Bresson, X.: A generalization of transformer networks to graphs. arXiv preprint arXiv:2012.09699 (2020)
Dwivedi, V.P., Joshi, C.K., Laurent, T., Bengio, Y., Bresson, X.: Benchmarking graph neural networks. arXiv preprint arXiv:2003.00982 (2020)
Dwivedi, V.P., Luu, A.T., Laurent, T., Bengio, Y., Bresson, X.: Graph neural networks with learnable structural and positional representations. In: Proceedings of the 10th International Conference on Learning Representations (2022)
Gao, C., Wang, X., He, X., Li, Y.: Graph neural networks for recommender system. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining (2022)
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: Proceedings of the 4th International Conference on Learning Representations (2015)
Grégoire, M., Dexiong, C., Margot, S., Julien, M.: GraphiT: encoding graph structure in transformers. arXiv preprint arXiv:2106.05667 (2021)
Haveliwala, T.H.: Topic-sensitive PageRank. In: Proceedings of the 11th International Conference on World Wide Web (2002)
He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., Wang, M.: LightGCN: simplifying and powering graph convolution network for recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)
KG, S., Sadasivam, G.S.: A survey on personalized recommendation techniques. Int. J. Recent Innov. Trends Comput. Commun. 2(6), 1385–1395 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (2014)
Klicpera, J., Bojchevski, A., Günnemann, S.: Predict then propagate: graph neural networks meet personalized PageRank. arXiv preprint arXiv:1810.05997 (2018)
Kreuzer, D., Beaini, D., Hamilton, W., Létourneau, V., Tossou, P.: Rethinking graph transformers with spectral attention. In: Advances in Neural Information Processing Systems, vol. 34 (2021)
Lee, D., Kang, S., Ju, H., Park, C., Yu, H.: Bootstrapping user and item representations for one-class collaborative filtering. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021)
Li, P., Wang, Y., Wang, H., Leskovec, J.: Distance encoding-design provably more powerful GNNs for structural representation learning. arXiv preprint arXiv:2009.00142 (2020)
Li, Q., Han, Z., Wu, X.M.: Deeper insights into graph convolutional networks for semi-supervised learning. In: Proceedings of the 32th AAAI Conference on Artificial Intelligence (2018)
Lim, D., et al.: Sign and basis invariant networks for spectral graph representation learning. arXiv preprint arXiv:2202.13013 (2022)
Liu, S., Ounis, I., Macdonald, C.: An MLP-based algorithm for efficient contrastive graph recommendations. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)
Liu, S., Ounis, I., Macdonald, C., Meng, Z.: A heterogeneous graph neural model for cold-start recommendation. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (2020)
Loukas, A.: What graph neural networks cannot learn: depth vs width. arXiv preprint arXiv:1907.03199 (2019)
Morris, C., et al.: Weisfeiler and Leman go neural: Higher-order graph neural networks. In: Proceedings of the 33th AAAI Conference on Artificial Intelligence (2019)
Murphy, R., Srinivasan, B., Rao, V., Ribeiro, B.: Relational pooling for graph representations. In: Proceedings of the 36th International Conference on Machine Learning (2019)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748 (2018)
Rampášek, L., Galkin, M., Dwivedi, V.P., Luu, A.T., Wolf, G., Beaini, D.: Recipe for a general, powerful, scalable graph transformer. In: Proceedings of the 36th Conference on Neural Information Processing Systems (2022)
Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (2009)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Wang, X., Ounis, I., Macdonald, C.: Leveraging review properties for effective recommendation. In: Proceedings of the 30th International Conference on World Wide Web (2021)
Wang, X., He, X., Wang, M., Feng, F., Chua, T.S.: Neural graph collaborative filtering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (2019)
Weisfeiler, B., Leman, A.: The reduction of a graph to canonical form and the algebra which appears therein. NTI, Series 2(9), 12–16 (1968)
Welling, M., Kipf, T.N.: Semi-supervised classification with graph convolutional networks. In: Proceedings of the 5th International Conference on Learning Representations (2016)
Wu, J., et al.: Self-supervised graph learning for recommendation. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (2021)
Xie, X., et al.: Contrastive learning for sequential recommendation. In: Proceedings of the 38th IEEE International Conference on Data Engineering (2022)
Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? In: Proceedings of the 6th International Conference on Learning Representations (2018)
Yi, Z., Wang, X., Ounis, I., Macdonald, C.: Multi-modal graph contrastive learning for micro-video recommendation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)
Ying, C., et al.: Do transformers really perform badly for graph representation? In: Advances in Neural Information Processing Systems, vol. 34 (2021)
You, J., Ying, R., Leskovec, J.: Position-aware graph neural networks. In: Proceedings of the 36th International Conference on Machine Learning (2019)
Yu, J., Yin, H., Li, J., Gao, M., Huang, Z., Cui, L.: Enhance social recommendation with adversarial graph convolutional networks. IEEE Trans. Knowl. Data Eng. 34, 3727–3739 (2020)
Yu, J., Yin, H., Li, J., Wang, Q., Hung, N.Q.V., Zhang, X.: Self-supervised multi-channel hypergraph convolutional network for social recommendation. In: Proceedings of the 30th International Conference on World Wide Web (2021)
Yu, J., Yin, H., Xia, X., Chen, T., Cui, L., Nguyen, Q.V.H.: Are graph augmentations necessary? Simple graph contrastive learning for recommendation. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)
Yu, J., Yin, H., Xia, X., Chen, T., Li, J., Huang, Z.: Self-supervised learning for recommender systems: a survey. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yi, Z., Ounis, I., Macdonald, C. (2023). Graph Contrastive Learning with Positional Representation for Recommendation. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_19
Download citation
DOI: https://doi.org/10.1007/978-3-031-28238-6_19
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-28237-9
Online ISBN: 978-3-031-28238-6
eBook Packages: Computer ScienceComputer Science (R0)