Abstract
CTR prediction tasks deal with the problem of evaluating the probability of users clicking on products, and have been widely deployed in many online recommendation and advertising platforms. Mainstream CTR models can be divided into two categories: the traditional machine learning models (e.g., GBDT [7]) that learn the linear feature combinations for prediction, and deep learning based algorithms (such as DeepFM [9]) for modeling the complex and sparse feature correlations. Some recent works proposed to fuse these two kinds of models for prediction. These fusion models either feed the intermediate results learned by one model into the second category or rely on the ensemble techniques to fuse two independently trained model outputs. In this paper, we propose a residual learning based fusion framework for CTR prediction. The key idea is that, we first train a model (e.g., GBDT), and let the second model (e.g., DeepFM) learn the residual part that can not be accurately predicted by the first model. The soundness of this framework is that: as the prediction power of these two kinds of models is complementary, it is easier to let the second model learn the residual output that can not be well captured by the first model. We show that our proposed framework is flexible and it is easier to train with faster convergence. Extensive experimental results on three real-world datasets show the effectiveness of our proposed framework.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abedalla, A., et al.: MTRecS-DLT: multi-modal transport recommender system using deep learning and tree models. In: 2019 SNAMS, pp. 274–278. IEEE (2019). https://doi.org/10.1109/SNAMS.2019.8931864
Chen, L., Wu, L., Hong, R., Zhang, K., Wang, M.: Revisiting graph based collaborative filtering: a linear residual graph convolutional network approach. In: AAAI2020, vol. 34, pp. 27–34 (2020)
Chen, T., Guestrin, C.: XGBoost: a scalable tree boosting system. In: KDD2016, KDD 2016, p. 785–794. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2939672.2939785
Cheng, H., Cantú-Paz, E.: Personalized click prediction in sponsored search. In: WSDM (2010). https://doi.org/10.1145/1718487.1718531
Cheng, H.T., et al.: Wide & deep learning for recommender systems. https://doi.org/10.1145/2988450.2988454
Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: Proceedings of the Second European Conferenceon Computational Learning Theory (1995). https://doi.org/10.1006/jcss.1997.1504
Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat., 1189–1232 (2001). https://doi.org/10.1214/aos/1013203451
Graepel, T., Borchert, T., Herbrich, R.: Web-scale Bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine (2010). https://doi.org/10.5555/3104322.3104326
Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: a factorization-machine based neural network for CTR prediction. https://doi.org/10.24963/ijcai.2017/239
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
He, X., et al.: Practical lessons from predicting clicks on ads at facebook. In: Proceedings of the Eighth International Workshop on Data Mining for Online Advertising, pp. 1–9 (2014). https://doi.org/10.1145/2648584.2648589
Huang, T., Zhang, Z., Zhang, J.: FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp. 169–177 (2019). https://doi.org/10.1145/3298689.3347043
Juan, Y., Lefortier, D., Chapelle, O.: Field-aware factorization machines in a real-world online advertising system. In: Proceedings of the 26th International Conference on World Wide Web Companion, pp. 680–688 (2017). https://doi.org/10.1145/3041021.3054185
Juan, Y., Zhuang, Y., Chin, W.S., Lin, C.J.: Field-aware factorization machines for CTR prediction. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 43–50 (2016). https://doi.org/10.1145/2959100.2959134
Ke, G., et al.: LightGBM: a highly efficient gradient boosting decision tree. In: Advances In Neural Information Processing Systems, pp. 3146–3154 (2017). https://doi.org/10.5555/3294996.3295074
Ke, G., Xu, Z., Zhang, J., Bian, J., Liu, T.Y.: DeepGBM: a deep learning framework distilled by GBDT for online prediction tasks. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 384–394 (2019). https://doi.org/10.1145/3292500.3330858
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., Sun, G.: xDeepFM: combining explicit and implicit feature interactions for recommender systems. https://doi.org/10.1145/3219819.3220023
Ling, X., Deng, W., Chen, G., Zhou, H., Cui, L., Feng, S.: Model ensemble for click prediction in bing search ads (2017). https://doi.org/10.1145/3041021.3054192
Rendle, S.: Factorization machines. In: 2010 IEEE International Conference on Data Mining, pp. 995–1000. IEEE (2010). https://doi.org/10.1109/ICDM.2010.127
Richardson, M., Dominowska, E., Ragno, R.: Predicting clicks: estimating the click-through rate for new ads. In: Proceedings of the 16th International Conference on World Wide Web, pp. 521–530 (2007). https://doi.org/10.1145/1242572.1242643
Trofimov, I., Kornetova, A., Topinskiy, V.: Using boosted trees for click-through rate prediction for sponsored search. In: Data Mining for Online Advertising and Internet Economy, pp. 1–6 (2012)
Wang, R., Fu, B., Fu, G., Wang, M.: Deep & cross network for ad click predictions. https://doi.org/10.1145/3124749.3124754
Wu, L., Sun, P., Fu, Y., Hong, R., Wang, X., Wang, M.: A neural influence diffusion model for social recommendation. In: SIGIR2019, pp. 235–244 (2019)
Yang, A.: A recommendation system based on fusing boosting model and DNN model. https://doi.org/10.32604/cmc.2019.07704
YuChin Juan, W.S.C., Zhuang, Y.: 3 Idiots’ Approach for Display Advertising Challenge (2014). https://github.com/ycjuan/kaggle-2014-criteo/
Acknowledgement
This work was supported in part by the National Natural Science Foundation of China (Grant No. 61972125, U19A2079), the Fundamental Research Funds for the Central Universities (Grant No. JZ2020HGPA0114), Zhejiang Lab (No. 2019KE0AB04) and the Foundation of Key Laboratory of Cognitive Intelligence, iFLYTEK, P.R., China (Grant No. COGOS-20190002).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bao, J., Ji, Y., Yang, Y., Wu, L., Fu, R. (2020). ResFusion: A Residual Learning Based Fusion Framework for CTR Prediction. In: Dou, Z., Miao, Q., Lu, W., Mao, J., Jia, G. (eds) Information Retrieval. CCIR 2020. Lecture Notes in Computer Science(), vol 12285. Springer, Cham. https://doi.org/10.1007/978-3-030-56725-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-56725-5_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-56724-8
Online ISBN: 978-3-030-56725-5
eBook Packages: Computer ScienceComputer Science (R0)