Abstract
Complex networks are often either too large for full exploration, partially accessible, or partially observed. Downstream learning tasks on these incomplete networks can produce low quality results. In addition, reducing the incompleteness of the network can be costly and nontrivial. As a result, network discovery algorithms optimized for specific downstream learning tasks given resource collection constraints are of great interest. In this paper, we formulate the task-specific network discovery problem in an incomplete network setting as a sequential decision making problem. Our downstream task is selective harvesting, the optimal collection of vertices with a particular attribute. We propose a framework, called Network Actor Critic (NAC), which learns a policy and notion of future reward in an offline setting via a deep reinforcement learning algorithm. A quantitative study is presented on several synthetic and real benchmarks. We show that offline models of reward and network discovery policies lead to significantly improved performance when compared to competitive online discovery algorithms.
DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited. This material is based upon work supported by the Under Secretary of Defense for Research and Engineering under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Under Secretary of Defense for Research and Engineering.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Heess, N., Dhruva, T.B., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Ali Eslami, S.M., Riedmiller, M.A., Silver, D.: Emergence of locomotion behaviours in rich environments. CoRR, abs/1707.02286 (2017)
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., Hassabis, D.: Mastering the game of Go without human knowledge. Nature 550, 354–359 (2017)
Wang, X., Garnett, R., Schneider, J.: Active search on graphs. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2013)
LaRock, T., Sakharov, T., Bhadra, S., Eliassi-Rad, T.: Reducing network incompleteness through online learning: a feasibility study. In: The 14th International Workshop on Mining and Learning with Graphs (2018)
Soundarajan, S., Eliassi-Rad, T., Gallagher, B., Pinar, A.: MaxOutProbe: an algorithm for increasing the size of partially observed networks. CoRR, abs/1511.06463 (2015)
Soundarajan, S., Eliassi-Rad, T., Gallagher, B., Pinar, A.: MaxReach: reducing network incompleteness through node probes. In: ASONAM, pp 152–157 (2016)
Avrachenkov, K., Basu, P., Neglia, G., Ribeiro, B., Towsley, D.: Pay few, influence most: online myopic network covering. In: IEEE Conference on Computer Communications Workshops, pp. 813–818 (2014)
Murai, F., Rennó, D., Ribeiro, B., Pappa, G.L., Towsley, D.F., Gile, K.: Selective harvesting over networks. Data Min. Knowl. Discov. 32(1), 187–217 (2017)
Zhang, Z., Cui, P., Zhu, W.: Deep learning on graphs: a survey. CoRR, abs/1812.04202 (2018)
You, J., Liu, B., Ying, R., Pande, V., Leskovec, J.: Graph convolutional policy network for goal-directed molecular graph generation. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, pp. 6412–6422 (2018)
Mofrad, M.H., Melhem, R., Hammoud, M.: Partitioning graphs for the cloud using reinforcement learning. CoRR, abs/1907.06768 (2019)
De Cao, N., Kipf, T.: MolGAN: an implicit generative model for small molecular graphs, CoRR, abs/1805.11973 (2018)
Dai, H., Khalil, E.B., Zhang, Y., Dilkina, B., Song, L.: Learning combinatorial optimization algorithms over graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, pp. 6351–6361 (2017)
Ho, C., Kochenderfer, M.J., Mehta, V., Caceres, R.S.: Control of epidemics on graphs. In: 54th IEEE Conference on Decision and Control, pp. 4202–4207 (2015)
Goindani, M., Neville, J.: Social reinforcement learning to combat fake news spread. In: Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence (2019)
Mittal, A., Dhawan, A., Medya, S., Ranu, S., Singh, A.K.: Learning heuristics over large graphs via deep reinforcement learning. CoRR, abs/1903.03332 (2019)
Haveliwala, T.H.: Topic-sensitive pagerank: a context-sensitive ranking algorithm for web search. IEEE Trans. Knowl. Data Eng. 15(4), 784–796 (2003)
Kloumann, I.M., Ugander, J., Kleinberg, J.: Block models and personalized PageRank. Proc. Natl. Acad. Sci. 114(1), 33–38 (2017)
Gleich, D.: PageRank beyond the web. SIAM Rev. 57 (2014). https://doi.org/10.1137/140976649
Lancichinetti, A., Fortunato, S., Radicchi, F.: Benchmark graphs for testing community detection algorithms. Phys. Rev. E 78, 046110 (2008)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. CoRR, abs/1707.06347 (2017)
Nadakuditi, R.R., Newman, M.E.J.: Graph spectra and the detectability of community structure in networks. CoRR, abs/1205.1813 (2012)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR, abs/1412.6980 (2014)
Holland, P.W., Laskey, K.B., Leinhardt, S.: Stochastic blockmodels: first steps. Soc. Netw. 5(2), 109–137 (1983)
Erdös, P., Rényi, A.: On random graphs. Publicationes Mathematicae 6, 290–297 (1959)
Rozemberczki, B., Davies, R., Sarkar, R., Sutton, C.A.: GEMSEC: graph embedding with self clustering. CoRR, abs/1802.03997 (2018)
Avrachenkov, K., Borkar, V.S., Kadavankandy, A., Sreedharan, J.K.: Comparison of random walk based techniques for estimating network averages. In: International Conference on Computational Social Networks, pp. 27–38 (2016)
Avrachenkov, K., Borkar, V.S.,Kadavankandy, A., Sreedharan, J.K.: Revisiting random walk based sampling in networks: evasion of burn-in period and frequent regenerations. Comput. Soc. Netw. (2018)
Avrachenkov, K., Litvak, N., Nemirovsky, D., Smirnova, E., Sokol, M.: Quick detection of top-k personalized pagerank lists. In: International Workshop on Algorithms and Models for The Web-Graph, pp. 50–61 (2011)
Borkar, V.S., Mathkar, A.S.: Reinforcement learning for matrix computations: pagerank as an example. In: International Conference on Distributed Computing and Internet Technology, pp. 14–24 (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Morales, P., Caceres, R.S., Eliassi-Rad, T. (2020). Deep Reinforcement Learning for Task-Driven Discovery of Incomplete Networks. In: Cherifi, H., Gaito, S., Mendes, J., Moro, E., Rocha, L. (eds) Complex Networks and Their Applications VIII. COMPLEX NETWORKS 2019. Studies in Computational Intelligence, vol 881. Springer, Cham. https://doi.org/10.1007/978-3-030-36687-2_75
Download citation
DOI: https://doi.org/10.1007/978-3-030-36687-2_75
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36686-5
Online ISBN: 978-3-030-36687-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)