Abstract
Neural Architecture Search (NAS) has become more and more prevalent in the field of deep learning in the past two years. Existing works often focus on image classification, and few works recently extend NAS to another computer vision task, such as semantic image segmentation. The semantic image segmentation is essentially a dense prediction for each pixel on whole image. Therefore, we choose the same basic primitive operations to build the search space for the two computer vision task respectively. Searching good neural network architectures and then training them from scratch is a regular procedure for NAS. In this paper, we design a prototype system that deploy search module and train module to collaborate with each other. Follow the former research, we initialize over-parameterized cells architecture and then transform to the continuous relaxation of the architecture to derive the good subnetwork by gradient descent. Our system can support any differential search algorithm, such as one-shot, DARTS or ProxylessNAS. We illustrate the effectiveness of our chosen primitive operations in the image classification and ability to transfer these operations to build search space for semantic image segmentation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Apache Hadoop. https://hadoop.apache.org/
Baker, B., Gupta, O., Naik, N., Raskar, R.: Designing neural network architectures using reinforcement learning. In: International Conference on Learning Representations (ICLR), pp. 1–18 (2017). https://doi.org/10.1080/0305215042000274942. http://arxiv.org/abs/1611.02167
Baker, B., Gupta, O., Raskar, R., Naik, N.: Accelerating neural architecture search using performance prediction. In: ICLR Workshop, vol. 2, pp. 1–7 (2018). http://metalearning.ml/papers/metalearn17_baker.pdf
Bender, G., Kindermans, P.J., Zoph, B., Vasudevan, V., Le, Q.: Understanding and simplifying one-shot architecture search. In: 35th International Conference on Machine Learning (ICML), vol. 80, pp. 549–558 (2018). https://doi.org/10.1109/TDEI.2009.5211872. http://proceedings.mlr.press/v80/bender18a.html
Cai, H., Chen, T., Zhang, W., Yu, Y., Wang, J.: Efficient architecture search by network transformation. In: AAAI (2018). http://arxiv.org/abs/1707.04873
Cai, H., Zhu, L., Han, S.: ProxylessNAS: direct neural architecture search on target task and hardware. In: 2019 International Conference on Learning Representations (ICLR) (2019)
Chen, L.C., et al.: Searching for efficient multi-scale architectures for dense image prediction. In: NeurIPS (2018)
Deng, B., Lin, D., Yan, J., Lin, D.: Peephole: predicting network performance before training (2017). http://arxiv.org/abs/1712.03351
Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: a survey. CoRR abs/1808.05377 (2018)
Everingham, M., Eslami, S.M.A., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The Pascal visual object classes challenge: a retrospective. Int. J. Comput. Vis. 111(1), 98–136 (2015). https://doi.org/10.1007/s11263-014-0733-5
Grathwohl, W., Creager, E., Kamyar, S., Ghasemipour, S., Zemel, R.: Gradient-based optimization of neural network architecture. In: International Conference on Learning Representations (ICLR), pp. 1–6 (2018)
Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269 (2017)
Jin, H., Song, Q., Hu, X.: Efficient neural architecture search with network morphism. CoRR abs/1806.10282 (2018)
Kandasamy, K., Neiswanger, W., Schneider, J., Poczos, B., Xing, E.: Neural architecture search with bayesian optimisation and optimal transport (2018). http://arxiv.org/abs/1802.07191
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
Klein, A., Falkner, S., Springenberg, J.T., Hutter, F.: Learning curve prediction with bayesian neural networks. In: International Conference on Learning Representations (ICLR), pp. 1–16. MCMC (2017)
Liu, C., et al.: Auto-DeepLab: hierarchical neural architecture search for semantic image segmentation. CoRR abs/1901.02985 (2019)
Liu, C., et al.: Progressive neural architecture search. In: ECCV (2018)
Liu, H., Simonyan, K., Vinyals, O., Fernando, C., Kavukcuoglu, K.: Hierarchical representations for efficient architecture search. In: 2018 International Conference on Learning Representations (ICLR) (2018). https://openreview.net/forum?id=BJQRKzbA-
Liu, H., Simonyan, K., Yang, Y.: DARTS: differentiable architecture search. In: 2019 International Conference on Learning Representations (ICLR) (2019). https://openreview.net/forum?id=S1eYHoC5FX
Miikkulainen, R., et al.: Evolving deep neural networks. In: Kozma, R., Alippi, C., Choe, Y., Morabito, F.C. (eds.) Artificial Intelligence in the Age of Neural Networks and Brain Computing. Elsevier, Amsterdam (2018). http://nn.cs.utexas.edu/?miikkulainen:chapter18
Milicchio, F., Gehrke, W.A.: Distributed Services with OpenAFS: For Enterprise and Education, 1st edn. Springer, Heidelberg (2010). Incorporated
Paszke, A., et al.: Automatic differentiation in PyTorch. In: NIPS, pp. 1–4 (2017)
Qing, Y.: FastDFS is an open source high performance distributed file system (DFS) (2013). https://github.com/happyfish100/fastdfs/tree/master
Sergeev, A., Balso, M.D.: Horovod: fast and easy distributed deep learning in TensorFlow. CoRR abs/1802.05799 (2018). http://arxiv.org/abs/1802.05799
Shin, R., Packer, C., Song, D.: Differentiable neural network architecture search. In: International Conference on Learning Representations (ICLR), pp. 1–4 (2018). No. 2017
Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009). https://doi.org/10.1162/artl.2009.15.2.15202. http://www.mitpressjournals.org/doi/10.1162/artl.2009.15.2.15202
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002). https://doi.org/10.1162/106365602320169811
Elsken, T., Metzen, J.H., Hutter, F.: Simple and efficient architecture search for convolutional neural networks. In: 2018 International Conference on Learning Representations (ICLR) (2018). https://openreview.net/forum?id=SySaJ0xCZ
Xie, L., Yuille, A.L.: Genetic CNN. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 1388–1397 (2017)
Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. In: International Conference on Learning Representations (ICLR), pp. 1–16 (2017). https://doi.org/10.1016/j.knosys.2015.01.010
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V.: Learning transferable architectures for scalable image recognition. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8697–8710 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Zhou, T., Weng, Y., Yang, G. (2019). CNASV: A Convolutional Neural Architecture Search-Train Prototype for Computer Vision Task. In: Wang, X., Gao, H., Iqbal, M., Min, G. (eds) Collaborative Computing: Networking, Applications and Worksharing. CollaborateCom 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 292. Springer, Cham. https://doi.org/10.1007/978-3-030-30146-0_26
Download citation
DOI: https://doi.org/10.1007/978-3-030-30146-0_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-30145-3
Online ISBN: 978-3-030-30146-0
eBook Packages: Computer ScienceComputer Science (R0)