Abstract
Skip connections are used in DenseNets recently and have significantly improved network performance. In this paper, we compare skip connections with the diffusion process of endogenous Nitric Oxide (NO) between neurons, and propose DiffusionNets by replacing skip connections with NO diffusion model. Each layer is considered as a point spreading signal to space as well as receiving signal from space. The whole network transmits information with a diffusing way. DiffusionNets have several advantages: (1) generate more discriminative features. (2) more similar to neural information transmission. (3) higher classification accuracy. DiffusionNets were evaluated on CIFAR10 and CIFAR100 and outperform the original DenseNets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alom, M.Z., Taha, T.M.: The history began from AlexNet: a comprehensive survey on deep learning approaches. arXiv preprint arXiv:1803.01164 (2018)
Srivastava, R.K., Greff, K.: Training very deep networks. In: NIPS, pp. 2377–2385 (2015)
He, K.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)
Huang, G.: Densely connected convolutional networks. In: CVPR, pp. 2269–2361 (2016)
Jia, P., Yin, J.: Retrograde adaptive resonance theory based on the role of nitric oxide in long-term potentiation. J. Comput. Neurosci. 23(1), 129–141 (2007)
Bredt, D.S.: Nitric oxide, a novel neuronal messenger. Neuron 8(1), 3–11 (1992)
Krekelberg, B., Taylor, J.G.: Nitric oxide: what can it compute? Netw. Comput. Neural Syst. 8(1), 1–16 (2009)
Philippides, A., Husbands, P.: Four-dimensional neuronal signaling by nitric oxide: a computational analysis. J. Neurosci. 20(3), 1199–1207 (2000)
Chen, S., Zhou, Z., Hu, D.: Diffusion and growing self-organizing map: a nitric oxide based neural model. In: Yin, F.-L., Wang, J., Guo, C. (eds.) ISNN 2004. LNCS, vol. 3173, pp. 199–204. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-540-28647-9_34
Yin, J., Hu, D., Chen, S.: DSOM: a novel self-organizing model based on NO dynamic diffusing mechanism. Sci. China 48(2), 247–262 (2005)
Sweeney, Y., Clopath, C.: Emergent spatial synaptic structure from diffusive plasticity. Eur. J. Neurosci. 45(8), 1057–1067 (2017)
Palomo, E.J., López-Rubio, E.: The growing hierarchical neural gas self-organizing neural network. IEEE Trans. Neural Netw. learn. Syst. 28(9), 2000–2009 (2017)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Technical report (2009)
He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 630–645. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_38
Acknowledgments
This work was supported by the National Science Foundation of China (61420106001 91420302 and 61773391).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Gao, K., Shen, H., Su, J., Hu, D. (2018). DiffusionNet: Establish Convolutional Networks with Nitric Oxide Diffusion Model. In: Peng, Y., Yu, K., Lu, J., Jiang, X. (eds) Intelligence Science and Big Data Engineering. IScIDE 2018. Lecture Notes in Computer Science(), vol 11266. Springer, Cham. https://doi.org/10.1007/978-3-030-02698-1_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-02698-1_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-02697-4
Online ISBN: 978-3-030-02698-1
eBook Packages: Computer ScienceComputer Science (R0)