Skip to main content
SpringerLink
Log in

Proposal of a declarative and parallelizable artificial neural network using the notification-oriented paradigm

  • Review
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Since the 1960s, artificial neural networks (ANNs) have been implemented and applied in various areas of knowledge. Most of these implementations had their development guided by imperative programming (IP), usually resulting in highly coupled programs. Thus, even though intrinsically parallel in theory, ANNs do not easily take an effective distribution on multiple processors when developed under IP. As an alternative, the notification-oriented paradigm (NOP) emerges as a new programming technique. NOP facilitates the development of decoupled and distributed systems, using abstraction of knowledge through logical–causal rules, as well as the generation of an optimized code. Both features are possible by means of a notification-oriented inference process, which avoids structural and temporal redundancies in the logic–causal evaluations. These advantages are relevant to systems that have parts decoupled in order to run in parallel, such as ANN. In this sense, this work presents the development of a multilayer perceptron ANN using backpropagation training algorithm based on the concepts of a NOP implementation. Such implementation allows, transparently from high-level programming, parallel code generation that runs on multicore platforms. Furthermore, the solution based on NOP, when compared against the equivalent on IP, presents a high level of decoupling and explicit use of logic–causal elements, which are, respectively, useful to distribution, understanding and improvement of the application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Adapted from [37]

Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Adapted from [36]

Fig. 14

Adapted from [36]

Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

References

  1. Nilsson NJ (2009) The quest for artificial intelligence: a history of ideas and achievements, 1st edn. Cambridge University Press, New York

    Book  Google Scholar 

  2. Haykin S (2008) Neural networks and learning machines, 2nd edn. Pearson, Porto Alegre

    Google Scholar 

  3. Baptista D, Abreu S, Freitas F et al (2013) A survey of software and hardware use in artificial neural networks. Neural Comput Appl 23:591–599. https://doi.org/10.1007/s00521-013-1406-y

    Article  Google Scholar 

  4. Park DC, El-Sharkawy MA, Marks RJ et al (1991) Electric load forecasting using an artificial neural network. IEEE Trans Power Syst 6:442–449. https://doi.org/10.1109/59.76685

    Article  Google Scholar 

  5. Krose B, Van Der Smagt P (1996) An introduction to neural networks, 8th edn. University of Amsterdan, Amsterdam

    Google Scholar 

  6. Mellit A, Pavan AM (2010) A 24-h forecast of solar irradiance using artificial neural network: application for performance prediction of a grid-connected PV plant at Trieste, Italy. Sol Energy 84:807–821. https://doi.org/10.1016/j.solener.2010.02.006

    Article  Google Scholar 

  7. Omondi AR, Rajapakse JC, Bajger M (2006) FPGA Neurocomputers. In: FPGA implementations of neural networks. Springer, Dordrecht

  8. Lapuschkin S (2016) The LRP toolbox for artificial neural networks. J Mach Learn Res 1:1–5

    MathSciNet  MATH  Google Scholar 

  9. Arvind August D, Pingali K et al (2010) Programming multicores: do applications programmers need to write explicitly parallel programs? IEEE Micro 30:19–33. https://doi.org/10.1109/MM.2010.54

    Article  Google Scholar 

  10. Belmonte DL, Linhares RR, Stadzisz PC, Simão JM (2016) A new method for dynamic balancing of workload and scalability in multicore systems. IEEE Lat Am Trans 14:3335–3344. https://doi.org/10.1109/TLA.2016.7587639

    Article  Google Scholar 

  11. Valerievich BA, Anatolievna PT, Alekseevna BM, et al (2017) Modern approaches to the development parallel programs for modern multicore processors. In: 2017 6th Mediterranean conference on embedded computing (MECO). pp 1–4

  12. Gabbrielli M, Martini S (2010) Programming languages: principles and paradigms, 1st edn. Springer, London

    Book  MATH  Google Scholar 

  13. Seiffert U (2004) Artificial neural networks on massively parallel computer hardware. Neurocomputing 57:135–150. https://doi.org/10.1016/j.neucom.2004.01.011

    Article  Google Scholar 

  14. Gupta G, Sohi GS (2011) Dataflow execution of sequential imperative programs on multicore architectures. In: Proceedings of the 44th annual IEEE/ACM international symposium on Microarchitecture. pp 59–70

  15. Borkar S, Chien AA (2011) The future of microprocessors. Commun ACM 54:67. https://doi.org/10.1145/1941487.1941507

    Article  Google Scholar 

  16. Linhares RR, Simão JM, Stadzisz PC (2015) NOCA: a notification-oriented computer architecture. IEEE Lat Am Trans 13:1593–1604. https://doi.org/10.1109/TLA.2015.7112020

    Article  Google Scholar 

  17. Simao JM, Stadzisz PC (2009) Inference based on notifications: a holonic metamodel applied to control issues. IEEE Trans Syst Man Cybern A Syst Hum 39:238–250. https://doi.org/10.1109/TSMCA.2008.2006371

    Article  Google Scholar 

  18. Simão JM, Banaszewski RF, Tacla CA, Stadzisz PC (2012) Notification oriented paradigm (NOP) and imperative paradigm: a comparative study. J Softw Eng Appl 5:402–416. https://doi.org/10.4236/jsea.2012.56047

    Article  Google Scholar 

  19. Simão JM (2005) A contribution to the development of a HMS simulation tool and proposition of a meta-model for holonic control. Ph.D. Thesis. CPGEI at UTFPR, Curitiba, PR, Brazil and CRAN at UHP, France

  20. Oliveira RN, Roth V, Henzen AF et al (2018) Notification oriented paradigm applied to ambient assisted living tool. IEEE Lat Am Trans 16:7. https://doi.org/10.1109/TLA.2018.8327425

    Google Scholar 

  21. Mendonça ITM, Simão JM, Wiecheteck LVB, Stadzisz PC (2015) Método para Desenvolvimento de Sistemas Orientados a Regras utilizando o Paradigma Orientado a Notificações. In: 12th Congresso Brasileiro de Inteligencia Computacional—CBIC. Curitiba, Paraná, Brazil

  22. Nissen S (2017) Fast artificial neural network library. http://libfann.github.io/fann/docs/files/fann-h.html. Accessed 12 Dec 2017

  23. Schütz F, Fabro JA, Lima CRE, et al (2015) Training of an artificial neural network with backpropagation algorithm using notification oriented paradigm. In: 2015 Latin-America congress on computational intelligence, LA-CCI 2015. IEEE Comput. Soc, Curitiba, Paraná, Brazil, pp 1–6

  24. Heaton J (2015) Artificial Intelligence for Humans, Volume 3: Deep Learning and Neural Networks, 1st edn. Heaton Research Inc, Chesterfield, MO

    Google Scholar 

  25. Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Hum Genet 7:179–188. https://doi.org/10.1111/j.1469-1809.1936.tb02137.x

    Google Scholar 

  26. Vyas S, Upadhyay D (2014) Identification of Iris plant using feed forward neural network on the basis of floral dimensions. IJIRSET 3:18200–18204. https://doi.org/10.15680/IJIRSET.2014.0312062

    Google Scholar 

  27. Simão JM, Stadzisz PC (2008) Notification Oriented Paradigm (NOP)—a notification oriented technique to software composition and execution. Patent Number PI0805518-1, INPI, Brazil

  28. Ronszcka AF, Banaszewski RF, Linhares RR et al (2015) Notification-oriented and rete network inference: a comparative study. In: Proceedings of 2015 IEEE international conference on systems, man, and cybernetics, SMC 2015. IEEE, Hong-Kong, China, pp 807–814

  29. Simão JM, Tacla CA, Stadzisz PC (2009) Holonic control meta-model. IEEE Trans Syst Man Cybern A Syst Hum 2(39):1126–1139. https://doi.org/10.1109/tsmca.2009.2022060

  30. Ronszcka AF, Valença GZ, Linhares RR et al (2017) Notification-oriented paradigm framework 2.0: an implementation based on design patterns. IEEE Lat Am Trans 15:2220–2231

    Article  Google Scholar 

  31. Ronszcka AF, Ferreira CA, Stadzisz PC et al (2017) Notification-oriented programming language and compiler. In: Simpósio Brasileiro de Engeharia de Sistemas Computacionais. Curitiba/PR

  32. Ferreira CA (2015) Linguagem e compilador para o Paradigma Orientado a Notificações (PON): avanços e comparações. M.Sc. Thesis. PPGCA at UTFPR, Curitiba, PR

  33. Peters E, Jasinski RP, Pedroni VA, Simão JM (2012) A new hardware coprocessor for accelerating notification-oriented applications. In: FPT 2012–2012 international conference on field-programmable technology. IEEE Computer Society, Seoul, Korea, pp 257–260

  34. Kerschbaumer R, Linhares RR, Simão JM et al (2017) Notification-oriented paradigm to implement digital hardware. J Circuits Syst Comput. https://doi.org/10.1142/S0218126618501244

    Google Scholar 

  35. Simão JM, Linhares RR, de Witt FA et al (2012) Paradigma Orientado a Notificações em Hardware Digital. 13p. Patent number BR 10 2012 026429 3, INPI, Brazil

  36. Pordeus LF, Kerschbaumer R, Linhares RR et al (2016) Notification oriented paradigm to digital hardware. Sodebras 11:116–122

    Google Scholar 

  37. Ronszcka AF (2012) Contribuição para a concepção de aplicações no paradigma orientado a notificações (PON) sob o viés de padrões. M.Sc. Thesis. CPGEI at UTFPR, Curitiba, PR

  38. Sudha N (2015) Multicore processor: architecture and programming. In: 2015 19th international symposium on VLSI design and test. IEEE, pp 1–2

  39. O’Sullivan B, Stewart D, Goerzen J (2008) Concurrent and multicore programming. In: Real world Haskell. O’Reilly Media, New York

  40. (2000) IEEE Standard for Information Technology—portable operating system interface (POSIX)—part 1: system application program interface (API)-Amendment J: Advanced Real-time Extensions [C Language]. IEEE Std 10031j-2000 0_1-88. https://doi.org/10.1109/ieeestd.2000.91855

  41. Izeboudjen N, Bouridane A, Farah A, Bessalah H (2012) Application of design reuse to artificial neural networks: case study of the back propagation algorithm. Neural Comput Appl 21:1531–1544. https://doi.org/10.1007/s00521-011-0764-6

    Article  Google Scholar 

  42. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536

    Article  MATH  Google Scholar 

  43. Atanassov K, Krawczak M, Sotirov S (2008) Generalized net model for parallel optimization of feed-forward neural network with variable learning rate backpropagation algorithm. In: 2008 4th international IEEE conference intelligent systems, IS 2008, pp 1616–1619

  44. Ludwig Jr O, Montgomery E (2007) Neural networks foundations and applications with C programs, 1st edn. Ed. Ciência Moderna

  45. Raymond ES (2003) The art of unix programming, 1st edn. Addison-Wesley, Boston

    Google Scholar 

  46. Hughes C, Hughes T (2003) Parallel and distributed programming using C++, 1st edn. Addison-Wesley, Boston

    MATH  Google Scholar 

  47. Barreto WRM, Vendrami ACBK, Simao JM (2018) Notification oriented paradigm for distributed systems. In: Anais do Computer on the Beach. Florianópolis/SC

  48. Pordeus LF (2017) Contribuição para o desenvolvimento de uma arquitetura de computação própria ao paradigma orientado a notificações. M.Sc. Thesis. CPGEI at UTFPR, Curitiba, PR

  49. Yun H, Gondi S, Biswas S, Corporation B (2016) BWLOCK: a dynamic memory access control framework for soft real-time applications on multicore platforms. Trans Comput 66:1247–1252. https://doi.org/10.1109/TC.2016.2640961

    Article  MathSciNet  Google Scholar 

  50. Akhter S, Roberts J (2006) Multi-core programming: increasing performance through software multi-threading. Intel press, USA

    Google Scholar 

  51. Pethick M, Liddle M, Werstein P, Huang Z (2003) Parallelization of a backpropagation neural network on a cluster computer. In: 15th IASTED international conference on parallel and distributed computing and systems. IASTED/ACTA Press, Marina Del Rey, California, pp 574–582

  52. Gargouri A, Krid M, Masmoudi DS (2013) Hardware implementation of new bell-shaped pulse mode neural network with on-chip learning and application to image processing. Int J High Perform Syst Archit 4:132–143. https://doi.org/10.1504/IJHPSA.2013.055224

    Article  Google Scholar 

  53. Lakshmi KP, Subadra M (2013) A survey on FPGA based MLP realization for on-chip learning. Int J Sci Eng Res 4:1–9

    Google Scholar 

  54. Rezvani R, Katiraee M, Jamalian AH et al (2012) A new method for hardware design of multi-layer perceptron neural networks with online training. In: Proceedings of 11th conference on cognitive informatics & cognitive computing, pp 527–534. https://doi.org/10.1109/icci-cc.2012.6311205

  55. Abrol S, Mahajan R (2015) Artificial neural network implementation on FPGA chip. Int J Comput Sci Inf Technol Res 3:11–18

    Google Scholar 

  56. Nedjah N, da Silva FP, de Sá AO et al (2016) A massively parallel pipelined reconfigurable design for M-PLN based neural networks for efficient image classification. Neurocomputing 183:39–55. https://doi.org/10.1016/j.neucom.2015.05.138

    Article  Google Scholar 

  57. Baptista FD, Morgado-Dias F (2017) Automatic general-purpose neural hardware generator. Neural Comput Appl 28:25–36. https://doi.org/10.1007/s00521-015-2034-5

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the original creators of NOP (Prof. J. M. Simão and Prof. P. C. Stadzisz) as well as the NOP research group particularly the developers of NOP Framework 2.0 (A. F. Ronszcka and G. Z. Valença). In addition, by funding researchers, the authors would like to acknowledge the Araucaria Foundation, Capes and UTFPR.

Author information

Authors and Affiliations

  1. CPGEI (Graduate Program on Electrical Engineering and Industrial Informatics), UTFPR (Federal University of Technology), Curitiba, PR, 80230-901, Brazil

    Fernando Schütz, Adriano F. Ronszcka, Paulo C. Stadzisz & Jean M. Simão

  2. PPGCA (Graduate Program on Applied Computing), UTFPR (Federal University of Technology), Curitiba, PR, 80230-901, Brazil

    João A. Fabro

Authors
  1. Fernando Schütz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João A. Fabro
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adriano F. Ronszcka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paulo C. Stadzisz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jean M. Simão
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Schütz.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schütz, F., Fabro, J.A., Ronszcka, A.F. et al. Proposal of a declarative and parallelizable artificial neural network using the notification-oriented paradigm. Neural Comput & Applic 30, 1715–1731 (2018). https://doi.org/10.1007/s00521-018-3517-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3517-y

Keywords

Search

Navigation