Abstract
Learning algorithms implemented for neural networks have generally being conceived for networks implemented in software. However algorithms which have been developed for software implementations typically require far greater accuracy for efficiently training the networks than can be easily implemented in hardware neural networks. Although some learning algorithms designed for software implementation can be successfully implemented in hardware it has become apparent that in hardware these algorithms are generally ill suited, failing to converge well (or at all). Particle Swarm Optimisation (PSO) is known to have a number of features that make it well suited to the training of neural hardware. In this paper the suitability of PSO to train limited precision neural hardware is investigated. Results show that the performance achieved with this algorithm does not degrade until the accuracy of the networks is reduced to a very small number of bits.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R. Battiti and G. Tecchiolli, „Training Neural Nets with the Reactive Tabu Search“, IEEE Transactions on Neural Networks, 6(5): 1185–1200, 1995.
K. Bennett and J. Blue, „A Support Vector Machine Approach to Decision Trees“, R.P.I Math Report No. 97-100, Rensselaer Polytechnic Institute, Troy, NY, 1997
C. Blake, E. Keogh, and C. Merz, „UCI Repository of Machine Learning Databases“, [http://www.ics.uci.edu/~mlearn/MLRepository.html], Irvine, CA, University of California, Department of Information and Computer Science, 1998
M. Clerc, „The swarm and the queen: towards a deterministic and adaptive particle swarm optimization“. Proc. 1999 Congress on Evolutionary Computation, pp 1951–1957, Washington, DC, USA, 1999.
T. Duong, „Learning in Neural Networks: VLSI Implementation Strategies“, Fuzzy Logic and Neural Network Handbook, 1995.
R. Eberhart and J. Kennedy, „A New Optimizer Using Particles Swarm Theory“, Proc. Sixth International Symposium on Micro Machine and Human Science Nagoya, Japan, pp. 39–43, 1995.
R. Eberhart and Y. Shi. „Comparing inertia weights and constriction factors in particle swarm optimisation“. Proceedings of the 2000 congress on evolutionary computation, pp. 84–88, 2000.
M. Glesner and W. Pöchmüller, „An Overview of Neural Networks in VLSI“, Chapman & Hall, London, 1994.
J. Kennedy, „The Particle Swarm: Social Adaptation of Knowledge“, Proc. IEEE International Conference on Evolutionary Computation Indianapolis, Indiana, USA, pp. 303–308, 1997.
J. Kennedy and R. Eberhart with Y. Shi. „Swarm Intelligence“, San Francisco: Morgan Kaufmann/ Academic Press, 2001.
R. King, C. Feng, and A. Sutherland, „STATLOG: comparison of classification algorithms on large real-world problems. Applied Artificial Intelligence, 9(3):289–334, 1995.
L. Prechelt, „Proben1-A Set of Neural Network Benchmarking Problems and Benchmarking Rules“. Technical Report Nr. 21/ 94. Faculty of Computer Science, University of Karlsruhe, Germany, 1994.
N. Shang and L. Breiman, „Distribution Based Trees Are More Accurate International Conference on Neural Information Processing“. ICONIP’96, p. 133, 1996.
Y. Shang, „Global Search Methods for Solving Nonlinear Optimization Problems„, Ph.D. Thesis, Dept. of Computer Science, Univ. of Illinois, Urbana, Illinois, August 1997.
Y. Shi and R.C. Eberhart, „A Modified Particle Swarm Optimizer“, IEEE International Conference on Evolutionary Computation, Anchorage, Alaska, USA, 1998.
J. Smith, J. Everhart, W. Dickson, W. Knowlerand and R. Johannes, „Using the ADAP learning algorithm to forecast the onset of diabetes mellitus“,. In Proceedings of the Symposium on Computer Applications and Medical Care, pp. 261–265, 1988.
W. Wolberg and O. Mangasarian, „Multisurface method of pattern separation for medical diagnosis applied to breast cytology“, Proceedings of the National Academy of Sciences, U.S.A., vol. 87, pp 9193–9196, 1990.
Y. Xie and M. Jabri. Training algorithms for limited precision feedforward neural nets. SEDAL technical report 1991-8-3, Department of Electrical Engineering, University of Sydney, NSW 2006, Australia, 1991.
Y. Xie and M. Jabri, „Analysis of effects of quantisation in multilayer neural networks using statistical models“, IEEE Trans. Neural Networks, 3(2):334–338, 1992.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Braendler, D., Hendtlass, T. (2002). The Suitability of Particle Swarm Optimisation for Training Neural Hardware. In: Hendtlass, T., Ali, M. (eds) Developments in Applied Artificial Intelligence. IEA/AIE 2002. Lecture Notes in Computer Science(), vol 2358. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-48035-8_19
Download citation
DOI: https://doi.org/10.1007/3-540-48035-8_19
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43781-9
Online ISBN: 978-3-540-48035-8
eBook Packages: Springer Book Archive