Abstract
The solution of intractable problems implies the use of heuristics. Quantum computers may find use for optimization problems, but have yet to solve any NP-hard problems. This paper demonstrates results in game theory for domain transference and the reuse of problem-solving knowledge through the application of learned heuristics. It goes on to explore the possibilities for the acquisition of heuristics for the solution of the NP-hard TSP problem. Here, it is found that simple heuristics (e.g., pairwise exchange) often work best in the context of more or less sophisticated experimental designs. Often, these problems are not amenable to exclusive logic solutions; but rather, require the application of hybrid approaches predicated on search. In general, such approaches are based on randomization and supported by parallel processing. This means that heuristic solutions emerge from attempts to randomize the search space. The paper goes on to present a constructive proof of the unbounded density of knowledge in support of the Semantic Randomization Theorem (SRT). It highlights this result and its potential impact upon the community of machine learning researchers.
Similar content being viewed by others
References
Behbood, V., Lu, J., & Zhang, G. (2011). Long term bank failure prediction using fuzzy refinement-based transductive transfer learning. In IEEE International Conference on Fuzzy Systems (pp. 2676–2683). Taiwan: IEEE.
Behbood, V., Lu, J., & Zhang, G. (2013a). Fuzzy bridged refinement domain adaptation: Long-term bank failure prediction. International Journal of Computational Intelligence and Applications, 12(01).
Behbood, V., Lu, J., & Zhang, G. (2013b). Text categorization by fuzzy domain adaptation. In IEEE International Conference on Fuzzy Systems. Hyderabad: IEEE.
Behbood, V., Lu, J., & Zhang, G. (2014). Fuzzy refinement domain adaptation for long term prediction in banking ecosystem. IEEE Transactions on Industrial Informatics, 10(2), 1637–1646.
Caruana, R. (1993). Multitask learning: a knowledge-based source of inductive bias. In Proceedings of the tenth international conference on machine learning (pp. 41–48). MA, USA.
Caruana, R. (1997). Multitask learning. Machine learning, 28(1), 41–75.
Celiberto, L.A., Matsuura, J.P., Lopez de Mantaras, R., & Bianchi, R.A.C. (2011). Using cases as heuristics in reinforcement learning: a transfer learning application. In International Joint Conference on Artificial Intelligence. Barcelona.
Chaitin, G.J. (1975). Randomness and mathematical proof. Scientific American, 232(5), 47–52.
Chopra, S., Balakrishnan, S., & Gopalan, R. (2013). DLID: Deep Learning for domain adaptation by interpolating between domains. In ICML Workshop on challenges in representation learning, Vol. 2. Atlanta.
Cireşan, D.C., Meier, U., & Schmidhuber, J. (2012). Transfer learning for latin and chinese characters with deep neural networks. In International Joint Conference on Neural Networks (IJCNN). Australia: IEEE.
Cousins, N., & Eccles, J.C. (1985). Nobel Prize Conversations with Sir John Eccles, Roger Sperry, Ilya Prigogine, Brian Josephson. CA: Saybrook Publishers.
Eccles, J.C. (1976). The understanding of the brain. New York: McGraw-Hill Co. 2d ed.
Feigenbaum, E.A., & McCorduck, P. (1983). The fifth generation: Artificial intelligence and Japan’s computer challenge to the world reading. MA: Addison-Wesley Pub. Co.
Fogel, D.B. (2001). Blondie24: Playing at the edge of AI. Mountain View: Morgan Kaufmann Publishers, Inc.
Huang, J.-T., Li, J., Yu, D., Deng, L., & Gong, Y. (2013). Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers. Vancouver: IEEE.
Kandaswamy, C., Silva, L.M., Alexandre, L.A., Santos, J.M., & de Sá, J.M. (2014). Improving deep neural network performance by reusing features trained with transductive transference. In 24Th international conference on artificial neural networks and machine learning–ICANN (pp. 265–272). Hampurg: Springer.
Kfoury, A.J., Moll, R.N., & Arbib, M.A. (1982). A programming approach to computability. New York: Springer Verlag Inc.
Koçer, B., & Arslan, A. (2010). Genetic transfer learning. Expert Systems with Applications, 37(10), 6997–7002.
Lin, J.-H., & Vitter, J.S. (1991). Complexity results on learning by neural nets. Machine Learning, 6(3), 211–230.
Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., & Zhang, G. (2015). Transfer learning using computational intelligence: a survey. Knowledge-Based Systems, 80, 14–23.
Luis, R., Sucar, L.E., & Morales, E.F. (2010). Inductive transfer for learning bayesian networks. Machine learning, 79(1-2), 227– 255.
Ma, Y., Luo, G., Zeng, X., & Chen, A. (2012). Transfer learning for cross-company software defect prediction. Information and Software Technology, 54(3), 248–256.
Michalski, R.S., Carbonell, J.G., & Mitchell, T.M. (1983). Machine learning: an artificial intelligence approach (volume 1). Palo Alto: Tioga Publishing Co.
Mitchell, T.M., Carbonell, J.G., & Michalski, R.S. (Eds.) (1986). Machine Learning: A guide to current research. New York: Springer-Verlag Inc.
Niculescu-Mizil, A., & Caruana, R. (2007). Inductive Transfer for bayesian network structure learning. In 11Th international Conference on Artificial Intelligence and Statistics, Puerto Rico (pp. 339–346).
Nilsson, N.J. (1980). Principles of artificial intelligence. Mountain View: Morgan Kaufmann Publishers Inc.
Oyen, D., & Lane, T. (2013). Bayesian discovery of multiple bayesian networks via transfer learning. In 13Th international conference on data mining (ICDM) (pp. 577–586). Dallas: IEEE.
Pan, S.J., & Yang, Q. (2010). A survey on transfer learning. IEEE Transactions Knowledge and Data Engineering, 22(10), 1345–1359.
Rubin, S.H. (2004). On the auto-randomization of knowledge. In: Information Reuse and Integration. In Proceedings of the 2004 IEEE International Conference On Information Reuse and Integration (pp. 308–313): IEEE.
Rubin, S.H. (2007). On randomization and discovery. Information Sciences, 177(1), 170–191.
Rubin, S.H. (2012). On creativity and intelligence in computational systems. In Advances in Reasoning-Based Image Processing Intelligent Systems (pp. 383–421). Berlin: Springer.
Rubin, S.H., & Bouabana-Tebibel, T. (2015a). Naval intelligent authentication and support through randomization and transformative search. In To appear New approaches in intelligent control and image analysis - techniques, methodologies and applications. Intelligent Systems Reference Library: Springer.
Rubin, S.H., & Bouabana-Tebibel, T. (2015b). NNCS: Randomization and informed search for novel naval cyber strategies. In Recent Advances in Computational Intelligence in Defense and Security. Studies in Computational Intelligence, (Vol. 621 pp. 193–223): Springer.
Rubin, S.H., Bouabana-Tebibel, T., Hoadjli, Y., Habib, K., & Belamiri, B.Y. (2015). On heuristic randomization and reuse as an enabler of domain transference. In The 16th IEEE international conference on information reuse and integration IEEE IRI 2015. San Francisco, USA, August 13–15 (pp. 411–418).
Rubin, S.H., Chen, S.-C., Law, J.B., & Lee, G.K. (2005). On the inherent necessity of heuristic proofs. In 2005 IEEE International Conference on Systems, Man and Cybernetics, (Vol. 4 pp. 3890–3896): IEEE.
Rubin, S.H., Murthy, S.N.J., Smith, M.H., & Trajković, L. (2004). Kaser: knowledge amplification by structured expert randomization. IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 34(6), 2317–2329.
Shell, J. (2013). Fuzzy transfer learning. PhD thesis: De Montfort University.
Shell, J., & Coupland, S. (2012). Towards fuzzy transfer learning for intelligent environments, (Vol. 7683 pp. 145–160): Springer.
Shell, J., & Coupland, S. (2015). Fuzzy transfer learning: methodology and application. Information Sciences, 293, 59–79.
Silver, D.L., & Poirier, R. (2007). Context-sensitive MTL networks for machine lifelong learning. In 20th florida artificial intelligence research society conference (pp. 628–633). Key West.
Swietojanski, P., Ghoshal, A., & Renals, S. (2012). Unsupervised cross-lingual knowledge transfer in DNN-based LVCSR. In IEEE Workshop on spoken language technology (SLT) (pp. 246–251). Miami: IEEE.
Tesar, B. (2000). Learnability in optimality theory: Mit Press.
Uspenskii, V.A. (1987). Gödel’s incompleteness theorem. Translated from Russian. Ves Mir Publishers: Moscow.
Acknowledgments
Stuart would like to extend special thanks to Michael Leyton, Rutgers Department of Mathematics, for reviewing his tremendously exciting Theory of Randomization (Rubin 2007) and to the kind genius, Sukarno Mertoguno for applicative discussions on cybersecurity. Stuart would also like to extend his thanks to the Office of Naval Research (ONR) for providing financial backing for this research with the support of his Branch Head, Ken Simonsen. This work was produced by a U.S. government employee as part of his official duties and is not subject to copyright. It is approved for public release with an unlimited distribution.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rubin, S.H., Bouabana-Tebibel, T. & Hoadjli, Y. On the empirical justification of theoretical heuristic transference and learning. Inf Syst Front 18, 981–994 (2016). https://doi.org/10.1007/s10796-016-9661-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10796-016-9661-y