A Review of Optimization and Machine Learning

Abstract

Before we dive into the models and components of deep learning in depth, it’s important to address the broader field it fits into, which is machine learning. But before that, I want to discuss, if only briefly, optimization. Optimization refers to the selection of a best element from some set of available alternatives. The objective of most machine learning algorithms is to find the optimal solution given a function with some set of inputs. As already mentioned, this often comes within the concept of a supervised learning problem or an unsupervised learning problem, though the procedures are roughly the same.