Automatic Differentiation for Modern Nonlinear Regression

Abstract

For modern nonlinear regression routines, the efficient computation of first and higher order derivatives is highly important. Automatic differentiation constitutes an opportunity to achieve both higher run-time efficiency and an increased feasibility of higher-order uncertainty analysis of complex models. In this article we present an overview of the derivative requirements of nonlinear regression routines. We further describe our experience in developing a C++ library for model analysis that uses the ADOL-C package for automatic differentiation. We show how the model analysis library, named MAP, has benefited from using automatic differentiation. Also a number of experiments are presented to show how more flexible and efficient execution trace management could further enhance the ease-of-use of ADOL-C.