Quantitative methods to evaluate and compare Soil Organic Matter (SOM) Models

Summary

Evaluation and comparison of the performance of soil organic matter models is often based upon visual/graphical comparison of the simulated values produced by the model with actual values from field experiments. Such methods provide an immediate qualitative description of the differences, highlighting trends, different types of errors and distribution patterns of simulated and measured values. However, model evaluations or comparisons should ideally incorporate both a qualitative visual/graphical assessment and a quantitative statistical appraisal. Statistical methods have been selected that are suitable for quantitative evaluation and comparison of soil organic matter models. The methods included each provide information on some distinct aspect of the accuracy of the simulation.

Quantifying association and coincidence provides two different measures of the overall similarity between the simulated and measured values. A high coincidence indicates that the simulated values closely correspond to the measured values, whereas a high association indicates that the shape of the simulated curve is similar to that measured. The total difference between the simulated values and measurements may be expressed in terms of consistent and inconsistent errors. Consistent errors quantify the extent of model bias towards either over- or under-prediction of observations. Inconsistent errors correspond to errors which cancel out because the model shows no inclination towards either over- or under-prediction. If the experiments have been replicated, the total difference between simulated values and measurements can be more usefully expressed in terms of systematic and random errors. Systematic errors represent the failure of the model to simulate differences between the experiments. Random errors represent experimental error. Finally, the maximum difference between any pair of simulated and measured values can be used to indicate where a small number of very large errors are contributing to a large proportion of the overall error calculated, so that other statistical values may become unreliable. If used in conjunction with visual/graphical methods, the statistical techniques described in this paper provide a rigorous method for the quantitatively evaluating the performance of soil organic matter models, or indeed other models.