Abstract
In general, the solution to a regression problem is the minimizer of a given loss criterion and depends on the specified loss function. The nonparametric isotonic regression problem is special, in that optimal solutions can be found by solely specifying a functional. These solutions will then be minimizers under all loss functions simultaneously as long as the loss functions have the requested functional as the Bayes act. For the functional, the only requirement is that it can be defined via an identification function, with examples including the expectation, quantile, and expectile functionals. Generalizing classical results, we characterize the optimal solutions to the isotonic regression problem for identifiable functionals by rigorously treating these functionals as set-valued. The results hold in the case of totally or partially ordered explanatory variables. For total orders, we show that any solution resulting from the pool-adjacent-violators algorithm is optimal.
Similar content being viewed by others
References
Ayer, M., Brunk, H. D., Ewing, G. M., Reid, W. T., Silverman, E. (1955). An empirical distribution function for sampling with incomplete information. Annals of Mathematical Statistics, 26, 641–647.
Barlow, R. E., Bartholomew, D. J., Bremner, J. M., Brunk, H. D. (1972). Statistical inference under order restrictions. London: Wiley.
Bartholomew, D. J. (1959a). A test of homogeneity for ordered alternatives. Biometrika, 46, 36–48.
Bartholomew, D. J. (1959b). A test of homogeneity for ordered alternatives. II. Biometrika, 46, 328–335.
Bellec, P. C. (2018). Sharp oracle inequalities for least squares estimators in shape restricted regression. The Annals of Statistics, 46, 745–780.
Brümmer, N., Du Preez, J. (2013). The PAV algorithm optimizes binary proper scoring rules. arXiv:1304.2331.
Brunk, H. D. (1955). Maximum likelihood estimates of monotone parameters. Annals of Mathematical Statistics, 26, 607–616.
Dawid, A. P. (2016). Contribution to the discussion of Of quantiles and expectiles: Consistent scoring functions, Choquet representations and forecast rankings by Ehm, W., Gneiting, T., Jordan, A. and Krüger, F. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 78, 505–562.
Ehm, W., Gneiting, T., Jordan, A., Krüger, F. (2016). Of quantiles and expectiles: Consistent scoring functions, Choquet representations and forecast rankings. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 78, 505–562.
Gneiting, T. (2011). Making and evaluating point forecasts. Journal of the American Statistical Association, 106, 746–762.
Groeneboom, P., Jongbloed, G. (2014). Nonparametric estimation under shape constraints. New York: Cambridge University Press.
Guntuboyina, A., Sen, B. (2018). Nonparametric shape-restricted regression. Statistical Science, 33, 568–594.
Gurney, A. J. T., Griffin, T. G. (2011). Pathfinding through congruences. Relational and Algebraic Methods in Computer Science (Vol. 6663, pp. 180–195). Heidelberg: Springer.
Han, Q., Wang, T., Chatterjee, S., Samworth, R. J. (2019). Isotonic regression in general dimensions. The Annals of Statistics, 47, 2440–2471.
Huber, P. J. (1964). Robust estimation of a location parameter. Annals of Mathematical Statistics, 35, 73–101.
Kyng, R., Rao, A., Sachdeva, S. (2015). Fast, provable algorithms for isotonic regression in all \({L}_{p}\)-norms. Advances in Neural Information Processing Systems 28 (pp. 2719–2727). Red Hook: Curran Associates Inc.
Luss, R., Rosset, S. (2014). Generalized isotonic regression. Journal of Computational and Graphical Statistics, 23, 192–210.
Luss, R., Rosset, S. (2017). Bounded isotonic regression. Electronic Journal of Statistics, 11, 4488–4514.
Miles, R. E. (1959). The complete amalgamation into blocks, by weighted means, of a finite set of real numbers. Biometrika, 46, 317–327.
Mösching, A., Dümbgen, L. (2020). Monotone least squares and isotonic quantiles. Electronic Journal of Statistics, 14, 24–49.
Newey, W. K., Powell, J. L. (1987). Asymmetric least squares estimation and testing. Econometrica, 55, 819–847.
Pardalos, P. M., Xue, G. (1999). Algorithms for a class of isotonic regression problems. Algorithmica, 23, 211–222.
Patton, A. J. (2011). Volatility forecast comparison using imperfect volatility proxies. Journal of Econometrics, 160, 246–256.
Patton, A. J. (2020). Comparing possibly misspecified forecasts. Journal of Business & Economic Statistics, 38, 796–809.
Polonik, W. (1998). The silhouette, concentration functions and ML-density estimation under order restrictions. The Annals of Statistics, 26, 1857–1877.
Robertson, T., Wright, F. T. (1973). Multiple isotonic median regression. The Annals of Statistics, 1, 422–432.
Robertson, T., Wright, F. T. (1980). Algorithms in order restricted statistical inference and the Cauchy mean value property. The Annals of Statistics, 8, 645–651.
Savage, L. J. (1971). Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66, 783–801.
Stout, Q. F. (2015). Isotonic regression for multiple independent variables. Algorithmica, 71, 450–470.
Sysoev, O., Burdakov, O., Grimvall, A. (2011). A segmentation-based algorithm for large-scale partially ordered monotonic regression. Computational Statistics & Data Analysis, 55, 2463–2476.
Thompson, W. A., Jr. (1962). The problem of negative estimates of variance components. Annals of Mathematical Statistics, 33, 273–289.
van Eeden, C. (1958). Testing and estimating ordered parameters of probability distributions. Amsterdam: Mathematical Centre.
Ziegel, J. F. (2016). Contribution to the discussion of Of quantiles and expectiles: Consistent scoring functions, Choquet representations and forecast rankings by Ehm, W., Gneiting, T., Jordan, A. and Krüger, F. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 78, 505–562.
Acknowledgements
We would like to thank two reviewers, Tilmann Gneiting, Alexandre Mösching and Lutz Dümbgen for inspiring discussions and valuable comments. Alexander I. Jordan acknowledges the support of the Klaus Tschira Foundation. Anja Mühlemann and Johanna F. Ziegel gratefully acknowledge financial support from the Swiss National Science Foundation.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Jordan, A.I., Mühlemann, A. & Ziegel, J.F. Characterizing the optimal solutions to the isotonic regression problem for identifiable functionals. Ann Inst Stat Math 74, 489–514 (2022). https://doi.org/10.1007/s10463-021-00808-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10463-021-00808-0