Skip to main content
SpringerLink
Log in

Estimating Dynamic Graphical Models from Multivariate Time-Series Data: Recent Methods and Results

  • Conference paper
  • First Online:
Advanced Analysis and Learning on Temporal Data (AALTD 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9785))

Abstract

Dynamic graphical models aim to describe the time-varying dependency structure of multiple time-series. In this article we review research focusing on the formulation and estimation of such models. The bulk of work in graphical structurelearning problems has focused in the stationary i.i.d setting, we present a brief overview of this work before introducing some dynamic extensions. In particular we focuson two classes of dynamic graphical model; continuous (smooth) models which are estimated via localised kernels, and piecewise models utilising regularisation based estimation. We give an overview of theoretical and empirical results regarding these models, before demonstrating their qualitative difference in the context of a real-world financial time-series dataset. We conclude with a discussion of the state of the field and future research directions.

D.B. Nelson—This work is funded by the Defence Science Technology Laboratory (Dstl) National PhD Scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For simplicity, in this paper we assume the mean parameter is zero \(\varvec{\mu }=\varvec{0}\).

  2. 2.

    Note: the model here refers to the sparsity pattern, rather than the fact that the distribution is Gaussian.

  3. 3.

    Note: we set the first changepoint (denoted by \(k=0\)) at \(\tau _{0}=1\) and the last \(K+1\)st changepoint is located at \(\tau _{K+1}=T\).

  4. 4.

    The matrix \(\varvec{\varSigma }^{o}\) is known as an oracle estimator, as it has access to the ground truth \(\varvec{\varSigma }_{*}\) through the risk function \(\mathcal {R}(\cdot )\).

  5. 5.

    We note that Harcharoui et al. perform their changepoint analysis in a reformulated lasso problem rather than considering directly jumps in \(\hat{\varvec{u}}\) as presented here.

  6. 6.

    In the univariate setting for functional approximation this is analogous to the fused lasso of Tibshirani et al. [40].

  7. 7.

    If we choose \(\hat{\varvec{S}}^{t}\) to be estimated through a localised kernel as in (14) we obtain the SINGLE estimator of Monti et al. [32], if we use a Dirac delta kernel i.e. \(\hat{\varvec{S}}^{t}=\varvec{y}^{\top }\varvec{y}/2\) then we recover the IFGL estimator of our previous work [18]. Note: the TESLA approach of Ahmed et al. [1] uses a regulariser term like \(R_{\mathrm {IFGL}}\) but with a logistic loss function for binary variables.

  8. 8.

    The data can be obtained from Ken French’s website http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html.

References

  1. Ahmed, A., Xing, E.P.: Recovering time-varying networks of dependencies in social and biological studies. Proc. Natl. Acad. Sci. USA 106, 11878–11883 (2009)

    Article  Google Scholar 

  2. Angelosante, D., Giannakis, G.B.: Sparse graphical modeling of piecewise-stationary time series. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2011)

    Google Scholar 

  3. Banerjee, O., Ghaoui, L.E., D’Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. J. Mach. Learn. 9, 485–516 (2008)

    MathSciNet  MATH  Google Scholar 

  4. Bleakley, K., Vert, J.P.: The group fused lasso for multiple change-point detection. Technical report HAL-00602121 (2011)

    Google Scholar 

  5. Boyd, S., Parikh, N., Chu, E.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3, 1–122 (2011)

    Article  MATH  Google Scholar 

  6. Boyd, S., Vandenberghe, L.: Convex Optimization. Cambridge University Press (2004)

    Google Scholar 

  7. Cai, T., Liu, W., Luo X.: A constrained L1 minimization approach to sparse precision matrix estimation. J. Am. Stat. Assoc. (2011)

    Google Scholar 

  8. Carvalho, C.M., West, M.: Dynamic matrix-variate graphical models matrix-variate dynamic linear models. Bayesian Anal. 2, 69–97 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  9. Cowell, R.G., Verrall, R.J., Yoon, Y.K.: Modelling operational risk with Bayesian networks. J. Risk Insur. 74, 795–827 (2007)

    Article  Google Scholar 

  10. Danaher, P., Wang, P., Witten, D.M.: The joint graphical lasso for inverse covariance estimation across multiple classes. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 76, 373–397 (2013)

    Article  MathSciNet  Google Scholar 

  11. Drton, M., Perlman, M.D.: Model selection for Gaussian concentration graphs. Biometrika 51, 591–602 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  12. Enikeeva, F., Harchaoui, Z.: High-dimensional change-point detection with sparse alternatives (2013). arXiv:1312.1900

  13. Foygel, R., Drton, M.: Extended Bayesian information criteria for Gaussian graphical models. In: Advances in Neural Information Processing Systems, vol. 23 (2010)

    Google Scholar 

  14. Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9, 432–441 (2008)

    Article  MATH  Google Scholar 

  15. Friedman, J., Hastie, T., Tibshirani, R.: Applications of the lasso and grouped lasso to the estimation of sparse graphical models (2010)

    Google Scholar 

  16. Gibberd, A.J., Nelson, J.D.B.: High dimensional changepoint detection with a dynamic graphical lasso. In: International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2014)

    Google Scholar 

  17. Gibberd, A.J., Nelson, J.D.B.: Estimating multiresolution dependency graphs within the stationary wavelet framework. In: IEEE Global Conference on Signal and Information Processing (GlobalSIP) (2015)

    Google Scholar 

  18. Gibberd, A.J., Nelson, J.D.B.: Regularized estimation of piecewise constant Gaussian graphical models: the group-fused graphical lasso (2015). arXiv:1512.06171

  19. Harchaoui, Z., Lévy-Leduc, C.: Multiple change-point estimation with a total variation penalty. J. Am. Stat. Assoc. 105, 1480–1493 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  20. Hastie, T., Tibshirani, R.: Varying-coefficient models. J. R. Stat. Soc. B 55, 757–796 (1993)

    MathSciNet  MATH  Google Scholar 

  21. Jordan, M.I.: Graphical models. Stat. Sci. 19, 140–155 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kolar, M., Xing, E.P: On time varying undirected graphs. In: Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS) (2011)

    Google Scholar 

  23. Kolar, M., Xing, E.P.: Estimating networks with jumps. Electron. J. Stat. 6, 2069–2106 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  24. Lafferty, J., Liu, H., Wasserman, L.: Sparse nonparametric graphical models. Stat. Sci. 27, 519–537 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  25. Lam, C., Fan, J.: Sparsistency and rates of convergence in large covariance matrix estimation. Ann. Stat. 37, 4254–4278 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  26. Lauritzen, S.L.: Graphical Models. Oxford University Press, Oxford (1996)

    MATH  Google Scholar 

  27. Lèbre, S., Becq, J., Devaux, F., Stumpf, M.P.H., Lelandais, G.: Statistical inference of the time-varying structure of gene-regulation networks. BMC Syst. Biol. 4, 130 (2010)

    Article  Google Scholar 

  28. Lee, J.D., Hastie, T.J.: Learning the structure of mixed graphical models. J. Comput. Graph. Stat. 24, 230–253 (2015)

    Article  MathSciNet  Google Scholar 

  29. Little, M.A., Jones, N.S.: Generalized methods and solvers for noise removal from piecewise constant signals. I. background theory. Proc. Math. Phys. Eng. Sci./R. Soc. 467, 3088–3114 (2011)

    Google Scholar 

  30. Loh, P., Wainwright, M.J.: Structure estimation for discrete graphical models: generalized covariance matrices and their inverses. In: Neural Information Processing Systems (NIPS) (2012)

    Google Scholar 

  31. Meinshausen, N., Bühlmann, P.: High-dimensional graphs and variable selection with the lasso. Ann. Stat. 34, 1436–1462 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  32. Monti, R.P., Hellyer, P., Sharp, D., Leech, R., Anagnostopoulos, C., Montana, G.: Estimating time-varying brain connectivity networks from functional MRI time series. NeuroImage 103, 427–443 (2014)

    Article  Google Scholar 

  33. Negahban, S.N., Ravikumar, P., Wainwright, M.J., Yu, B.: A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Stat. Sci. 27, 538–557 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  34. Ravikumar, P., Wainwright, M.J., Lafferty, J.D.: High-dimensional Ising model selection using \({l_1}\)-regularized logistic regression. Ann. Stat. 38, 1287–1319 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  35. Ravikumar, P., Wainwright, M.J., Raskutti, G., Yu, B.: High-dimensional covariance estimation by minimizing \({l_1}\)-penalized log-determinant divergence. Electron. J. Stat. 5, 935–980 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  36. Rothman, A.J., Bickel, P.J., Levina, E., Zhu, J.: Sparse permutation invariant covariance estimation. Electron. J. Stat. 2, 494–515 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  37. Roy, S., Atchad, Y., Michailidis, G.: Change-point estimation in high-dimensional Markov random field models (2015). arXiv:1405.6176v2

  38. Talih, M., Hengarter, N.: Structural learning with time-varying components: tracking the cross-section of financial time series. J. R. Stat. Soc. B 67, 321–341 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  39. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 73, 273–282 (1996)

    Article  MathSciNet  MATH  Google Scholar 

  40. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J., Knight, K.: Sparsity, smoothness via the fused lasso. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 67, 91–108 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  41. Wang, H.: Bayesian graphical lasso models and efficient posterior computation. Bayesian Anal. 7, 867–886 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  42. Xuan, X., Murphy, K.: Modeling changing dependency structure in multivariate time series. In: International Conference on Machine Learning (2007)

    Google Scholar 

  43. Yang, E., Ravikumar, P.K., Allen, G.I., Liu, Z.: On Poisson graphical models. In: Advances in Neural Information Processing Systems (NIPS) (2013)

    Google Scholar 

  44. Yang, S., Pan, Z., Shen, X., Wonka, P., Ye, J.: Fused multiple graphical lasso (2012)

    Google Scholar 

  45. Yi-Ching, Y., Au, S.T.: Least-squares estimation of a step function. Indian J. Stat. Ser. A 51, 370–381 (1989)

    MathSciNet  MATH  Google Scholar 

  46. Yuan, X.: Alternating direction method for covariance selection models. J. Sci. Comput. 51, 261–273 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang, B., Geng, J., Lai, L.: Multiple change-points estimation in linear regression models via sparse group lasso. IEEE Trans. Signal Process. 63, 2209–2224 (2014)

    Article  MathSciNet  Google Scholar 

  48. Zhou, S., Lafferty, J., Wasserman, L.: Time varying undirected graphs. Mach. Learn. 80, 295–319 (2010)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistical Science, University College London, Gower Street, London, WC1E 6BT, UK

    Alex J. Gibberd & James D. B. Nelson

  2. Department of Security and Crime Science, University College London, London, UK

    Alex J. Gibberd

Authors
  1. Alex J. Gibberd
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James D. B. Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alex J. Gibberd .

Editor information

Editors and Affiliations

  1. Lab. d'Informatique de Grenoble, Université Grenoble, Grenoble, France

    Ahlame Douzal-Chouakria

  2. Universidade da Coruna, Coruna, Spain

    José A. Vilar

  3. IRISA, Université de Bretagne-Sud, Vannes, France

    Pierre-François Marteau

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Gibberd, A.J., Nelson, J.D.B. (2016). Estimating Dynamic Graphical Models from Multivariate Time-Series Data: Recent Methods and Results. In: Douzal-Chouakria, A., Vilar, J., Marteau, PF. (eds) Advanced Analysis and Learning on Temporal Data. AALTD 2015. Lecture Notes in Computer Science(), vol 9785. Springer, Cham. https://doi.org/10.1007/978-3-319-44412-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-44412-3_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-44411-6

  • Online ISBN: 978-3-319-44412-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation