Abstract
The filtering method developed by Kim et al. (SIAM Rev 51:339–360, 2009), \(\ell _{1}\) trend filtering, is attractive because it enables us to estimate a continuous piecewise linear trend. This paper introduces a new filtering method closely related to \(\ell _{1}\) trend filtering in order to contribute to the accumulation of knowledge on \(\ell _{1}\) trend filtering. We show that the piecewise linearity, which is the key feature of \(\ell _{1}\) trend filtering, is derived from the new filtering. For this reason, we refer to the filtering as ‘pure’ \(\ell _{1}\) trend filtering. We also demonstrate some other miscellaneous results concerning the new filtering.
Similar content being viewed by others
Notes
Similarly to (6), we may consider the following filtering:
$$\begin{aligned} \widehat{{{\varvec{z}}}}_{\text {HP}}={{\varvec{D}}}'({{\varvec{D}}}{{\varvec{D}}}')^{-1}\widehat{{{\varvec{\zeta }}}}, \end{aligned}$$where
$$\begin{aligned} \widehat{{{\varvec{\zeta }}}}=\mathop {{\text {arg min}}}\limits _{{{\varvec{\zeta }}}\in \mathbb {R}^{(T-2)}}\,\left( \Vert {{\varvec{y}}}-{{\varvec{D}}}'({{\varvec{D}}}{{\varvec{D}}}')^{-1}{{\varvec{\zeta }}}\Vert _{2}^{2}+\phi \Vert {{\varvec{\zeta }}}\Vert _{2}^{2}\right) . \end{aligned}$$As shown in Yamada (2015), \(\widehat{{{\varvec{z}}}}_{\text {HP}}\), which is referred to as pure HP trend, satisfies \(\widehat{{{\varvec{x}}}}_{\text {HP}}=\widehat{{{\varvec{\tau }}}}+\widehat{{{\varvec{z}}}}_{\text {HP}}\). See also Yamada (2018).
The author is indebted to Kazuhiko Hayakawa for deriving this formula. In addition, Dow (2003) provides the exact expression of \(({{\varvec{D}}}{{\varvec{D}}}')^{-1}\).
References
Dow M (2003) Explicit inverses of Toeplitz and associated matrices. ANZIAM J 44:185–215
Grant M. Boyd S. (2013): CVX: Matlab software for disciplined convex programming, version 2.0 beta. (http://cvxr.com/cvx)
Han C (2007) Determinants of covariance matrices of differenced AR(1) processes. Econom Theory 23:1248–1253
Harchaoui Z, Lévy-Leduc C (2010) Multiple change-point estimation with a total variation penalty. J Am Stat Assoc 105(492):1480–1493
Hatanaka M, Yamada H (2003) Co-trending: a statistical system analysis of economic trends. Springer, Tokyo
Hodrick RJ, Prescott EC (1997) Postwar U.S. business cycles: an empirical investigation. J Money Credit Bank 29(1):1–16
Kim S, Koh K, Boyd S, Gorinevsky D (2009) \(\ell _{1}\) trend filtering. SIAM Rev 51(2):339–360
Osborne MR, Presnell B, Turlach BA (2000) On the LASSO and its dual. J Comput Gr Stat 9(2):319–337
Perron P (1989) The great crash, the oil price shock, and the unit root hypothesis. Econometrica 57(6):1361–1401
Rappoport P, Reichlin L (1989) Segmented trends and non-stationary time series. Econ J 99:168–177
Sato K (2002) From fast to last: the Japanese economy in the 1990s. J Asian Econ 13(2):213–235
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc B 58(1):267–288
Tibshirani RJ, Taylor J (2011) The solution path of the generalized lasso. Ann Stat 39(3):1335–1371
Tibshirani RJ (2014) Adaptive piecewise polynomial estimation via trend filtering. Ann Stat 42(1):285–323
Yamada H (2015) Ridge regression representations of the generalized Hodrick–Prescott filter. J Jpn Stat Soc 45:121–128
Yamada H (2017) Estimating the trend in US real GDP using the \(\ell _{1}\) trend filtering. Appl Econ Lett 24(10):713–716
Yamada H (2018): Why does the trend extracted by the Hodrick–Prescott filtering seem to be more plausible than the linear trend?. Appl Econ Lett 25(2):102–105
Yamada H, Jin L (2013) Japan’s output gap estimation and \(\ell _{1}\) trend filtering. Empir Econ 45(1):81–88
Yamada H, Yoon G (2014) When Grilli and Yang meet Prebisch and Singer: piecewise linear trends in primary commodity prices. J Int Money Finance 42:193–207
Yamada H, Yoon G (2016a) Measuring the US NAIRU as a step function. Empir Econ 51(4):1679–1688
Yamada H, Yoon G (2016b) Selecting the tuning parameter of the \(\ell _{1}\) trend filter. Stud Nonlinear Dyn Econom 20(1):97–105
Author information
Authors and Affiliations
Corresponding author
Additional information
The author would like to thank Peter C. B. Phillips, Kazuhiko Hayakawa, Chirok Han, Hiro Y. Toda, Hiroshi Kurata, and Hirofumi Wakaki for their useful suggestions and comments. He also appreciates an anonymous referee and an associate editor for their valuable comments. The usual caveat applies. The Japan Society for the Promotion of Science supported this work through KAKENHI Grant Number 15K13010.
Appendix A: MATLAB function for calculating \(\widehat{{{\varvec{z}}}}\) in (5)
Rights and permissions
About this article
Cite this article
Yamada, H. A trend filtering method closely related to \(\ell _{1}\) trend filtering. Empir Econ 55, 1413–1423 (2018). https://doi.org/10.1007/s00181-017-1349-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00181-017-1349-8
Keywords
- \(\ell _{1}\) trend filtering
- Generalized lasso regression
- Hodrick–Prescott filtering
- Ridge regression
- Penalized least squares
- 1d fused lasso
- Total variation denoising