Abstract
It is a familiar scheme to show least-squares linear regression as resolving sample vectors of dependent variables into two orthogonal components, a regressional component lying in a space spanned by sample vectors of the independent variables, and a residual component lying in the orthogonal complement of that space. This scheme was probably first introduced by Kolmogorov (1946). But the step from this to considering the linear transformations which obtain these orthogonal components of sample vectors, these being the complementary pair of symmetric idempotents which are the orthogonal projectors on the space and its orthogonal complement, and then to analysis of relations between variables directly in terms of these projectors, is one that does not appear to have been explored. The advantage of this step apart from its showing the way to some possibly new concepts is that it makes an enlargement of didactic methods in a natural formalism for handling some familiar theory with a direct display of its algebraical-geometrical meaning, and it gives the framework for simple proofs and formulae. Proofs of some relevant algebraical propositions have already been given elsewhere (Afriat, 1957 and 1956). The view taken of regression analysis is one that does not depend on distribution concepts. Orthogonal projection is presented as the fundamental principle and the least-squares principle is derived as a property of it. The association of a distribution with a sample is a method that yields
This research was supported by the National Science Foundation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Afriat, S.N, On the latent vectors and characteristic values of products of pairs of symmetric idempotents, Quart. j, Math, (2), 7 (1956), 76–78.
Afriat, S.N., Orthogonal and oblique projectors and the characteristics of pairs of vector spaces, Proc. Cambridge Phil. Soc. 53, 4 (1957), 800–816.
Afriat, S.N., Statistical projection, Research Memorandum No. 16 (June 1960), Econometric Research Program, Princeton University.
Aitken, A.C., On least squares and linear combinations of observations, Proc. Roy. Soc. Edin. 55 (1945), 42.
Anderson, T.W., Estimating linear restrictions on regression coefficients for multivariate normal distributions, Ann. Math. Stat. 22 (1951), 327–351.
Anderson, T.W., An Introduction to Multivariate Statistical Analysis (New York, 1958 ).
Bellman, Richard, Introduction to Matrix Analysis (New York, 1960 ).
Cochran, W.G., The distribution of quadratic forms in a normal system, with applications to the analysis of variance, Proc. Cambridge Phil. Soc. 30 (1934), 178–191.
Dixmier, Jacques, Position relative de deu variétés linéares fermées dans us espace de Hilbert, Rev. Sci. Fasc. 7 (1948), 387–399.
Flanders, D.A., Angles between flat subspaces of a real n-dimensional Euclidean space, Studies and Essays Presented to R. Courant (New York, 1948 ).
Hadamard, Jacques, Resolution d’une question relative aux déterminants, Bull. Sci. Math. (2), 17 (1893), 240–246.
Halmos, P.R., Introduction to Hilbert Space and the Theory of Spectral Multiplicity (New York, 1951 ).
Hannan, E.J., The general theory of canonical correlation and its relation to functional analysis, J. Australian Math. Soc. 2 (1961), 229–242.
Hooper, J.W., Simultaneous equations and canonical correlation theory, Econometrica 27 (1959), 245–256.
Jordan, C., Essai sur la géométrie an dimensions, Bull. Soc. Math.Paris 3 (1875), 103–174; 4 (1877), 92.
Julia, G., Sur les projections des systemes orthonormaux de l’espace hilbertien, C.R. Ac. Sc. 218 (1944), 892.
Kolmogorov, A.N., Nauk (N.S.), On the proof of the method of least squares, Uspeki Matem, 1 (11), 1 (1946), 57–70. (M.R. 8 (1947), 523–3 ).
Mann, H.B., The algebra of the linear hypothesis, Ann. Math. Stat. 31 (1960), 1–15.
Mirsky, L., Introduction to Linear Algebra, ( Oxford, 1955 ), p. 420.
Penrose, R., A generalized inverse for matrices, Proc. Cambridge Phil. Soc. 51 (1955), 406–413.
Rosen, J.B., The gradient projection method for nonlinear programming. Part I. Linear constraints, J. Soc. Indust. Appl. Math. 8 (1960), 181–217.
Roy, S.N., Some Aspects of Multivariate Analysis (New York, 1957 ). Scheffe, Henry, The Analysis of Variance (New York, 1959 ).
Schoute, P.H., On the equations determining the angles of two polydimensional spaces, Proc. Sci. K. Akad. Wet. 7 (1905), 340–341.
Somerville, D.M.Y., An Introduction to the Geometry of N Dimensions (London, 1929 ).
Wilks, S.S., Certain generalizations in the analysis of variance, Biometrica, 24 (1932), 471–494.
Wilks, S.S., Mathematical Statistics (Princeton, 1943 ).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1969 Springer-Verlag Berlin · Heidelberg
About this chapter
Cite this chapter
Afriat, S.N. (1969). Regression and Projection. In: Fox, K.A., Sengupta, J.K., Narasimham, G.V.L. (eds) Economic Models, Estimation and Risk Programming: Essays in Honor of Gerhard Tintner. Lecture Notes in Operations Research and Mathematical Economics, vol 15. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-46198-9_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-46198-9_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-04638-7
Online ISBN: 978-3-642-46198-9
eBook Packages: Springer Book Archive