Abstract
In this chapter we present an overview of a number of related iterative methods for the solution of linear systems of equations. These methods are of Krylov projection type and include such popular methods as Conjugate Gradients, Bi-Conjugate Gradients, LSQR, and GMRES. We will sketch how these methods can be derived from simple basic iteration formulas, and how they are interrelated.
Iterative schemes are usually considered as alternatives for the solution of linear sparse systems, like those arising in, e.g., finite element or finite difference approximation of (systems of) partial differential equations. The structure of the operators plays no explicit role in any of these schemes, and the operator may be given even as a rule or a subroutine.
Although these methods seem to be almost trivially parallelizable at first glance, parallelism is sometimes a point of concern because of the inner products involved. We will consider this point in some detail.
Iterative methods are usually applied in combination with so-called preconditioning operators in order to improve convergence properties. This aspect receives more attention in our other chapter in this volume.
The work of this author was partially supported by the National Science Foundation under contract ASC 92-01266, the Army Research Office under contracts DAAL03-91-G-0150 and DAAL03-91-C-0047 (Univ. Tenn. subcontract ORA4466.04 Amendment 1), and the Office of Naval Research under contract ONR-N00014-92-J-1890.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bank, R. and Chan, T. F., 1993. “An analysis of the composite step bi-conjugate gradient method,” Numer. Math. 66, pp. 295–319.
Barrett, R., Berry, M., Chan, T., Demmel, J., Donato, J., Dongarra, J., Eijkhout, V., Pozo, R., Romine, C., and Van der Vorst, H., 1994. Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, SIAM, Philadelphia.
Brezinski, C. and Redivo-Zaglia, M., 1994. “Treatment of near breakdown in the CGS algorithm,” Numerical Algorithms 7, pp. 33–73.
Chan, T. F. and Szeto, T., 1994a. “A composite step conjugate gradient squared algorithm for solving nonsymmetric linear systems,” Numerical Algorithms 7, pp. 16–32.
Chan, T. F. and Szeto, T., 1994b. “Composite step product methods for solving nonsymmetric linear systems,” Technical Report CAM-94-18, UCLA, Department of Mathematics, Los Angeles.
Chronopoulos, A. T. and Kim, S. K., 1990. “s-Step Orthomin and GM RES implemented on parallel computers,” Technical Report 90/43R, UMSI, Minneapolis.
De Sturler, E., 1991. “A parallel restructured version of GMRES(m), ” Technical Report 91-85, Delft University of Technology, Delft.
De Sturler, E. and Fokkema, D. R., 1993. “Nested Krylov methods and preserving the orthogonality,” Preprint 796, Utrecht University, Department of Mathematics, Utrecht.
Demmel, J., Heath, M., and Van der Vorst, H., 1993. “Parallel linear algebra,” Acta Numerica 2, Cambridge University Press, Cambridge.
Dongarra, J. J., 1990. “Performance of various computers using standard linear equations software in a Fortran environment,” Technical Report CS-89-85, University of Tennessee, Knoxville.
Dongarra, J. J., Duff, I. S., Sorensen, D. C., and Van der Vorst, H. A., 1991. Solving Linear Systems on Vector and Shared Memory Computers, SIAM, Philadelphia.
Dongarra, J. J. and Van der Vorst, H. A., 1992. “Performance of various computers using standard sparse linear equations solving techniques,” Supercomputer 9, pp. 17–29.
Fletcher, R., 1976. “Conjugate gradient methods for indefinite systems,” Lecture Notes in Mathematics 506, Springer Verlag, Berlin, pp. 73–89.
Freund, R. W., Golub, G. H., and Nachtigal, N. M., 1992. “Iterative solution of linear systems,” Acta Numerica 1, Cambridge University Press, Cambridge, pp. 57–100.
Freund, R. W., 1993. “A transpose-free quasi-minimal residual algorithm for non-Hermitian linear systems,” SIAM J. Sci. Comp. 14, pp. 470–482.
Golub, G. H. and Van Loan, C. F., 1989. Matrix Computations, The Johns Hopkins University Press, Baltimore.
Gutknecht, M. H., 1993. “Variants of BICGSTAB for matrices with complex spectrum,” SIAM J. Sci. Comp. 14, pp. 1020–1033.
Hestenes, M. R. and Stiefel, E., 1954. “Methods of conjugate gradients for solving linear systems,” J. Res. Natl. Bur. Stand. 49, pp. 409–436.
Lanczos, C., 1952. “Solution of systems of linear equations by minimized iterations,” J. Res. Natl. Bur. Stand 49, pp. 33–53.
Ortega, J. M., 1988. Introduction to Parallel and Vector Solution of Linear Systems, Plenum Press, New York.
Paige, C. C. and Saunders, M. A., 1975. “Solution of sparse indefinite systems of linear equations,” SIAM J. Numer. Anal. 12, pp. 617–629.
Paige, C. C. and Saunders, M. A., 1982. “LSQR: An algorithm for sparse linear equations and sparse least squares,” ACM Trans. Math. Soft. 8, pp. 43–71.
Parlett, B. N., Taylor, D. R., and Liu, Z. A., 1985. “A look-ahead Lanczos algorithm for unsymmetric matrices,” Math. Comp. 44, pp. 105–124.
Pommerell, C., 1992. “Solution of large unsymmetric systems of linear equations,” Ph.D. thesis, Swiss Federal Institute of Technology, Zürich.
Saad, Y., 1993. “A flexible inner-outer preconditioned GMRES algorithm,” SIAM J. Sci. Comp. 14, pp. 461–469.
Saad, Y. and Schultz, M. H., 1986. “GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems,” SIAM J. Sci. Stat. Comp. 7, pp. 856–869.
Simon, H. D., 1989. “Direct sparse matrix methods,” in Modern Numerical Algorithms for Supercomputers, Almond, J. C. and Young, D. M., eds., University of Texas at Austin, Center for High Performance Computing, pp. 325–444.
Sleijpen, G. L. G., Van der Vorst, H. A., and Fokkema, D. R., 1994. “Bi-CGSTAB(l) and other hybrid bi-cg methods,” Numerical Algorithms 7, pp. 75–109.
Sleijpen, G. L. G. and Fokkema, D. R., 1993. “BICGSTAB(l) for linear equations involving unsymmetric matrices with complex spectrum,” ETNA 1, pp. 11–32.
Sonneveld, P. 1989. “CGS: A fast Lanczos-type solver for non-symmetric linear systems,” SIAM J. Sci. Stat. Comp. 10, pp. 36–52.
Van der Vorst, H. A., 1992. “Bi-CGSTAB: A fast and smoothly converging variant of Bi-CG for the solution of non-symmetric linear systems,” SIAM J. Sci. Stat. Comp. 13, pp. 631–644.
Van der Vorst, H. A. and Vuik, C., 1994. “GMRESR: A family of nested GMRES methods,” Num. Lin. Alg. Applies. 1, pp. 369–386.
Varga, R. S., 1962. Matrix Iterative Analysis, Prentice Hall, Engle-wood Cliffs.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1997 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Van der Vorst, H.A., Chan, T.F. (1997). Linear System Solvers: Sparse Iterative Methods. In: Keyes, D.E., Sameh, A., Venkatakrishnan, V. (eds) Parallel Numerical Algorithms. ICASE/LaRC Interdisciplinary Series in Science and Engineering, vol 4. Springer, Dordrecht. https://doi.org/10.1007/978-94-011-5412-3_4
Download citation
DOI: https://doi.org/10.1007/978-94-011-5412-3_4
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-6277-0
Online ISBN: 978-94-011-5412-3
eBook Packages: Springer Book Archive