Abstract
Computational models are increasingly used for diagnosis and treatment of cardiovascular disease. To provide a quantitative hemodynamic understanding that can be effectively used in the clinic, it is crucial to quantify the variability in the outputs from these models due to multiple sources of uncertainty. To quantify this variability, the analyst invariably needs to generate a large collection of high-fidelity model solutions, typically requiring a substantial computational effort. In this paper, we show how an explicit-in-time ensemble cardiovascular solver offers superior performance with respect to the embarrassingly parallel solution with implicit-in-time algorithms, typical of an inner-outer loop paradigm for non-intrusive uncertainty propagation. We discuss in detail the numerics and efficient distributed implementation of a segregated FSI cardiovascular solver on both CPU and GPU systems, and demonstrate its applicability to idealized and patient-specific cardiovascular models, analyzed under steady and pulsatile flow conditions.
Similar content being viewed by others
References
Askes H, Nguyen D, Tyas A (2011) Increasing the critical time step: micro-inertia, inertia penalties and mass scaling. Comput Mech 47(6):657–667
Ayachit U (2015) The paraview guide: a parallel visualization application. Kitware Inc, New York
Bartezzaghi A, Cremonesi M, Parolini N, Perego U (2015) An explicit dynamics GPU structural solver for thin shell finite elements. Comput Struct 154:29–40
Bäumler K, Vedula V, Sailer A, Seo J, Chiu P, Mistelbauer G, Chan F, Fischbein M, Marsden A, Fleischmann D (2020) Fluid-structure interaction simulations of patient-specific aortic dissection. Biomech Model Mechanobiol 19:1602–1628
Bazilevs Y, Calo V, Cottrell J, Hughes T, Reali A, Scovazzi G (2007) Variational multiscale residual-based turbulence modeling for large eddy simulation of incompressible flows. Comput Methods Appl Mech Eng 197(1):173–201
Behnel S, Bradshaw R, Citro C, Dalcin L, Seljebotn D, Smith K (2011) Cython: the best of both worlds. Comput Sci Eng 13(2):31–39. https://doi.org/10.1109/MCSE.2010.118
Bell N, Garland M (2009) Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: Proceedings of the conference on high performance computing networking, storage and analysis, pp 1–11
Bolin D, Lindgren F (2013) A comparison between Markov approximations and other methods for large spatial data sets. Comput Stat Data Anal 61:7–21
Buluç A, Fineman J, Frigo M, Gilbert J, Leiserson C (2009) Parallel sparse matrix-vector and matrix-transpose-vector multiplication using compressed sparse blocks. In: Proceedings of the twenty-first annual symposium on Parallelism in algorithms and architectures, pp 233–244
Chiastra C, Morlacchi S, Gallo D, Morbiducci U, Cárdenes R, Larrabide I, Migliavacca F (2013) Computational fluid dynamic simulations of image-based stented coronary bifurcation models. J R Soc Interface 10(84):20130193
Chiastra C, Dubini G, Migliavacca F (2020) Modeling the stent deployment in coronary arteries and coronary bifurcations. In: Biomechanics of coronary atherosclerotic plaque. Elsevier, pp 579–597
Figueroa C, Vignon-Clementel I, Jansen K, Hughes T, Taylor C (2006) A coupled momentum method for modeling blood flow in three-dimensional deformable arteries. Comput Methods Appl Mech Eng 195(41–43):5685–5706
Fiordilino J (2018) Ensemble time-stepping algorithms for the heat equation with uncertain conductivity. Numer Methods Partial Differ Equ 34(6):1901–1916
Garland M (2008) Sparse matrix computations on manycore GPUs. In: Proceedings of the 45th annual design automation conference, pp 2–6
Greathouse J, Daga M (2014) Efficient sparse matrix-vector multiplication on GPUs using the CSR storage format. In: SC’14: Proceedings of the international conference for high performance computing, networking, storage and analysis. IEEE, pp 769–780
Gunzburger M, Jiang N, Wang Z (2019) An efficient algorithm for simulating ensembles of parameterized flow problems. IMA J Numer Anal 39(3):1180–1205
Jansen K, Whiting C, Hulbert G (2000) A generalized-\(\alpha \) method for integrating the filtered Navier–Stokes equations with a stabilized finite element method. Comput Methods Appl Mech Eng 190(3–4):305–319
Jiang N (2015) A higher order ensemble simulation algorithm for fluid flows. J Sci Comput 64(1):264–288
Jiang N (2017) A second-order ensemble method based on a blended backward differentiation formula timestepping scheme for time-dependent Navier–Stokes equations. Numer Methods Partial Differ Equ 33(1):34–61
Jiang N, Layton W (2014) An algorithm for fast calculation of flow ensembles. Int J Uncertain Quantif 4(4):273–301
Jiang N, Layton W (2015) Numerical analysis of two ensemble eddy viscosity numerical regularizations of fluid motion. Numer Methods Partial Differ Equ 31(3):630–651
Karypis G, Kumar V (2009) MeTis: unstructured graph partitioning and sparse matrix ordering system, Version 4.0. http://www.cs.umn.edu/~metis
Klöckner A, Pinto N, Lee Y, Catanzaro B, Ivanov P, Fasih A (2012) Pycuda and pyopencl: a scripting-based approach to GPU run-time code generation. Parallel Comput 38(3):157–174
Lan H, Updegrove A, Wilson N, Maher G, Shadden S, Marsden A (2018) A re-engineered software interface and workflow for the open-source simvascular cardiovascular modeling package. J Biomech Eng 140(2):0245011
Lindgren F, Rue H, Lindström J (2011) An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach. J R Stat Soc Ser B (Stat Methodol) 73(4):423–498. https://doi.org/10.1111/j.1467-9868.2011.00777.x
Luo Y, Wang Z (2018) An ensemble algorithm for numerical solutions to deterministic and random parabolic PDEs. SIAM J Numer Anal 56(2):859–876
Mohebujjaman M, Rebholz L (2017) An efficient algorithm for computation of MHD flow ensembles. Comput Methods Appl Math 17(1):121–137
Morlacchi S, Chiastra C, Gastaldi D, Pennati G, Dubini G, Migliavacca F (2011) Sequential structural and fluid dynamic numerical simulations of a stented bifurcated coronary artery. J Biomech Eng 133(12):121010
Schroeder W, Martin K, Lorensen B (2006) The visualization toolkit, 4th edn. Kitware, New York
Seo J, Schiavazzi D, Marsden A (2019) Performance of preconditioned iterative linear solvers for cardiovascular simulations in rigid and deformable vessels. Comput Mech. https://doi.org/10.1007/s00466-019-01678-3
Stein M (2012) Interpolation of spatial data: some theory for kriging. Springer, Berlin
Stone J, Gohara D, Shi G (2010) Opencl: a parallel programming standard for heterogeneous computing systems. Comput Sci Eng 12(3):66–73
Strbac V, Pierce D, Vander Sloten J, Famaey N (2017) GPGPU-based explicit finite element computations for applications in biomechanics: the performance of material models, element technologies, and hardware generations. Comput Methods Biomech Biomed Eng 20(16):1643–1657
Takhirov A, Neda M, Waters J (2016) Time relaxation algorithm for flow ensembles. Numer Methods Partial Differ Equ 32(3):757–777
Tran J, Schiavazzi D, Kahn A, Marsden A (2019) Uncertainty quantification of simulated biomechanical stimuli in coronary artery bypass grafts. Comput Methods Appl Mech Eng 345:402–428. https://doi.org/10.1016/j.cma.2018.10.024
Whittle P (1954) On stationary processes in the plane. Biometrika 41:434–449
Whittle P (1963) Stochastic-processes in several dimensions. Bull Int Stat Inst 40(2):974–994
Acknowledgements
This work was supported by a National Science Foundation award #1942662 CAREER: Bayesian Inference Networks for Model Ensembles (PI Daniele E. Schiavazzi). This research used computational resources provided through the Center for Research Computing at the University of Notre Dame. We also acknowledge support from the open source SimVascular project at www.simvascular.org.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Li, X., Schiavazzi, D.E. An ensemble solver for segregated cardiovascular FSI. Comput Mech 68, 1421–1436 (2021). https://doi.org/10.1007/s00466-021-02076-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00466-021-02076-4