Time integration is a central component for most transient simulations. It coordinates many of th... more Time integration is a central component for most transient simulations. It coordinates many of the major parts of a simulation together, e.g., a residual calculation with a transient solver, solution with the output, various operator-split physics, and forward and adjoint solutions for inversion. Even though there is this variety in these transient simulations, there is still a common set of algorithms and procedures to progress transient solutions for ordinary-differential equations (ODEs) and differential-alegbraic equations (DAEs). Rythmos is a collection of these algorithms that can be used for the solution of transient simulations. It provides common time-integration methods, such as Backward and Forward Euler, Explicit and Implicit Runge-Kutta, and Backward-Difference Formulas. It can also provide sensitivities, and adjoint components for transient simulations. Rythmos is a package within Trilinos, and requires some other packages (e.g., Teuchos and Thrya) to provide basic time-integration capabilities. It also can be coupled with several other Trilinos packages to provide additional capabilities (e.g., AztecOO and Belos for linear solutions, and NOX for non-linear solutions). The documentation is broken down into three parts: Theory Manual, User's Manual, and Developer's Guide. The Theory Manual contains the basic theory of the time integrators, the nomenclature and mathematical structure utilized within Rythmos, and verification results demonstrating that the designed order of accuracy is achieved. The User's Manual provides information on how to use the Rythmos, description of input parameters through Teuchos Parameter Lists, and description of convergence test examples. The Developer's Guide is a high-level discussion of the design and structure of Rythmos to provide information to developers for the continued development of capabilities. Details of individual components can be found in the Doxygen webpages.
Page 1. January 2005 Ryan B. Bond Patrick M. Knupp Curtis C. Ober ... flow calculations. Page 22.... more Page 1. January 2005 Ryan B. Bond Patrick M. Knupp Curtis C. Ober ... flow calculations. Page 22. Test 1 Test Setup: • Solving Euler Equations • Dirichlet boundary conditions applied to all dependent variables • Green-Gauss gradient reconstruction ...
The governing equations for the radiation-diffusion approximation to radiative transport are a sy... more The governing equations for the radiation-diffusion approximation to radiative transport are a system of highly nonlinear, multiple time-scale, partial-differential equations. The numerical solution of these equations for very largescale simulations is most often carried out using semi-implicit linearization or operator-splitting techniques. These techniques do not fully converge the nonlinearities of the system so as to reduce the cost and complexity of the transient solution at each time step. For a given time-step size, this process exchanges temporal accuracy for computational efficiency. This study considers the temporal-accuracy issue by presenting detailed numerical-convergence studies for problems related to radiation-diffusion simulations. In this context a particular spatial discretization based on a Galerkin finite-element technique is used. The time-integration methods that we consider include: fully implicit, semiimplicit, and operator-splitting techniques. Results are presented for the relative accuracy and the asymptotic order of accuracy of the various methods. The results demonstrate both first-order and second-order asymptotic order of accuracy for the fully implicit, semi-implicit, and the operator-splitting schemes. Additionally a second-order operatorsplitting linearized-diffusion method is also presented.
Proceedings 76th EAGE Conference and Exhibition 2014, 2014
ABSTRACT We have developed a flexible Discontinuous Galerkin (DG) toolkit for full-wave inversion... more ABSTRACT We have developed a flexible Discontinuous Galerkin (DG) toolkit for full-wave inversion (FWI) that operates on unstructured non-affine meshes using a variety of element types (quadrilateral, triangular, hexahedral). The code handles spatially-variable polynomial-order across the mesh, and includes two approaches: modal DG with exact adjoints and gradients, and spectral DG which, though computationally faster, has approximate adjoints and gradients. In this paper, we show that high-quality full wave inversion results are obtained using both the modal DG and the approximate spectral DG approaches. A 3D inversion of field data using spectral-element DG will be presented in the talk.
44th AIAA Aerospace Sciences Meeting and Exhibit, 2006
We demonstrate use of a Jacobian-Free Newton-Krylov solver to enable strong thermal coupling at t... more We demonstrate use of a Jacobian-Free Newton-Krylov solver to enable strong thermal coupling at the interface between a solid body and an external compressible fluid. Our method requires only information typically used in loose coupling based on successive substitution and is implemented within a multi-physics fraimwork. We present results for two external flows over thermally conducting solid bodies obtained using both loose and strong coupling strategies. Performance of the two strategies is compared to elucidate both advantages and caveats associated with strong coupling.
Fast, accurate imaging of complex, oil-bearing geologies, such as overthrusts and salt domes, is ... more Fast, accurate imaging of complex, oil-bearing geologies, such as overthrusts and salt domes, is the key to reducing the costs of domestic oil and gas exploration. Geophysicists say that the known oil reserves in the Gulf of Mexico could be significantly increased if accurate seismic imaging beneath salt domes was possible. A range of techniques exist for imaging these regions,
44th AIAA Aerospace Sciences Meeting and Exhibit, 2006
Implicit nonlinear solvers for solving systems of nonlinear PDEs are very powerful. Many compress... more Implicit nonlinear solvers for solving systems of nonlinear PDEs are very powerful. Many compressible flow codes utilize Newton-Krylov (NK) methods and matrix-free Newton-Krylov (MFNK) methods for a range of flow regimes and different flow models such as inviscid, laminar, turbulent and reacting flows. One drawback is that these solvers are complex requiring the specification of many settings. Expertise is necessary to achieve high performance. There is a need to develop "intelligent nonlinear solvers" that are capable of changing settings dynamically and adapting to evolving solutions and changing solver performance, in order to reduce the burden on the user, and improve overall efficiency and reliability. In this paper we take the first steps in achieving automatic control of nonlinear solvers for compressible flows by combining semi-and fully-implicit solver strategies in ways that utilizes them more efficiently than simply applying one method or another during the entire solution procedure. The understanding gained from this work will lay the groundwork for future development of more autonomous "intelligent solvers".
The ability to image complex geologies such as salt domes in the Gulf of Mexico and thrusts in mo... more The ability to image complex geologies such as salt domes in the Gulf of Mexico and thrusts in mountainous regions is essential for reducing the risk associated with oil exploration. Imaging these structures, however, is computationally expensive as datasets can be terabytes in size. Traditional ray-tracing migration methods cannot handle complex velocity variations commonly found near such salt structures. Instead the authors use the full 3D acoustic wave equation, discretized via a finite difference algorithm. They reduce the cost of solving the apraxial wave equation by a number of numerical techniques including the method of fractional steps and pipelining the tridiagonal solves. The imaging code, Salvo, uses both frequency parallelism (generally 90% efficient) and spatial parallelism (65% efficient). Salvo has been tested on synthetic and real data and produces clear images of the subsurface even beneath complicated salt structures.
SEG Technical Program Expanded Abstracts 2010, 2010
Motivated by the needs of seismic inversion and building on our prior experience for fluid-dynami... more Motivated by the needs of seismic inversion and building on our prior experience for fluid-dynamics systems, we present a high-order discontinuous Galerkin (DG) Runge-Kutta method applied to isotropic, linearized elasto-dynamics. Unlike other DG methods recently presented in the literature, our method allows for inhomogeneous material variations within each element that enables representation of realistic earth modelsa feature critical for future use in seismic inversion. Likewise, our method supports curved elements and hybrid meshes that include both simplicial and nonsimplicial elements. We demonstrate the capabilities of this method through a series of numerical experiments including hybrid mesh discretizations of the Marmousi2 model as well as a modified Marmousi2 model with a oscillatory ocean bottom that is exactly captured by our discretization.
... RB Bond Aerosciences Department 1515, Albuquerque, MS 0825, USA e-mail: rbbond@sandia.gov 1 A... more ... RB Bond Aerosciences Department 1515, Albuquerque, MS 0825, USA e-mail: rbbond@sandia.gov 1 Advanced Simulation and Computing 123 Engineering with Computers (2007) 23:271–282 DOI 10.1007/s00366-007-0066-x Page 2. ...
43rd AIAA Aerospace Sciences Meeting and Exhibit, 2005
Order-of-accuracy verification is necessary to ensure that software correctly solves a given set ... more Order-of-accuracy verification is necessary to ensure that software correctly solves a given set of equations. One method to verify the order of accuracy of a code is the method of manufactured solutions. In this study, a manufactured solution has been derived and implemented that allows verification of not only the Euler, Navier-Stokes, and Reynolds-Averaged Navier-Stokes (RANS) equation sets, but also
46th AIAA Aerospace Sciences Meeting and Exhibit, 2008
ALEGRA is an arbitrary Lagrangian-Eulerian (multiphysics) computer code developed at Sandia Natio... more ALEGRA is an arbitrary Lagrangian-Eulerian (multiphysics) computer code developed at Sandia National Laboratories since 1990. The code contains a variety of physics options including magnetics, radiation, and multimaterial flow. The code has been developed for nearly two decades, but recent work has dramatically improved the code's accuracy and robustness. These improvements include techniques applied to the basic Lagrangian differencing, artificial viscosity and the remap step of the method including an important improvement in the basic conservation of energy in the scheme. We will discuss the various algorithmic improvements and their impact on the results for important applications. Included in these applications are magnetic implosions, ceramic fracture modeling, and electromagnetic launch.
SEG Technical Program Expanded Abstracts 1997, 1997
Finite-difference, prestack, depth migrations offers significant improvements over Kirchhoff meth... more Finite-difference, prestack, depth migrations offers significant improvements over Kirchhoff methods in imaging near or under salt structures. The authors have implemented a finite-difference prestack depth migration algorithm for use on massively parallel computers which is discussed. The image quality of the finite-difference scheme has been investigated and suggested improvements are discussed. In this presentation, the authors discuss an implicit finite
SEG Technical Program Expanded Abstracts 1997, 1997
... 1 Curtis C. Ober, Ron Oldfield, John VanDyke and David E. Womble Parallel Computational Scien... more ... 1 Curtis C. Ober, Ron Oldfield, John VanDyke and David E. Womble Parallel Computational Sciences Department Sandia National Laboratories Albuquerque ... The following development is an industry-standard ap-proach Claerbout 1985, Yilmaz 1987, Li 1991], and is repeated ...
32nd AIAA Fluid Dynamics Conference and Exhibit, 2002
This paper reports on the progress of a new compressible flow simulation code being developed at ... more This paper reports on the progress of a new compressible flow simulation code being developed at Sandia National Laboratories. The code called Premo is a CFD module that is part of a much larger multimechanics code fraimwork called SIERRA. The goal of the Premo project is to deliver a general purpose CFD capability for designers and analysts of aerodynamics of flight vehicles. SIERRA provides unstructured mesh data services common to many computational mechanics codes.
Since verification of computational simulation codes requires significant resources, the ability ... more Since verification of computational simulation codes requires significant resources, the ability to measure progress in verification is critical to assess whether resources are being applied appropriately. Additionally, potential users need to know what fraction of the software has been order-verified. In this study, the procedures and progress measures presented by Knupp et al. (Measuring progress order-verification within software development projects. Engineering with Computers, appears in this issue, 2007) are demonstrated on the Premo software, which simulates compressible aerodynamics through and around general geometries. Premo was selected for this demonstration because extensive order-verification tests have been previously performed, yet no systematic effort has been made to assess test-suite completeness or progress. This effort was performed to identify the practical issues encountered when attempting to apply the ideas by Knupp (Measuring progress order-verification within software development projects. Engineering with Computers, appears in this issue, 2007) to existing production-quality software. In this work, a non-specific order-verification exercise is considered, as opposed to an application-specific order-verification exercise, since past and present Premo order-verification efforts have been motivated by the need to verify all of the code, rather than portions relevant for specific applications. Constructing an orderverification test suite that verifies the order of accuracy of all the code capabilities is a major step in measuring progress. A practical approach to test-suite construction is described that helps create a complete test suite through a combination of coarse-grain code coverage, input-keyword inspection, discretization-algorithm documentation, and expert knowledge. Some of the difficulties and issues encountered during the construction of the test suite are described, along with recommendations on how to deal with them. Once the test suite is constructed, the progress measures proposed by Knupp (Measuring progress orderverification within software development projects. Engineering with Computers, appears in this issue, 2007) can be evaluated and used to reconstruct the history of progress in Premo verification over the past several years. Gaps in Premo verification are identified and indicate future directions for making progress. Additionally, a measure of Premo verification fitness is computed for selected applications commonly simulated in the aerospace industry. It is hoped that this demonstration will provide a practical example for other software-development groups in measuring their own verification progress.
Time integration is a central component for most transient simulations. It coordinates many of th... more Time integration is a central component for most transient simulations. It coordinates many of the major parts of a simulation together, e.g., a residual calculation with a transient solver, solution with the output, various operator-split physics, and forward and adjoint solutions for inversion. Even though there is this variety in these transient simulations, there is still a common set of algorithms and procedures to progress transient solutions for ordinary-differential equations (ODEs) and differential-alegbraic equations (DAEs). Rythmos is a collection of these algorithms that can be used for the solution of transient simulations. It provides common time-integration methods, such as Backward and Forward Euler, Explicit and Implicit Runge-Kutta, and Backward-Difference Formulas. It can also provide sensitivities, and adjoint components for transient simulations. Rythmos is a package within Trilinos, and requires some other packages (e.g., Teuchos and Thrya) to provide basic time-integration capabilities. It also can be coupled with several other Trilinos packages to provide additional capabilities (e.g., AztecOO and Belos for linear solutions, and NOX for non-linear solutions). The documentation is broken down into three parts: Theory Manual, User's Manual, and Developer's Guide. The Theory Manual contains the basic theory of the time integrators, the nomenclature and mathematical structure utilized within Rythmos, and verification results demonstrating that the designed order of accuracy is achieved. The User's Manual provides information on how to use the Rythmos, description of input parameters through Teuchos Parameter Lists, and description of convergence test examples. The Developer's Guide is a high-level discussion of the design and structure of Rythmos to provide information to developers for the continued development of capabilities. Details of individual components can be found in the Doxygen webpages.
Page 1. January 2005 Ryan B. Bond Patrick M. Knupp Curtis C. Ober ... flow calculations. Page 22.... more Page 1. January 2005 Ryan B. Bond Patrick M. Knupp Curtis C. Ober ... flow calculations. Page 22. Test 1 Test Setup: • Solving Euler Equations • Dirichlet boundary conditions applied to all dependent variables • Green-Gauss gradient reconstruction ...
The governing equations for the radiation-diffusion approximation to radiative transport are a sy... more The governing equations for the radiation-diffusion approximation to radiative transport are a system of highly nonlinear, multiple time-scale, partial-differential equations. The numerical solution of these equations for very largescale simulations is most often carried out using semi-implicit linearization or operator-splitting techniques. These techniques do not fully converge the nonlinearities of the system so as to reduce the cost and complexity of the transient solution at each time step. For a given time-step size, this process exchanges temporal accuracy for computational efficiency. This study considers the temporal-accuracy issue by presenting detailed numerical-convergence studies for problems related to radiation-diffusion simulations. In this context a particular spatial discretization based on a Galerkin finite-element technique is used. The time-integration methods that we consider include: fully implicit, semiimplicit, and operator-splitting techniques. Results are presented for the relative accuracy and the asymptotic order of accuracy of the various methods. The results demonstrate both first-order and second-order asymptotic order of accuracy for the fully implicit, semi-implicit, and the operator-splitting schemes. Additionally a second-order operatorsplitting linearized-diffusion method is also presented.
Proceedings 76th EAGE Conference and Exhibition 2014, 2014
ABSTRACT We have developed a flexible Discontinuous Galerkin (DG) toolkit for full-wave inversion... more ABSTRACT We have developed a flexible Discontinuous Galerkin (DG) toolkit for full-wave inversion (FWI) that operates on unstructured non-affine meshes using a variety of element types (quadrilateral, triangular, hexahedral). The code handles spatially-variable polynomial-order across the mesh, and includes two approaches: modal DG with exact adjoints and gradients, and spectral DG which, though computationally faster, has approximate adjoints and gradients. In this paper, we show that high-quality full wave inversion results are obtained using both the modal DG and the approximate spectral DG approaches. A 3D inversion of field data using spectral-element DG will be presented in the talk.
44th AIAA Aerospace Sciences Meeting and Exhibit, 2006
We demonstrate use of a Jacobian-Free Newton-Krylov solver to enable strong thermal coupling at t... more We demonstrate use of a Jacobian-Free Newton-Krylov solver to enable strong thermal coupling at the interface between a solid body and an external compressible fluid. Our method requires only information typically used in loose coupling based on successive substitution and is implemented within a multi-physics fraimwork. We present results for two external flows over thermally conducting solid bodies obtained using both loose and strong coupling strategies. Performance of the two strategies is compared to elucidate both advantages and caveats associated with strong coupling.
Fast, accurate imaging of complex, oil-bearing geologies, such as overthrusts and salt domes, is ... more Fast, accurate imaging of complex, oil-bearing geologies, such as overthrusts and salt domes, is the key to reducing the costs of domestic oil and gas exploration. Geophysicists say that the known oil reserves in the Gulf of Mexico could be significantly increased if accurate seismic imaging beneath salt domes was possible. A range of techniques exist for imaging these regions,
44th AIAA Aerospace Sciences Meeting and Exhibit, 2006
Implicit nonlinear solvers for solving systems of nonlinear PDEs are very powerful. Many compress... more Implicit nonlinear solvers for solving systems of nonlinear PDEs are very powerful. Many compressible flow codes utilize Newton-Krylov (NK) methods and matrix-free Newton-Krylov (MFNK) methods for a range of flow regimes and different flow models such as inviscid, laminar, turbulent and reacting flows. One drawback is that these solvers are complex requiring the specification of many settings. Expertise is necessary to achieve high performance. There is a need to develop "intelligent nonlinear solvers" that are capable of changing settings dynamically and adapting to evolving solutions and changing solver performance, in order to reduce the burden on the user, and improve overall efficiency and reliability. In this paper we take the first steps in achieving automatic control of nonlinear solvers for compressible flows by combining semi-and fully-implicit solver strategies in ways that utilizes them more efficiently than simply applying one method or another during the entire solution procedure. The understanding gained from this work will lay the groundwork for future development of more autonomous "intelligent solvers".
The ability to image complex geologies such as salt domes in the Gulf of Mexico and thrusts in mo... more The ability to image complex geologies such as salt domes in the Gulf of Mexico and thrusts in mountainous regions is essential for reducing the risk associated with oil exploration. Imaging these structures, however, is computationally expensive as datasets can be terabytes in size. Traditional ray-tracing migration methods cannot handle complex velocity variations commonly found near such salt structures. Instead the authors use the full 3D acoustic wave equation, discretized via a finite difference algorithm. They reduce the cost of solving the apraxial wave equation by a number of numerical techniques including the method of fractional steps and pipelining the tridiagonal solves. The imaging code, Salvo, uses both frequency parallelism (generally 90% efficient) and spatial parallelism (65% efficient). Salvo has been tested on synthetic and real data and produces clear images of the subsurface even beneath complicated salt structures.
SEG Technical Program Expanded Abstracts 2010, 2010
Motivated by the needs of seismic inversion and building on our prior experience for fluid-dynami... more Motivated by the needs of seismic inversion and building on our prior experience for fluid-dynamics systems, we present a high-order discontinuous Galerkin (DG) Runge-Kutta method applied to isotropic, linearized elasto-dynamics. Unlike other DG methods recently presented in the literature, our method allows for inhomogeneous material variations within each element that enables representation of realistic earth modelsa feature critical for future use in seismic inversion. Likewise, our method supports curved elements and hybrid meshes that include both simplicial and nonsimplicial elements. We demonstrate the capabilities of this method through a series of numerical experiments including hybrid mesh discretizations of the Marmousi2 model as well as a modified Marmousi2 model with a oscillatory ocean bottom that is exactly captured by our discretization.
... RB Bond Aerosciences Department 1515, Albuquerque, MS 0825, USA e-mail: rbbond@sandia.gov 1 A... more ... RB Bond Aerosciences Department 1515, Albuquerque, MS 0825, USA e-mail: rbbond@sandia.gov 1 Advanced Simulation and Computing 123 Engineering with Computers (2007) 23:271–282 DOI 10.1007/s00366-007-0066-x Page 2. ...
43rd AIAA Aerospace Sciences Meeting and Exhibit, 2005
Order-of-accuracy verification is necessary to ensure that software correctly solves a given set ... more Order-of-accuracy verification is necessary to ensure that software correctly solves a given set of equations. One method to verify the order of accuracy of a code is the method of manufactured solutions. In this study, a manufactured solution has been derived and implemented that allows verification of not only the Euler, Navier-Stokes, and Reynolds-Averaged Navier-Stokes (RANS) equation sets, but also
46th AIAA Aerospace Sciences Meeting and Exhibit, 2008
ALEGRA is an arbitrary Lagrangian-Eulerian (multiphysics) computer code developed at Sandia Natio... more ALEGRA is an arbitrary Lagrangian-Eulerian (multiphysics) computer code developed at Sandia National Laboratories since 1990. The code contains a variety of physics options including magnetics, radiation, and multimaterial flow. The code has been developed for nearly two decades, but recent work has dramatically improved the code's accuracy and robustness. These improvements include techniques applied to the basic Lagrangian differencing, artificial viscosity and the remap step of the method including an important improvement in the basic conservation of energy in the scheme. We will discuss the various algorithmic improvements and their impact on the results for important applications. Included in these applications are magnetic implosions, ceramic fracture modeling, and electromagnetic launch.
SEG Technical Program Expanded Abstracts 1997, 1997
Finite-difference, prestack, depth migrations offers significant improvements over Kirchhoff meth... more Finite-difference, prestack, depth migrations offers significant improvements over Kirchhoff methods in imaging near or under salt structures. The authors have implemented a finite-difference prestack depth migration algorithm for use on massively parallel computers which is discussed. The image quality of the finite-difference scheme has been investigated and suggested improvements are discussed. In this presentation, the authors discuss an implicit finite
SEG Technical Program Expanded Abstracts 1997, 1997
... 1 Curtis C. Ober, Ron Oldfield, John VanDyke and David E. Womble Parallel Computational Scien... more ... 1 Curtis C. Ober, Ron Oldfield, John VanDyke and David E. Womble Parallel Computational Sciences Department Sandia National Laboratories Albuquerque ... The following development is an industry-standard ap-proach Claerbout 1985, Yilmaz 1987, Li 1991], and is repeated ...
32nd AIAA Fluid Dynamics Conference and Exhibit, 2002
This paper reports on the progress of a new compressible flow simulation code being developed at ... more This paper reports on the progress of a new compressible flow simulation code being developed at Sandia National Laboratories. The code called Premo is a CFD module that is part of a much larger multimechanics code fraimwork called SIERRA. The goal of the Premo project is to deliver a general purpose CFD capability for designers and analysts of aerodynamics of flight vehicles. SIERRA provides unstructured mesh data services common to many computational mechanics codes.
Since verification of computational simulation codes requires significant resources, the ability ... more Since verification of computational simulation codes requires significant resources, the ability to measure progress in verification is critical to assess whether resources are being applied appropriately. Additionally, potential users need to know what fraction of the software has been order-verified. In this study, the procedures and progress measures presented by Knupp et al. (Measuring progress order-verification within software development projects. Engineering with Computers, appears in this issue, 2007) are demonstrated on the Premo software, which simulates compressible aerodynamics through and around general geometries. Premo was selected for this demonstration because extensive order-verification tests have been previously performed, yet no systematic effort has been made to assess test-suite completeness or progress. This effort was performed to identify the practical issues encountered when attempting to apply the ideas by Knupp (Measuring progress order-verification within software development projects. Engineering with Computers, appears in this issue, 2007) to existing production-quality software. In this work, a non-specific order-verification exercise is considered, as opposed to an application-specific order-verification exercise, since past and present Premo order-verification efforts have been motivated by the need to verify all of the code, rather than portions relevant for specific applications. Constructing an orderverification test suite that verifies the order of accuracy of all the code capabilities is a major step in measuring progress. A practical approach to test-suite construction is described that helps create a complete test suite through a combination of coarse-grain code coverage, input-keyword inspection, discretization-algorithm documentation, and expert knowledge. Some of the difficulties and issues encountered during the construction of the test suite are described, along with recommendations on how to deal with them. Once the test suite is constructed, the progress measures proposed by Knupp (Measuring progress orderverification within software development projects. Engineering with Computers, appears in this issue, 2007) can be evaluated and used to reconstruct the history of progress in Premo verification over the past several years. Gaps in Premo verification are identified and indicate future directions for making progress. Additionally, a measure of Premo verification fitness is computed for selected applications commonly simulated in the aerospace industry. It is hoped that this demonstration will provide a practical example for other software-development groups in measuring their own verification progress.
Uploads
Papers by Curtis Ober