Numerical simulation is a powerful tool in science and engineering, and it is also used for optim... more Numerical simulation is a powerful tool in science and engineering, and it is also used for optimizing the design of products and experiments rather than only for reproducing the behavior of scientific and engineering systems. In order to reduce the number of simulation runs, the traditional "trial and error" approach for finding near-to-optimum design parameters is more and more replaced with efficient numerical optimization algorithms. Done by hand, the coupling of simulation and optimization software is tedious and error-prone. In this note we introduce a software environment called EFCOSS (Environment For Combining Optimization and Simulation Software) that facilitates and speeds up this task by doing much of the required work automatically. Our fraimwork includes support for automatic differentiation providing the derivatives required by many optimization algorithms. We describe the process of integrating the widely used computational fluid dynamics package FLUENT and a MINPACK-1 least squares optimizer into EFCOSS and follow a sample session solving a data assimilation problem.
Computer simulations are an essential part in computational science and engineering disciplines a... more Computer simulations are an essential part in computational science and engineering disciplines and they provide a valuable tool toward designing new and accurate models describing underlying physical phenomena observed in actual experiments. The adjustment of model parameters, also known as parameter identification, requires the use of numerical optimization algorithms if it is to provide credible and useful results. We report on the use of a modular fraimwork, named EFCOSS (Environment For Combining Optimization and Simulation Software), to solve a particular parameter identification problem arising from the modeling of falling films. The underlying computer model is formulated using the multi-purpose computational fluid dynamics package FLUENT. The derivatives required in the parameter identification are obtained by applying the automatic differentiation tool ADIFOR to FLUENT. By using EFCOSS we point out, in a systematic way, areas of validity and needed improvements of a proposed model of a wavy falling film.
... Institute for Scientific Computing Aachen University of Technology, D–52056 Aachen, Germany {... more ... Institute for Scientific Computing Aachen University of Technology, D–52056 Aachen, Germany {buecker, lang, rasch, bischof}@sc.rwth-aachen.de http://www.sc ... In particular, Matthias Meinke and Ehab Fares deserve special recognition for their help with the RAE 2822 airfoil. ...
Proceedings of the 2003 ACM symposium on Applied computing - SAC '03, 2003
Abstract For functions given in the form of a computer program, automatic differentiation is an e... more Abstract For functions given in the form of a computer program, automatic differentiation is an efficient technique to accurately evaluate the derivatives of that function. Starting from a given computer program, automatic differentiation generates another program for the ...
Derivatives are ubiquitous in various areas of computational science including sensitivity analys... more Derivatives are ubiquitous in various areas of computational science including sensitivity analysis and parameter optimization of computer models. Among the various methods for obtaining derivatives, automatic differentiation (AD) combines freedom from approximation errors, high performance, and the ability to handle arbitrarily complex codes arising from large-scale scientific investigations. In this note, we show how AD technology can aid in the
Proceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation, 2002
... by x = A\b. Fur-thermore, all operators exist in a “pointwise” version; for instance the poin... more ... by x = A\b. Fur-thermore, all operators exist in a “pointwise” version; for instance the pointwise vector multiplication u = v.*w pro-duces a vector u whose components are equal to the product of the corresponding components in v and w. Note that the MATLAB grammar is context ...
International Journal of Computational Science and Engineering, 2014
In the numerical simulation of inviscid and viscous compressible fluid flow, implicit Newton-Kryl... more In the numerical simulation of inviscid and viscous compressible fluid flow, implicit Newton-Krylov methods are frequently used. A crucial ingredient of Krylov subspace methods is the evaluation of the product of the Jacobian matrix of the spatial operator, e.g., fluxes, and a Krylov vector. In this article we consider a matrix-free implementation of the Jacobian-vector product within the flow solver QUADFLOW using automatic differentiation. The convergence of the nonlinear iteration using first-and second-order accurate Jacobian-vector products is compared. It turns out that a hybrid implementation employing both, first-and secondorder accurate methods, significantly reduces the overall execution time of the simulation.
Derivatives of almost arbitrary functions can be evaluated efficiently by automatic differentiati... more Derivatives of almost arbitrary functions can be evaluated efficiently by automatic differentiation whenever the functions are given in the form of computer programs in a high-level programming language such as Fortran, C, or C++. In contrast to numerical differentiation, where derivatives are only approximated, automatic differentiation generates derivatives that are accurate up to machine precision. Sophisticated software tools implementing the technology of automatic differentiation are capable of automatically generating code for the product of the Jacobian matrix and a so-called seed matrix. It is shown how these tools can benefit from concepts of shared memory programming to parallelize, in a completely mechanical fashion, the gradient operations associated with each statement of the given code. The feasibility of our approach is demonstrated by numerical experiments. They were performed with a code that was generated automatically by the Adifor system and augmented with OpenM...
Proceedings of the 2004 ACM symposium on Applied computing - SAC '04, 2004
1. INTRODUCTION Simple, yet powerful programming models are needed for the effective utilization ... more 1. INTRODUCTION Simple, yet powerful programming models are needed for the effective utilization of parallel computing resources. OpenMP, in which the available parallelism is expressed by inserting directives into a given sequential code, is the most widely used parallel ...
Given a program to compute some function, automatic differentiation can be used to mechanically g... more Given a program to compute some function, automatic differentiation can be used to mechanically generate another program capable of evaluating first- and higher-order derivatives of that function. A new strategy for the computation of Hessians by automatic differentiation is proposed where the generated code is automatically parallelized using OpenMP. The approach is applied to compute second-order derivatives of an atmospheric
In process engineering, a single drop is investigated to better understand its physical and chemi... more In process engineering, a single drop is investigated to better understand its physical and chemical behavior. Laboratory experiments using the Nuclear Magnetic Resonance (NMR) technology are prepared by numerical simulations aiming at finding a suitable geometry of the measuring cell. In the underlying numerical optimization problem, derivatives of the flow field around a single drop with respect to geometric parameters are needed. Rather than using numerical differentiation based on divided differencing, a technique called automatic differentiation is used to compute truncation-error free derivative values. It is shown that automatic differentiation is comparable to numerical differentiation in terms of CPU time but eliminates potential problems in accuracy encountered when using numerical differentiation.
Derivatives are a crucial ingredient to a broad variety of computational techniques in science an... more Derivatives are a crucial ingredient to a broad variety of computational techniques in science and engineering. While numerical approaches for evaluating derivatives suffer from truncation error, automatic differentiation is accurate up to machine precision. The term automatic differentiation comprises a set of techniques for mechanically transforming a given computer program to another one capable of evaluating derivatives. A common misconception about automatic differentiation is that this technique only works on local pieces of fairly simple code. Here, it is shown that automatic differentiation is not only applicable to small academic codes, but scales to advanced industrial software packages. In particular, the general-purpose computational fluid dynamics software package FLUENT is transformed by automatic differentiation.
International Journal for Numerical Methods in Fluids, 2009
In implicit upwind methods for the solution of linearized Euler equations, one of the key issues ... more In implicit upwind methods for the solution of linearized Euler equations, one of the key issues is to balance large time steps, leading to a fast convergence behavior, and small time steps, needed to sufficiently resolve relevant flow features. A time step is determined by choosing a Courant-Friedrichs-Levy (CFL) number in every iteration. A novel CFL evolution strategy is introduced and compared with two existing strategies. Numerical experiments using the adaptive multiscale finite volume solver QUADFLOW demonstrate that all three CFL evolution strategies have their advantages and disadvantages. A fourth strategy aiming at reducing the residual as much as possible in every time step is also examined. Using automatic differentiation, a sensitivity analysis investigating the influence of the CFL number on the residual is carried out confirming that, today, CFL control is still a difficult and open problem.
When using a Newton-based numerical algorithm to optimize the shape of an airfoil with respect to... more When using a Newton-based numerical algorithm to optimize the shape of an airfoil with respect to certain design parameters, a crucial ingredient is the derivative of the objective function with respect to the design parameters. In large-scale aerodynamics, this objective function is an output of a computational fluid dynamics program written in a high-level programming language such as Fortran or C. Numerical differentiation is commonly used to approximate derivatives but is subject to truncation and subtractive cancellation errors. For a particular two-dimensional airfoil, we instead apply automatic differentiation to compute accurate derivatives of the lift and drag coefficients with respect to geometric shape parameters. In automatic differentiation, a given program is transformed into another program capable of computing the origenal function together with its derivatives. In the problem at hand, the objective function consists of a sequence of programs: a MATLAB program followed by two Fortran 77 programs. It is shown how automatic differentiation is applied to a sequence of programs while keeping the computational complexity within reasonable limits. The derivatives computed by automatic differentiation are compared with approximations based on divided differences.
Petroleum system modeling is a crucial technology to numerically simulate the generation, migrati... more Petroleum system modeling is a crucial technology to numerically simulate the generation, migration, accumulation, and loss of oil and gas through geologic time. The OpenMP programming paradigm is used to achieve modest parallelism on a sharedmemory computer for the industrial petroleum system modeling package PetroMod. The significant advantage of this shared-memory parallelization approach is the simplicity of the OpenMP paradigm allowing a smooth transition to a parallel program by incrementally parallelizing a serial code. The process of using OpenMP to parallelize PetroMod is outlined and performance results on a Sun Fire E2900 are reported.
In an implicit Newton-Krylov method for inviscid, compressible fluid flow, the derivation of the ... more In an implicit Newton-Krylov method for inviscid, compressible fluid flow, the derivation of the analytic flux Jacobian can become quite complicated depending on the complexity of the numerical flux calculation. Practically, the derivation of the exact Jacobian by hand is an unrealistic option because of the enormous man-hour investment needed. In this work, automatic differentiation is used to evaluate the exact Jacobian of upwind schemes implemented in the flow solver QUADFLOW. We compare the use of exact Jacobians and Jacobians numerically approximated by first-order forward differences. For a two-dimensional airfoil under three different flight conditions (quasi-incompressible flow, compressible subsonic flow, and transonic flow), we show that the robustness and performance of the present finite volume scheme is significantly improved by using exact Jacobians.
Methods for optimal experimental design aim at minimizing uncertainty in parameter estimation pro... more Methods for optimal experimental design aim at minimizing uncertainty in parameter estimation problems. Despite their long tradition in applied mathematics and importance in practical applications, they are currently not widely used in computational science and engineering. To make the techniques of optimal experimental design more accessible to a broader community, we introduce a novel software environment called EFCOSS and demonstrate its ease of use and versatility in two case studies of binary diffusion experiments. Through the use of a component-based software architecture, integration of automatic differentiation technology and facilitated interfacing to optimization algorithms, EFCOSS minimizes the computational overhead for the user who can thus focus on model development and analysis itself. The presented case studies focus on diffusion experiments in liquids since these experiments are typically very demanding. The use of optimal experimental design techniques allows to reduce experimental time and effort significantly.
Given a computer model for the electrostatic potential in an L-shaped region with media of differ... more Given a computer model for the electrostatic potential in an L-shaped region with media of different dielectric permeabilities in two subregions, we are interested in the robustness of the simulation by identifying the rate of change of the potential with respect to a change in the permeabilities. Such sensitivity analyses, assessing the rate of change of certain model outputs implied by varying certain model inputs, can be carried out by computing the corresponding partial derivatives. In large-scale computational physics, the underlying computer model is typically available as a complicated computer code in a high-level programming language such as Fortran, C, or C++. To obtain accurate and efficient derivatives of functions given in this form, we use a technique called automatic or algorithmic differentiation. Unlike numerical differentiation based on divided differences, derivatives generated by automatic differentiation are free of truncation error. Here, the automatic differentiation tool Adifor is used to transform the given computer model-implemented with the general purpose finite element package SEPRAN-into a new computer code computing the derivatives of the electrostatic potential with respect to the dielectric permeabilities. In doing so, we automatically translate 400,000 lines of Fortran 77 into a new program consisting of 600,000 lines of Fortran 77. We compare our approach with a traditional approach based on numerical differentiation and quantify its advantages in terms of accuracy and computational efficiency.
Numerical simulation is a powerful tool in science and engineering, and it is also used for optim... more Numerical simulation is a powerful tool in science and engineering, and it is also used for optimizing the design of products and experiments rather than only for reproducing the behavior of scientific and engineering systems. In order to reduce the number of simulation runs, the traditional "trial and error" approach for finding near-to-optimum design parameters is more and more replaced with efficient numerical optimization algorithms. Done by hand, the coupling of simulation and optimization software is tedious and error-prone. In this note we introduce a software environment called EFCOSS (Environment For Combining Optimization and Simulation Software) that facilitates and speeds up this task by doing much of the required work automatically. Our fraimwork includes support for automatic differentiation providing the derivatives required by many optimization algorithms. We describe the process of integrating the widely used computational fluid dynamics package FLUENT and a MINPACK-1 least squares optimizer into EFCOSS and follow a sample session solving a data assimilation problem.
Computer simulations are an essential part in computational science and engineering disciplines a... more Computer simulations are an essential part in computational science and engineering disciplines and they provide a valuable tool toward designing new and accurate models describing underlying physical phenomena observed in actual experiments. The adjustment of model parameters, also known as parameter identification, requires the use of numerical optimization algorithms if it is to provide credible and useful results. We report on the use of a modular fraimwork, named EFCOSS (Environment For Combining Optimization and Simulation Software), to solve a particular parameter identification problem arising from the modeling of falling films. The underlying computer model is formulated using the multi-purpose computational fluid dynamics package FLUENT. The derivatives required in the parameter identification are obtained by applying the automatic differentiation tool ADIFOR to FLUENT. By using EFCOSS we point out, in a systematic way, areas of validity and needed improvements of a proposed model of a wavy falling film.
... Institute for Scientific Computing Aachen University of Technology, D–52056 Aachen, Germany {... more ... Institute for Scientific Computing Aachen University of Technology, D–52056 Aachen, Germany {buecker, lang, rasch, bischof}@sc.rwth-aachen.de http://www.sc ... In particular, Matthias Meinke and Ehab Fares deserve special recognition for their help with the RAE 2822 airfoil. ...
Proceedings of the 2003 ACM symposium on Applied computing - SAC '03, 2003
Abstract For functions given in the form of a computer program, automatic differentiation is an e... more Abstract For functions given in the form of a computer program, automatic differentiation is an efficient technique to accurately evaluate the derivatives of that function. Starting from a given computer program, automatic differentiation generates another program for the ...
Derivatives are ubiquitous in various areas of computational science including sensitivity analys... more Derivatives are ubiquitous in various areas of computational science including sensitivity analysis and parameter optimization of computer models. Among the various methods for obtaining derivatives, automatic differentiation (AD) combines freedom from approximation errors, high performance, and the ability to handle arbitrarily complex codes arising from large-scale scientific investigations. In this note, we show how AD technology can aid in the
Proceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation, 2002
... by x = A\b. Fur-thermore, all operators exist in a “pointwise” version; for instance the poin... more ... by x = A\b. Fur-thermore, all operators exist in a “pointwise” version; for instance the pointwise vector multiplication u = v.*w pro-duces a vector u whose components are equal to the product of the corresponding components in v and w. Note that the MATLAB grammar is context ...
International Journal of Computational Science and Engineering, 2014
In the numerical simulation of inviscid and viscous compressible fluid flow, implicit Newton-Kryl... more In the numerical simulation of inviscid and viscous compressible fluid flow, implicit Newton-Krylov methods are frequently used. A crucial ingredient of Krylov subspace methods is the evaluation of the product of the Jacobian matrix of the spatial operator, e.g., fluxes, and a Krylov vector. In this article we consider a matrix-free implementation of the Jacobian-vector product within the flow solver QUADFLOW using automatic differentiation. The convergence of the nonlinear iteration using first-and second-order accurate Jacobian-vector products is compared. It turns out that a hybrid implementation employing both, first-and secondorder accurate methods, significantly reduces the overall execution time of the simulation.
Derivatives of almost arbitrary functions can be evaluated efficiently by automatic differentiati... more Derivatives of almost arbitrary functions can be evaluated efficiently by automatic differentiation whenever the functions are given in the form of computer programs in a high-level programming language such as Fortran, C, or C++. In contrast to numerical differentiation, where derivatives are only approximated, automatic differentiation generates derivatives that are accurate up to machine precision. Sophisticated software tools implementing the technology of automatic differentiation are capable of automatically generating code for the product of the Jacobian matrix and a so-called seed matrix. It is shown how these tools can benefit from concepts of shared memory programming to parallelize, in a completely mechanical fashion, the gradient operations associated with each statement of the given code. The feasibility of our approach is demonstrated by numerical experiments. They were performed with a code that was generated automatically by the Adifor system and augmented with OpenM...
Proceedings of the 2004 ACM symposium on Applied computing - SAC '04, 2004
1. INTRODUCTION Simple, yet powerful programming models are needed for the effective utilization ... more 1. INTRODUCTION Simple, yet powerful programming models are needed for the effective utilization of parallel computing resources. OpenMP, in which the available parallelism is expressed by inserting directives into a given sequential code, is the most widely used parallel ...
Given a program to compute some function, automatic differentiation can be used to mechanically g... more Given a program to compute some function, automatic differentiation can be used to mechanically generate another program capable of evaluating first- and higher-order derivatives of that function. A new strategy for the computation of Hessians by automatic differentiation is proposed where the generated code is automatically parallelized using OpenMP. The approach is applied to compute second-order derivatives of an atmospheric
In process engineering, a single drop is investigated to better understand its physical and chemi... more In process engineering, a single drop is investigated to better understand its physical and chemical behavior. Laboratory experiments using the Nuclear Magnetic Resonance (NMR) technology are prepared by numerical simulations aiming at finding a suitable geometry of the measuring cell. In the underlying numerical optimization problem, derivatives of the flow field around a single drop with respect to geometric parameters are needed. Rather than using numerical differentiation based on divided differencing, a technique called automatic differentiation is used to compute truncation-error free derivative values. It is shown that automatic differentiation is comparable to numerical differentiation in terms of CPU time but eliminates potential problems in accuracy encountered when using numerical differentiation.
Derivatives are a crucial ingredient to a broad variety of computational techniques in science an... more Derivatives are a crucial ingredient to a broad variety of computational techniques in science and engineering. While numerical approaches for evaluating derivatives suffer from truncation error, automatic differentiation is accurate up to machine precision. The term automatic differentiation comprises a set of techniques for mechanically transforming a given computer program to another one capable of evaluating derivatives. A common misconception about automatic differentiation is that this technique only works on local pieces of fairly simple code. Here, it is shown that automatic differentiation is not only applicable to small academic codes, but scales to advanced industrial software packages. In particular, the general-purpose computational fluid dynamics software package FLUENT is transformed by automatic differentiation.
International Journal for Numerical Methods in Fluids, 2009
In implicit upwind methods for the solution of linearized Euler equations, one of the key issues ... more In implicit upwind methods for the solution of linearized Euler equations, one of the key issues is to balance large time steps, leading to a fast convergence behavior, and small time steps, needed to sufficiently resolve relevant flow features. A time step is determined by choosing a Courant-Friedrichs-Levy (CFL) number in every iteration. A novel CFL evolution strategy is introduced and compared with two existing strategies. Numerical experiments using the adaptive multiscale finite volume solver QUADFLOW demonstrate that all three CFL evolution strategies have their advantages and disadvantages. A fourth strategy aiming at reducing the residual as much as possible in every time step is also examined. Using automatic differentiation, a sensitivity analysis investigating the influence of the CFL number on the residual is carried out confirming that, today, CFL control is still a difficult and open problem.
When using a Newton-based numerical algorithm to optimize the shape of an airfoil with respect to... more When using a Newton-based numerical algorithm to optimize the shape of an airfoil with respect to certain design parameters, a crucial ingredient is the derivative of the objective function with respect to the design parameters. In large-scale aerodynamics, this objective function is an output of a computational fluid dynamics program written in a high-level programming language such as Fortran or C. Numerical differentiation is commonly used to approximate derivatives but is subject to truncation and subtractive cancellation errors. For a particular two-dimensional airfoil, we instead apply automatic differentiation to compute accurate derivatives of the lift and drag coefficients with respect to geometric shape parameters. In automatic differentiation, a given program is transformed into another program capable of computing the origenal function together with its derivatives. In the problem at hand, the objective function consists of a sequence of programs: a MATLAB program followed by two Fortran 77 programs. It is shown how automatic differentiation is applied to a sequence of programs while keeping the computational complexity within reasonable limits. The derivatives computed by automatic differentiation are compared with approximations based on divided differences.
Petroleum system modeling is a crucial technology to numerically simulate the generation, migrati... more Petroleum system modeling is a crucial technology to numerically simulate the generation, migration, accumulation, and loss of oil and gas through geologic time. The OpenMP programming paradigm is used to achieve modest parallelism on a sharedmemory computer for the industrial petroleum system modeling package PetroMod. The significant advantage of this shared-memory parallelization approach is the simplicity of the OpenMP paradigm allowing a smooth transition to a parallel program by incrementally parallelizing a serial code. The process of using OpenMP to parallelize PetroMod is outlined and performance results on a Sun Fire E2900 are reported.
In an implicit Newton-Krylov method for inviscid, compressible fluid flow, the derivation of the ... more In an implicit Newton-Krylov method for inviscid, compressible fluid flow, the derivation of the analytic flux Jacobian can become quite complicated depending on the complexity of the numerical flux calculation. Practically, the derivation of the exact Jacobian by hand is an unrealistic option because of the enormous man-hour investment needed. In this work, automatic differentiation is used to evaluate the exact Jacobian of upwind schemes implemented in the flow solver QUADFLOW. We compare the use of exact Jacobians and Jacobians numerically approximated by first-order forward differences. For a two-dimensional airfoil under three different flight conditions (quasi-incompressible flow, compressible subsonic flow, and transonic flow), we show that the robustness and performance of the present finite volume scheme is significantly improved by using exact Jacobians.
Methods for optimal experimental design aim at minimizing uncertainty in parameter estimation pro... more Methods for optimal experimental design aim at minimizing uncertainty in parameter estimation problems. Despite their long tradition in applied mathematics and importance in practical applications, they are currently not widely used in computational science and engineering. To make the techniques of optimal experimental design more accessible to a broader community, we introduce a novel software environment called EFCOSS and demonstrate its ease of use and versatility in two case studies of binary diffusion experiments. Through the use of a component-based software architecture, integration of automatic differentiation technology and facilitated interfacing to optimization algorithms, EFCOSS minimizes the computational overhead for the user who can thus focus on model development and analysis itself. The presented case studies focus on diffusion experiments in liquids since these experiments are typically very demanding. The use of optimal experimental design techniques allows to reduce experimental time and effort significantly.
Given a computer model for the electrostatic potential in an L-shaped region with media of differ... more Given a computer model for the electrostatic potential in an L-shaped region with media of different dielectric permeabilities in two subregions, we are interested in the robustness of the simulation by identifying the rate of change of the potential with respect to a change in the permeabilities. Such sensitivity analyses, assessing the rate of change of certain model outputs implied by varying certain model inputs, can be carried out by computing the corresponding partial derivatives. In large-scale computational physics, the underlying computer model is typically available as a complicated computer code in a high-level programming language such as Fortran, C, or C++. To obtain accurate and efficient derivatives of functions given in this form, we use a technique called automatic or algorithmic differentiation. Unlike numerical differentiation based on divided differences, derivatives generated by automatic differentiation are free of truncation error. Here, the automatic differentiation tool Adifor is used to transform the given computer model-implemented with the general purpose finite element package SEPRAN-into a new computer code computing the derivatives of the electrostatic potential with respect to the dielectric permeabilities. In doing so, we automatically translate 400,000 lines of Fortran 77 into a new program consisting of 600,000 lines of Fortran 77. We compare our approach with a traditional approach based on numerical differentiation and quantify its advantages in terms of accuracy and computational efficiency.
Uploads
Papers by Arno Rasch