PETROPHYSICS, VOL. 61, NO. 1 (FEBRUARY 2020); PAGES 86–98; 11 FIGURES; 4 TABLES. DOI: 10.

30632/PJV61N1-2020a3

A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems1

Yuchen Jin2, Qiuyang Shen2, Xuqing Wu2, Jiefu Chen2, and Yueqin Huang3

ABSTRACT
Geosteering inversion, which can be viewed as a providing a fast, accurate surrogate to solve the inverse
nonlinear inverse problem, is an important technique problem. Particularly, leveraged by the forward physical
used by directional drilling. Traditional methods that rely model and 1D convolutional neural network (1D-CNN),
on iterative procedures and regularization are sensitive the proposed method provides more reliable solutions
to the selection of initial values and can be slow due to to the inverse problem with improved performance. In
convergence issues. In industrial applications, a lookup addition, a new physics-driven loss function is introduced
table is used to produce fast predictions. However, to accommodate both the model mis¿t and the data mis¿t.
performance is not guaranteed by this approach due to Our experiments demonstrate the effectiveness of our
the limitation of the hardware. In this paper, we propose method.
a novel physics-driven deep-learning framework for

INTRODUCTION and phase shift. To reconstruct the resistivity pro¿le,

the interpretation of multichannel measurements is
Accurate interpretation of geosteering data obtained indispensable. Since the EM wave can be synthesized
by an electromagnetic (EM) logging-while-drilling by simulation using a physical forward model , the
(LWD) tool in high-angle and horizontal wells is critical interpretation can be viewed as an inverse problem, ,
for formation evaluation and well placement. Among all where we need to reconstruct the formation properties from
common sensing technologies used for geosteering, which the observed measurements.
include gamma-ray, neutron density, and EM resistivity,
resistivity measurement is preferred since it provides large Conventional Methods
depth of investigation (DoI). An EM resistivity tool takes the The inverse problem has been well studied by both
measurement of earth’s electrical properties by propagating academia and the oil and gas industry. The lookup table is
the electromagnetic wave into the formation. Equipped with one of the methods used in industrial practices. The lookup
tilted or transverse antennas, the tool can distinguish the table reÀects a one-to-one mapping relationship between the
direction of the boundary-crossing as well as the formation measurements and the resistivity pro¿le calculated via the
anisotropy (Edwards, 2000; Li et al., 2005; Omeragic et al., forward model . When new measurements are observed,
2006). In recent years, the DoI of azimuthal resistivity tools an exhaustive search of the table would be performed to
can be extended to over 100 feet from the borehole owing to locate the best matching pro¿le (Tchakarov et al., 2016;
the advent of ultradeep azimuthal resistivity tools (Seydoux Itskovich and Nikitenko, 2017). A signi¿cant drawback of
et al., 2014, Ezioba and Denichou, 2014; Wang et al., 2018). the lookup table method is its lack of scalability. The size
The new tools have similar designs to the conventional ones, of the lookup table grows exponentially as the dimension
which are con¿gured with multiple transmitter-receiver of parameters of the formation resistivity pro¿le increases.
pairs working at multiple frequencies. Figure 1 demonstrates Limited by the storage space, the lookup table method is
a schematic diagram of an azimuthal resistivity tool. It unable to provide accurate inversion results for complicated
expands the application of geosteering to the reservoir-scale. formation structures.
Interpretation of the azimuthal resistivity measurements, Iterative methods have also been used to solve the
however, is not straightforward. The induced receiver inverse problem. The so-called deterministic inversion
captures the EM wave and records the signal attenuation iteratively solves an optimization problem by minimizing

Manuscript received by the Editor August 19, 2019; revised manuscript received November 11, 2019; manuscript accepted November 24, 2019.
2
University of Houston, N308 Engineering Bldg 1, 4726 Calhoun Rd, Houston, TX 77204-4005; yjin4@uh.edu; qshen4@uh.edu;
xwu8@central.uh.edu; jchen84@uh.edu
3
Cyentech Consulting LLC, General Services Building (GSB), 4211 Elgin St., #140, Houston, TX, 77204-1002; yueqinhuang@cyentech.com

86 PETROPHYSICS February 2020

 A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Fig. 1— Schematic of an azimuthal resistivity tool. T1, T2, T3 and T4 are z-direction transmitting antennas, T5 and T6 are x-direction transmitting
antennas. R1 and R2 are z-direction receiving antennas, and R3 and R4 are x-direction receiving antennas

the difference between the observed measurements and inversion (Ramuhalli et al., 2002), eddy current inversion
synthesized responses obtained via a forward model (Behun et al., 2018), radar backscattering model inversion
built upon rigorous physics. A variety of iterative algorithms (Farah et al., 2006), spectral optical model inversion (Malkiel
is available, including the gradient descent method, et al., 2018) and geosteering earth model inversion (Xu et
Gauss-Newton method, and the Levenberg-Marquardt al., 2018). Instead of ¿nding a numerical solution through
algorithm (LMA) (Levenberg, 1944; Marquardt, 1963). the physical modeling, these implementations try to learn an
The deterministic methods are usually gradient-based and inverse mapping driven by the historical data. The feasibility
searching for the best earth model is ef¿cient. However, due of learning a physical model via the data-driven approach
to the high nonlinearity of the forward model, deterministic has been supported by Lerer et al. (2016), who propose a
methods suffer from the local minima problem and are segmentation network to predict the movement of block
very sensitive to the initial values. In addition, additional towers. In Fragkiadaki et al. (2016), the visualization of a
regularizing terms are required since inverse problems are billboard game and the agent’s applied forces are input to a
typically ill-posed (Key, 2009; Thiel et al., 2018). recurrent neural network for predicting the movement of the
Stochastic inversion is another popular approach objects.
for solving inverse problems. The stochastic global Different from the aforementioned approaches, the
optimization is governed by the Bayesian theorem. Under inverse mapping can be learned by combining the deep-
the Bayesian framework, the observation model or the so- learning framework with conventional iterative algorithms.
called likelihood is usually built upon the forward model Adler and Oktem (2017) proposed that gradients produced
and some knowledge about the errors (e.g. measurement by the gradient descent algorithm could be adjusted by a
noise). The Markov chain Monte Carlo (MCMC) sampling well-trained arti¿cial neural network. Huang et al. (2018)
method is used to sample the posterior distribution of model proposed a similar workÀow for full waveform inversion
parameters given the measurements. Stochastic inversion (FWI). In this work, the DNN predicts the update step for
is computationally expensive. Although some efforts have the seismic velocity model from the gradient calculated
been made to improve the sampling ef¿ciency through the from the forward modeling. This work aims at improving
multichain sampling (Shen et al., 2017), hybrid MCMC the updating direction and avoiding the local minima during
(Shen et al., 2018b), and nonparametric sampling (Shen the inversion. Multiple works also explored the idea of
et al., 2018a), the overall ef¿ciency is much lower than applying physical constraints to the DNN by using analytical
deterministic methods. and differential models. For example, the inverse discrete
cosine transform is used to construct the loss function by
Deep-Learning Approaches Johnston et al. (2017). Jaderberg et al. (2015) formulated the
In recent years, deep-learning approaches have been af¿ne transform as an analytical function and constructed a
used to solve inverse problems. The basic idea is to train spatial transformer network. Whitening and coloring feature
a deep neural network (DNN) to map measurements into transforms are used to transfer the style features extracted
the model parameters. In this case, the network serves as by the auto-encoder in Li et al. (2017. Wu et al. (2017)
a surrogate for the inverse mapping . Examples of proposed a novel data mis¿t function for training an image
applications include geoacoustic model inversion (Benson segmentation network by including a differential physical
et al., 1998), EM nondestructive evaluation (NDE) signal model. Zhang et al. (2019) built a physics-constrained CNN
for predicting seismic responses. In this work, the network is

February 2020 PETROPHYSICS 87

 Jin et al.

guided by a data loss and a physical loss simultaneously. The model at every single logging station. Figure 2 presents a
physical loss is formulated by the physical constraints of a schematic diagram of a 1D layered earth model, which
dynamic system. The introduction of the physical constraint assumes that the formation property remains constant within
mitigates the over¿tting problem and reduces the dependency each horizontal layer and all layers are stacked vertically.
on a large training set. Jiang and Fan (2019) constructed a
generative network for con¿guring metasurfaces, which are The 1D layered model is de¿ned by the electrical
subwavelength-structured arti¿cial media that can shape and resistivity, , of each layer, and the true vertical depth (TVD),
localize EM waves in unique ways. The network is trained , of the boundary between adjacent and layers.
to learn the relationship between device geometry and We hereby parameterize the earth model of layers by a
optical response. The training process is driven by the data vector as follows:
mis¿t, which measures the difference between the ground-
truth and the synthetic output obtained through the forward (1)
simulation. The back-propagation of the forward simulated
is estimated by the adjoint state method. Given a forward model, the synthetic measurements can
Similarly, we have included the data mis¿t as a part be obtained by:
of the loss function. Our forward model is differentiable
but requires numerical estimation of the Jacobian matrix. (2)
The technique developed in this work can be applied to an
arbitrary differentiable forward model. Where indicates the forward model. The inverse
modeling can be written as:
METHODOLOGY
(3)
Geosteering modeling and inversion requires a time-
sensitive workÀow to satisfy the real-time data interpretation Where de¿nes the inverse mapping and is the
and decision-making. Since 3D modeling is computationally observed measurements.
expensive, the formation structure is parameterized as a 1D
In this article, the azimuthal resistivity LWD tool we
synthesize adopts the design shown in Fig. 1. The receivers
capture the induced EM ¿eld and record the voltage change.
A full mutual inductance tensor with nine components
can be calculated. Hence, different types of measurements
can be further derived through the captured response. We
simulate the EM response in 1D layered space by using
multiple pairs of triaxial antennas. In detail, the channels
are divided by different frequencies and T-R spacings. The
lowest frequency is 1 kHz with the longest spacing of 160
ft. Given a 1D formation model, in our synthetic case, 92
measurements are generated.
The relationship among the measurement, the source,
and the medium is essentially governed by physics. An
analytical inversion process relies on the forward model to
guide its search for solutions. However, the solution could be
suboptimal due to local minima problems, noise, and other
unknown interference. In this paper, we propose a novel
physics-driven DNN (PhyDNN) framework for solving
Fig. 2—An illustration of the underground formation. Divided by several the geosteering inverse problem. The notations used in the
homogeneous layers, the underground formation could be described by
resistivities of each layer and boundary depth between each two layers. following part are outlined in Table 1. Figure 3 demonstrates
the framework diagram of the PhyDNN.

88 PETROPHYSICS February 2020

 A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Table 1—List of Notations for the Proposed Methods

Fig. 3—An overview of the framework for the proposed PhyDNN.

Modern Deep Neural Network (DNN) Structure nine 1D convolutional layers, ¿ve average pooling layers,
In deep-learning approaches, a DNN serves as a and one fully-connected layer. The observed measurements
surrogate for the whole iterative optimization. Generally, a are normalized before fed into the convolutional layers. The
DNN could be viewed as a stack of no-linear feed-forward output of the fully connected layer is sliced into two parts
operators , i.e. “layers”. A DNN with layers could be and scaled into the resistivity domain and the TVD boundary
formulated as: domain respectively. The whole network contains 1.83 × 106
trainable parameters. Each network layer is a composition
(4) of convolution, normalization, and activation and can be
formulated as:
In our work, the DNN is adapted from the VGG16
network (Simonyan and Zisserman, 2015). The detailed (5)
network layout is shown in Fig. 4. Our network consists of

February 2020 PETROPHYSICS 89

 Jin et al.

Deep neural network

Fig. 4—Illustration of the architecture details of the DNN adapted from the VGG16.

where is a learnable 1D convolutional kernel. are where is a sample that is randomly selected from the
mean and standard deviation estimated from . In the training set. The “model mis¿t” in Eq. 7 measures the
following, all learnable network parameters are represented difference between the predicted earth model and the true
by . Unique features included earth model . Once the model is well trained, we can use
in this network design include: the feed-forward network prediction to estimate
x 1D convolutional layers for 1D inputs (Van Den Oord the unknown earth model for a new group of measurements
et al., 2016; Ince et al., 2016). .
x Instance normalization (Ulyanov et al., 2016; He et In a data-driven DNN, the convergence of the network
al. 2016; Zhang et al.,2018). is guided by the distribution of the parameter of the earth
x PReLU (He et al., 2015) as the activation function. model alone. Due to the randomness in selecting the data
batch and the nonlinearity of the governing physics, the
The last layer of this network is a scaling layer, which descending path will be long and Àuctuate. Therefore, a data-
is de¿ned as driven DNN could either oversmooth or over¿t the inverse
mapping function between the measurement and the earth
model. A differentiable forward model, on the other hand,
(6) can be used to regulate the back-propagation process.

Physics-Driven Method and Data Mis¿t

where is the output of the hidden layer . and The idea of the PhyDNN is inspired by the auto-encoder
are the upper bound and lower bound of the prediction structure, which has been applied for unsupervised learning
respectively. Both and are learnable vectors. The scaling (Baldi, 2012). Generally, the auto-encoder comprises two
layer serves as a constraint to enforce the prediction’s value parts, an encoder and a decoder. The encoder maps the input
range, i.e. . into the encoded space and the decoder would recover the
The DNN is designed to learn the mapping function input data of the encoder. In most cases, the encoder and
between the observed measurements (input of the network) the decoder are designed in a symmetric structure. In Fig.
and predicted earth model (output of the network). 3, the 1D convolutional DNN serves as an encoder, which
can be viewed as a surrogate for the inverse mapping .
Data-Driven Method and Model Mis¿t The decoder is replaced by the prede¿ned physical forward-
Most deep-learning methods for solving an inverse modeling function . The objective function of the
problem are data driven. In those methods, the deep network PhyDNN training is de¿ned as:
learns the inverse mapping from the statistical
distribution of the dataset, which is generated by following
physical constraints. During the training process, the network (8)
learns these constraints from the dataset empirically. The
objective function for training used by a data-driven network
is as follows: According to Eq. 8, the loss function consists of two
terms, model mis¿t and data mis¿t . Similar to
(7) the traditional iterative optimization, the data mis¿t, ,
quanti¿es the difference between observed measurements

90 PETROPHYSICS February 2020

 A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

and synthetic tool responses obtained via a forward function Earth Model Description and Dataset
for a given earth model. Unlike a data-driven DNN, In the experiment, we assume that the underground
whose convergence is guided by the model mis¿t alone, a formation can be represented by a three-layer model of
differentiable forward model is used by PhyDNN to regulate ¿ve parameters, three resistivity values (R1, R2, R3) and the
the back-propagation process by minimizing the data mis¿t. distance to the upper and lower boundaries (Dup and Ddn). To
A weighting factor, , is added to balance the two simplify the problem, the relative dip angle of the logging
mis¿t terms. Compared to the data-driven DNN and iterative tool is assumed to be ¿xed. Figure 5 shows the schematic
optimization, PhyDNN will deliver an accurate inversion graph of the three-layer model.
result with high measurement compliance and more success
in avoiding local minima problems.
The implementation for PhyDNN requires
backpropagating the gradients from the data mis¿t. Similar
to deterministic methods, we could formulate the ¿rst-order
gradient of data mis¿t by using the chain rule:

(9) Fig. 5—A schematic diagram for a 3-layer earth model.

The training data set is sampled from a lookup table

generated by using the forward model function π(.) Each
entry of the lookup table (m, d) is called a “point”, where
For forward models without analytical form of m = {R1, R2, R3 Dup, Ddn}is the model parameter and d the
derivatives, the Jacobian matrix in Eq. 9 is estimated corresponding tool responses (measurements). The range of
numerically as: the model parameters is de¿ned as: R1, R2, R3 ‫( א‬10–1, 102)
[ȍ . m], Dup ‫–( א‬25.0, 0) [ft] and Ddn ‫( א‬0, 25.0) [ft]. We ¿xed
(10) the dip angle at 90° in all cases. Such a con¿guration ensures
that (1) Dup< Ddn, (2) the logging tool is kept horizontally
EXPERIMENTS in the middle layer, and (3) two boundaries are in the
sensing range of the logging tool. To sample the resistivity
In this part, we train and test the network to demonstrate and boundary values, we divide the logarithmic resistivity
its effectiveness in solving the geosteering inverse problem. range into 16 intervals and the depth range into 24 intervals.
The experiment is deployed on Python 3.5+ and TensorFlow Therefore, we have 163 × 242, i.e., 2.36 × 106, points in the
r1.13+. The forward-model function is implemented in C++ lookup table. For each measurement, the forward model
with a Python-C-API wrapper. We use OpenMP to enable outputs 92 values. The total size of the lookup table for
the parallel computing for the forward model to accelerate training is approximately 1.61 GB.
the CPU computation. During the training and testing To minimize the training workload and demonstrate the
phases, the whole program, except the implementation of the effectiveness of the model, we selected a three-layer model
forward-model function, can be run on both CPU and GPU. and keep the tool horizontal in the middle layer. During the
To accelerate the implementation cycle and reduce experiment, 90% of the dataset is used for training and the
coding efforts, we provide an open source library: Modern rest 10% is reserved for validation.
Deep Network Toolkits (MDNT) (Jin, 2018). MDNT can
wrap an arbitrary external forward-model function as a Training
TensorFlow operator. The modern convolutional layers and The training is performed on a NVIDIA DGX
the scaling layer are also implemented in MDNT. workstation, equipped with Intel Xeon E5-2698 v4 CPU
(20-core) and 4 Tesla V100 GPUs. Only one GPU is used

February 2020 PETROPHYSICS 91

 Jin et al.

for training the model. The GPU has 5,120 CUDA cores, is needed for inversion. The end-to-end pretrained feed-
640 tensor cores, 32 GB GPU RAM and 900 GB/s RAM forward network takes measurements as the input and outputs
bandwidth. the predicted model parameters as the inversion result. In
The network is trained for a total of 5 epochs (Goodfellow order to validate the ef¿ciency of the proposed approach,
et al., 2016). At each step, we randomly select 64 pairs of (m, we compare the inversion performance by using PhyDNN,
d) from the training set as a minibatch during the batchwise data-driven DNN, and the lookup-table methods. We use
optimization. We train the network with ADAM optimizer the same lookup table as the training and no interpolation is
(Kingma and Ba, 2015). During the ¿rst 2 epochs, the DNN performed for the table search.
is trained exclusively by minimizing ࣦml alone. Figure 7 shows the inversion result by applying different
The DNN is then further optimized by minimizing both methods to a case where the logging tool locates in a layer
ࣦml and ࣦdl in the next 3 epochs. As a comparison, we also with high resistivity. The green line represents the path of the
trained a data-driven DNN by minimizing the model mis¿t LWD tool. Figure 8 compares the measurement discrepancy
loss ࣦml for 5 epochs. The training loss is measured by the between observed and synthetic values for different
mean squared error (MSE). inversion approaches. Figure 9 shows two more examples
Figure 6a shows that the training losses with regard with different resistivity distributions. In all cases, PhyDNN
to the model mis¿t are statistically the same for both data- delivers the best inversion performance with regard to both
driven DNN and PhyDNN. However, compared to the model accuracy and measurement compliance. Figure 10
data-driven DNN, PhyDNN has a much smaller training compares model mis¿t and data mis¿t by averaging testing
error with regard to the data mis¿t. Figure 6 also compares results for all three approaches and the result shows that
training loss with validation errors, which indicates that our (1) the lookup-table method has the highest model and data
network training has converged. mis¿ts on average, (2) the prediction from the data-driven
network returns small model mis¿t but high data mis¿t, and
Testing (3) PhyDNN has the lowest overall mis¿t.
Once the network is well trained, only the encoder part

(a) (b)
Fig. 6—Training and validation curves for both PhyDNN and data-driven DNN. (a) The curves denoting model mis¿t (ࣦml) for both data-driven DNN and
PhyDNN. Both approaches have similar training and validation errors with regard to the model mis¿t. (b) The curves denoting data mis¿t (ࣦdl) for both
data-driven DNN, and PhyDNN. Compared to data-driven DNN, PhyDNN has much lower training and validation errors with regard to the data mis¿t.

92 PETROPHYSICS February 2020

 A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Fig. 7—A three-layer case by using different inversion methods. The x-axis represents the distance along the horizontal direction. z-axis is along the
vertical direction. The green line represents the trajectory of the wellbore, and different resistivities are represented by different colors.

(a) Lookup Table (b) Data-Driven DNN (c) Proposed PhyDNN

Fig. 8—Comparison between predicted measurements and ground-truth measurements with different methods. In each sub¿gure, the axis represents
the distance along the horizontal direction and we pick ¿ve representative measurements.

February 2020 PETROPHYSICS 93

 Jin et al.

(c)
Fig. 9—Two inversion examples using PhyDNN. Numerical tests show that PhyDNN works well for a three-layer model with different resistivity
distributions.

DISCUSSION

Evaluation of the Computational Ef¿ciency

In this section, we evaluate the computational ef¿ciency
of three different inversion methods: lookup table, PhyDNN,
and LMA, an iterative deterministic algorithm. Both the
time and memory consumption are taken into consideration.
The time consumption is the elapsed time consumed by the
evaluated method. The memory consumption reÀects the
required memory for running the method. Table 2 and Table
3 list the time consumption and the memory consumption
respectively.
Table 4 shows the average inversion accuracy for
the different methods. The overall inversion accuracy of
PhyDNN is comparable to LMA. In practice, the PhyDNN
Fig. 10—A comparison of data mis¿ts and model mis¿ts for all testing result can be used as the initial guess for the deterministic
cases by using different inversion methods. method, which will help the system converge much faster.

Table 2— Time Consumption for Interpreting 80 Logging Points

10K and 2M indicate different lookup-table sizes.

Table 3— Memory Consumption for Interpreting 80 Logging Points

10K and 2M indicate different lookup-table sizes.

94 PETROPHYSICS February 2020

 A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Table 4—Comparison of Average Data and Model Mis¿ts for Using Different Inversion Methods

Sensitivity Analysis the regularization method. NR-PhyDNN is preferred for

In order to improve and validate the robustness of the interpreting noisy ¿eld data.
model, we added noise to the input of the network during
the training. As Goodfellow et al. (2016) and Loshchilov CONCLUSIONS
and Hutter (2019) pointed out, noise injection is an effective
data augmentation method and serves as a replacement of In this work, we proposed a novel physics-driven DNN
the traditional Tikhonov regularization (Bishop, 1995). to solve the geosteering inverse problem. The proposed
In this test, we retrained both PhyDNN and data-driven PhyDNN is implemented by combining the DNN with a
DNN with the same training set but replaced the network forward-model function that follows the physics. Compared
input with d + ‫ܭ‬, where d is the observed measurements to other methods, we conclude that:
and ‫ ܭ‬is white noise. ‫ ܭ‬follows a normal distribution ࣨ (0, x PhyDNN is an ef¿cient method with high inversion
ı where the standard deviation (std.) ı is also a random accuracy and low training losses.
variable and sampled from a uniform distribution U (0, 0.4). x PhyDNN is computationally cheap compared to
The “noise-robust” version of PhyDNN and data-driven iterative optimization or lookup-table solutions.
DNN is denoted as NR-PhyDNN and NR-data-driven DNN. x PhyDNN is robust against noisy measurement after
Figure 11 concludes the test result with noisy measurements. training with data augmentation.
NR-PhyDNN achieved low model mis¿t and data mis¿t x The deep structure of PhyDNN prevents the
values on all noise levels. NR-data-driven DNN, however, under¿tting problem encountered by a shallow
performed worse than plain PhyDNN when the noise level structure and provides a scalable solution for more
is low. challenging inverse problems with complicated
In short, the above experiments show that PhyDNN is formation structures.
more robust after augmenting the training processing with

(a) (b)

Fig. 11—Comparison of data mis¿ts and model mis¿ts between networks trained with and without noise augmentation. (a) Model mis¿t vs. noise level
(x-axis). As the noise level increases, NR-PhyDNN and NR-data-driven DNN has much lower model mis¿ts. (b) Data mis¿t vs. noise level (x-axis). As
the noise level increases, both NR-PhyDNN and NR-data-driven DNN deliver lower data mis¿ts than corresponding networks trained without noise
augmentation. A plain PhyDNN shows good resistance to the noise when the noise level is low.

February 2020 PETROPHYSICS 95

 Jin et al.

x PhyDNN has the potential to enable downhole ࣠ (.) = a physical forward model
computing for an LWD tool. J= the Jacobian matrix
ࣦml, ࣦdl = the model mis¿t and the data mis¿t
ACKNOWLEDGMENTS m= a parameterized earth model
m= a predicted earth model
This material is based upon work supported by the wj = the kernel parameter of the j th layer
U.S. Department of Energy, Of¿ce of Science, and Of¿ce ‫=׋‬ a scaling layer
of Advanced Science Computing Research, under Award —j, ıj = the average and the standard error of the j th layer
Numbers DE-SC0017033. {Į, ȕ}= a weighting factor for mis¿ts
= an operator for convolution
DISCLAIMER = an operator for function composition
dl
= an operator for the data-based gradient
This report was prepared as an account of work Ȗj , ȕj = the scaling factors for the j th layer
sponsored by an agency of the United States Government.
Neither the United States Government nor any agency REFERENCES
thereof, nor any of their employees, make any warranty,
express or implied, or assumes any legal liability or Adler, J., and Öktem, O., 2017, Solving Ill-Posed Inverse Problems
responsibility for the accuracy, completeness, or usefulness Using Iterative Deep Neural Networks, Inverse Problems,
of any information, apparatus, product, or process disclosed, 33(12), 124007. DOI: 10.1088/1361-6420/aa9581.
or represents that its use would not infringe privately owned Baldi, P., 2012, Autoencoders, Unsupervised Learning, and Deep
rights. Reference herein to any speci¿c commercial product, Architectures, Journal of Machine Learning Research,
process, or service by trade name, trademark, manufacturer, Workshop and Conference Proceedings, 27, 37–49. http://
proceedings.mlr.press/v27/baldi12a/baldi12a.pdf. Accessed
or otherwise does not necessarily constitute or imply its
January 3, 2020.
endorsement, recommendation, or favoring by the United Behun, L., Smetana, M., and Capova, K., 2018, Comparison of
States Government or any agency thereof. The views and Detection Abilities Between Fluxgate and GMR Magnetometer
opinions of authors expressed herein do not necessarily in Inverse ECT of Deep Lying Cracks, 2018 ELEKTRO, 1–4,
state or reÀect those of the United States Government or any DOI: 10.1109/ELEKTRO.2018.8398332.
agency thereof. Benson, J., Chapman, N.R., and Antoniou, A., 1998, Geoacoustic
Model Inversion With Arti¿cial Neural Networks,
NOMENCLATURE Proceedings, 1998 IEEE Symposium on Advances in Digital
Abbreviations Filtering and Signal Processing, , 121–125, DOI: 10.1109/
ADFSP.1998.685708.
CNN = convolutional neural network
Bishop, C.M., 1995, Training With Noise is Equivalent to Tikhonov
DNN = deep neural network Regularization, Neural Computation, 7(1), 108–116. DOI:
DoI = depth of investigation 10.1162/neco.1995.7.1.108.
EM = electromagnetic Edwards, J., 2000, Geosteering Examples Using Modeled 2-MHz
FWI = full waveform inversion LWD Response in the Presence of Anisotropy, Paper N,
LMA= Levenberg-Marquardt algorithm Transactions, SPWLA 41st Annual Logging Symposium,
LWD = logging while drilling Dallas, Texas, USA, 4–7 June.
MCMC = Markov chain Monte Carlo Ezioba, U., and Denichou, J.-M., 2014, Mapping-While-Drilling
MDNT = modern deep network toolkits system improves well placement and ¿eld development,
MSE = mean squared error Journal of Petroleum Technology, 66(8), 32–35. DOI:
10.2118/0814-0032-JPT.
NDE = nondestructive evaluation
Farah, L.B., Farah, I.R., Bennaceur, R., Belhadj, Z., and Boussema,
PhyDNN = physics-driven DNN M.R., 2006, A Neural Network Approach for the Inversion
TVD = true vertical depth of Multi-Scale Roughness Parameters and Soil Moisture,
Proceedings, 2006 2nd Information and Communication
Symbols Technologies (ICTTA’06), 406–411, DOI: 10.1109/
Aj = the jth layer of the DNN ICTTA.2006.1684404.
d = a measurement Fragkiadaki, K., Agrawal, P., Levine, S., and Malik, J., 2016,
d = a synthetic measurement produced by the Learning Visual Predictive Models of Physics for Playing
forward model Billiards, presented at the 4th International Conference on
ࣞĬ = a DNN with tunable parameter Ĭ Learning Representations (ICLR 2016). https://arxiv.org/
ॱ [.] = an operator for expectation pdf/1511.07404.pdf. Accessed January 5, 2020.

96 PETROPHYSICS February 2020

 A Physics-Driven Deep-Learning Network for Solving Nonlinear Inverse Problems

Goodfellow, I., Bengio, Y., and Courville, A., 2016, Deep Li, Q., Omeragic, D., Chou, L., Yang, L., Duong, K., Smits, J.,
Learning. MIT Press. ISBN: 978-0262035613. http://www. Yang, J., Lau, T., Liu, C., Dworak, R., Dreuillault, V., and
deeplearningbook.org/. Accessed January 5, 2020. Ye., H., 2005, New Directional Electromagnetic Tool for
He, K., Zhang, X., Ren, S., and Sun, J., 2015, Delving Deep Into Proactive Geosteering and Accurate Formation Evaluation
Recti¿ers: Surpassing Human-Level Performance on Imagenet While Drilling, Paper UU, Transactions, SPWLA 46th Annual
Classi¿cation, Proceedings, 2015 IEEE international Logging Symposium, New Orleans, Louisiana, USA, 26–29
Conference On Computer Vision, 1026–1034, DOI: 10.1109/ June.
ICCV.2015.123. Li, Y., Fang, C., Yang, J., Wang, Z., Lu, X., and Yang, M.-H., 2017,
He, K., Zhang, X., Ren, S., and Sun, J., 2016, Identity Mappings Universal Style Transfer Via Feature Transforms, Advances
in Deep Residual Networks, Proceedings, European 14th in Neural Information Processing Systems, 386–396. https://
Conference on Computer Vision (ECCV 2016), 4, 630–645. arxiv.org/pdf/1705.08086.pdf. Accessed January 5, 2020.
DOI: 10.1007/978-3-319-46493-0_38. https://arxiv.org/ Loshchilov, I., and Hutter, F., 2019, Decoupled Weight Decay
pdf/1603.05027.pdf. Accessed January 5, 2020. Regularization, presented at the 7th International Conference
Huang, L., Polanco, M., and Clee, E.T., 2018, Initial Experiments on Learning Representations (ICLR 2019). https://arxiv.org/
on Improving Seismic Data Inversion with Deep Learning, pdf/1711.05101.pdf. Accessed January 5, 2020.
Proceedings, New York Scienti¿c Data Summit (NYSDS), 1–3, Malkiel, I., Mrejen, M., Nagler, A., Arieli, U., Wolf, L., and
DOI: 10.1109/NYSDS.2018.8538956. Suchowski, H., 2018, Deep Learning for the Design of Nano-
Ince, T., Kiranyaz, S., Eren, L., Askar, M., and Gabbouj, M., 2016, Photonic Structures, presented at the 2018 IEEE International
Real-Time Motor Fault Detection By 1-D Convolutional Conference on Computational Photography (ICCP), 1-14,
Neural Networks, IEEE Transactions on Industrial Electronics, DOI: 10.1109/ICCPHOT.2018.8368462.
63(11), 7067–7075. DOI: 10.1109/TIE.2016.2582729. Marquardt, D.W., 1963, An Algorithm for Least-Squares
Itskovich, G.B., and Nikitenko, M.N., 2017, Fast Inversion of Estimation of Nonlinear Parameters, Journal of the Society for
MWD Transient EM Data Excited by a Pulse of an Arbitrary Industrial and Applied Mathematics, 11(2), 431–441. DOI:
Shape, U.S. Patent No. 9,562,990, granted February 7, 2017. 10.1137/0111030.
Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, Omeragic, D., Dumont, A., Esmersoy, C., Habashy, T., Li, Q.,
K., 2015, Spatial Transformer Networks, Advances in Neural Minerbo, G., Rosthal, R., Smits, and Tabanou, J., 2006,
Information Processing Systems, 28, 2017–2025, https:// Sensitivities of Directional Electromagnetic Measurements
papers.nips.cc/paper/5854-spatial-transformer-networks.pdf. for Well Placement and Formation Evaluation While Drilling,
Accessed January 5, 2020. SEG Technical Program Expanded Abstracts, 1630–1634,
Jiang, J., and Fan, J.A., 2019, Global Optimization of Dielectric DOI: 10.1190/1.2369833.
Metasurfaces Using a Physics-Driven Neural Network, Ramuhalli, P., Udpa, L., and Udpa, S.S., 2002, Electromagnetic
Nano Letters, 19(8), 5366–5372. DOI: 10.1021/acs. NDE Signal Inversion by Function-Approximation Neural
nanolett.9b01857. Networks, IEEE Transactions on Magnetics, 38(6), 3633–
Jin, Y., 2018, Modern Deep Network Toolkits for TensorÀow- 3642. DOI: 10.1109/TMAG.2002.804817.
Keras, MDNT. https://github.com/cainmagi/MDNT. Accessed Seydoux, J., Legendre, E., Mirto, E., Dupuis, C., Denichou, J.-
January 5, 2020. M., Bennett, N., Kutiev, G., Kuchenbecker, M., Morriss, C.,
Johnston, A., Garg, R., Carneiro, G., and Reid, I.D., 2017, Scaling and Yang, L., 2014, Full 3D Deep Directional Resistivity
CNNs for High Resolution Volumetric Reconstruction From Measurements Optimize Well Placement and Provide
a Single Image, Proceedings, 2017 IEEE International Reservoir-Scale Imaging While Drilling, Paper LLL,
Conference on Computer Vision Workshops, 930–939. DOI: Transactions, SPWLA 55th Annual Logging Symposium, Abu
10.1109/ICCVW.2017.114. Dhabi, UAE, 18–22 May.
Key, K., 2009, 1D Inversion of Multicomponent, Multifrequency Shen, Q., Chen, J., and Wang, H., 2018a, Data-Driven
Marine CSEM Data: Methodology and Synthetic Studies for Interpretation of Ultradeep Azimuthal Propagation Resistivity
Resolving Thin Resistive Layers, Geophysics, 74(2), F9–F20. Measurements: Transdimensional Stochastic Inversion and
DOI: 10.1190/1.3058434. Uncertainty Quanti¿cation. Petrophysics, 59(6), 786–798.
Kingma, D.P., and Ba, J., 2015, Adam: A method For Stochastic DOI: 10.30632/PJV59N6-2018a4.
Optimization, presented at the 3rd International Conference Shen, Q., Wu, X., Chen, J., and Han, Z., 2017, Distributed
on Learning Representation (ICLR 2015). https://arxiv.org/ Markov Chain Monte Carlo Method on Big-Data Platform
pdf/1412.6980.pdf. Acccessed January 5, 2020. for Large-Scale Geosteering Inversion Using Directional
Lerer, A., Gross, S., and Fergus, R., 2016, Learning Physical Electromagnetic Well Logging Measurements, Applied
Intuition of Block Towers by Example, Proceedings, 33rd Computational Electromagnetics Society Journal, 32(5),
International Conference on Machine Learning, 48(9), 430– 405–412. https://aces-society.org/includes/downloadpaper.
438. https://arxiv.org/pdf/1603.01312.pdf. Accessed January php?of=ACES_Journal_May_2017_Paper_5&nf=17-5-5.
5, 2020. Accessed January 5, 2020.
Levenberg, K., 1944, A Method for the Solution of Certain Non- Shen, Q., Wu, X., Chen, J., Han, Z., and Huang, Y., 2018b, Solving
Linear Problems in Least Squares, Quarterly of Applied Geosteering Inverse Problems By Stochastic Hybrid Monte
Mathematics, 2(2), 164–168. DOI: 10.1090/qam/10666. Carlo Method, Journal of Petroleum Science and Engineering,

February 2020 PETROPHYSICS 97

 Jin et al.

161, 9–16. DOI: 10.1016/j.petrol.2017.11.031. problem, optimization, seismic processing, and signal
Simonyan, K., and Zisserman, A., 2015, Very Deep Convolutional processing.
Networks for Large-Scale Image Recognition, presented at
the 3rd International Conference on Learning Representations Qiuyang Shen is a Research Scientist with Cyentech
(ICLR 2015). https://arxiv.org/pdf/1409.1556.pdf. Accessed
Consulting LLC. He received his PhD degree in Electrical
January 5, 2020.
Tchakarov, B.J., Wang, T., Guenther, R.S., Cao, T.H., and Tang, C., Engineering from the University of Houston in 2019. His
2016, Downhole Closed-Loop Geosterring Methodology, US current research interests include geophysical inversions,
Patent 9,273,517 granted March 1, 2016. statistical data analysis, and uncertainty quanti¿cations.
Thiel, M., Bower, M., and Omeragic, D., 2018, 2D Reservoir
Imaging Using Deep Directional Resistivity Measurements, Xuqing Wu is an Assistant Professor of the Department
Petrophysics, 59(2), 218–233. DOI: 10.30632/PJV59N2- of Information and Logistics Technology at the University
2018a7. of Houston. Prior to joining the University of Houston in
Ulyanov, D., Vedaldi, A., and Lempitsky, V., 2016, Instance 2015, he was a data scientist and modeling engineer at
Normalization: The Missing Ingredient for Fast Stylization.
Schlumberger. His research interests include machine
https://arxiv.org/pdf/1607.08022.pdf. Accessed January 5,
learning, statistical inference, subsurface sensing, and
2020.
Van Den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, computer vision. Dr. Wu received his PhD degree in
O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, Computer Science from the University of Houston in 2011.
K., 2016, WaveNet: A Generative Model for Raw Audio,
presented at the 9th ISCA Speech Synthesis Workshop, 125. Jiefu Chen is an Assistant Professor at the Department
https://arxiv.org/pdf/1609.03499.pdf. Access January 5, 2020. of Electrical and Computer Engineering, the University of
Wang, H., Shen, Q., and Chen, J., 2018, Sensitivity Study and Houston. Previously, he worked for four and a half years as
Uncertainty Quanti¿cation of Azimuthal Propagation a Staff Scientist for Weatherford International. Dr. Chen’s
Resistivity Measurements, Paper R, Transactions, SPWLA research interests include computational electromagnetics,
59th Annual Logging Symposium, London, England, UK, 2–6
multiphysics modeling and inversion, scienti¿c
June.
Wu, J., Lu, E., Kohli, P., Freeman, B., and Tenenbaum, J., 2017, machine learning, underground and underwater wireless
Learning to See Physics Via Visual De-Animation, Advances communication, and well logging. He holds a BS degree in
in Neural Information Processing Systems (NIPS 2017), 30, engineering mechanics and a MS degree in dynamics and
153–164. https://papers.nips.cc/paper/6620-learning-to-see- control, both from Dalian University of Technology, and a
physics-via-visual-de-animation.pdf. Accessed January 5, PhD degree in Electrical Engineering from Duke University.
2020.
Xu, Y., Sun, K., Xie, H., Zhong, X., Mirto, E., Feng, Y., and Hong, Yueqin Huang is the Chief Scientist of Cyentech
X., 2018, Borehole Resistivity Measurement Modeling Using Consulting LLC. Previously she worked as a Geophysicist
Machine-Learning Techniques, Petrophysics, 59(6), 778–785.
for DownUnder GeoSolutions, and as an Assistant Professor
DOI: 10.30632/PJV59N6-2018a3.
for Xiamen University. Dr. Huang’s research interests
Zhang, R., Liu, Y., and Sun, H., 2019, Physics-guided Convolutional
Neural Network (PhyCNN) for Data-Driven Seismic include computational acoustics and seismology, ground
Response Modeling. https://arxiv.org/pdf/1909.08118.pdf. penetrating radar, and well logging. She received her PhD
Accessed January 5, 2020. degree in Electrical Engineering from Xiamen University in
Zhang, X., Zhou, X., Lin, M., and Sun, J., 2018, ShufÀenet: An 2011.
Extremely Ef¿cient Convolutional Neural Network for
Mobile Devices, Proceedings, 2018 IEEE/CVF Conference
on Computer Vision and Pattern Recognition, 6848–6856.
DOI: 10.1109/CVPR.2018.00716.

ABOUT THE AUTHORS

Yuchen Jin is a third-year PhD student at the Department

of Electrical and Computer Engineering, University of
Houston, co-advised by Dr. Jiefu Chen and Dr. Xuqing Wu.
He received his BS degree in Electronic Information from
Huazhong University of Science and Technology in 2017.
His research interests include machine learning, inverse

98 PETROPHYSICS February 2020