Experiment Number: 2

Aim: Using MATLAB to generate images with the training data set model
generative network monitoring system.

A) Image Generation Using Generative Adversarial Network (GAN)

This example shows how to create a custom training experiment to train a
generative adversarial network (GAN) that generates images of flowers. For a
custom training experiment, you explicitly define the training procedure used
by Experiment Manager. In this example, you implement a custom training
loop to train a GAN, a type of deep learning network that can generate data
with similar characteristics as the input real data. For more information on
generating images using a GAN, see Train Generative Adversarial Network
(GAN). For more information on custom training
experiments, see Configure Custom Training Experiment.

Fig.2.1

a. Input

• params is a structure with fields from the Experiment Manager hyperparameter

table.
• monitor is an experiments. Monitor object that you can use to track the progress of
the training, update information fields in the results table, record values of the
metrics used by the training, and produce training plots.
 b. Output

• output is a structure that contains the trained generator network, the trained
discriminator network, and the execution environment used for training.
Experiment Manager saves this output, so you can export it to the MATLAB
workspace when the training is complete.

CODE: -
function output = ImageGenerationExperiment_training(params,monitor)
1) Initialize Output:

Initialize the networks as []. Use the execution environment setting "auto" to train the GAN on a
GPU if one is available.

CODE: -

output.generator = [];
output.discriminator = [];
output.executionEnvironment = "auto";

2) Load Training Data:

Load the training data as an imageDatastore. The training data consists of 3670 images from the
Flowers data set. Augment the data to include random horizontal flipping and resize the images
to have size 64-by-64. For more information on this data set, see Data Sets for Deep Learning.

CODE: -

monitor.Status = "Loading Data";

url =
"http://download.tensorflow.org/example_images/flower_photos.tgz";
downloadFolder = tempdir; filename =
fullfile(downloadFolder,"flower_dataset.tgz");

imageFolder =
fullfile(downloadFolder,"flower_photos"); if
~exist(imageFolder,"dir") websave(filename,url);
untar(filename,downloadFolder) end

datasetFolder = fullfile(imageFolder);
imdsTrain = imageDatastore(datasetFolder, ...
IncludeSubfolders=true);

augmenter = imageDataAugmenter(RandXReflection=true);
augimdsTrain = augmentedImageDatastore([64 64],imdsTrain, ...
DataAugmentation=augmenter);
3) Define Generator Network:
 Define the architecture for the generator network as a layer graph that generates images from 1-
by1-by-100 arrays of random values. To train the network with a custom training loop and enable
automatic differentiation, convert the layer graph to a dlnetwork object.

CODE: -

monitor.Status = "Creating Generator";

filterSize = 5; numFilters
= 64; numLatentInputs =
100; projectionSize = [4 4
512];

layersGenerator = [
featureInputLayer(numLatentInputs)
projectAndReshapeLayer(projectionSize,Name
="proj")
transposedConv2dLayer(filterSize,4*numFilters
) batchNormalizationLayer reluLayer
transposedConv2dLayer(filterSize,2*numFilters
,Stride=2,Cropping="same")
batchNormalizationLayer reluLayer

transposedConv2dLayer(filterSize,numFilters,S
tride=2,Cropping="same")
batchNormalizationLayer reluLayer
transposedConv2dLayer(filterSize,3,Stride=2,C
ropping="same") tanhLayer];

lgraphGenerator =
layerGraph(layersGenerator); output.generator
= dlnetwork(lgraphGenerator);
4) Define Discriminator Network:

Define the architecture for the discriminator network as a layer graph that classifies real and
generated 64-by-64-by-3 images. For the dropout layer, read the dropout probability defined in
the Experiment Manager hyperparameter table by accessing the argument value
params.dropoutProb. To train the network with a custom training loop and enable automatic
differentiation, convert the layer graph to a dlnetwork object.

CODE: -

monitor.Status = "Creating Discriminator";

filterSize = 5;
numFilters = 64;
inputSize = [64 64 3];
dropoutProb = params.dropoutProb;
 scale = 0.2;

layersDiscriminator = [
imageInputLayer(inputSize,Normalization="none")
dropoutLayer(dropoutProb)
convolution2dLayer(filterSize,numFilters,Stride=2,Padding="same")
leakyReluLayer(scale)

convolution2dLayer(filterSize,2*numFilters,Stride=2,Padding="same")
batchNormalizationLayer leakyReluLayer(scale)
convolution2dLayer(filterSize,4*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
batchNormalizationLayer
leakyReluLayer(scale)
convolution2dLayer(4,1)
sigmoidLayer];

lgraphDiscriminator = layerGraph(layersDiscriminator);
output.discriminator = dlnetwork(lgraphDiscriminator);
leakyReluLayer(scale)
convolution2dLayer(filterSize,8*numFilters,Stride=2,Padding="same")
5) Specify Training Options:

Train with a mini-batch size of 128 for 50 epochs. For both networks, specify an initial learning
rate of 0.0002, a gradient decay factor of 0.5, and a squared gradient decay factor of 0.999.
Initialize the trailing average gradient and trailing average gradient-square decay rates as []. If the
discriminator learns to discriminate between real and generated images too quickly, then the
generator can fail to train. To better balance the learning of the discriminator and the generator,
add noise to the real data by randomly flipping the labels. Specify the fraction of real labels to
flip by reading the value of flipFactor defined in the Experiment Manager hyperparameter table.

Training can take some time to run. For better results, consider increasing the training time to
500 epochs.

CODE: -

numEpochs = 50; miniBatchSize =

128; learnRate = 0.0002;
gradientDecayFactor = 0.5;
squaredGradientDecayFactor = 0.999;
trailingAvgG = []; trailingAvgSqG =
[]; trailingAvgD = []; trailingAvgSqD
= []; flipFactor = params.flipFactor;

6) Train Model:

Use the input argument monitor to track the progress of the training and display the metric values
(scoreGenerator, scoreDiscriminator, and scoreCombined) in the experiment results table. Group
the scoreGenerator and scoreDiscriminator metrics in a single subplot of the training plot.

CODE: -

monitor.Metrics = ["scoreGenerator","scoreDiscriminator","scoreCombined"];
monitor.XLabel = "Iteration";
groupSubPlot(monitor,"Combined Score","scoreCombined");
groupSubPlot(monitor,"Generator and Discriminator Scores", ...
["scoreGenerator","scoreDiscriminator"]);
monitor.Status = "Training";
Use minibatchqueue to process and manage the mini-batches of images. For each mini-batch:
• Use the mini-batch preprocessing helper function preprocess Minibatch to rescale the images
in the range [-1,1].
• Discard any partial mini-batches with fewer than 128 observations.
• Format the image data with the dimension labels 'SSCB' (spatial, spatial, channel, batch). By
default, the minibatch queue object converts the data to dl array objects with underlying type
single.

CODE: -
augimdsTrain.MiniBatchSize = miniBatchSize;
mbq = minibatchqueue(augimdsTrain,...
MiniBatchSize=miniBatchSize,...
PartialMiniBatch="discard",...
MiniBatchFcn=@preprocessMiniBatch,...
MiniBatchFormat="SSCB",...

OutputEnvironment=output.executionEnvironment);
Train the model using a custom training loop. For each epoch, shuffle the datastore and loop over
mini batches of data. If training on a GPU, then convert latent inputs to gpuArray. Evaluate the
model gradients and update the discriminator and generator network parameters. After each
iteration of the custom training loop, save the trained network, record the metric values, and
update the training progress. Exit the custom training loop if you click Stop on the Experiment
Manager toolstrip.

CODE: -

iteration = 0;
for epoch = 1:numEpochs
shuffle(mbq); while
hasdata(mbq)
iteration = iteration + 1;
X = next(mbq);

Z = randn(numLatentInputs,miniBatchSize,"single");
Z = dlarray(Z,"CB");

if (output.executionEnvironment == "auto" && canUseGPU) || ...

output.executionEnvironment ==
"gpu" Z = gpuArray(Z); end
[~,~,gradientsG,gradientsD,stateG,scoreG,scoreD] = ...

dlfeval(@modelLoss,output.generator,output.discriminator,X,Z,flipFactor);
output.generator.State = stateG;

[output.discriminator,trailingAvgD,trailingAvgSqD] = adamupdate( ...

output.discriminator,gradientsD, ...
trailingAvgD,trailingAvgSqD,iteration, ...
learnRate,gradientDecayFactor,squaredGradientDecayFactor);

[output.generator,trailingAvgG,trailingAvgSqG] = adamupdate( ...

output.generator,gradientsG, ...
trailingAvgG,trailingAvgSqG,iteration, ...
learnRate,gradientDecayFactor,squaredGradientDecayFactor);

[6]
 scoreG = double(gather(extractdata(scoreG))); scoreD =
double(gather(extractdata(scoreD)));
scoreCombinedValue = 1-2*max(abs(scoreD-0.5),abs(scoreG-0.5));

recordMetrics(monitor,iteration, ...
scoreGenerator=scoreG, ... scoreDiscriminator=scoreD, ...
scoreCombined=scoreCombinedValue);

if monitor.Stop || isnan(scoreG) || isnan(scoreD) return; end end

monitor.Progress = (epoch/numEpochs)*100; end

7) Generate Test Images:

Create a batch of 25 random vectors to input to the generator network and display the resulting
images in a figure. When the training is complete, the Review Results gallery in the toolstrip
displays a button for the figure. The Name property of the figure specifies the name of the button.
You can click the button to display the figure in the Visualizations pane. Use this figure to check
that the generator produces a variety of images without many duplicates. If the images have little
diversity and some of them are almost identical, then your generator is likely affected by mode
collapse.

CODE: -

numLatentInputs = 100;
numTestImages = 25;

ZTest = randn(numLatentInputs,numTestImages,"single");
ZTest = dlarray(ZTest,"CB");

if (output.executionEnvironment == "auto" && canUseGPU)

|| ...
output.executionEnvironment ==
"gpu" ZTest = gpuArray(ZTest); end

XGenTest = predict(output.generator,ZTest);

figure(Name="Test Images")
I = imtile(extractdata(XGenTest));
I = rescale(I); image(I) xticks([])
yticks([]) title("Generated Test
Images")
End of training function.
end

8) Helper Functions:
The modelLoss function takes as input the generator and discriminator dlnetwork objects (netG
and netD), a mini-batch of input data (X), an array of random values (Z), and the percentage of
real labels to flip (flipProb), and returns the loss values, the gradients of the loss values with
respect to the learnable parameters in the networks, the generator state, and the scores of the two
networks.

CODE: -

function [lossG,lossD,gradientsG,gradientsD,stateG,scoreG,scoreD] = ...

modelLoss(netG,netD,X,Z,flipProb)

YReal = forward(netD,X);

[XGenerated,stateG] = forward(netG,Z);
YGenerated = forward(netD,XGenerated);

scoreD = (mean(YReal) + mean(1-YGenerated)) / 2;

scoreG = mean(YGenerated);

numObservations = size(YReal,4);
idx = rand(1,numObservations) < flipProb;
YReal(:,:,:,idx) = 1 - YReal(:,:,:,idx);

[lossG, lossD] = GANLoss(YReal,YGenerated);

gradientsG =
dlgradient(lossG,netG.Learnables,RetainData=true); gradientsD
= dlgradient(lossD,netD.Learnables); end
The GANLoss function returns the loss for the discriminator and generator networks.

CODE: -

function [lossG,lossD] = GANLoss(YReal,YGenerated)

lossD = -mean(log(YReal))-mean(log(1-YGenerated));
lossG = -mean(log(YGenerated)); end

The preprocessMiniBatch function preprocesses the data by extracting the image data from the
incoming cell array, concatenating the images into a numeric array, and rescaling the images to
be in the range [-1,1].

CODE: -

function X = preprocessMiniBatch(data)
X = cat(4,data{:});
X = rescale(X,-1,1,InputMin=0,InputMax=255);
end

[8]
9) Output:

Fig2.2

Fig2.3
[10]