Recognize Flowers with TensorFlow

Slide
Lite on 1
Android

Prepared by : Wafaa Alayoubi

Ibrahim Isleem

Supervised by : Dr. Akram Abu Garad

 ABSTRACT
Slide 3
Convolutional neural networks

• Recently Convolutional Neural Networks (CNNs) have been shown to achieve state-of-
the-art performance on various classification tasks. In this paper, we present for the first
time a place recognition technique based on CNN models, by combining the powerful
features learnt by CNNs with a spatial and sequential filter. Applying the system to a 70
km benchmark place recognition dataset we achieve a 75% increase in recall at 100%
precision, significantly outperforming all previous state of the art techniques. We also
conduct a comprehensive performance comparison of the utility of features from all 21
layers for place recognition, both for the benchmark dataset and for a second dataset
with more significant viewpoint changes.

Your Coffee Shop 2

 ABSTRACT
Slide 3
TensorFlow

• TensorFlow offers multiple levels of abstraction so you can choose the right one for your
needs. Build and train models by using the high-level Keras API, which makes getting
started with TensorFlow and machine learning easy.

• If you need more flexibility, eager execution allows for immediate iteration and intuitive
debugging. For large ML training tasks, use the Distribution Strategy API for distributed
training on different hardware configurations without changing the model definition.

Your Coffee Shop 3

 ABSTRACT
Slide 3
Google Colab

• Google is quite aggressive in AI research. Over many years, Google developed AI framework called
TensorFlow and a development tool called Colaboratory. Today TensorFlow is open-sourced and
since 2017, Google made Colaboratory free for public use. Colaboratory is now known as Google
Colab or simply Colab.

• Another attractive feature that Google offers to the developers is the use of GPU. Colab supports
GPU and it is totally free. The reasons for making it free for public could be to make its software a
standard in the academics for teaching machine learning and data science. It may also have a long
term perspective of building a customer base for Google Cloud APIs which are sold per-use basis.

• Irrespective of the reasons, the introduction of Colab has eased the learning and development of
machine learning applications.

Your Coffee Shop 4

 INTRODUCTION
Slide 3
Convolutional neural networks

• Since their introduction in the early 1990s, Convolutional Neural

Networks (CNNs) have been used to achieve excellent performance on
a variety of tasks such as handwriting recognition and face detection.
More recently, supervised deep convolutional neural networks have
been shown to deliver high level performance on more challenging
classification tasks.

Your Coffee Shop 5

 OUTLINE
Slide 3
1. Convolutional neural networks.
• What is Convolutional neural networks ?
2. TensorFlow
• What is TensorFlow?
• Why TensorFlow ?
• TensorFlow Architecture
3. Colab Notebooks
• What is Colab?
• What Colab Offers You?
4. Example of Recognize Flowers using Transfer Learning
• Python part.
• Android part.
Your Coffee Shop 6
 Convolutional neural networks
Slide 3
What is Convolutional neural networks ?

• The name “convolutional neural network” indicates that the network employs a
mathematical operation called convolution. Convolution is a specialized kind of
linear operation. Convolutional networks are simply neural networks that use
convolution in place of general matrix multiplication in at least one of their layers.

• Convolutional neural networks are multi-layer supervised networks which can

learn features automatically from datasets .For the last few years, CNNs have
achieved state-of-the-art performance in almost all important classification tasks.
Their primary disadvantage is that they require very large amounts of training
data.
Your Coffee Shop 7
 TENSORFLOW
Slide 3
What is TensorFlow?

• TensorFlow is an open-source library for Deep Learning and Machine learning .

• Developed by the Google Brain team and released in November 2015.
• TensorFlow is mainly used for: Classification, Perception, Understanding, Discovering,
Prediction and Creation.
• Currently, the most famous deep learning library in the world is Google's TensorFlow.
TensorFlow makes it easy for beginners and experts to create machine learning models
for desktop, mobile, web, and cloud. See the sections below to get started.

Your Coffee Shop 8

 TENSORFLOW
Slide 3
Why TensorFlow ?
Runs Everywhere

Runs on desktop and mobile devices such as:

• Linux
• macOS
• iOS
• Android
• Raspberry pi
• And Windows

Your Coffee Shop 9

 TENSORFLOW
Slide 3
Why TensorFlow ?
Flexibility:
• Python API offers flexibility to create all sorts of
computations(Including any neural network architecture we
can think of)
• Includes highly efficient C++ implementations of many ML
operations

Your Coffee Shop 10

 TENSORFLOW
Slide 3
Why TensorFlow ?
Large community:
• One the most popular open source projects on GitHub.
• It has a dedicated team of passionate and helpful
developers.
• Growing community contributing to improve it.

Your Coffee Shop 11

 TENSORFLOW
Slide 3
Why TensorFlow ?
Google products

It powers many of Google’s large-scale services, such as:

• Google Cloud Speech
• Google Photos and
• Google Search

Your Coffee Shop 12

 TENSORFLOW
ARCHITECTURE
Slide 3
TensorFlow architecture works in three parts:
• Preprocessing the data
• Build the model
• Train and estimate the model
It is called TensorFlow because it takes input as a multi-dimensional array,
also known as tensors.
You can construct a sort of flowchart of operations that you want to
perform on that input.
This is why it is called TensorFlow because the tensor goes in it flows
through a list of operations, and then it comes out the other side.

Your Coffee Shop 13

 COLAB NOTEBOOKS
Slide 3
What is Colab?
Google is quite aggressive in AI research. Over many years, Google
developed AI framework called TensorFlow and a development tool
called Colaboratory. Today TensorFlow is open-sourced and since
2017, Google made Colaboratory free for public use. Colaboratory is
now known as Google Colab or simply Colab.

Another attractive feature that Google offers to the developers is

the use of GPU.
Colab supports GPU and it is totally free.

Your Coffee Shop 14

 COLAB NOTEBOOKS
Slide 3
What is Colab?
 Colab is a free notebook environment that runs entirely in the cloud.
Most importantly, it does not require a setup and the notebooks that
you create can be simultaneously edited by your team members -
just the way you edit documents in Google Docs. Colab supports
many popular machine learning libraries which can be easily loaded
in your notebook.

Your Coffee Shop 15

 COLAB NOTEBOOKS
Slide 3
What Colab Offers You?
As a programmer, you can perform the following using Google Colab.
• Write and execute code in Python.
• Document your code that supports mathematical equations.
• Create/Upload/Share notebooks.
• Import/Save notebooks from/to Google Drive.
• Import/Publish notebooks from GitHub.
• Import external datasets e.g. from Kaggle.
• Integrate PyTorch, TensorFlow, Keras, OpenCV.
• Free Cloud service with free GPU.

Your Coffee Shop 16

 EXAMPLE
Slide 3
Train a flower recognizer using Colab

What you will build?

A simple camera app that runs a TensorFlow
image recognition program to identify flowers.

Your Coffee Shop 17

 EXAMPLE
Slide 3
Setup

Install TensorFlow
Clone the git repository

Your Coffee Shop 18

 EXAMPLE
Slide 3
Recognize Flowers using Transfer Learning

1. Import the required packages.

Your Coffee Shop 19

 EXAMPLE
Slide 3
2. Setup Input Pipeline

Download the flowers dataset.

Your Coffee Shop 20

 EXAMPLE
Slide 3
2. Setup Input Pipeline

• Use ImageDataGenerator to rescale the images.

• Create the train generator and specify where the train dataset directory,
image size, batch size.

• Create the validation generator with similar approach as the train

generator with the flow_from_directory() method.

Your Coffee Shop 21

 EXAMPLE
Slide 3
2. Setup Input Pipeline

Your Coffee Shop 22

 EXAMPLE
Slide 3
2. Setup Input Pipeline

Save the labels in a file which will be downloaded later.

Your Coffee Shop 23

 EXAMPLE
Slide 3
3. Create the base model from the pre-trained convnets
• Create the base model from the MobileNet V2 model developed at Google, and pre-
trained on the ImageNet dataset, a large dataset of 1.4M images and 1000 classes of
web images.

• First, pick which intermediate layer of MobileNet V2 will be used for feature extraction.
A common practice is to use the output of the very last layer before the flatten
operation, the so-called "bottleneck layer". The reasoning here is that the following fully-
connected layers will be too specialized to the task the network was trained on, and
thus the features learned by these layers won't be very useful for a new task. The
bottleneck features, however, retain much generality.

Your Coffee Shop 24

 EXAMPLE
Slide 3
3. Create the base model from the pre-trained convnets

• Let's instantiate an MobileNet V2 model pre-loaded with weights trained on ImageNet.

By specifying the include_top=False argument, we load a network that doesn't include
the classification layers at the top, which is ideal for feature extraction.

Your Coffee Shop 25

 EXAMPLE
Slide 3
4. Feature extraction
You will freeze the convolutional base created from the previous step and use that as a feature
extractor, add a classifier on top of it and train the top-level classifier.

5. Add a classification head

Your Coffee Shop 26

 EXAMPLE
Slide 3
6. Compile the model
You must compile the model before training it. Since there are two classes, use
a binary cross-entropy loss.

Your Coffee Shop 27

 EXAMPLE
Slide 3
6. Compile the model
You must compile the model before training it. Since there are two classes, use
a binary cross-entropy loss.

Your Coffee Shop 28

 EXAMPLE
Slide 3
7. Train the model

Your Coffee Shop 29

 EXAMPLE
Slide 3
8. Learning curves
Let's take a look at the learning curves of the training and validation accuracy/loss
when using the MobileNet V2 base model as a fixed feature extractor.

Your Coffee Shop 30

 EXAMPLE
Slide 3
9.Fine tuning

• In our feature extraction experiment, you were only training a few layers
on top of an MobileNet V2 base model. The weights of the pre-trained
network were not updated during training.

• One way to increase performance even further is to train (or "fine-tune")

the weights of the top layers of the pre-trained model alongside the
training of the classifier you added. The training process will force the
weights to be tuned from generic features maps to features associated
specifically to our dataset.

Your Coffee Shop 31

 EXAMPLE
Slide 3
10. Un-freeze the top layers of the model
All you need to do is unfreeze the base_model and set the bottom layers be un-trainable.
Then, recompile the model (necessary for these changes to take effect), and resume training.

Your Coffee Shop 32

 EXAMPLE
Slide 3
11. Compile the model
Compile the model using a much lower training rate.

Your Coffee Shop 33

 EXAMPLE
Slide 3
12. Continue Train the model

13. Convert to TFLite

Saved the model using tf.saved_model.save and then convert the saved model to
a tf lite compatible format.

Your Coffee Shop 34

 EXAMPLE
Slide 3
13. Convert to TFLite
Download the converted model and labels

• Let's take a look at the learning curves of the training and validation accuracy/loss, when fine
tuning the last few layers of the MobileNet V2 base model and training the classifier on top of it.
The validation loss is much higher than the training loss, so you may get some overfitting.

• Let's take a look at the learning curves of the training and validation accuracy/loss, when fine tuning the
last few layers of the MobileNet V2 base model and training the classifier on top of it. The validation loss
is much higher than the training loss, so you may get some overfitting.

Your Coffee Shop 35

 EXAMPLE
Slide 3
13. Convert to TFLite

Your Coffee Shop 36

 EXAMPLE
Slide 3
Summary:
•Using a pre-trained model for feature extraction: When working with a small dataset, it is common to take
advantage of features learned by a model trained on a larger dataset in the same domain. This is done by
instantiating the pre-trained model and adding a fully-connected classifier on top. The pre-trained model is
"frozen" and only the weights of the classifier get updated during training. In this case, the convolutional
base extracted all the features associated with each image and you just trained a classifier that determines
the image class given that set of extracted features.

•Fine-tuning a pre-trained model: To further improve performance, one might want to repurpose the top-
level layers of the pre-trained models to the new dataset via fine-tuning. In this case, you tuned your
weights such that your model learned high-level features specific to the dataset. This technique is usually
recommended when the training dataset is large and very similar to the orginial dataset that the pre-trained
model was trained on

Your Coffee Shop 37

 ANDROID
Slide 3
1- Open the project with Android Studio
1. Open Android Studio. After it loads select " Open an existing
Android Studio project" from this popup:
2. In the file selector, choose
examples/lite/codelabs/flower_classification/start from your
working directory
3. You will get a "Gradle Sync" popup, the first time you open the
project, asking about using gradle wrapper. Click "OK".

Your Coffee Shop 38

 ANDROID
Slide 3
2- Add TensorFlow Lite to the Android app

Your Coffee Shop 39

 ANDROID
Slide 3
3- Update build.gradle

Your Coffee Shop 40

 ANDROID
Slide 3
3- Update build.gradle

Your Coffee Shop 41

 ANDROID
Slide 3
4 - Initialize a TensorFlow Lite interpreter

Your Coffee Shop 42

 ANDROID
Slide 3
4 - Initialize a TensorFlow Lite interpreter

Your Coffee Shop 43

 ANDROID
Slide 3
4 - Initialize a TensorFlow Lite interpreter

Your Coffee Shop 44

 ANDROID
Slide 3
5 - Test run the app

• The app can run on either a real Android device or in the Android
Studio Emulator.
Set up an Android device
• You can't load the app from Android Studio onto your phone unless
you activate "developer mode" and "USB Debugging". This is a one
time setup process.

Follow these instructions.

Your Coffee Shop 45

 Slide 15

THANK YOU!

Your Coffee Shop 46

 REFERENCES
Slide 3
• https://codelabs.developers.google.com/codelabs/recognize-flowers-
with-tensorflow-on-android/#0
• https://colab.research.google.com/github/tensorflow/examples/blob
/master/community/en/flowers_tf_lite.ipynb
• https://www.tutorialspoint.com/google_colab/index.htm
• https://en.wikipedia.org/wiki/Convolutional_neural_network?fbclid=I
wAR2sA2LGBPQtjjEJzfPJLOUQTZbywoT94N9Gviv3LGUxUtp9h-Eoq5ej6
Ss#Definition
• https://www.tensorflow.org/about

Your Coffee Shop 47