VISVESVARAYA TECHNOLOGICAL UNIVERSITY

“Jnana Sangama”, Belagavi- 590 018

PROJECT PHASE I REPORT

ON
“IOT BASED SMART COLD STORAGE SYSTEM FOR
EFFICIENT STOCK MANAGEMENT”

Submitted in partial fulfillment of the requirements for the degree of

BACHELOR OF ENGINEERING
IN
COMPUTER SCIENCE & ENGINEERING

Under the Guidance of

Mr. Chethan P B.E., M.Tech.
Assistant Professor,
Department of CS & E
Adichunchanagiri Institute of Technology
Chikkamagaluru

Submitted by
Manoj R (4AI18CS404) Mohammed Ibrahim Safiulla (4AI18CS405)
Sharath C R (4AI17CS084) Nitish Kumar N H (4AI8CS406)

DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING

ADICHUNCHANAGIRI INSTITUTE OF TECHNOLOGY
(Affiliated to V.T.U., Accredited by NBA)
CHIKKAMAGALURU-577102, KARNATAKA
2020- 2021
 ADICHUNCHANAGIRI INSTITUTE OF TECHNOLOGY
(Affiliated to Visvesvaraya Technological University, Belagavi)
Chikkamagaluru, Karnataka, India-577102.
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

CERTIFICATE
This is to certify that the Project Phase-I (17CSP78) work entitled “IOT Based Smart
Cold Storage System for Efficient Stock Management” is a bonafide work carried out by
Manoj R (4AI18CS404), Mohammed Ibrahim Safiulla (4AI18CS405), Sharath C R
(4AI17CS084), Nitish Kumar N H (4AI18CS406) students of 7th semester B.E, in partial fulfillment
for the award of Degree of Bachelor of Engineering in Computer Science and Engineering of the
Visvesvaraya Technological University, Belagavi during the academic year 2020-21. It is certified that all
corrections and suggestions indicated for Internal Assessment have been incorporated in the report
deposited in the department library. The project report has been approved, as it satisfies the academic
requirements in respect of Project Work prescribed for the said Degree.

Signature of the Guide

Mrs. Chethan P B.E., M.Tech.
Assistant Professor
Dept. of CS&E

Signature of the Coordinator Signature of the Coordinator

Mrs. Arpitha C N B.E., M.Tech. Mrs. Sunitha M R B.E., M.Tech., (Ph.D)
Assistant Professor Assistant Professor
Dept. of CS&E Dept. of CS&E

Signature of the HOD Signature of the Principal

Dr. Pushpa Ravikumar B.E., M.Tech., Ph.D., LMISTE Dr. C.T Jayadeva B.E., M.Tech., Ph.D
Professor & Head Principal,
Dept. of CS&E A.I.T, Chikkamagaluru
 ABSTRACT
Internet of Things (IOT) plays the role of an expert’s technical tool by empowering
physical resources into smart entities through existing network infrastructures. The IOT paradigm
is aimed at formulating a complex information system with the combination of sensor data
acquisition, efficient data exchange through networking, machine learning, artificial intelligence,
big data, and clouds. Conversely, collecting information and maintaining the confidentiality of an
independent entity, and then running together with privacy and security provision in IOT is the
main concerning issue.
In this approach, firstly, the objects placed inside the smart cold storage will be detected
and identified using a pi-camera. Load cell with HX711 IC driver is used to calculate the Weight
of the objects. Raspberry Pi-3 B+ collects data from the Arduino and analyse the data using
python programming and transmit the stock information to the users through mobile application.
It gives an alert to the users to place an order if the weight falls below the threshold value, i.e., If
there is any shortage or out of stock of the objects. DHT11 Temperature sensor is used to monitor
the Temperature of the storage system and fans are used to adjust the temperature inside the
Storage box and Sprinklers are used to adjust the humidity. When the storage box door was
open/close then notification will be sent to the mobile application, here we used IR Sensor to
monitor the door of the Storage box. When user keep Stock inside the box or remove the stock
from the box then Stock keeping time and stock removing time will be displayed.

i
 ACKNOWLEDGEMENTS
We express our humble Pranamas to his holiness Parama Poojya Jagadguru
Padmabushana Sri Sri Sri Dr. Balagangadharanatha Mahaswamiji and Parama Poojya
Jagadguru Sri Sri Sri Dr. Nirmalanandanatha Mahaswamiji and also to Sri Sri
Gunanatha Swamiji Sringeri Branch Chikkamagaluru who have showered their blessings on us
for framing our career successfully.

We are deeply indebted to our honorable Director Dr. C K Subbaraya for creating the
right kind of care and ambience.

We are Thankful to our beloved Principal Dr. C T Jayadeva for inspiring us to achieve
greater endeavors in all aspects of Learning.

We express our deepest gratitude to Dr. Pushpa Ravikumar, Professor & Head,
Department of Computer Science & Engineering for her valuable guidance, suggestions and
constant encouragement without which success of our Project Phase-I work would have been
difficult.

We are Grateful to our coordinators Mr. Sunitha M R and Mrs. Arpitha C N for their
excellent guidance, constant encouragement, support and constructive suggestions.

We are thankful to our guide Mrs. Chethan P Asst. Professor, Dept. of Computer
Science & Engineering AIT, Chikkamagaluru for his inspiration and lively correspondence for
carrying our project phase-I work.

We would like to thank our beloved parents for their support, encouragement and
blessings.

And last but not least, I would be pleased to express our heartfelt thanks to all teaching
and non-teaching staff of CS&E Department and our friends who have rendered their help,
motivation and support.
Manoj R (4AI18CS404)
Sharath C R (4AI17CS084)
Nitish Kumar N H (4AI18CS406)
Mohammed Ibrahim Safiulla (4AI18CS405)
 Table of Contents
CHAPTER TITLE PAGE No.

Abstract i
Acknowledgements ii
Table of Contents iii
List of Figures vi

Chapter 1 Introduction 1

1.1 Introduction 1
1.2 Motivation 5
1.3 Problem Statement 6
1.4 Scope of the Project 6
1.5 Objectives 7
1.6 Advantages 7
1.7 Review of Literature 7
1.8 Organization of the Report 11
1.9 Summary 11

Chapter 2 System Requirement Specification 12

2.1 Specific Requirements 12
2.2 Hardware Requirements 12
2.3 Software Requirements 12
2.4 Functional Requirements 13
2.5 Summary 13

Chapter 3 High Level Design 14

3.1 Design Consideration 14
3.2 System Architecture 15
3.2.1 Stock Detection 16

iii
 3.2.2 Stock Recognition 16
3.2.3 Weight Identification
3.2.4 Temperature Identification 17
3.2.5 Notification 17
3.3 Use Case Diagram 18
3.4 Module Specification 19
3.4.1 Image Pre-Processing module 19

3.4.2 Segmentation module 20

3.4.3 Feature Extraction module 21

3.4.4 Classification module 22
3.4.5 Stock Recognition module 23
3.5 Data Flow Diagram 25

3.5.1 Image Pre-Processing 25

3.5.2 Segmentation 27
3.5.1 Feature extraction 28
3.5.1 Classification 30
3.5.1 Stock recognition 30
3.6 State Chart Diagram 31
3.7 Summary 32

Chapter Detailed Design

4.1 Structural Chart of the Proposed System
4.2 Flowchart of Image pre-processing
4.3 Flowchart of Segmentation
4.4 Flowchart of Feature Extraction
4.5 Flowchart of Classification
4.6 Detailed Description of each module
4.6.1 RGB color image
4.6.2 Greyscale

iv
4.6.5 Feature Extraction
 4.6.6 Classification
4.6.7 Stock Recognition
4.7 Summary

REFERENCE
 List of Figures
FIGURE NAME PAGE NO
Architectural Diagram of the Proposed System 13
The Process of Stock notifications 16
Use Case Diagram to monitor Smart Cold Storage 17
Use Case Diagram of Stock Detection and Recognition 20
Use Case Diagram of Raspberry Pi Module 21
Use Case Diagram of Weight Identification Module 22
Use Case Diagram of Temperature Identification Module 23
Use Case Diagram of Notification Module 24
Data Flow Diagram of Proposed System 26
Data Flow Diagram of Image Pre-processing 27
Conversion of RGB Image to Gray Scale Image 28
Conversion of Gray Scale Image to Binary Image 28
Conversion of RGB Image to RGB Matrix 29
Conversion of Gray Scale Image to Gray Scale Matrix 29
Data Flow Diagram of Weight Identification 30
Data Flow Diagram of Temperature Identification 31
Data Flow Diagram of Storage Box Analysis 31
Structural Diagram of Proposed System 33
Flowchart of Proposed System 34
Flowchart of Stock Detection and Recognition 36
Flowchart of Weight Identification 37
Flowchart of Identification 38
Flowchart of Notification Module 39

v
 IOT Based Smart Cold Storage System for Efficient Stock Management

CHAPTER 1

INTRODUCTION

1.1 Introduction

Smart cold storage has the ability to connect to the internet through Wi-Fi to provide
special features. It is a smart way of dealing with the stock or items present in the storage. It
includes internal cameras, more flexible user-controlled cooling options, and the ability for to
interact with its features using smart phone or tablet. This proto type has the ability to know the
quantity of items present. It will provide notifications to the user and the dealer or shopkeeper, if
they are out of stock, Notifications include temperature, as well as information about the goods.
This prototype uses a pi-cam to detect the presence of objects. Arduino is used as weight scale
controller and Raspberry Pi3 as a home server. The controller will collect all information and send
them to Raspberry Pi-3. Using python programming the weight of the object when placed the
storage box is identified.

Raspberry PI consists of in built WIFI along with LAN connection. An SD card will be
inserted in raspberry pi by installing an operating system called Raspbian OS. It also has a camera
module to which the pi-cam is connected. The Arduino is connected to the GPIO of the Raspberry
pi. Weight scales attached with the load cell are connected to HX711IC. HX711IC acts as a driver
for storing analog output weight. The analog electrical output is given to HX711IC and output of
HX711IC is given to Arduino. The temperature sensor DHT11 is also connected to Arduino. The
temperature sensor DHT11 detects temperature inside the storage box. Arduino acts as an interface
between analog output from weight scales, temperature sensor and digital input of raspberry pi.
Raspberry is programmed in such a way that it captures 50 Samples of each product using pi-cam.
The samples of the product taken are stored in the database. This process is known as training.
Weight scales are placed inside the storage box.

B.E., Dept. of CS&E, AIT, Chikkamagaluru. Page 2

 IOT Based Smart Cold Storage System for Efficient Stock Management

Whenever a product is kept in the storage box, the image of it is captured using pi-cam. The
captured image is detected using python programming. Then weight of the product on each scale is
converted into electrical output using wheat stone bridge circuit. This circuit consists of fixed
resistances and unknown resistances so as to know the change in resistance using strain gauge.

This electrical output given to Arduino, Arduino consists of data regarding temperature and
weight of the products inside the storage box. Arduino is programmed so that the data present in it
is given to raspberry pi. Using python programming the raspberry pi analyses the data. An app is
created to notify the user and shopkeeper. The app consists of data base of product images of
products present in the storage box, weight present on each weight scale and temperature inside
the storage box. Raspberry is programmed to have a threshold value of each weight scale.
Whenever the weight of the products reduce beyond the threshold value the user and shopkeeper
are notified using mobile application. This app displays weight of each object along with its image
so as to make user have knowledge regarding presence of the products and weight of it if present.
In case of absence of any product or reduction beyond given threshold value shopkeeper is
notified. The app is updated automatically with details of temperature and products to make it easy
for the user.

When the Storage Box door was open, then notification will be sent to the Mobile application.
And when the Storage Box door was close, then notification will be sent to the mobile application.
Here we use IR sensor to monitor the cold storage box. IR sensor is connected to the Arduino,
Then This Arduino will help to send the notifications. This will helpful to keep the stock in cold
space, if the storage box door was open then coldness inside the storage box was not maintained.
This leads to the spoilage of items. DHT11 Temperature sensor are used to monitor the
temperature inside the storage box, if the Storage box door was open the temperature inside the
storage box will be increased. Here we used Fans to adjusting the temperature inside the Storage
box and Sprinklers are used to maintain the humidity. When we keep stock inside the storage box
then the stock keeping time will be displayed and when we remove the stock from the storage box
then the stock removing time will also be displayed.

B.E., Dept. of CS&E, AIT, Chikkamagaluru. Page 2

 IOT Based Smart Cold Storage System for Efficient Stock Management
1.2 Motivation

The aim in this project “Smart Cold Storage” by leveraging the latest supply chain
technology and the IOT, which will serve as a hub to improve the efficiency and speedup the process
throughout the entire supply chain. The cold storage facilities available are mostly for a single
commodity like potato, oranges, apple, flowers etc. which results in poor capacity utilization. The
existing cold storage system lacks storage capacity of fruits and vegetables partially responsible for
the non-availability of seasonal fruits and vegetables. A cold storage unit incorporates a refrigeration
system but cannot store for a longer time. It includes internal cameras, more flexible user controlled
cooling options, and the ability for to interact with its features using smart phone or tablet. This proto
type has the ability to know the quantity of items present. It will provide notifications to the user and
the dealer or shopkeeper, if they are out of stock, Notifications include temperature, as well as
information about the goods. This project can be implemented in medical field also to store and
maintain the medical product.
The smart refrigerator is constantly evolving, most of the functions of intelligent
refrigerators is only limited to the original traditional refrigerator, but with the development of
information technology, if the smart refrigerator and information technology can be combined,
then Smart refrigerator will make people's lives more convenient undoutly.
 The existing cold storage system lacks storage capacity of fruits and vegetables partially
responsible for the non-availability of seasonal fruits and vegetables.
 This proto type has the ability to know the quantity of items present. It will provide
notifications to the user and the dealer or shopkeeper, if they are out of stock, Notifications
include temperature, as well as information about the goods.
 Our smart cold storage system is more accurate and is capable of notifying the details about
the contents of the storage and their weights to the user and the product supplier, when the
contents are running out of stock. Temperature inside the storage system will also be indicated
in the specified app through the user’s smart phone.

1.3 Problem statement

To Design and implement an efficient Smart Cold Storage system for Stock Management
using IOT to Preserve the Stock & Medicine for a longer days and to reduce the power Consumption,
when Stock was not available in the storage Box.

B.E., Dept. of CS&E, AIT, Chikkamagaluru. Page 2

 IOT Based Smart Cold Storage System for Efficient Stock Management

Input:
 Stock Image detection by pi-Camera.
 Temperature detection inside the storage Box by DHT11 Sensor.
 Weight of the Stock can be calculated by load cell with HX711 IC driver.

Processing:
 The process consists of detection of any Stock inside the Storage Box by pi-Camera &
Capture the Image of stock inside the Storage Box & Identifying which type of stock it
is. If stock is not available, then send message like please refill the Box to the user
through the android application.
 Another process is to identifying the internal temperature of the Storage Box by
DHT11 Sensors.
 Weight of the Stock can be calculated by load cell with HX711 IC driver.

Output:
 Display the image of the Stock inside the Storage Box on the mobile application, If we
took Stock from the Storage Box then it display message like Stock is exhausted. If
stock is not available, then notification send the user to refill the Storage Box.
 Display the date and time of Stock inserting and Stock removing.
 Display the temperature Inside Storage Box and adjusting the temperature, when the
temperature inside the box was high.
 Display the net weight of the Stock inside the Storage Box.
 Send the notification to the user through the mobile application, when the storage box
door open/close.


1.4 Scope of the Project

B.E., Dept. of CS&E, AIT, Chikkamagaluru. Page 2
 IOT Based Smart Cold Storage System for Efficient Stock Management
The Scope of the project are,
 Research Field:- To prevent the spoilage of chemicals & can be preserve safely for longer
days.
 Medical Field:-To prevent the spoilage of medicine/drugs and preserve safely.
 Industrial Field:-To prevent manufactured items and avoid spoilage.
 Shipping Container:-To be used in storage areas, where the transformation is less.
 Home, Malls, Hotel, Fishery:- To prevent storage of food, vegetables, fruits, meat etc.

B.E., Dept. of CS&E, AIT, Chikkamagaluru. Page 2

 1.5 Objectives
The Objectives of the project are,
 To display the Image of the stock along with it’s name, when they kept stock inside the
Storage Box.
 To display the temperature Inside the storage Box.
 To adjusting the temperature inside the box, when the temperature inside the box was high.
 To display the net weight of the stock.
 To display the date & time, when they kept stock inside & when they took Stock from the
Storage Box.
 To get notifications on mobile application, when the storage box door was open/close.
 To monitor the Cold storage Box through the android application.
 To get notifications, when they kept any Stock inside the storage Box and when they took any
Stock from the Storage Box.
 To reduce the unnecessary Power Consumption.

1.6 Review of Literature

[1] Title: Intelligent Refrigerator based on Internet of Things
Author: Shouming Qiao, Hongzhen Zhu, Lijuan Zheng, Jianrui Ding.
Description:
Based on the technology of radio frequency identification (RFID), combined with Internet
and information processing technology, intelligent refrigerator for food management is developed,
food records the within the refrigerator is achieved. These data will be automatically uploaded to the
production of intelligent refrigerator cloud data service platform, easy to be viewed by the user. And
according to the food inside the refrigerator it can provide recipes. Wireless communication module
is used to achieve the reception of Internet information. A week before shelf life of the food, a
warning will be sent to the user. When the food over the shelf life, there will be usage warning to the
user. When the food is reduced or not available through the information on the Internet, different
businesses will be compared to choose the best food. At the same time the wireless communication
module can also be used to connect intelligent terminal and the refrigerator, the situation inside the
refrigerator is checked.
 [2] Title: Smart Refrigerator using Internet Of Things.
Author: Krutik Panchal, Harshil Patel, Prithvi Bhatt, Vaishali Gaikwad (Mohite)
Description:
Smart appliances with multimedia capability have been emerging into our daily life. Smart
appliances include washing machine, television, refrigerator etc. In this modern era, human beings are
used to being intouch with this technology or we can say it as internet of things (IoT). As we look
around ourselves we see modernization with superior technology, for example cell phones, kitchen,
appliances and many more. It uses advancement of computing technology and the wide use of the
Internet. Smart home is one of the most prominent areas of intelligent appliances.Refrigerator is one
of the intelligent appliances that is mostly used. With smart sensing technology, the refrigerator will
keep check on the expiry of food products and the spoilage of eatable items.
 Organization of the report

The report has been organized into the below chapters.

Chapter 1-Introduction: This chapter presents a brief description about the IOT
Based Smart Cold Storage System for Efficient Stock Management.

Chapter 2-System requirement specification: As the name suggested the

second chapter consisting of specific requirement, software and hardware
requirements that used in this project. Also, we summarize this chapter at the end.

Chapter 3-High Level Design: This chapter contains design consideration,

architecture of the proposed system, and use case diagram.

Chapter 4- Detail design: The chapter 4 explains about the detail functionalities
and description of each module.

Summary

The first chapter describes the Cold Storage System for efficient Stock Management
using Internet of Things. The motivation of the project is discussed in section 1.2. Problem
statement of the project explained in section 1.3. And the Scope of the project is described in
the section 1.4 and objectives are presented in the section 1.5. and section 1.6 gives details of
the literature survey reviews, the important papers referred.
 CHAPTER 2

SYSTEM REQUIREMENT SPECIFICATION

System requirement specifications gathered by extracting the appropriate information

to implement the system. It is the elaborative conditions which the system needs to attain.
Moreover, the SRS delivers a complete knowledge of the system to understand what this
project is going to achieve without any constraints on how to achieve this goal. This SRS
does not providing the information to outside characters but it hides the plan.

2.1 Specific requirements

 Arduino IDE

Arduino IDE where IDE stands for Integrated Development Environment – An

official software introduced by Arduino.cc, that is mainly used for writing, compiling and
uploading the code in the Arduino Device. Almost all Arduino modules are compatible with
this software that is an open source and is readily available to install and start compiling the
code on the go.
Arduino IDE is an open source software that is mainly used for writing and compiling
the code into the Arduino Module. It is an official Arduino software, making code
compilation too easy that even a common person with no prior technical knowledge can get
their feet wet with the learning process. It is easily available for operating systems like MAC,
Windows, Linux and runs on the Java Platform that comes with inbuilt functions and
commands that play a vital role for debugging, editing and compiling the code in the
environment.

 OpenCV
OpenCV supports a wide variety of programming languages such as C++, Python, Java,
etc., and is available on different platforms including Windows, Linux, OS X, Android, and
iOS. Interfaces for high speed GPU operations based on CUDA and OpenCL are also under
 active development. OpenCV-Python is the Python API for OpenCV, combining the best
qualities of the OpenCV C++ API and the Python language.
OpenCV-Python is a library of Python bindings designed to solve computer vision
problems. Python is a general purpose programming language started by Guido van Rossum
that became very popular very quickly, mainly because of its simplicity and code readability.
It enables the programmer to express ideas in fewer lines of code without reducing readability.

2.2 Hardware requirements

 Sensor: Rasberry Pi, DHT11 Temperature sensor, IR Sensor.
 Microcontroller: Arduino Uno R3, NodeMCU.
 Extra Requirements: Battery, Jumpers , HX711IC load cell, SD card and Pi-Camera.

2.3 Software requirements

 Programming Language: Python and embedded c++.
 Software: opencv and Arduino IDE.
 Operating system: Rasbian os.

2.4 Functional requirements

The set of images is used to train the system to recognize which type of stock it is.
This project needs images of Stocks.

 The Stock present inside the Storage box are captured using a camera.
 The captured Stock images are detected and Recognized using Tenser-flow
algorithm .
 Temperature inside the Storage box was detected by DHT11 temperature Sensor
and adjusting the temperature inside the box by fans, when the temperature was
high..
 Weight of the Stock can be calculated by load cell with HX711 IC driver.
 Receive notifications on mobile application, when the Storage box door was
open/close.
Summary

The chapter 2 considers all the system requirements which is required to develop
this proposed system. The specific requirements for this project have been explained in
section 2.1. The hardware requirements for this project have been explained in section
2.2. The backend software is clearly explained in section 2.3. The functional
requirements are explained in the section 2.4.
 CHAPTER 3

HIGH LEVEL DESIGN

High-level design (HLD) explains the architecture that would be used for developing a
software product. The architecture diagram provides an overview of an entire system,
identifying the main components that would be developed for the product and their interfaces.
The HLD uses possibly non-technical to mildly technical terms that should be understandable
to the administrators of the system. In contrast low level design further exposes the logical
detailed design of each of these elements for programmers.

High level design is the design which is used to design the software related
requirements. In this chapter complete system design is generated and shows how the
modules, sub modules and the flow of the data between them are done and integrated. It is
very simple phase that shows the implementation process. The errors done here will be
modified in the coming processes.

3.1 Design consideration

The design consideration briefs about how the system behaves for cold storage and
classify the corresponding notation for the given stock image. Stock Detection, Stock
Recognition, Weight Identification, Temperature identification, Notifications.

 Stock Detection: The goal of object detection is to detect all the instances from the
known samples. It mainly focuses on the Stock detection and tracking based on its
color, i.e., the input to the proto type will be an image.

 Stock Recognition: It is the process of recognizing the product. The process of training
the pi-cam is programmed to capture 50 samples and stores them in database. It
compares the detected image with the stored samples in the database as samples. Here
Pre-processing, segmentation, extraction, classification process are takes place by using
the tenser-flow algorithm.
 Weight Identification: Weight identification involves identifying the weight of the
object which is recognized in the process of stock recognition. To identify the
weight of the product placed in the storage box HX711 IC with load cell are used. .
Using python programming the weight of the stock when placed the storage box is
identified.

 Temperature Identification: The temperature inside the storage box is also

notified to the user through Blynk app. DHT11 temperature sensor is used. It gives
linear relationship between output voltage and Celsius temperature. Arduino is
connected to Raspberry pi, which is programmed to send notification to the user
using app.

 Notifications: The information regarding the presence of the product and image of
the product, weight of it and temperature inside the storage box are updated
automatically in the mobile Application. If the product is not available inside the
box then user to refill the box this kind of notifications will also send to the user.
When the door of the box will open/close the will also notifications send to the
user. When user keep/remove the stock then also notification send.
 IOT Based Smart Cold Storage System for Efficient Stock Management

3.2 System architecture

Figure 3.1: Architectural Diagram of the Proposed System.

The figure 3.1 shows the system architecture for the proposed method. Here we can
see that the stock image is captured by a pi-camera and the image input is fed into the stock
recognition system, in which it is processed and compared efficiently. The virtual stock is
updated accordingly.

The system which is being used currently performs the task in stock recognition and
classification of it. The input image (stock image) is matched with the images in the dataset
using Tensor-flow algorithm. So, the matched image with minimum value is recognized and
classified to give the respective output.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 15

 IOT Based Smart Cold Storage System for Efficient Stock Management

The input image is pre-processed and converted to grey scale image to find the
Threshold value based on input image. Based on Threshold value further binary and
segmented images are obtained.

The proposed system has the following steps for Smart Cold Storage:
1. Stock Detection.
2. Stock Recognition
2.1 Pre-processing
2.2 Segmentation
2.3 Feature extraction
2.4 Classification
3. Weight Identification.
4. Temperature Identification.
5. Notifications.

3.2.1 Stock Detection:

The goal of object detection is to detect all the instances from the known samples. As shown in
Fig. 3.2 the process involves Stock detection, recognition, weight identification, temperature
recognition and notifications. It mainly focuses on the Stock detection and tracking based on its
colour, i.e., the input to the proto type will be an image, which is captured by a pi-cam. A pi-cam will
detect the stock and tracks it continuously. Each sample that is specified in the python script is
detected when placed in the storage area using pi-cam. The visual data captured by the pi-cam is
processed with Raspberry Pi and the stock is detected based on the color or shape. Each detection is
reported with information like name or ID of the stock.

3.2.2 Stock Recognition:

It is the process of recognizing the product. As mentioned above in the process of training the
pi-cam is programmed to capture 50 samples and stores them in database. Stored images are specified
according their identity. After completion of stock detection with the help python script, the pi-cam
recognizes the product placed in the storage box. It compares the detected image with the stored
samples in the database as samples are saved along with the number. The Raspberry Pi identifies the

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 16

 IOT Based Smart Cold Storage System for Efficient Stock Management

number of the sample matched with the detected image. The numbering of each samples is linked to
specifications the product. Specifications consist of name of the product. Using these parameters and
the detection of the stock the product placed is recognized along with its identity i.e. Name.

3.2.3 Weight Identification:

Weight identification involves identifying the weight of the object which is recognized in the
process of stock recognition. Each sample saved is specified with number, name and threshold value.
These parameters are used in the weight identification. To identify the weight of the product placed in
the storage box weight scales are used. These weight scales are constructed with load cell with a
capacity of 5kgs. Whenever an stock is placed on the weight scales, the resistance of strain gauge
changes and produces proportional electrical output. The output pins of the load cell is connected to
HX711 IC. This IC acts as a driver for storing the output of weight scales. This output is given to
Arduino through GPIOs. As Arduino is connected to Raspberry Pi-3 model B+. Data regarding the
weight of the object is sent to it. Using python programming the weight of the stock when placed the
storage box is identified.

3.2.4 Temperature Identification:

The temperature inside the storage box is also notified to the user through Blynk app. So as
to make the user know regarding the temperature in the box. DHT11 temperature sensor is used. It
gives linear relationship between output voltage and Celsius temperature. This electrical output is
given to Arduino, which is interfaced to Raspberry Pi. Raspberry Pi is programmed to send
notification to the user using app. This makes the user aware of the temperature inside the storage
box.

3.2.5 Notifications:
After the identification of the data regarding the weight of the products and temperature inside
the storage box the user is notified through an app. An app named all things maker is created and
linked to the program using its ID. Using this mobile application, the image of the product identified
and the weight of it is updated in the app. Whenever the weight of any product is reduced beyond its
threshold value the user and shopkeeper are notified through an alert notification. The information

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 17

 IOT Based Smart Cold Storage System for Efficient Stock Management

regarding the presence of the product, weight of it and temperature inside the storage box are updated
automatically in the mobile Application.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 18

 IOT Based Smart Cold Storage System for Efficient Stock Management

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 18

3.3 Specification using Use case diagram

Figure 3.2: Use Case Diagram for Recognition and Classification of image.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 19

 Hand Gesture Recognition and Classification for Deaf and Dumb

The use case diagram for Smart cold storage system is as depicted in the above figure
3.3. Initially the set of captured images are stored in SD Card in the Raspberry Pi. The
obtained RGB image is converted in to gray scale image to reduce complexity. Then the
pre-processing techniques are applied on the obtained gray scale image. Then Grey Scale
image to Binary image conversion segmentation process can be takes place, the
significant features are extracted using Tensor-flow Algorithm. Further, we classify the
which type of stock it is, we match the closest image in database and recognize the
required Stock. It is complete process of stock detection.

The process of Raspberry Pi is to capture the image using the Pi-camera and stock
detection process is to identify which type of stock it is, then raspberry pi will help to
display the captured image in the Blynk application.

The process of HX711 IC with Load cell is to just calculating the weight of the
stock, which is already present in the Storage box. Pi-camera will capture the image of
the Stock and ten load cell will activate and weigh the Stock weight. And that weight will
display on the Blynk Application.

The process of DHT11 Temperature sensor is to just shows the temperature inside
the storage box. DHT11 temperature sensor measure the temperature inside the storage
box and then temperature in degree Celsius will be display on the mobile application.

3.4 Module Specification

Module Specification is the way to improve the structure design by breaking

down the system into modules and solving it as independent task. By doing so the
complexity is reduced and the modules can be tested independently. The number of
modules for this model is five, Stock detection module, Raspberry Pi module, HX711 IC
with Load cell module, DHT11 temperature sensor module and notification module.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 20

 3.4.1 Stock Detection and
Stock Recognition Module
Name of the Module:Stock detection and Stock
Recognition.

Actors: Stock.
Use Cases: Image capture, RGB Image Pre-processing, Binary Image Segmentation,
Segmented Image feature Extraction, Image Classification.

Functionality: The main functionality of this module is to convert the RGB image to binary
format for faster processing.

Description: Figure 3.4 shows the use case diagram of the Stock detection and Stock
Recognition. In this use case diagram, there are five use cases and one actor. In the first
use case, image is captured and is used as input for this module. In second use case, the
captured image is converted to RGB matrix. RGB image into Gray scale image to reduce
complexity. Then grey image is converted to binary image. In the third use case Binary
image segmentation process takes place and segmented matrix formed. In the fourth use
case, feature extraction from the segmented matrix. At last image can be classified and
display which type of stock it is.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 21

Figure 3.3: Use Case Diagram of Stock Detection and Stock Recognition .

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 22

 Hand Gesture Recognition and Classification for Deaf and Dumb

3.4.2 Raspberry Pi Module

Name of the Module: Raspberry

Pi.

Segmentation Actors: Raspberry Pi.

Use Cases: Image capture, Notification display.

Functionality: The main functionality of this module is to capture the Stock Image,
which is present inside the storage box and Display the capture on the Blynk application.

Description: Figure 3.5 shows the use case diagram of Raspberry Pi module. In this use
case diagram, there are two use cases and one actor. In the first use case, the Raspberry Pi
capture the image of Stock, which is present inside the storage box as input. In second
use case, captured image will further going to Stock detection and stock recognition
process and identity which type of stock it is, then stock name as well as captured stock
image will be displayed on the blynk application.

Figure 3.4: Use Case Diagram of Raspberry Pi Module.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 20

 Hand Gesture Recognition and Classification for Deaf and Dumb

3.4.3 Weight Identification

Module

Name of the Module: Weight

Identification.

Actors: HX711IC with Load cell.

Use Cases: Image capture and Notification display.

Functionality: The main functionality of this module is to Identify the weight of the
particular stock which is available inside the storage box.

Description: Figure 3.6 shows the use case diagram of Weight Identification module. In
this use case diagram, there are two use cases and one actor. In the first use case, the

Pi-camera will capture the stock image and which type of stock it is will be done stock
detection and stock recognition module. Calculate the Weight of particular stock is done
by using the HX711 IC with Load cell. In The second use case, weight of the stock will be
display in the Blynk application.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 21

 Hand Gesture Recognition and Classification for Deaf and Dumb

Figure 3.5: Use Case Diagram of Weight Identification Module.

3.4.4 Temperature

Identification Module.

Name of the Module: Temperature

Identification.

Actors: DHT11 Temperature

Sensor.

Use Cases: Temperature measure, Notification display.

Functionality: The main functionality of this module is to calculate Temperature inside

the Storage box in degree Celsius.

Description: Figure 3.7 shows the use case diagram of Temperature Identification

module. In this use case diagram, there are two use cases and one actor. In the first use
case, the DHT11 Temperature sensor will measure the temperature inside the storage box
in degree Celsius. In the second use case, measured temperature will be displays on the
Blynk application. If the Temperature raised in the storage box, then fans will turn on to
reduce the temperature inside the storage box.

[Type text] Page 22

 Hand Gesture Recognition and Classification for Deaf and Dumb

Figure 3.6: Use Case Diagram of Temperature Identification Module.

[Type text] Page 23

 IOT Based Smart Cold Storage System for Efficient Stock Management

3.4.5 Notification Module

Name of the Module: Notification.

Actors: Blynk Application.

Use Cases: Stock display, Weight display, Temperature display, Door open/close, Date and
Time display.

Functionality: The main functionality of this module is to Display the messages on the Blynk
Application.

B.E., Dept. of CS&E, AIT, Chikkamagaluru. Page 23

Description: Figure 3.8 shows the use case diagram of Notification Module. In this use
case diagram, there are five use cases and one actor. In the first use case, When pi camera
capture the stock image then that image will be display on the Blynk Application. In the
second use case, When user keep the stock inside the storage box, then HX711 IC with
load cell will calculate the weight of the particular stock and that weight will be display
on the Blynk Application. . In the third use case, Temperature inside the storage box will
be measured in degree Celsius and that temperature will be displayed on the Blynk
Application. In the fourth use case, if the storage box door was open/close then the
notification will be send to the user. In the fifth use case, Date and Time of stock
keeping/removing will be displayed on the Blynk application.

Figure 3.7: Use Case Diagram of Notification Module

Data Flow Diagram

As the name specifies so to the meaning of the words, it is the process which is explained

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 24

in detail like how the data flows between the different processes. The below figure 3.10
depicts the flow diagram and is composed of the input, process and the output. After each
process the data which is flown between the systems need to be specified and hence
called the data flow diagram. In most of the times it is the initial step for designing any of
the systems to be implemented. It also shows from where the data originated, where the
data flowed and where it got stored. The obtained RGB image is converted in to gray
scale image to reduce complexity. The detected input hand image is classified once after
pre-processing, segmentation is done to extract the significant features which are then
matched with images in dataset and the two images(the input image and the image
already in database) having minimum value on applying Euclidean distance is the
expected result.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 25

 Figure 3.8: Data Flow Diagram for the proposed system

Image pre-processing.

Input image through camera is captured and it can be used to store as dataset for
training or as input image to recognize the character. The image is captured and stored in
any supported format specified by the device.

As shown in the below figure 3.3 initially the set of captured images are stored in
a temporary file in MATLAB. The storage is linked to the file set account from which the
data is accessed. The obtained RGB image is converted in to gray scale image to reduce
complexity.
 Figure 3.9: Data Flow Diagram of RGB to Grey Conversion

Pre-processing is required on every image to enhance the functionality of image

processing. Captured images are in the RGB format. The pixel values and the
dimensionality of the captured images is very high. As images are matrices and
mathematical operations are performed on images are the mathematical operations on
matrices. So, we convert the RGB image into Gray image using “rgb2gray” function and
thus converting the Gray image into Binary image. Image segmentation technique is used
on the Binary image to detect the hand region.
 Hand Gesture Recognition and Classification for Deaf and Dumb

Figure 3.10: RGB to gray conversion.

Segmentation

Segmentation is done to divide image into two regions, background and the
foreground containing region of interest i.e. hand region. The segmented image has the
hand region with the pixel value ‘1’ and the background as the ‘0’. The image is then
used as a mask to get the hand region from the RGB image by multiplying the black and
white image i.e. binary image with the original RGB image. The image is resized to
reduce size of the matrix used for the recognition process. The images are then converted
into column matrix for feature extraction.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 27

 Hand Gesture Recognition and Classification for Deaf and Dumb

Figure 3.11: Data Flow Diagram of Segmentation

As shown in the above figure 3.5 binary image is masked using the threshold value
to obtain the region of interest.

Figure 3.12: Gray to Binary Conversion

Feature Extraction

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 28

 Feature extraction is the most significant step in recognition stage as the size of
the data dimensionality is reduced in this stage. Here, features are extracted using
Principle Component Analysis (PCA), where we can get Eigen Values and Eigen
Vectors. To calculate this Eigen Values and Eigen Vectors, column matrix of all the
images is formed and concatenated to form single matrix, mean of this matrix is
calculated and it is subtracted for normalization.

Figure 3.13: Data Flow Diagram of Feature extraction

As shown in the above figure 3.7 Principle component analysis is applied to

segmented matrix to obtain Eigen values. An Eigen vector of a square matrix A is a non-
zero vector v that, when the matrix is multiplied by v, yields a constant multiple of v, the
multiplier being commonly denoted by ƛ. The number ƛ is called the Eigen value of A
corresponding to Eigen vector v.

Eigen vectors are set of basic functions which describes variability of data. Also,
Eigen vectors are also a kind of coordinate system for which the covariance matrix
becomes diagonal for which the new coordinate system is uncorrelated. Eigen values
measures the variance of data of new coordinate system. Eigen values and Eigen vectors
are a part of linear transformations. Eigen vectors are the directions along which the
linear transformation acts by stretching, compressing or flipping and Eigen values gives
the factor by which the compression or stretching occurs. The more the Eigen vectors the
better the information obtained from the linear transformation.

The mean of the matrix is calculated as:

𝑀
μ= 𝑛=1 𝑇𝑛 where, M is the number of column matrix.
 Mean from each of the column vector of the database Ti is subtracted as: Temp = A- μ
Where, A is column matrix & μ is the mean.

Classification

Euclidean Distance is used to classify the gestures of hand. Feature vectors

calculated for the dataset and the Feature vector calculated for the input image are used to
classify the character related to the input gesture. Classification of character is based on
the maximum score. Maximum score is the maximum matching score i.e. minimum
distance of input gesture with the dataset image of the corresponding character.

Figure 3.14: Data Flow Diagram of Classification Process

Sign recognition

Figure 3.15: Data Flow Diagram of Sign recognition process

The obtained feature vectors are matched with the values in the dataset. The
minimum value between the input image and segmented image is obtained after
matching. As a result, thus, the sign is recognized.
 State Chart Diagram

A state chart diagram is also named as state diagram. It is popular one among five
UML diagrams and it is used to model the dynamic nature of the system. State chart
defines various states of an object during its lifetime. The state chart diagram is a
composition of finite number of states and the functionalities describe the functioning of
each module in the system. It is a graph where each state is a directed edge and
represented by node and these directed edges represents transition between states.

Hand Gesture Recognition System

Segmentation
Input Pre-Processing Do:
Image Binary Image Removal of ROI
Do: Masking
RGB to Grey Scale
Grey to binary
Segmented
Image

Feature Extraction
Do:
Classification Principal Component Eigen ValueAnalysis (PCA)
Do:
Euclidean Distance
Feature Vector

Sign Recognition
Minimum Do:
Distance Matching with the dataset X
Recognised Sign

Figure 3.16: State Chart Diagram of the proposed system

 Hand Gesture Recognition and Classification for Deaf and Dumb

The figure 3.11 shows the state chart diagram of the Hand recognition and
classification. The process starts with the solid circle, the first state is the reading the
hand gesture image as input and the second state is pre-processing to convert RGB to
gray and then to Binary image. In third state segmentation is done on binary image to
remove the ROI by masking process. In fourth level, we extract the significant features
using PCA and later in Fifth level we use Euclidian distance to find the minimum
distance value and finally we recognize the sign which has closest match with images in
database.

Summary

In third chapter, high level design of the proposed method is discussed. Section
3.1 presents the design considerations for the project. Section 3.2 discusses the system
architecture of proposed system. The next section 3.3 describes use case diagram. Section
3.4 describes module specification for all the modules. The data flow diagram for each
module is explained in section 3.5. The State Chart diagram for the system is explained in
section 3.6.

B.E., Dept. of CS&E, AIT, Chikmagaluru. Page 32

 Hand Gesture Recognition and Classification for Deaf and Dumb

CHAPTER 4

DETAILED DESIGN

A detail design is the process of each individual module which is completed in the
earlier stage than implementation. It is the second phase of the project first is to design
phase and second phase is individual design of each phase options. It saves more time and
another plus point is to make implementation easier.

Structural chart of the proposed system

Fig 4.1: Structural chart for the proposed method.

The figure 4.1 shows the structural chart of hand gesture recognition system. The
chart is composed of five modules namely Image pre-processing, segmentation, feature
extraction, classification and sign recognition. Hand gesture is the input to the system and
the output is recognized sign in text and speech format.

B.E., Dept. of CS&E, A.I.T., Chikkamagaluru. Page 33

 Hand Gesture Recognition and Classification for Deaf and Dumb

Flow chart of Image Pre-Processing

The figure 4.2 shows the flowchart of image pre-processing where at first image is
captured in RGB format. To make processing much simpler RGB image must be converted
for gray scale format. Further grayscale image is converted to binary image to reduce the
processing time.

B.E., Dept. of CS&E, A.I.T., Chikkamagaluru. Page 34

Flow chart of Image Segmentation
The figure 4.3 shows the flowchart of image segmentation where input is binary
image to which masking is performed to obtain region of interest. From the region of
interest segmented matrix is obtained.
 Fig 4.3: Flowchart for image segmentation.

Flow chart of Feature Extraction

The figure 4.2 shows the flowchart of feature extraction process using principle
component analysis in which eigen value is calculated by converting the segmented matrix
in column matrices and calculating their mean. Finally eigen value is obtained by
subtracting all column matrices from the mean value. Further eigen value is used for the
classification purpose.
Fig 4.4: Flowchart for Feature Extraction
Flow chart of Classification
The figure 4.4 shows the flowchart of feature extraction process using principle
component analysis in which eigen value is calculated by converting the segmented matrix
in column matrices and calculating their mean. Finally eigen value is obtained by
subtracting all column matrices from the mean value. Further eigen value is used for the
classification purpose.
Fig 4.5: Flowchart for Classification
Detailed Description of each module
This section gives the detailed description of each module which includes pre-
processing, segmentation, feature extraction, classification and sign recognition
RGB Color Image

The RGB color model is additive colour model in which red, green, and blue light
are added together in various ways to reproduce a broad array of colors. The name of the
model comes from the initials of the three additive primary colours, red, green, and blue.
The main purpose of the RGB color model is for the sensing, representation, and display of
images in electronic systems, such as televisions and computers, though it has also been
used in conventional photography.

Before the electronic age, the RGB color model already had a solid theory behind it,
based in human perception of colors. RGB is a device-dependent color model: different
devices detect or reproduce a given RGB value differently, since the color elements (such
as phosphors or dyes) and their response to the individual R, G, and B levels vary from
manufacturer to manufacturer, or even in the same device over time. Thus an RGB value
does not define the same color across devices without some kind of color management.

Typical RGB input devices are color TV and video cameras, image scanners, and
digital cameras. Typical RGB output devices are TV sets of various technologies (CRT,
LCD, plasma), computer and mobile phone displays, video projectors, multicolour LED
displays, large screens such as Jumbo Tron. Color printers, on the other hand, are not
RGB devices, but subtractive color devices (typically CMYK color model).

Grayscale

In photography and computing, a grayscale or greyscale digital image is an image

in which the value of each pixel is a single sample, that is, it carries only intensity
information. Images of this sort, also known as black-and-white, are composed
exclusively of shades of grey, varying from black at the weakest intensity to white at the
strongest.

Grayscale images are distinct from one-bit bi-tonal black-and-white images, which
in the context of computer imaging are images with only the two colors, black, and
white (also called bi-level or binary images). Grayscale images have many shades of grey
in between. Grayscale images are also called monochromatic, denoting the presence of
only one (mono) color (chrome).

Grayscale images are often the result of measuring the intensity of light at each
pixel in a single band of the electromagnetic spectrum (e.g. infrared, visible light,
ultraviolet), and in such cases they are monochromatic proper when only a given frequency
is captured. But also, they can be synthesized from a full color image.

RGB to Gray conversion

Image obtained by digital camera is in RGB format. But for post processing, this
image needs to be converted into gray scale format. This makes processing much simpler.
The need for converting to Grayscale is to reduce the processing time for the Algorithms,
RGB Images are not required to Processing,

Intensity = 0.2989*red + 0.5870*green + 0.1140*blue

Need for Gray scale conversion

A grayscale or grayscale digital image is an image in which the value of each pixel
is a single sample, that is, it carries only intensity information. Images of this sort, also
known as black-and-white, are composed exclusively of shades of gray, varying from black
at the weakest intensity to white at the strongest.

Grayscale images are often the result of measuring the intensity of light at each
pixel in a single band of the electromagnetic spectrum (e.g. infrared, visible light,
ultraviolet, etc.), and in such cases they are monochromatic proper when only a given
frequency is captured. But also they can be synthesized from a full color image; see the
section about converting to grayscale.

The intensity of a pixel is expressed within a given range between a minimum and a
maximum, inclusive. This range is represented in an abstract way as a range from 0 (total
absence, black) and 1 (total presence, white), with any fractional values in between. This
notation is used in academic papers, but this does not define what "black" or "white" is in
terms of colorimetry.

Another convention is to employ percentages, so the scale is then from 0% to 100%.

This is used for a more intuitive approach, but if only integer values are used, the range
encompasses a total of only 101 intensities, which are insufficient to represent a broad
gradient of grays. Also, the percentile notation is used in printing to denote how much ink
is employed in half toning, but then the scale is reversed, being 0% the paper white (no ink)
and 100% a solid black (full ink).

In computing, although the grayscale can be computed through rational numbers,

image pixels are stored in binary, quantized form. Some early grayscale monitors can only
show up to sixteen (4-bit) different shades, but today grayscale images (as photographs)
intended for visual display (both on screen and printed) are commonly stored with 8 bits
per sampled pixel, which allows 256 different intensities (i.e., shades of grey) to be
recorded, typically on a non-linear scale. The precision provided by this format is barely
sufficient to avoid visible banding artefacts, but very convenient for programming due to
the fact that a single pixel then occupies a single byte.

Technical uses (e.g. in medical imaging or remote sensing applications) often require
more levels, to make full use of the sensor accuracy (typically 10 or 12 bits per sample) and
to guard against round off errors in computations. Sixteen bits per sample (65,536 levels) is a
convenient choice for such uses, as computers manage 16-bit words efficiently. The TIFF
and the PNG (among other) image file formats support 16-bit grayscale natively, although
browsers and many imaging programs tend to ignore the low order 8 bits of each pixel. No
matter what pixel depth is used, the binary representations assume that 0 is black and the
maximum value (255 at 8 bpp, 65,535 at 16 bpp, etc.) is white, if not otherwise noted

Segmentation

In this stage, prior to the gray scale image which was obtained is further converted
into a binary image (consisting of 0 and 1). Here, the binary image is obtained from the
calculated threshold value using the grey scale matrix. Based on threshold value (if the pixel
value in grey scale matrix is greater than threshold value, we then plot 1 in binary matrix else
we tend
to plot 0. Once, after the binary matrix is obtained, we perform masking operation
(multiplying grey scale matrix with the binary matrix) to obtain segmented matrix.
The segmented matrix is thus obtained to get our region of interest (ROI). The
region of interest is ideally the binary matrix consisting of value 1. The matrix having 0
value is ignored as we are not concerned about it.
Feature Extraction

Principle Component Analysis (PCA) is a common feature extraction method in data

science. It is a statistical procedure that uses an orthogonal transformation to convert a set of
observations of possibly correlated variables (entities each of which takes on various
numerical values) into a set of values of linearly uncorrelated variables called principal
components. This transformation is defined in such a way that the first principal component
has the largest possible variance (that is, accounts for as much of the variability in the data as
possible), and each succeeding component in turn has the highest variance possible under the
constraint that it is orthogonal to the preceding components. The resulting vectors (each
being a linear combination of the variables and containing n observations) are an
uncorrelated orthogonal basis set. PCA is sensitive to the relative scaling of the original
variables.

Feature extraction is the most significant step in recognition stage as the size of the
data dimensionality is reduced in this stage. Also, it can be used for signal processing,
pattern matching. In Principle Component Analysis only the unique and significant features
are extracted. Later, we can get Eigen Values and Eigen Vectors. To calculate this Eigen
Values and Eigen Vectors, column matrix of all the images is formed and concatenated to
form single matrix, mean of this matrix is calculated and it is subtracted for normalization.
The mean of the matrix is calculated as:

𝑀
μ= 𝑛=1 𝑇𝑛 where, M is the number of column matrix.

Mean from each of the column vector of the database Ti is subtracted as:

Temp = A- μ

Where, A is column matrix & μ is the mean.

 Classification

Euclidean Distance is used to classify the gestures of hand. Feature vectors

calculated for the dataset and the Feature vector calculated for the input image are used to
classify the character related to the input gesture. Classification of character is based on the
maximum score. Maximum score is the maximum matching score i.e. minimum distance of
input gesture with the dataset image of the corresponding character.

2
𝑛
Ed = √∑𝑖=1(𝑞𝑖,𝑗− 𝑝𝑖,𝑗 )

Where, j =1, 2…, m, n=total pixels in single image m= total number of images in the dataset.

Sign Recognition

Detection of human hand in white background will enhance the performance of

image pre-processing, in terms of accuracy and speed. Features are extracted from the
acquired image using PCA and comparison of the features is done using Euclidean
distance. If match is found sign will be recognized and corresponding output is displayed
along with audio.

Summary
The fourth chapter gives the detailed design for the Hand gesture Recognition
system. In section 4.1 structural chart for the proposed system is shown. Section 4.2 the
flowchart for image processing is shown. Section 4.3 gives the flowchart for segmentation.
Section 4.4 gives the flowchart for feature extraction. Section 4.5 gives the flowchart for
classification. Section 4.6 gives the detailed description of each module.
 REFERENCES
[1] Mrs. Neela Harish , Dr. S .Poonguzhali, “Design and development of hand gesture
recognition system for speech impaired people”, International Conference on Industrial
Instrumentation and Control (ICIC) College of Engineering Pune, India. May 28-30, 2015.

[2] Suganya R, Dr.T. Meeradevi, “Design Of A Communication Aid For Physically

Challenged” ,in IEEE Sponsored 2nd International Conference On Electronics And
Communication System (Icecs 2015),2015.

[3] Meenakshi Panwar and Pawan Singh Mehra, “Hand Gesture Recognition for Human
Computer Interaction”, International Conference on Computing, Communication and
Applications (ICCCA), Din Digul, Tamilnadu , India, February 2012.

[4] M. Ebrahim AI-Ahdal & Nooritawati Md Tahir," Review in Sign Language Recognition
Systems" Symposium on Computer & Informatics(ISCI),pp:52-57, IEEE ,2012 Sruthi
Upendran, Thamizharasi. A, “American Sign Language Interpreter System for Deaf and
Dumb Individuals” in International Conference on Control, Instrumentation, Communication
and Computational Technologies (ICCICCT), 2014.

[5] Andrew Wilson and Aaron Bobick, “Learning visual behavior for gesture analysis,” In
Proceedings of the IEEE Symposium on Computer Vision, Coral Gables, Florida, pp. 19-21,
November 2015.

[6] Nitesh S. Soni, Prof. Dr. M. S. Nagmode, Mr. R. D. Komati, “Hand Gesture Recognition
&Classification for Deaf & Dumb”, International Conference on Inventive Computation
Technology (ICICT) , Coimbatore, India ,2016.