Assignment No.

Title: Study of Deep learning Packages: Tensorflow, Keras, Theano and PyTorch. Document
the
distinct features and functionality of the packages.
Aim: Study and installation of following Deep learning Packages:
i. Tensor Flow
ii. Keras
iii. Theno
iv. PyTorch
Theory:

1. TensorFlow

Developed by: Google Brain Team

Release Year: 2015

Key Features:

• Graph-based computation: TensorFlow builds computational graphs where each

node represents an operation, and edges are the data flowing between them. The
separation of the computational description (graph) from the actual computations is an
advantage when optimizing for large-scale distributed systems.
• Scalability: TensorFlow scales easily across CPUs, GPUs, and even TPUs (Tensor
Processing Units). It’s designed for high-performance computation and can handle
very large datasets.
• TensorFlow Serving: A feature that makes it easier to deploy and serve machine
learning models.
• TensorBoard: A visualization toolkit to track and monitor the training process,
performance metrics, and graphs.
• TensorFlow Lite: Optimized for mobile and edge device deployments, making it a
great choice for real-time applications like voice recognition on mobile devices.
• TensorFlow Extended (TFX): An end-to-end platform for managing machine
learning production pipelines.
• Strong support for production environments: TensorFlow’s large ecosystem,
including TensorFlow Hub (for reusable models) and TensorFlow.js (for running
models in browsers), makes it suitable for enterprise-scale applications.

Pros:

• Rich ecosystem with tools for model building, training, serving, and deployment.
• Good community support and extensive documentation.
• Suitable for large-scale, distributed training.
• Continuous updates and improvements by Google.
Cons:

• Steeper learning curve compared to libraries like Keras and PyTorch.

• More verbose syntax, which can sometimes make model building less intuitive.

2. Keras

Developed by: François Chollet

Release Year: 2015 (initially as a standalone library, later integrated with TensorFlow)

Key Features:

• User-friendly API: Keras is known for its simple, intuitive, and easy-to-learn API.
It’s especially beneficial for beginners and rapid prototyping.
• Runs on top of other backends: Initially, Keras supported multiple backends such as
Theano, TensorFlow, and CNTK. However, from version 2.3 onwards, Keras is
integrated as an official part of TensorFlow.
• High-level library: Keras provides high-level building blocks for designing deep
learning models, like layers, loss functions, and optimizers, while hiding the
complexities of the underlying implementations.
• Modularity and composability: Every model in Keras is a sequence or a graph of
standalone modules that can be combined in different ways.
• Wide support for model types: It supports both Convolutional Neural Networks
(CNNs) and Recurrent Neural Networks (RNNs), as well as their combinations.
• Pre-trained models: Keras offers easy access to pre-trained models through the
keras.applications module.

Pros:

• Simplifies model building with its high-level API.

• Great for fast experimentation and prototyping.
• Seamless integration with TensorFlow.
• Has multiple pre-built datasets and utilities for easy dataset handling.

Cons:

• Limited flexibility in comparison to lower-level libraries like PyTorch and

TensorFlow when building custom or complex models.
• Performance may not be as optimized as lower-level libraries in very fine-tuned tasks.

3. Theano

Developed by: Université de Montréal

Release Year: 2007
Discontinued: 2017 (No longer under active development)
Key Features:

• Symbolic expression: Theano excels at symbolically defining expressions and then

optimizing and evaluating them efficiently.
• Tight integration with NumPy: Theano supports efficient evaluation of NumPy-
based expressions, which allows for easy integration with traditional scientific
computing workflows.
• GPU acceleration: Theano was one of the first libraries to enable computations on
GPUs, which dramatically improved performance for deep learning tasks.
• Automatic differentiation: Theano provides efficient and precise automatic
differentiation, which is essential for backpropagation in neural networks.

Pros:

• Efficient use of both CPU and GPU for numerical computation.

• Focuses on optimizing mathematical expressions, which can lead to faster
computations.
• Can serve as a backend for higher-level libraries like Keras.

Cons:

• Development was discontinued in 2017, so it's no longer receiving updates or new

features.
• Higher-level abstraction libraries like TensorFlow and PyTorch have since surpassed
Theano in terms of ease of use and community support.
• More complex syntax compared to newer frameworks.

4. PyTorch

Developed by: Facebook's AI Research Lab (FAIR)

Release Year: 2016

Key Features:

• Dynamic computation graph: Unlike TensorFlow’s static graph, PyTorch uses a

dynamic computation graph, meaning the graph is generated on-the-fly. This allows
for greater flexibility in model building and makes debugging much easier.
• Pythonic nature: PyTorch is very Python-friendly, making it intuitive for Python
developers. It feels more like using NumPy with GPU support.
• Automatic differentiation with Autograd: PyTorch’s Autograd system automates
the computation of gradients, which is key for backpropagation in neural networks.
• Flexible and modular: It provides flexibility for researchers to experiment with
novel architectures and customize every layer, loss function, and optimization
strategy.
• TorchScript: Allows models to be transformed from eager execution (imperative
mode) to static graph mode for optimized production deployment.
• Strong community and research focus: PyTorch is widely used in academia, and
many state-of-the-art models are developed with PyTorch.
Pros:

• Very intuitive and flexible, great for research and experimentation.

• Easier debugging due to dynamic computation graphs.
• PyTorch Lightning: A higher-level framework for simplifying training and scaling
models.
• Excellent community support and strong momentum in academic research.

Cons:

• Less mature production ecosystem compared to TensorFlow (though this is

improving).
• Larger models and production systems may require additional tools like TorchServe
or third-party solutions for model serving.

Installation of Tensorflow On Ubntu:

1. Install the Python Development Environment:
You need to download Python, the PIP package, and a virtual environment. If these packages
are already
installed, you can skip this step. You can download and install what is needed by visiting the
following links:
https://www.python.org/
https://pip.pypa.io/en/stable/installing/
https://docs.python.org/3/library/venv.html
To install these packages, run the following commands in the terminal:
sudo apt update
sudo apt install python3-dev python3-pip python3-venv

2. Create a Virtual Environment:

Navigate to the directory where you want to store your Python 3.0 virtual environment. It can
be in your
home directory, or any other directory where your user can read and write permissions.
mkdir tensorflow_files
cd tensorflow_files
Now, you are inside the directory. Run the following command to create a virtual
environment: