Data-Engineering Compressed

Download as pdf or txt
Download as pdf or txt
You are on page 1of 20

Exceling in Data Engineering:

A Comprehensive Course Syllabus

www.sevenmentor.com
DATA ENGINEERING
Course Mission
SevenMentor Institute leads the way in providing cutting-edge
IT Training and Skill Development across India. We have strived to
establish an ideal learning atmosphere at all our training centers. We
prepare our students to become dependable future professionals.
Our institute aspires to promote universal access to learning for
students. To achieve our mission of promoting better learning we
invite all students to enroll in our Data Engineering Course. Join us
today to make a fulfilling career in Data Engineering.

02
DATA ENGINEERING
STATISTICS:

75% Companies will use some form of


Data Engineering based Application

INDUSTRY
INSIGHTS 58% Year on Year Growth For All
Data Engineering Jobs

Learn Data Engineering and


Be in Demand Always!
Data Engineering, the unsung hero of the data ecosystem, is the
discipline dedicated to the design, construction, and maintenance of
the infrastructure that enables the seamless flow and storage of data.
Think of it as the architect and builder of the data world, responsible
for constructing the pipelines and structures that house the
information underpinning the digital landscape. It's the backbone of
data-driven decision-making, ensuring that data is collected,
processed, and made accessible to data analysts, scientists, and
decision-makers. In the realm of data engineering, data becomes your
raw material, and tools like ETL processes and data warehouses are
your building blocks, constructing the foundations of data-driven
enterprises

03
Unlock the Potential of Data Engineering:
Achieve Boundless Innovation

If your aspiration is to attain expertise in Data Engineering, a realm


revered for its limitless possibilities and far-reaching influence, it is
essential to set out on a perpetual voyage dedicated to continuous
growth and competence within this ever-evolving field. Data
Engineering stands out for its flexibility and extensive applicability
across a myriad of sectors, enabling enthusiasts to craft a diverse
range of ingenious, forward-thinking, and game-changing solutions.

04
Be Prepared For Every Scenario!
Gain practical Data Engineering experiences through well designed
courses, latest tools and excellent teachers.

Experienced Faculty Flexible Scheduling Hands-On Learning

Mock Interview Sessions Real-World Projects Career Support

Comprehensive Curriculum Lifetime Access

05
PYTHON

Ÿ What is Python and history of Python?


Ÿ Installing Anaconda, Jupyter Notebook
Ÿ First Program
Ÿ Python Identifiers, Keywords and Indentation
Ÿ Comments
Ÿ Getting User Input
Ÿ Python Data Types
Ÿ What are keywords
Ÿ What are variables?
Ÿ Python Inbuilt Functions

Control-Flow Statements: Data Structures:

Ÿ If-else Ÿ What are Data Structures?


Ÿ Elif Ÿ Lists in Python
Ÿ While loop Ÿ Code Walkthrough on Lists
Ÿ For loop Ÿ Understanding Iterators
Ÿ Range Function Ÿ Tuple in Python
Ÿ Break Ÿ Code Walkthrough on Tuple
Ÿ Continue Ÿ Dictionaries in Python
Ÿ Assert Ÿ Code Walkthrough on Dictionaries
Ÿ Pass Ÿ Sets in Python
Ÿ Return Ÿ Code Walkthrough on Sets
Ÿ Coding Assignment Ÿ More examples on Data Structures

Functions:

Ÿ What are functions in python? Ÿ Map Function


Ÿ Defining and Calling Functions Ÿ Filter Function
Ÿ Inbuilt Functions Ÿ Format Function
Ÿ User Defined Functions Ÿ Code Walkthrough on User Define
Ÿ Lambda Function Functions
Ÿ Split Function Ÿ Regular Expressions Basics
Ÿ Strip Function

06
Object Oriented Programming:

Ÿ Why do we need Object Oriented Programming?


Ÿ What is a class?
Ÿ What is an object?
Ÿ What is Self?
Ÿ Constructors
Ÿ Global and local variables
Ÿ Static and Dynamic Variables
Ÿ Abstraction
Ÿ Inheritance
Ÿ Encapsulation
Ÿ Polymorphism
Ÿ Code Walkthroughs on OOP
Ÿ Polymorphism
Ÿ Code Walkthroughs on OOP

Exception Handling and GUI:

Ÿ Why do we need to handle exceptions?


Ÿ Errors in Python
Ÿ Compile-Time Errors
Ÿ Runtime Errors
Ÿ What is Exception?
Ÿ try....except...else
Ÿ try-finally clause
Ÿ Raising an exceptions
Ÿ User Defined Exceptions
Ÿ Graphical User Interface in python
Ÿ Tkinter

07
Miscellaneous Topics:

Ÿ SQL connection with Python using SQLITE Library


Ÿ Multi-Threading and Multi-Processing
Ÿ Introduction to Web-scraping
Ÿ BeautifulSoup Library
Ÿ Numpy Library for Data Analysis
Ÿ Code Walkthrough On Numpy Library
Ÿ Pandas Library for Data Analysis
Ÿ Code Walkthrough On Pandas Library
Ÿ Matplotlib Library for Data Analysis
Ÿ Code Walkthrough On Matplotlib Library
Ÿ Revision Sessions
Ÿ Assignment Discussions

Project Discussion:

Ÿ Defining the Business Problem


Ÿ Constraints
Ÿ Flow Diagram
Ÿ Libraries Used
Ÿ Results and Conclusion
Ÿ Future Scope
Ÿ References

SQL

Ÿ What is SQL? Ÿ Introduction to MySQL


Ÿ Why do we need SQL? Ÿ Installation of MySQL server
Ÿ What is Data Base Management Ÿ Download sample database
System? Ÿ Load sample database to work.
Ÿ Types of DBMS
Ÿ Execution Of SQL query
Ÿ Difference Between SQL & MYSQL

08
Basic SQL Keywords:

Ÿ Basic SELECT Statement


Ÿ Limit/Offset
Ÿ OrderBy
Ÿ Distinct
Ÿ Where
Ÿ Comparison Operators
Ÿ Null
Ÿ Logical Operators
Ÿ Aggregate Operators(Count, Max, Min, Avg, Sum)
Ÿ Group By
Ÿ Having
Ÿ Order Of Keywords
Ÿ Wildcard Operators

JOINS: DML/DDL:

Ÿ What are Joins? Ÿ DML:Insert


Ÿ Inner Join Ÿ DML:Update, Delete
Ÿ Outer Join Ÿ DDL:Create Table
Ÿ Left Join
Ÿ DDL:Alter:Add,Drop,Modify
Ÿ Right Join
Ÿ DDL:Drop Table,Truncate,Delete
Ÿ Self Join
Ÿ DCL:Data Control Language:
Ÿ SubQueries/NestedQueries/Inner
Queries
GRANT,REVOKE
Ÿ Triggers
Ÿ Stored Procedures

PL-SQL

Ÿ Informal introduction to PL/SQL Advantages of PL/SQL


Ÿ Datatypes in PL/SQL
Ÿ Program structure of PL/SQL Embedding SQL statements
Ÿ Using conditional statements and loops
Ÿ What is cursor?
Ÿ How to create cursor?
Ÿ Using cursors in PL/SQL
Ÿ How to create explicit cursor? Creation of for loop cursor What are cursor
parameters?
Ÿ How to use for update clause? What is ref cursors?
Ÿ How to use implicit cursors?

09
Understanding Exception Handling:

Ÿ What is an Exception?
Ÿ Describing Exception types Handling system defined exceptions
Handling user defined
Ÿ exceptions? Sql code vs Sql errm
Ÿ Pragma exception_init

Creation Of Stored Procedures:

Ÿ Creating procedures in Pl/SQL


Ÿ Working with procedure parameters
Ÿ IN parameter
Ÿ OUT parameter
Ÿ IN OUT parameter
Ÿ How to create procedures with cursors
Ÿ How procedures return records?
Ÿ What is Pragma autonomous transaction?

Creating & Using Functions:

Ÿ Importance of function
Ÿ How to create functions?
Ÿ Difference between procedures and functions
Ÿ How to use inline functions?

Creating & Using Packages:

Ÿ What is a Package?
Ÿ Reasons to use packages
Ÿ What is package specification?
Ÿ What is package body?
Ÿ How to instantiate package?
Ÿ How to initialise instantiated package? What are all the package state?

10
Collections In PL/SQL:

Ÿ What is collection?
Ÿ How to use arrays?
Ÿ Using nested tables
Ÿ How to use index by value?
Ÿ Listing types of collection methods.
Ÿ General overview and discussion about DBA Concepts

Introduction To Data Engineering/Data Warehouse

Ÿ What is Data Engineering?


Ÿ Use Cases, and Applications?
Ÿ Data Engineer or Data Scientist?
Ÿ What is DataWarehouse?
Ÿ Data Lakes
Ÿ Data Engineering Problems Tools of a Data Engineer
Ÿ Working with Different Databases Processing Tasks,
Ÿ Scheduling Tools, and Different Cloud Providers
Ÿ Why Cloud Computing, Use Cases, and Applications?
Ÿ Different Cloud Services

AWS

AWS Data Engineering Tools : Data Ingestion Tools :


Ÿ CORE python Ÿ Amazon Kinesis Firehose
Ÿ Sql and no sql Ÿ AWS Snowball
Ÿ Data Storage Tools Ÿ AWS Storage Gateway
Ÿ Data Integration Tools
Ÿ Data Warehouse Tools
Ÿ Data Visualization Tools AWS Storage Gateway : Data Warehouse Tools :

AWS Glue Amazon Redshift


Data Storage Tools:

Amazon S3 Data Visualization Tools :

Amazon QuickSight

11
MongoDB

Ÿ What Is MongoDB? Installation and Configuration MongoDB Data


Modelling Introduction to
Ÿ NoSQL Architecture with MongoDB MongoDB Advantages
Ÿ MongoDB Tools, Collection and Documents

CRUD and the MongoDB Shell:

Ÿ Introduction to CRUD
Ÿ Introduction to the MongoDB API
Ÿ Creating a Database, Collection and Documents

Data Modelling & Schema Design: Administration:

Ÿ MongoDB Database References Ÿ MongoDB Deployment and


Ÿ Model Tree Structures Cluster setup
Ÿ MongoDB Analysing Queries Ÿ MongoDB GridFS
Ÿ MongoDB Atomic Operations Ÿ Trident Spout
Ÿ MongoDB Map Reduce Ÿ Working with Replica Sets
Ÿ MongoDB Text Search Ÿ MongoDB Sharding
Ÿ MongoDB Regular Expression
Ÿ MongoDB Capped Collections

Indexing:

Ÿ Indexing and Aggregation


Ÿ Indexing, query profiling and the query optimisers
Ÿ GeoSpatial Indexes
Ÿ Index types, Index Properties
Ÿ MongoDB Advanced Indexing
Ÿ MongoDB Indexing Limitations
Ÿ Aggregation Introduction

12
SPARK

Hadoop Overview:

Ÿ Hadoop Overview:
Ÿ Need of Hadoop technology
Ÿ Hadoop Cluster and Racks in detail
Ÿ Overview of Map Reduce
Ÿ Big data Concepts and data types
Ÿ Concept of Streaming data and different tools utilisation
Ÿ HDFS and Basic Hadoop commands

Scala Programming:

Ÿ Scala overview and Environment Setup


Ÿ Oops concepts in scala
Ÿ Control Structure and Functions
Ÿ Closures and Collections
Ÿ Exception Handling in scala

Apache Spark 2 x Installation:

Ÿ Download release and set


Ÿ Working with eclipse
Ÿ Installing Scala IDE with spark
Ÿ Testing with different OS

Working with Apache Spark:

Ÿ RDD and its Transformations


Ÿ Working with Eclipse Maven, Spark context and RDD
Ÿ Working with different file formats
Ÿ Introduction to Spark Data Frame
Ÿ Data Frames and RDD's with with Spark 1.x and 2.x style
Ÿ Creating Multiple Spark Context and Spark Sessions
Ÿ Applying Own Schema to the Data Frame and basic operations.
Ÿ Creating Datasets and its basic operations
Ÿ Dataset vs DataFrame Performance

13
Working with Apache Spark:

Ÿ Running Spark Job in Yarn/cluster Mode From IDE


Ÿ Spark with Mysql, transformations On MySQL
Ÿ Table - DataFrame API 12.Query Push Down to MySQL Database
Ÿ Creating Partitioned Table with Spark
Ÿ Spark built-in functions and UDF
Ÿ Examples with spark Sql and RDD's
Ÿ Spark job submit

Spark Streaming:

Ÿ Working with data stream 2. Example of network


Ÿ Twitter data stream
Ÿ Twitter data analysis cases

Kafka:

Ÿ Fundamentals of kafka, Work Flow and Basic Operations


Ÿ Creating Topics, Partition, Replication, Broker and Kafka cluster
Ÿ Working with Producer and Consumer Examples
Ÿ Creating Consumer Group, Leaders, Followers
Ÿ Starting brokers, Listing and modifying topics
Ÿ Single Node-Multiple Brokers Configuration and Creating Producer,
Consumer and Consumer
Ÿ group application
Ÿ Running a jar files from terminal

Real time case studies:

Ÿ Working on different data sets

SparkMlib:

Ÿ Classification algorithm
Ÿ Clustering algorithm

14
Get Skills To Fulfill Every Role:
Every student at SevenMentor gets personalized guidance,
Mentorship, and ample opportunities to address individual
questions and concerns. All our sessions are designed to be
engaging, interactive, and tailored to your learning pace,
ensuring you grasp each concept with clarity.

Aniket Kulkarni

He have theorotical as well as practical experience in the


field of ML. He did M Tech and have 10+ years of Industrial
and Academic experience.
Currently working as Data Science Trainer. He have trained
2000 + professionals and students for the course of Python,
SQL, Power BI, Machine Learning and Deep Learning.

Karishma Pawar

Karishma holds a PhD in Computer Engineering,


specializing in cutting-edge fields including Artificial
Intelligence and Machine Learning. In addition to more
than 8 years of experience in Data Science, remarkable
publications in peer-reviewed journals and conferences
are to her credit. She has previously worked with Avaya
and Infosys, and demonstrated a commitment to
advancing the frontiers of knowledge.

15
Get Skills To Fulfill Every Role:
Our Data Engineering Courses are designed for a wide range of people
looking for skills and opportunities across all major IT sectors

Hands-On Projects: Gain practical Career Support: We're dedicated to


experience by working on real- your success! Benefit from career
world projects, building a robust guidance, resume building,
portfolio that will impress potential interview prep, and job placement
employers.

Flexibility: Our flexible schedule Community: Join a vibrant


options allow you to learn at your community of like-minded learners,
own pace, making it perfect for where you can collaborate, share
both beginners and experienced ideas, and network with peers.
developers looking to upskill.

Our Students are at reputed Tech Companies

16
DATA ENGINEERING
JOBS ARE ALSO VERY STABLE!
The demand for Data Engineering professionals is growing rapidly, so there is a lot
of job security in this field. This can be a great motivator for people who are looking
for a stable career.

Learners’ Industry 65%


Background Product

Others 10%

25%

Services

Learners’ Work
Experience

2-3 Years 0-1 Years 1-2 Years 5+ Years

17
Non-IT and Others
Learners’ Tech
Expertise Data Engineering

Professionals

10% 21% 24% 45%

BOOST YOUR CAREER TO


NEW HEIGHTS:
The global Data Engineering job market is expected to grow by
44% from 2021 to 2030, creating 3.5 million new jobs.
In India it is expected that 309,000 new Data Engineering jobs will
be available by 2030, accounting for 9% of the global demand. The
average salary for an Data Engineering professional in India is
approximately Rs. 05 to Rs. 14 Lakhs per annum.

18
Affordable Training without Compromise:

We recognize that realizing your aspirations in the field of


Data Engineering should not strain your finances. At our
institution, affordability is at the heart of our Data
Engineering Program. We firmly believe that exceptional
education should be within reach for all, and we've
meticulously designed our curriculum with this principle in
mind. Our Data Engineering Program promises an
exhilarating learning journey at a remarkably affordable cost.

HOW TO START YOUR


CAREER IN DATA ENGINEERING

Ÿ Enroll at SevenMentor Institute

Ÿ Get hands-on training from Experienced Teachers

Ÿ Receive Industry-recognized Data Engineering


Ÿ Certification

Ÿ Work for leading MNCs through our on-campus


interviews

19
WE ARE THERE FOR YOU
If you are interested in learning more about
Data Engineering training, please contact us. Our
team would be happy to answer any questions you
have and help you find the right training for you.

Request For Call Back

020 7117 1500

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy