1 TKK - ETL Testing/Big Data Testing | Road Map | 2024

ROAD MAP FROM NON-TECH TO BIG DATA TESTING

1. IT – already working in IT companies or some IT related role.
2. Non-IT
I. I will consider all – from BA, B. Com, BCA & other degree as non-IT)
II. Even those who are working in non-it related roles like Support
III. IT support or HR, or Facility etc.
2. It is difficult directly to enter Big Data space.
3. I would recommend you all to start with ETL/DWH & then the next step should be either cloud
or Big Data.

BASIC TERMINOLOGIES
Learn basic terminologies used in IT. For example – what is a table, what is frontend, what is
backend. What is column, row, metadata, flat file, BI tool, users, DBMS, Database, RDBMS. Etc.
etc.

LEARN DWH CONCEPTS

1. What is DWH? History?
2. Languages used to deal with Data.
3. Need of DWH?
4. OLTP, OLAP. - https://www.youtube.com/playlist?list=PLhXtefn-
T4nj9tSlufJRC0xWtvSPUHvQ5
5. What is ETL? -
6. Characteristics of DWH. - https://www.youtube.com/watch?v=Lbx2_OYEUGE
7. Normalisation, denormalization.
8. ETL architecture.
9. Types of Data models - https://www.youtube.com/playlist?list=PLhXtefn-
T4nj9tSlufJRC0xWtvSPUHvQ5
10. Types of schemas – star, snowflake - https://www.youtube.com/playlist?list=PLhXtefn-
T4nj9tSlufJRC0xWtvSPUHvQ5
11. ETL Project architecture – I have explained that in the previous video.

PLAYLIST - https://www.youtube.com/playlist?list=PLhXtefn-T4nj9tSlufJRC0xWtvSPUHvQ5

www.TechieKrishnaKayaking.com 1
2 TKK - ETL Testing/Big Data Testing | Road Map | 2024

PLAYLIST -
https://www.youtube.com/playlist?list=PLUDwpEzHYYLtW8gTLTr5IB9e724Wlc312

www.TechieKrishnaKayaking.com 2
3 TKK - ETL Testing/Big Data Testing | Road Map | 2024

SQL
1. What is SQL?
2. Learn about Datatype, variable, syntax.
3. Learn about DDL, DML, TCL, DCL.
4. Create, alter, drop, etc. practice them.
5. DMLs – insert, update, delete, select, truncate.
6. As a tester we will mostly be working on Select commands, but we should have good
knowledge of other DDLs, like create, update delete.
7. Clauses – distinct, order by, where, group by, having.
8. AND & OR conditions.
9. FUNCTIONS
I. Character/String functions
II. Aggregate functions.
III. Analytical functions.
IV. Date functions.
10. SET OPERATORS – union, union all, interest, minus etc.
11. CONSTRAININTS – unique, not null, primary key, foreign key, etc.
12. JOINS – inner, cross, self, outer (left, right, full)
13. SUB-QUERIES
14. VIEWS

PLAYLIST -
https://www.youtube.com/playlist?list=PLuavSVFLG71sW63iz8hKC5l8nGYrLpqTt

www.TechieKrishnaKayaking.com 3
4 TKK - ETL Testing/Big Data Testing | Road Map | 2024

PLAYLIST - https://www.youtube.com/playlist?list=PLhXtefn-T4nix9ORxAMAjwCjtlj6rNbpc

www.TechieKrishnaKayaking.com 4
5 TKK - ETL Testing/Big Data Testing | Road Map | 2024

UNIX
1. This is like the lungs of ETL/DWH/Big Data tester.
2. SQL is the heart & UNIX is the lungs.
3. As I mentioned you do not have to be an expert, but you should have good knowledge.
4. I would recommend you to please install virtual box & install any of the Linux environment &
practice.
5. Please comment if you need tutorial on UNIX commands & setting up environment.

PLAYLIST - https://www.youtube.com/playlist?list=PLhXtefn-T4nj0UAxvV6ZZKJQ9awzMfxrn

TESTING CONCEPTS
1. SDLC – Software Development Life Cycle & Types

a. Sequential Model - Water fall & V-Model

b. Iterative Model - Agile Model
c. Advantages & Disadvantages

2. Learn the details of AGILE METHODOLOGY

a. All the different phases in detail

b. This is one of the most important question
www.TechieKrishnaKayaking.com 5
6 TKK - ETL Testing/Big Data Testing | Road Map | 2024

c. Learn about the different roles –

d. Comment “AGILE” if you would like to me do one video separately on this topic

3. Learn Scrum Framework

a. Comment “SCRUM Framework” if you would like me to make a separate video on this.

4. STLC – Software Testing Life cycle

a. All the different phases

b. Since now you will be giving interview as tester, so you must be very good at this.
c. Comment STLC, if you would like to me to make a video on this.

5. Refer Testing documents like

a. BRD – Comment the Full form

b. Test plan
c. Test Report
d. Test summary/coverage

6. Defect life cycle – Comment DLC, if you want me to make a separate video

7. Black box, White box, Grey box – basic knowledge

8. Different Roles in a project.

9. Levels of testing

a. Basic knowledge of all of it

b. Unit, Smoke, Sanity, System, Regression, UAT etc.
c. Comment “Levels of Testing” if you want to me to make one separate video on this
topic.

10. Test Case template.

a. I will make a sperate video on this, on how I design test cases & what template I use.
b. Comment.

www.TechieKrishnaKayaking.com 6
7 TKK - ETL Testing/Big Data Testing | Road Map | 2024

11. Requirement Traceability matrix

12. Test Strategy

13. Black box testing

a. Equivalence class partition

b. Boundary value Analysis
c. Decision Table Testing
d. Cosmetic testing
e. Exploratory Testing

ETL TESTING CONCEPTS

1. What is ETL
i. Extract (DBs, Flat file, XML file, Excel file, Parquet file)
ii. Transform
iii. Load
iv. https://www.youtube.com/watch?v=v-CuglZL6gw&t=515s
2. Test strategy - https://www.youtube.com/watch?v=29AIg81An3A&t=5s
i. Data completeness
ii. Data Transformation
iii. Data Quality
iv. Initial Load | Full load | First load
v. Incremental load | Second load | Delta load
vi. Presentation layer (BI/Report)
vii. Job Dependency
3. Learn one ETL Tool (from QA/Testing perspective)
i. Informatica PC
ii. Talend
iii. SSIS
iv. IBM DS
v. Any other ETL Tool
vi. Comment some of the famous ETL Tools
4. STM – Source Target Mapping Document
i. What is STM

www.TechieKrishnaKayaking.com 7
8 TKK - ETL Testing/Big Data Testing | Road Map | 2024

ii. Importance of STM

5. Data Transformation Test
i. Data Cleansing
ii. Data Merging
iii. Data Scrubbing
iv. Data Aggregation
6. TRANSFORMATIONS - https://www.youtube.com/watch?v=wgXft655kYQ
i. Aggregation transformation
ii. Expression transformation
iii. Filter transformation
iv. Joiner transformation
v. Router transformation
vi. Sequence transformation
vii. Sorter transformation
viii. Lookup transformation
7. Challenges in DWH/ETL Testing
8. File comparison testing approach
i. There are multiple ways of doing this
ii. Comment if you want me to make a video on this
iii. This is a must question in any interview
iv. Around 60% of the interviewers love this question
v. https://www.youtube.com/playlist?list=PLhXtefn-
T4njTQlqQhMx02cKDP41n4WMa
9. SCD – Slowly Changing Dimension
i. Comment if you want me to make a video on this topic
ii. One of the most interesting topic (all interviewers like this one)
iii. https://www.youtube.com/playlist?list=PLhXtefn-
T4nj9tSlufJRC0xWtvSPUHvQ5
10. LOADING
i. Initial load | Full load | First load Testing
ii. Incremental load | Second load | Delta load Testing
11. Job Dependency Testing
i. Not so important
ii. Good to know
iii. Job Dependency document
12. Active & Passive Transformations - https://www.youtube.com/watch?v=wgXft655kYQ

13. Defect Life cycle – from ETL/DWH QA Perspective -

https://www.youtube.com/watch?v=63guoKq22Ls

www.TechieKrishnaKayaking.com 8
9 TKK - ETL Testing/Big Data Testing | Road Map | 2024

PLAYLIST -
https://www.youtube.com/playlist?list=PLUDwpEzHYYLtW8gTLTr5IB9e724Wlc312

www.TechieKrishnaKayaking.com 9
10 TKK - ETL Testing/Big Data Testing | Road Map | 2024

www.TechieKrishnaKayaking.com 10
11 TKK - ETL Testing/Big Data Testing | Road Map | 2024

BIG DATA TESTING CONCEPTS

1. What is Hadoop?
2. Hadoop architecture.
3. What is MapReduce?
4. HDFS, Hadoop distributed file system.
5. How Hadoop works?
6. Advantages and disadvantages of Hadoop?
7. Hadoop ecosystem
8. Apache Hive
9. Apache Pig.
10. Apache Hbase.
11. Apache Mahout
12. HDFS services, Architecture.
13. Very, very important characteristics of big data. The 5V's.
a. Volume
b. Velocity
c. Variety
d. Value
e. Veracity
14. Types of data.
a. Structured.
b. Unstructured
c. Semi structured.
15. Linux commands Our Unix commands.
16. HDFS commands
17. Hive
a. Built-in functions
b. Aggregate functions.
c. Joins

www.TechieKrishnaKayaking.com 11
12 TKK - ETL Testing/Big Data Testing | Road Map | 2024

d. User defined functions.

18. Scoop Jobs, How to run scope jobs?
19. Pig -
a. Grouping and joining.
b. Built-in functions.
c. String functions. Max line modes of execution.
20. Big data testing -
a. Data staging validation.
b. MapReduce validation.
c. Target data validation.
21. Tools used in big data

PLAYLIST - https://www.youtube.com/playlist?list=PLUDwpEzHYYLvFc8ic8eAsN7aFki9YlvRL

www.TechieKrishnaKayaking.com 12
13 TKK - ETL Testing/Big Data Testing | Road Map | 2024

www.TechieKrishnaKayaking.com 13
14 TKK - ETL Testing/Big Data Testing | Road Map | 2024

CLOUD DATA TESTING CONCEPTS

1. AWS – Glue jobs

a. How to run
b. How to check the logs
c. How to debug
d. How to monitor jobs
2. AWS – S3
a. File system
b. ETL/ELT – how it works in S3
3. AWS Database [stg, dwh]
e. Redshift
f. AWS RDS
g. AWS SQL Server
h. Dynamo DB
i. AWS Oracle
j. AWS MySQL
4. AWS QuickSight
a. How it works?
b. How can you do QA on BI

www.TechieKrishnaKayaking.com 14
15 TKK - ETL Testing/Big Data Testing | Road Map | 2024

1. Azure – Data Factory

a. How to run the jobs
b. How to check the logs
c. How to debug
d. How to monitor jobs
2. Azure – Blob storage
a. File system
b. Naming conventions
3. Azure Database/SQL
a. ETL/ELT
b. Staging layer
c. DWH
4. Power BI

www.TechieKrishnaKayaking.com 15