Scribd
Upload
41 views17 pages

Data_Engineer_Interview_1737983905

Uploaded by

ash.austyn99
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
41 views17 pages

Data_Engineer_Interview_1737983905

Uploaded by

ash.austyn99
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

\

Azure Data Factory


Interview Scenario
Interviewer:
Can you explain how you would
handle a scenario in Azure Data
Factory where you need to
dynamically load multiple files based
on certain conditions, like a yes/no
logic?
Candidate:

We can handle such scenarios using a combination of


for-each loops, if-else activities, and dynamic content

Here's a step-by-step explanation to achieve this:

Get List of Files: Use a Get Metadata activity to retrieve


the list of files from a specified storage location.

For-Each Loop: Iterate through each file using the For


Each activity.

If Condition Activity: Within the For Each activity, use


an If Condition activity to implement the yes/no logic.
Interviewer:

How would you configure the Get


Metadata activity to retrieve the list of
files?
Candidate:

Get Metadata activity is configured to point


to the directory containing the files. You set
the field list to 'Child Items' which retrieves
names of all files and folders in the
directory. The output is then passed to the
For Each activity.
Interviewer:

How do you set up the For Each


activity to iterate through files?
Candidate:

In the For Each activity, you set the items


property to the output of the Get Metadata
activity, specifically the 'childItems' array. This
configuration ensures that the For Each activity
will iterate through each file in the directory
Interviewer:

How do you use the If Condition


activity to implement the yes/no logic
for each file?
Candidate:
Inside the For Each activity, you place an If
Condition activity. The condition expression can
use pipeline parameters or the file metadata to
evaluate whether to proceed with processing the
file or not. For example:

@if(equals(activity('Get
Metadata').output.itemName, 'Yes'))

If the condition is true, the pipeline will execute


the 'If True' activities; otherwise, it will execute
the 'If False' activities.
Interviewer:

Can you give an example of how the


yes/no logic might work in practice?
Candidate:

Suppose we have a metadata file that


indicates whether each data file should be
processed. The metadata file contains entries
with the file name and a 'Process' flag
(Yes/No). The If Condition activity checks this
flag for each file:
If 'Process' is 'Yes', the pipeline proceeds to
load the file.
If 'Process' is 'No', the pipeline skips the file.
Interviewer:

How do you dynamically load the files


if the condition is met?
Candidate:

If the condition is met (Process = Yes), you


can use a Copy Data activity within the 'If
True' branch of the If Condition activity. This
activity copies the file from the source to the
destination. The source and destination
settings can be parameterized to handle
different files dynamically.
Interviewer:

What happens if the condition is not


met?
Candidate:

If the condition is not met (Process = No), the


'If False' branch can either contain no
activities (effectively skipping the file) or log
the file name and reason for skipping it. This
can be done using a Set Variable or Append
Variable activity to store the information for
later use.
FOR CAREER GUIDANCE,
CHECK OUT OUR PAGE
www.nityacloudtech.com

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy