Scribd
Upload
1 views

AWS s3 to snowflake data load_v01

The document outlines the steps to load data from AWS S3 into Snowflake, including creating a table, uploading a file, and setting up IAM roles and storage integration. It details the creation of a Snowflake stage and the execution of a copy command to load the data. Additionally, it provides instructions for setting up a Snowpipe for automated data loading with event notifications in S3.

Uploaded by

manoj
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
1 views

AWS s3 to snowflake data load_v01

The document outlines the steps to load data from AWS S3 into Snowflake, including creating a table, uploading a file, and setting up IAM roles and storage integration. It details the creation of a Snowflake stage and the execution of a copy command to load the data. Additionally, it provides instructions for setting up a Snowpipe for automated data loading with event notifications in S3.

Uploaded by

manoj
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 7

AWS s3 to snowflake data load

Steps
1. Create a table to accept the data from the s3
2. Upload a file in the s3 bucket
3. Create the Role and give the permissions and create the trusted policy
4. Create the storage integration in snowflake
5. Create the stage in snowflake
6. Run the copy command to load the data

Create a table in snowflake to accept the data from the s3


Create or replace TABLE MY_DB.public.EMP_LOAD (

EMPID NUMBER (38,0),

ENAME VARCHAR (20),

SALARY NUMBER (38,0),

DEPTID NUMBER (38,0)

);

Create a file in the s3 bucket


Create a folder inside the s3 and upload the csv files

Now we are loaded the file to the cloud AWS- s3 bucket

Create IAM for access creations

Select Role in the Left side and create Role


Select AWS Account

Click On Next and choose amazon s3 full access

Click the Check box and then


Give the Rolename and the description and create Role

Again goto the Roles in the left pane and select the created Role

ARN Number has been highlighted here and copy and paste it somewhere this has to be added in the
snowflake DB while creation of the storage integration

Create the storage integration in snowflake


Create or replace storage integration aws_storage

type = external_stage

storage_provider = s3

storage_aws_role_arn = 'arn:aws:iam::759489707541:role/AWS_S3_INTEGRATION'

enabled = true

storage_allowed_locations = ('s3://aimore-snowflake-s3/EMP01/');

Red highlighted number is the ARN number that must be pasted from the AWS
Green Highlighted id is the allowed locations , this has to be copied from the s3 (refer the below
image)

After creation of the storage integration

Just execute the below in snowflake

desc integration aws_storage;

Copy this ARN and paste it into the trusted policy of AWS and update the trusted policy
Create the stage in snowflake
create or replace stage aws_Stage

url= 's3://aimore-snowflake-s3/EMP01/'

storage_integration= aws_storage

Run the copy command to load the data


Copy into tableanme

From @stagename;

Snow Pipe Extra contents


1. Extra we need to create the pipe object in Snowflake
2. Execute the show pipes command and see the output column called the notification channel
3. Copy the value of notification channel
4. Open the s3 bucket and select it and see the properties tab and go down and see the event
notification
5. Use Create button to create the new event notification and give any of the event name!
6. Select the radio button all object create events

From the above image choose those options and give the notification channel value which is
copied from the snowflake and create the event!

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy