Window Functions
Window Functions
Window Functions
Data Transformation
WINDOW
FUNCTIONS
DHANESH SARPALE
BASIC EXAMPLE
DHANESH SARPALE
GENERIC SYNTAX
MySQL
SELECT <column_list>,
<aggregate_function>(<column_expression>) OVER
(
PARTITION BY <partition_expression>
ORDER BY <order_expression>
ROWS <window_frame>
) AS <alias>
FROM <table_name>;
Python
import pandas as pd
df['<alias>'] = df['<column_expression>'].<aggregate_function>
().\
groupby(<partition_expression>).\
<transform_function>()
DHANESH SARPALE
LIST OF SQL WINDOW FUNCTIONS
DHANESH SARPALE
DATA ENGINEERING COMMON
OPERATIONS WITH WINDOW
FUNCTIONS
aggregating, transforming, and analyzing data
within precise partitions or windows
1. Data Aggregation
2. Data Cleansing
3. Data Enrichment
4. Data Partitioning
5. Data Ordering
DHANESH SARPALE
1. DATA AGGREGATION
DHANESH SARPALE
1. DATA AGGREGATION
MySQL
Python
import pandas as pd
# Assume you already have the data loaded into a pandas DataFrame called
'df'
# Calculating the sum and average sales for each product and category, and
overall sum and average
df['category_total_sales'] = df.groupby('category')['sales'].transform('sum')
df['category_avg_sales'] = df.groupby('category')['sales'].transform('mean')
df['overall_total_sales'] = df['sales'].sum()
df['overall_avg_sales'] = df['sales'].mean()
DHANESH SARPALE
2. DATA CLEANSINS
DHANESH SARPALE
2. DATA CLEANSING
MySQL
Python
import pandas as pd
DHANESH SARPALE
3.DATA ENRICHMENT
DHANESH SARPALE
3.DATA ENRICHMENT
MySQL
SELECT product_id, sales,
AVG(sales) OVER (ORDER BY date_column ROWS
BETWEEN 2 PRECEDING AND CURRENT ROW) AS
moving_average,
SUM(sales) OVER (ORDER BY date_column) AS
running_total,
SUM(sales) OVER (ORDER BY date_column) AS
cumulative_sum
FROM sales_data;
Python
# Calculate the 3-day moving average of sales for each
product
df['moving_average'] = df['sales'].rolling(window=3,
min_periods=1).mean()
DHANESH SARPALE
4. DATA PARTITIONANING
DHANESH SARPALE
5.DATA ORDERING
DHANESH SARPALE
Thank you for taking the time to read
this document! If you found it valuable,
I would greatly appreciate it if you
could show your support by liking and
sharing it with your network. I am
eager to connect with you on LinkedIn,
Let's connect and collaborate to foster
growth together!
DHANESH SARPALE