ADHIYAMAAN COLLEGE OF ENGINEERING

(Autonomous)

DEPARTMENT OF COMPUTER SCIENCE AND

ENGINEERING

818CIT01 – BIG DATA ANALYTICS

ASSIGNMENT - I

Submitted by:
ILANTHENRAL V
6176AC21UCS046
IV – CSE – A
1. Retail Sales Analysis using Data Warehousing
Retail businesses generate an enormous amount of transactional data every day. This data
is crucial for analyzing sales trends, managing inventory, and understanding customer
behavior. A data warehouse provides an efficient way to store, retrieve, and analyze this
data for better decision-making.
i. Data Warehouse Schema Design
A star schema is one of the most effective and commonly used designs for sales analysis. It
consists of a central fact table that contains sales transaction data, surrounded by multiple
dimension tables that provide additional details about products, customers, stores, and
time.
A star schema is widely used in retail sales analytics due to its simplicity and efficiency.
Fact Table: Sales_Fact

Column Name Data Type Description

sale_id INT (PK) Unique identifier for each sale

date_id INT (FK) Foreign key to Date dimension

product_id INT (FK) Foreign key to Product dimension

store_id INT (FK) Foreign key to Store dimension

customer_id INT (FK) Foreign key to Customer dimension

quantity_sold INT Number of units sold

total_sales DECIMAL Total revenue from the sale

discount DECIMAL Discount applied

Dimension Tables:
1. Date Dimension (Date_Dim): Contains fields such as date_id, date, month, year,
quarter, day_of_week.
2. Product Dimension (Product_Dim): Holds product-related details such as
product_id, product_name, category, brand, price, and supplier.
3. Store Dimension (Store_Dim): Includes store_id, store_name, location, region,
store_type, and manager.
4. Customer Dimension (Customer_Dim): Consists of customer_id, customer_name,
age, gender, loyalty_status, purchase_frequency.
ii. Data Preprocessing and Transformation
Before data is stored in the warehouse, it needs to be cleaned and transformed for
accurate reporting and analysis. The ETL (Extract, Transform, Load) process ensures that
data is collected from various sources, transformed into a consistent format, and loaded
into the warehouse.
Steps in Data Preprocessing:
1. Data Extraction:
o Collects sales transactions, customer information, and product details from
multiple sources such as POS systems, online stores, and CRM systems.
2. Data Cleaning:
o Handles missing values, removes duplicate records, and corrects
inconsistencies in product names, dates, and prices.
3. Data Normalization:
o Converts different currency formats, standardizes units (e.g., kilograms to
grams), and encodes categorical data (e.g., converting gender as Male=1,
Female=0).
4. Data Aggregation:
o Summarizes data to create new features like total revenue per store,
average basket size, and seasonal sales trends.
5. Data Loading:
o Stores the cleaned and structured data into the data warehouse for future
queries and analysis.
iii. Sales Analysis Metrics
The data warehouse enables advanced analysis to improve sales performance and
business strategies.
Key Metrics for Sales Analysis:
1. Total Revenue per Store, Product, and Region
o Helps in identifying high-performing stores and products.
2. Customer Segmentation Based on Purchase Behavior
o Groups customers into categories such as frequent buyers, occasional
buyers, and inactive customers.
3. Trend Analysis Over Time
o Analyzes how seasonality affects sales (e.g., higher sales in December due
to holiday shopping).
 4. Inventory Optimization
o Helps businesses prevent overstocking or stockouts by predicting demand
trends.
5. Profit Margin Analysis
o Identifies products with the highest and lowest profit margins.
iv. Business Benefits of Using Data Warehousing in Retail
1. Improved Decision-Making
• Store managers can use real-time sales reports to optimize staffing and inventory.
2. Personalized Marketing and Promotions
• Based on customer purchase history, businesses can send targeted offers and
promotions to increase sales.
3. Fraud Detection and Prevention
• Abnormal sales patterns (e.g., sudden high refunds or unusual discounts) can be
flagged to prevent fraud.
4. Supply Chain Optimization
• Helps in determining optimal reorder levels and minimizing supply chain
disruptions.

Diagram: Retail Data Warehouse Workflow

Sales Transactions ---> Data Cleaning & Transformation ---> Data Warehouse
| |
Customer Profiles Aggregated Reports for Analysis
| |
Store Performance Inventory & Demand Forecasting
|
Business Intelligence & Decision-Making

v. Case Study: Walmart’s Use of Data Warehousing

One of the best examples of data warehousing in retail is Walmart.
• Walmart collects over 2.5 petabytes of data per hour from millions of transactions
across its stores.
 • The company uses data warehouses and big data analytics to optimize pricing
strategies, predict demand, and manage supply chains efficiently.
• By analyzing weather patterns, Walmart discovered that before hurricanes,
customers buy more flashlights and Pop-Tarts—this helped them stock stores
appropriately and maximize sales.

2. Customer Loyalty Classification using Machine Learning

Customer loyalty is a critical factor for business success. Loyal customers contribute to
repeat sales, brand advocacy, and higher customer lifetime value (CLV). Machine learning
(ML) can help businesses predict whether a customer is loyal or not based on their
purchase behavior, demographics, and engagement history.i. Data Collection for Loyalty
Classification
i. Data Collection for Customer Loyalty Classification
To build a machine learning model for customer loyalty classification, we need various
data sources.
1. Transactional Data
• Purchase history: Number of purchases, total spending, average spending per
order.
• Discount usage: Whether customers frequently use discount coupons or loyalty
points.
• Purchase frequency: How often the customer makes purchases (e.g., weekly,
monthly, yearly).
2. Customer Demographics
• Age, gender, location, income level.
• Customer type: New, returning, VIP.
3. Behavioral Data
• Time since last purchase (Recency).
• Product categories purchased (e.g., electronics vs. groceries).
• Website behavior: Time spent browsing, abandoned carts.
• Engagement metrics: Email open rates, responses to promotions.
ii. Feature Engineering
Feature engineering involves creating new variables that improve model performance.
1. Important Features for Classification:
• purchase_frequency = total_orders / months_active
• avg_spent_per_order = total_spent / total_orders
• recency = days_since_last_purchase
• discount_usage_rate = total_discount_used / total_orders
• loyalty_score = weighted_sum(purchase_frequency, avg_spent_per_order, recency,
discount_usage_rate)
2. Feature Scaling and Encoding:
• Numerical features (e.g., total_spent, avg_spent_per_order) are normalized using
Min-Max Scaling.
• Categorical features (e.g., customer type, location) are one-hot encoded.
iii. Machine Learning Model Selection
Different machine learning algorithms can be used to classify customers into loyal or not
loyal.
1. Logistic Regression (Baseline Model)
• Predicts the probability of a customer being loyal based on their transaction
history.
• Works well for interpretable models but may not capture complex relationships.
2. Decision Trees & Random Forest
• Decision Trees split customers into different categories based on spending and
behavior.
• Random Forest improves accuracy by combining multiple trees to reduce
overfitting.
3. XGBoost (Extreme Gradient Boosting)
• A powerful model that handles large datasets efficiently.
• Often used in real-world classification problems due to high accuracy.
4. Neural Networks (Deep Learning)
• Useful when there are complex customer interactions to analyze.
• Requires large amounts of data for training.
iv. Model Training and Evaluation
1. Train-Test Split
• Dataset is divided into 80% training and 20% testing for model evaluation.
2. Performance Metrics
• Accuracy: Percentage of correctly classified customers.
• Precision: How many predicted loyal customers were actually loyal.
• Recall: How many actual loyal customers were correctly identified.
• F1-Score: Balances precision and recall.

v. Business Impact of Customer Loyalty Prediction

1. Personalized Marketing Campaigns
• Predict which customers are at risk of churning and offer targeted discounts.
2. Customer Retention Strategies
• Identify customers who frequently shop but may leave soon and send personalized
offers.
3. Increase Revenue through Segmentation
• Loyal customers can be offered premium products, while non-loyal customers can
be given incentives to stay.
4. Reduce Customer Acquisition Costs
• Retaining existing customers is 5X cheaper than acquiring new ones. ML models
help optimize customer engagement strategies.

vi. Case Study: Amazon's Customer Loyalty Prediction

Amazon uses AI-driven customer segmentation to classify shoppers based on their buying
patterns, browsing history, and engagement levels.
• Customers with high engagement but low purchases receive personalized
discounts.
• Customers with high-value purchases get exclusive offers.
• Inactive customers are targeted with email campaigns and limited-time deals
3. Predicting Patient Outcomes using Machine Learning
Predicting patient outcomes is one of the most impactful applications of machine learning
in healthcare. Hospitals and healthcare organizations use predictive models to assess
recovery chances, readmission risks, mortality rates, and disease progression. By analyzing
historical patient data, ML models can assist doctors in early diagnosis, treatment
planning, and improving patient care.

i. Data Collection for Patient Outcome Prediction

To develop an accurate predictive model, we need diverse patient data sources.
1. Demographic Data
• Age, Gender, Ethnicity (Certain conditions affect different populations differently).
• Socioeconomic Factors (Income level, access to healthcare, diet, lifestyle).
2. Clinical Data
• Vital signs (Heart rate, blood pressure, oxygen levels).
• Medical history (Chronic diseases, past surgeries, allergies).
• Lab test results (Blood sugar, cholesterol, WBC count).
• Imaging data (X-rays, MRIs, CT scans).
3. Treatment Data
• Medication prescribed (Dosage, frequency, effectiveness).
• Surgical procedures performed (Post-surgery recovery rates).
• Length of hospital stay (Long stays indicate severe cases).
4. Outcome Labels
• Recovery (Did the patient recover fully or partially?).
• Readmission Risk (Did the patient return to the hospital within 30 days?).
• Mortality (Whether the patient survived or not).

ii. Data Preprocessing

Healthcare data is often incomplete and noisy, so preprocessing is essential before training
an ML model.
1. Handling Missing Data
• Imputation (Fill missing values using median, mean, or predictive methods).
• Removing incomplete records if too many fields are missing.
2. Normalization of Numerical Features
• Vital signs and lab results are standardized to ensure consistent scaling.
3. Encoding Categorical Variables
• Example: Converting disease types into numerical values using One-Hot Encoding.
4. Feature Selection
• Eliminating irrelevant attributes (e.g., patient names).
• Keeping only the most important predictors of patient outcomes.

iii. Machine Learning Model Selection

Different ML models can predict patient outcomes based on historical data.
1. Logistic Regression (For Binary Predictions: Recovery or Not)
• Used for classifying whether a patient will recover or not based on input
parameters.
• Works well for interpretable predictions (e.g., effect of age on recovery).
2. Decision Trees & Random Forest
• Decision Trees analyze patient records and classify outcomes.
• Random Forest improves accuracy by combining multiple trees to reduce
overfitting.
3. Support Vector Machines (SVM)
• Works well for smaller datasets.
• Identifies clear separations between patient classes.
4. XGBoost (Best for Large Medical Datasets)
• Used in predicting readmission risk, mortality rates, and recovery chances.
• Highly accurate and efficient for structured healthcare data.
5. Neural Networks (Deep Learning)
• Used for analyzing complex medical data (e.g., MRI scans, X-ray images).
• Requires a large amount of data but provides high accuracy.
iv. Model Training and Evaluation
1. Splitting the Dataset
• Training Set (80%) – Used to train the ML model.
• Test Set (20%) – Used to evaluate model performance.
2. Performance Metrics
• Accuracy – Measures overall prediction correctness.
• Precision & Recall – Important for imbalanced datasets (e.g., rare diseases).
• ROC-AUC Score – Measures how well the model distinguishes between recovery
and non-recovery cases.

v. Building a Data Pipeline for Patient Data Processing

A data pipeline automates the collection, transformation, and analysis of patient data.
Step-by-Step Pipeline Design:
1. Data Ingestion
o Collect patient data from Electronic Health Records (EHR), wearable devices,
and hospital systems.
o Store it in a secure data warehouse (e.g., AWS S3, Google Cloud Storage).
2. Data Cleaning & Preprocessing
o Remove duplicate entries.
o Standardize medical records.
3. Feature Engineering
o Extract key medical indicators from raw data.
o Normalize and encode categorical variables.

vi. Business & Clinical Benefits of Patient Outcome Prediction

1. Early Disease Detection
• Predict high-risk conditions like heart attacks, diabetes, and cancer before they
worsen.
2. Optimized Hospital Resource Management
• Hospitals can allocate ICU beds, staff, and equipment based on predictions.

vii. Case Study: Predicting Patient Readmission at a U.S. Hospital

A major U.S. hospital used XGBoost-based ML models to predict patient readmissions after
discharge.
• 20,000+ patient records were analyzed.
• The model identified key risk factors such as age, pre-existing conditions, and past
hospitalization history.
• Results:
o 15% reduction in unnecessary hospital readmissions.
o $2 million in healthcare cost savings.

Conclusion
Big Data Analytics and Machine Learning revolutionize industries like retail and healthcare
by providing actionable insights. Data warehousing enables large-scale sales analysis,
while machine learning enhances customer retention strategies and patient care
predictions. Implementing these technologies leads to better business decisions, improved
patient outcomes, and a more efficient data-driven future.