Project Report

PREDICTIVE-ANALYSIS

course code: INT234

Submitted in partial fulfillment of the requirements for the

award of the degree of Bachelor of Technology
School of computer science and Engineering

PROJECT-REPORT

Submitted By:
Name: Aditya Sah

Section: K21RM
Reg no: 12111525
Roll No: 27

Submitted to: Tanima Thakur mam

Introduction:
This project explores predictive analysis on a synthetic breast cancer dataset,
focusing on identifying patterns in cell characteristics that differentiate benign
from malignant tumors. By applying five machine learning algorithms—K-Nearest
Neighbors (KNN), Linear Regression, Polynomial Regression, K-Means Clustering,
and Support Vector Machine (SVM)—we aim to evaluate and compare their
performance in classifying tumor characteristics. Each model's suitability for
breast cancer diagnosis is analyzed through visualizations and performance
metrics, offering insights into the best algorithmic approach for such data. This
analysis has potential applications in early detection, assisting healthcare
professionals in diagnosing and predicting cancer progression based on cellular
attributes.

Dataset Used:
The dataset consists of synthetic data generated to reflect “real-world breast
cancer” cell characteristics.
The main attributes include:
• cell_size: Describes the size of the cells, which can vary between benign and
malignant samples.
• cell_shape: Reflects the shape of the cells, an essential feature as malignant
cells tend to have irregular shapes.
• smoothness: Indicates the smoothness of cell borders, which can differ
significantly in cancerous cells.
• cell_density: Density metric for cell formation.
• symmetry: Symmetry attribute of the cell nuclei.
Since the primary task is predictive analysis, this dataset seems suitable for a
binary classification model predicting the diagnosis based on other cell
characteristics.
Data Preprocessing:
Preprocessing is crucial to ensure that models interpret the data uniformly,
reducing noise and optimizing for accurate predictions. Here are the
preprocessing steps applied to the dataset:
• Normalization: Cell attributes were normalized to rescale the data between
0 and 1. This minimizes the influence of extreme values and ensures each
feature contributes proportionally.
• Encoding: The target variable, diagnosis, was converted into a binary factor
with labels "Benign" and "Malignant." This step simplifies classification,
allowing algorithms to learn the difference between the two classes.
• Splitting: The dataset was split into training and testing subsets. This
method is essential for evaluating model performance, as it allows us to
train models on one subset and test them on another to prevent overfitting.

Algorithms Used:
1. K-Nearest Neighbors (KNN):
KNN is a simple yet effective classification algorithm. It classifies a sample based
on the class of its nearest neighbors. The distance between samples is calculated,
and the majority class among the k-nearest samples determines the classification.
For this project, we used a k value of 21, selected based on exploratory tests to
optimize accuracy.
KNN Workflow
1. Training: The model was trained on normalized data (449 samples).
2. Testing: 100 samples from the test set were used for evaluation.
3. Evaluation: A confusion matrix was generated to examine classification
accuracy, precision, and recall.
KNN's main advantage is simplicity, but it can be sensitive to the choice of k and
the distribution of the data.

2. Linear Regression:
Linear regression is typically used for predictive modeling on continuous data. For
this project, we implemented a linear regression model to predict cell_shape
based on cell_size. This model explores the relationship between these features,
which may help identify trends in cell characteristics that correlate with
malignancy.
Linear Regression Workflow
1. Model Fit: A linear equation is fitted to the data, attempting to minimize the
difference between predicted and actual values of cell_shape.
2. Training and Testing: The training and test sets were used to observe how
well the model generalizes.
3. Visualization: The linear relationship was visualized, showing how
cell_shape varies with cell_size.
Linear regression performs well for simple, linearly-related data. However, it
struggles to capture complex patterns, which led to exploring polynomial
regression.

3. Polynomial Regression:
Polynomial regression extends linear regression by incorporating higher-degree
terms. This approach allows the model to capture non-linear relationships. We
introduced cell_size^2 and cell_size^3 terms to improve the model fit for non-
linear variations in the data.
Polynomial Regression Workflow
1. Model Fit: Polynomial terms were added, producing a more flexible curve
that fits the data more accurately than a straight line.
2. Training and Testing: The polynomial model was tested to verify improved
performance over the linear model.
 3. Visualization: We plotted the polynomial regression results, showing a more
nuanced curve that captures non-linearity between cell_size and
cell_shape.
Polynomial regression is beneficial when relationships are complex, as seen in
biological data like this dataset. However, higher-degree terms may lead to
overfitting, so careful tuning is essential.

4. K-Means Clustering:
K-Means clustering is an unsupervised learning algorithm that segments data into
clusters based on similarity. Here, it was used to group data points into clusters to
explore patterns in cell characteristics without relying on labeled data.
K-Means Workflow
1. Elbow Method: This method was used to identify an optimal number of
clusters (6 in this case).
2. Cluster Assignment: Each data point was assigned to one of the six clusters.
 3. Visualization: Clusters were plotted to observe segmentation, which
provides insight into potential patterns in cell characteristics across the
dataset (6 in this case).
K-Means clustering is valuable for identifying inherent structure within data,
making it useful in exploring cell attribute patterns without labels.

5. Support Vector Machine (SVM):

SVM is a robust classification algorithm that maximizes the margin between
classes. It performs well on high-dimensional data and was used here to classify
samples based on clusters generated by K-Means. A linear kernel was applied for
simplicity.
SVM Workflow
1. Data Preparation: Cluster labels were added to the dataset as a response
variable.
2. Training and Testing: Data was split, and features were scaled to ensure
uniformity.
 3. Classification and Visualization: SVM decision boundaries were visualized,
showing the separation between clusters.
SVM's strength lies in its effectiveness on complex data, making it suitable for
high-dimensional classification tasks. However, it can be computationally
intensive, particularly with larger datasets.
Performance Comparison:

Algorithm Accuracy/Results Key Observations

K-Nearest High accuracy for binary Effective for classification but

Neighbors classification sensitive to k value.

Linear Useful for linear trends, but limited

Moderate fit
Regression with complex relationships.

Polynomial Better for non-linear relationships;

Improved fit over linear
Regression suited to this dataset.

K-Means Effective clustering, 5 Elbow method found 5 clusters;

Clustering clusters useful for initial grouping.
Algorithm Accuracy/Results Key Observations

Support Vector Accurate on cluster Strong at boundary definition;

Machine classification suitable for cluster labels.

In this project, each algorithm demonstrated unique strengths and limitations. K-

Nearest Neighbors (KNN) was effective and easy to understand, but its accuracy
depended on the chosen k value and was computationally intensive. Linear
Regression provided basic insights into feature relationships, but it couldn't
capture complex patterns, which Polynomial Regression handled better by fitting
non-linear trends. K-Means Clustering identified natural groupings in the dataset,
useful for discovering potential tumor subtypes, though it required careful
selection of the number of clusters. Finally, Support Vector Machine (SVM)
achieved high classification accuracy with clear decision boundaries, showing its
effectiveness in high-dimensional data but requiring careful tuning. Overall,
combining these algorithms gave a comprehensive analysis, highlighting each
model’s suitability for different aspects of tumor classification and pattern
recognition.

Overall Project Findings:

Each algorithm contributed unique insights:
The synthetic breast cancer dataset analysis highlighted key insights across several
machine learning models:
➢ KNN and SVM: Both models were highly effective in predicting benign
versus malignant cases. KNN provided simple, interpretable results but was
sensitive to the choice of 𝑘, while SVM offered a precise boundary for
classification, showing strong potential for diagnostic applications.
 ➢ Feature Relationships: Linear and polynomial regression models explored
relationships between features (e.g., cell size and shape), revealing that
more complex, non-linear models can better capture these biological
interactions.

➢ Clustering Insights: K-means clustering identified natural subgroups within

the data, which could be relevant for discovering potential subtypes within
the benign and malignant categories. This clustering could guide further
research into subtype-specific treatment or prognosis.

➢ Combined Modeling Approach: The findings suggest that using classification

models (KNN or SVM) for prediction, with regression insights to inform
feature selection and clustering to reveal data structure, could lead to a
well-rounded and interpretable predictive framework for cancer
diagnostics.
Each technique added unique insights, reinforcing that a multi-model approach is
beneficial for complex healthcare data, balancing predictive accuracy with
interpretability. In conclusion, combining models such as SVM or KNN with
regression-based insights and clustering could lead to a highly effective predictive
analysis framework for breast cancer diagnosis.

Conclusion:

The multi-algorithm approach provided valuable insights into the dataset's

structure and the relationships within. KNN and SVM demonstrated strong
performance in classification, while polynomial regression effectively modeled
non-linear relationships. K-Means clustering enabled the identification of distinct
cell characteristic patterns, highlighting possible tumor subtypes. The study
indicates that for similar datasets, using a combination of supervised and
unsupervised algorithms allows for a comprehensive understanding, supporting
early detection and diagnosis efforts.

In conclusion, the predictive analysis techniques applied here lay a foundation for
further research on cell characteristic analysis, potentially aiding healthcare
professionals in early cancer detection. Future work could explore additional
features, larger datasets, or other advanced algorithms to enhance predictive
accuracy and reliability further.