Name Surname –

Semester: Fall 2024/2025

Lecturer: Shahnaz Shahbazova
Subject: Machine Learning
Group: E 27-24

(1) Data Preprocessing: You are tasked with building a machine learning model
using data from multiple sources with different formats. How would you
preprocess and standardize this data for analysis?

Data preprocessing is a crucial step in building machine learning models, especially when
dealing with data from multiple sources with different formats. The goal is to clean, transform,
and standardize the data so that it can be effectively used for analysis and modeling. Here’s
how I would approach preprocessing and standardizing data for analysis: First of all, to
understand the data sources and formats, I would do before preprocessing the data, it's
important to understand the structure, content, and quality of the data from different sources.
Data might come in various formats like: - CSV, JSON, XML, SQL databases, or APIs. Different
encodings or delimiters. Time series data, categorical, or textual data. Begin by examining the
data, looking for any inconsistencies or patterns that might indicate the necessary steps for
transformation. For data cleaning, data from multiple sources often have missing values,
outliers, or incorrect entries. The following steps we can write and it can help clean the data.
Beginning with missing Data Handling, we can first identify missing values using functions like
`isnull()` or `isna()` in pandas. Secondly, we can handle missing values by either imputing them
or dropping rows or columns depending on the extent of missing data. Also, we can do outliers
detection. To detect outliers using statistical methods or visual methods like box plots. We can
then decide whether to transform, cap, or remove outliers depending on the nature of the data
and the problem at hand; similarly, for fixing Incorrect Data, we can look for incorrect entries
such as typos or values that don't make sense in context.
(2) Feature Engineering: You are working on a project to predict student
performance based on factors such as study habits, hours of sleep, and
extracurricular activities. How would you engineer features from these variables?

To engineer features from the given variables, we can apply the following steps: Categorical
features: To reate categories based on study habits, such as “good,” “average,” and “poor,”
depending on whether the student studies regularly, occasionally, or rarely. This could be
derived from a rating scale or self-reported data. – Secondly, study time in terms of if available,
create a feature representing the number of hours spent studying per day/week. – Consistency
is needed to take into consideration to create a binary or continuous feature indicating the
consistency of study habits. For the study material type: we can categorize or encode the types
of study materials the student uses.

Also, in order to make a project to predict student performance in terms of feature

engineering, we can take into consideration these factors and decide the activities. Hours of
Sleep, Continuous feature: Directly use the number of hours of sleep as a continuous feature.
Quality of Sleep If data on sleep quality is available, create a derived feature like "sleep quality"
based on responses to a self-reported scale. Coming to sleep Duration Categories, creating
categorical features for different sleep durationsConsistency in Sleep: If there’s data on the
variability in the student’s sleep schedule, create a feature representing sleep consistency.

Extracurricular ActivitiesParticipation: we can create a binary feature indicating whether the

student participates in extracurricular activities. In terms of number of activities, we count the
number of extracurricular activities the student is involved in and use that as a continuous
feature. For activity type create categorical features based on the type of extracurricular
activities. This could be a one-hot encoding or group them into fewer categories if necessary.
Additional, time spent on activities can be taken into consideration if data on time spent in
extracurricular activities is available, include it as a feature Activity Intensity: Create an
intensity score if there is information on how demanding or time-consuming the activities are.

Interaction Features:Study and Sleep Interaction: Create features that capture the interaction
between study habits and sleep, as they might impact performance together. For example,
combine "hours of study" and "hours of sleep" to see how a balance between the two affects
performance. Study and Extracurricular Activities Interaction: Similarly, create interaction terms
between study habits and extracurricular involvement to assess whether balancing these
factors influences performance. Normalization and Transformation: Normalize continuous
features like hours of sleep and study time to a common scale to ensure they have equal
influence on predictive models. Apply feature transformations for skewed features Additional
Derived FeaturesStudy-to-Extracurricular RatioCreate a ratio of study time to extracurricular
activity time to evaluate the balance between academic focus and outside activities. Similarly,
create a ratio of sleep to study time to assess the relationship between rest and study time. By
carefully transforming these variables into meaningful features, we can provide the model with
a well-rounded view of the student's lifestyle, which should help improve the prediction of
student performance.

(3) Model Selection: You need to build a model to detect fraudulent transactions
in real-time for a financial institution. Which machine learning model would you
choose, and how would you balance accuracy with computation in time?

There are machine learning algorithms to detect fraudulent transactions, and this kind of
fraudulent catch mechanism is needed nowadays important than anything else. There are
several reasons to emphasize the importance of the model selection. First of all, beginning with
the machine learning algorithms, I would choose a model in a way that it can catch anomalies.
By emphasizing anomalies, I would like to data into consideration because data in these models
are important. There can be anomalies that can be catched by using machine learning
algorithms. Also, there are a lot of scenarios that can happen if these anomalies are not taken
into seriusness. For example, let say that e-commerce site has been created and a lot of data
are added to the website. Customers can buy these items and also decide which payment
method they would like to use. Here is the scenario that if the anomaly happens, it can be
detected easily using these machine learning algorithms. Going into detail of the question,
when selecting a machine learning model to detect fraudulent transactions in real-time for a
financial institution, there are several key factors to consider: model accuracy, computation
time, interpretability, and scalability. Here's how I would approach this problem: Model
Selection Given that this is a real-time fraud detection problem, I would focus on models that
can balance high predictive accuracy with low latency (quick inference time). Logistic
Regression: It is simple, interpretable, and works well with high-dimensional data, especially if
we have engineered features from transaction history However, it may not always capture
complex relationships in the data. Random Forest / Gradient Boosting These are more
sophisticated ensemble methods that can capture complex patterns and provide strong
performance in classification tasks like fraud detection. XGBoost and LightGBM, in particular,
have optimizations for faster training and inference, making them more suitable for real-time
predictions. Neural Networks. These models tend to perform well when large amounts of data
are available and there are complex relationships between features. However, neural networks
are generally slower to train and require more computational resources, which might impact
real-time performance, unless specifically optimized. Anomaly Detection (Isolation Forest, One-
Class SVM): If the fraudulent transactions are much rarer compared to legitimate transactions
an unsupervised anomaly detection model could work well. Isolation Forest, for example, is
lightweight and efficient in identifying outliers, which could represent fraudulent transactions.
Given these considerations, Gradient Boosting models would be a good balance between
accuracy and computational efficiency. These models are highly effective for structured data
and provide scalable, accurate predictions with relatively fast inference times.
(4) Cross-Validation: You are given a time-series dataset of sales data over the past
five years. How would you implement cross-validation for this time-series data,
and which validation technique would you use?

For time-series data, traditional cross-validation techniques like k-fold cross-validation aren't
suitable because they don't account for the temporal order of the data. Instead, we can use a
time-series specific validation technique such as TimeSeriesSplit orWalk Forward Validation. We
can would implement cross-validation for time-series data as taking timeSeriesSplit validation
into consideration. This is one of the most common techniques that we can use for time-series
cross-validation. The key idea here is to split the data into training and test sets while
preserving the temporal order of the observations.Steps. We can divide the dataset into a
series of train-test splits, but ensure that the training set always consists of the data up to the
test set, and the test set is always a future set of observations.

For example, if we have data points 1 to 100: - Split 1: Train on 1–70, test on 71–80. - Split 2:
Train on 1–80, test on 81–90. - Split 3: Train on 1–90, test on 91–100. By making this way, we
can train on increasing amounts of data and each model is tested on a future set by avoiding
data leakage from future information into the training process.

Similarly, cross-validation to test data points, we can train with increasing amount of data and
coming to specific modes, data leakage for training process is needed to taken into process
scenarios.

(5) Hyperparameter Tuning: You are training a support vector machine (SVM) for
image classification, but your model is underperforming. Describe how would you
tune the kernel and other hyperparameters to improve its accuracy.
SVM is a support vector machine that is used to improve the accuracy of your Support Vector
Machine for image classification, hyperparameter tuning is crucial. We can approach tuning the
kernel and other key hyperparameters: Kernel Selection The choice of kernel determines the
transformation of the feature space. The three main types of kernels in SVM are. First one is
linear kernel as this is best when the data is linearly separable. It is faster and less
computationally expensive. Second one is polynomial kernel. It is useful when data is not
linearly separable but can be separated with a polynomial decision boundary. Another one is
radial basis function kernel. It is most commonly used for non-linear problems as it maps data
into a higher-dimensional space where it can be separated with a hyperplane. There is also
sigmoid kernel as it is similar to the neural network activation function, but less commonly
used. Tuning the Kernel Start by experimenting with the RBF kernel, as it is typically a good
default choice. Using cross-validation to compare the performance of different kernels. In case
of RBF, experiment with other kernels if it is suspected a different transformation is better for
the data. For hyperparameters for kernel. If we use an RBF kernel, the following
hyperparameters should be tune. C (Regularization Parameter) as it controls the trade-off
between a smooth decision boundary and classifying training points correctly. A small value
allows more margin violations, while a large value places more emphasis on correct
classification. There is a tuning approach by starting with a small value 0.1 and gradually
increase it (e.g., 1, 10, 100) to check if it improves performance. Larger values can lead to
overfitting, while smaller values can result in underfitting. Gamma can be defined the influence
of a single training example. A small gamma means a larger influence, resulting in a smoother
decision boundary. A large gamma can cause overfitting by capturing noise in the data. Tuning
approach is good to start with a default value such as scale or auto, and experiment with
different values of gamma like 0.01, 0.1, 1, 10, etc., to find the optimal value.

Other Hyperparameters Degree: If you choose a polynomial kernel, this controls the degree of
the polynomial. Higher degrees can capture more complex decision boundaries but might lead
to overfitting. There is also tuning approach by starting with degree 3 (default) and try values
from 1 to 5, checking performance on the validation set. Moving to class weight, if the classes
are imbalanced, the SVM might prioritize the majority class. Adjusting the `class_weight` to
"balanced" or manually setting weights for classes can help improve performance for minority
classes. Additional, tuning approach can be emphasized to set the `class_weight` parameter to
`'balanced'`, or manually experiment with class weights based on the class distribution in your
dataset. Grid Search is used for grid search to search for the best combination of
hyperparameters systematically. Furthermore, random search can be taken into consideration,
as an alternative, random search can sometimes find better results faster than grid search,
especially when the search space is large.Cross-Validation. We can use k-fold cross-validation to
evaluate the performance of your SVM with different hyperparameter settings. This helps
prevent overfitting and ensures that your model generalizes well to unseen data. Feature
Scaling: Ensure that your data is preprocessed properly before training the SVM. SVMs are
sensitive to the scale of features, so standardizing or normalizing the image pixel values is
essential for achieving optimal performance. Lastly, we can evaluation in a way that after tuning
the hyperparameters, evaluate the model on a separate test set to verify if the improvements
made during training generalize to unseen data. By systematically tuning the kernel type,
regularization parameter, gamma, and other SVM hyperparameters, we can significantly
improve your model's accuracy in image classification tasks.