CST8333 Programming Language Research Project

Reflective Report
Name-Raghav Dhir
Student No.-

Program Changes -
Throughout the course of the CST8333 Programming Language Research Project, the design,
features, and functionality of the program evolved significantly. These changes were the result
of my increasing familiarity with Python and its libraries, the practical challenges encountered
during development, and the feedback I received from testing and earlier iterations of the
program.

Below is a comprehensive breakdown of the major program changes that occurred throughout
the project:

1. Initial Design and Features

In the beginning, the program was relatively simple. It was built to handle basic data
processing tasks such as loading data from a CSV file and displaying it to the user. The program
lacked interactivity and didn't offer any options for users to manipulate the data or engage
with the dataset beyond viewing it. The data visualization component was minimal, with only
basic plots generated using the matplotlib library.
Key features of the initial version:
 Data Loading: Users could load a CSV file containing a dataset into the program.
 Data Display: The program displayed the raw data from the CSV file without any
significant modifications or transformations.
 Basic Visualization: Using matplotlib, the program generated simple, static plots to give
a basic understanding of the dataset's characteristics.
 Limited Customization: The plots lacked customization, meaning users couldn't adjust
aspects like titles, axis labels, colors, or any other visual details.
While functional in its simplest form, this version lacked any interactive features and did not
cater to the challenges of handling real-world, messy data.

Page 1 of 7
2. Addition of Data Cleaning Features
Upon gaining a deeper understanding of data analysis, I realized that working with raw data
without cleaning or preprocessing it would lead to unreliable or incomplete analysis results. As
a result, the program was updated to include a data cleaning module that allowed for better
management of missing values and irrelevant data.
Key Changes Made:
 Missing Data Handling: I integrated functions such as fillna() and dropna() from the
pandas library. This allowed users to either fill missing values with specific constants or
drop rows and columns that contained missing or null data.
 Data Filtering: I implemented features that allowed for the removal of unnecessary
columns or rows. This filtering process ensured that only relevant data was considered
in analysis and visualizations.
 Type Conversion: I added functionality to convert data types (e.g., from strings to
integers or floats) to ensure the dataset was consistent and suitable for operations such
as statistical analysis and machine learning.
This data cleaning functionality was crucial, as it made the program more useful in real-world
scenarios, where data can often be incomplete or noisy. The ability to clean data before
proceeding with analysis made the program more robust and versatile.

3. Enhancing Data Visualization

Page 2 of 7
One of the most gratifying and visually impactful changes I made was in the enhancement of
data visualization. Initially, the program provided basic, generic plots that didn’t offer much
depth in terms of customization. As I explored the matplotlib and seaborn libraries further, I
was able to improve the plotting functionality, allowing users to gain more insights from the
data.
Key Changes Made:
 Advanced Plot Customization: I added the ability to modify plots with titles, axis labels,
color schemes, and other customization features. This was crucial in making the
visualizations more informative and user-friendly.
 Diverse Plot Types: I extended the program to support a variety of plot types, such as
histograms, boxplots, and line charts. Each of these provided a different perspective on
the data. For example, histograms allowed users to understand the distribution of data
points, while boxplots helped identify outliers.
 Plot Exporting: I added the ability to save the generated plots as image files (e.g., PNG,
JPEG). This was an essential feature for users who wanted to include visualizations in
reports or presentations.
These enhancements made the program much more intuitive, allowing users to interact with
data in a meaningful way and easily extract insights from it.

4. Database Integration

Page 3 of 7
A major shift in the project occurred when I decided to integrate a database into the program.
Initially, the program only handled data in-memory, which was sufficient for small datasets.
However, as I worked with larger and more complex datasets, I realized that the program
needed persistent storage. This led to the integration of SQLite, a lightweight relational
database, which enabled the program to handle larger datasets more efficiently.
Key Changes Made:
 SQLite Integration: I utilized the sqlite3 module in Python to connect the program to an
SQLite database. This allowed the program to store data in a persistent manner,
reducing the need to load everything into memory at once.
 Database Creation: I wrote SQL commands to create a database, define tables, and
specify their structure. This involved creating tables for storing datasets, as well as
indexing and creating relationships between different data entities.
 Data Retrieval and Insertion: The program was updated to allow users to insert new
data into the database and retrieve data as needed. The database handled both data
storage and querying.
This change made the program much more scalable and efficient, especially when working
with large amounts of data that would have been cumbersome to process without a database.

5. CRUD Operations Implementation

Page 4 of 7
With the integration of the database, the next logical step was to implement CRUD (Create,
Read, Update, Delete) operations. These operations are fundamental to many software
applications, and they significantly increased the interactivity and functionality of the program.
Key Changes Made:
 Create: I implemented a feature that allowed users to add new records to the dataset
(either by manually entering data or importing data from other sources).
 Read: The program allowed users to query the database and display data in a user-
friendly manner, with options to filter and sort the records.
 Update: I enabled users to modify existing records. For instance, they could change
values in certain columns or rows based on user input.
 Delete: Users could delete records from the database, either by specifying individual
rows or by applying filters to delete multiple rows at once.
The implementation of CRUD operations turned the program into a more interactive tool.
Users could now modify and manipulate data dynamically, which made the program much
more practical for real-world applications.

6. Threading for Performance Optimization

Page 5 of 7
As the program grew in complexity and began handling larger datasets, I noticed performance
issues. Tasks like data loading, processing, and visualization were taking a considerable
amount of time, which degraded the user experience. To address this, I integrated threading
to improve performance.
Key Changes Made:
 Parallel Processing: By using Python’s threading module, I enabled the program to run
multiple tasks concurrently. For example, while the data was being loaded or processed,
the program could simultaneously update the user interface, display progress indicators,
and execute other non-blocking tasks.
 Improved Responsiveness: Threading helped reduce the time users had to wait for the
program to complete tasks, especially when dealing with large datasets or complex
visualizations.
 User Experience: The use of threading ensured that the program remained responsive
and didn’t freeze or become unresponsive during long operations, significantly
improving the overall user experience.
The introduction of threading optimized the program’s performance, allowing it to scale more
efficiently and handle large datasets without causing lags or delays.

7. Final Version and Summary of Changes

Page 6 of 7
By the conclusion of the project, the program had transformed from a simple data viewer into
a robust, comprehensive application capable of:
 Cleaning and preprocessing datasets
 Visualizing data through advanced plots
 Storing and querying data using an SQLite database
 Performing CRUD operations on datasets
 Running tasks concurrently for improved performance
Each change I made was motivated by the need to enhance the program's functionality,
efficiency, and usability. These modifications were not only driven by the project's technical
requirements but also by my personal growth as a developer. The lessons learned throughout
the project, from data manipulation to threading, were invaluable in improving my
programming skills. The iterative process of designing, testing, and refining the program
taught me the importance of continual improvement and learning from feedback.

Page 7 of 7