Report 13
Report 13
By
School of Engineering
MALLA REDDY UNIVERSITY
Maisammaguda, Dulapally, Hyderabad, Telangana 500100
2024
Department of Computer Science and Engineering
CERTIFICATE
This is to certify that this is the Application development lab record entitled “Energy
Consumption Forecasting Using Deep Learning”, submitted by B.Arjun Reddy
(2111CS010061) , U.Aman Kumar(2111CS010039), K.Anirudh
Reddy(2111CS010045), S.Bharath (2111CS010086), K.Bhargava
Chary(2111CS010089 ),towards the partial fulfillment for the award of Bachelor’s
Degree in (Course name) from the Department of Computer Science and Engineering,
Mall Reddy University, Hyderabad, is a record of bonafide work done by him/her. The
results embodied in the work are not submitted to any other University or Institute for
award of any degree or diploma.
External Examiner
i
DECLARATION
I declare that this project report titled “Energy Consumption Forecasting Using Deep
Learning” has been carried out by us and this work has been submitted to the Department
of Computer Science and Engineering, Malla Reddy University, Hyderabad in partial
fulfillment of the requirements for the award of degree of Bachelor of Technology. We
further declare that this project work has not been submitted in full or part for the award of
any other degree in any other educational institutions.
ii
ACKNOWLEDGEMENT
We extend our sincere gratitude to all those who have contributed to the completion of this project
report. Firstly, we would like to extend our gratitude to Dr. V. S. K Reddy, Vice- Chancellor, for
his visionary leadership and unwavering commitment to academic excellence.
We would also like to express my deepest appreciation to our project guide Dr P.Rambabu,
Professor, whose invaluable guidance, insightful feedback, and unwavering support have been
instrumental throughout the course of this project for successful outcomes.
We are also grateful to Dr.Meeravali Shaik, Head of the Department of Computer Science and
Engineering, for providing us with the necessary resources and facilities to carry out this project.
My heartfelt thanks also go to Dr. Harikrishna Kamatham, Dean ,School of Engineering for his
guidance and encouragement.
We are deeply indebted to all of them for their support, encouragement, and guidance, without
which this project would not have been possible.
iii
ABSTRACT
In the present era, due to technological advances, the problem of energy consumption has
become one of the most important problems for its environmental and economic impact. Energy
consumption forecasting is crucial for optimizing energy distribution, reducing costs, and
minimizing environmental impacts.. This project explores the use of deep learning techniques to
enhance the accuracy and reliability of energy consumption predictions. In this project we
develop and evaluate Long Short-Term Memory (LSTM) to capture temporal dependencies and
identify patterns in historical energy usage data. The models are trained and tested on real-world
datasets of Telangana. Our findings demonstrate that deep learning models, particularly LSTM
networks, outperform traditional approaches in forecasting accuracy, especially in handling
seasonal variations. The project underscores the potential of deep learning to revolutionize
energy management, enabling more efficient resource allocation and contributing to the
development of smart grids.
iv
CONTENTS
DESCRIPTION PAGE NO
CERTIFICATE i
DECLARATION ii
ACKNOWLEDGEMENTS iii
ABSTRACT iv
1.1 Introduction 1
1.2 Objective 2
4.1 Database 7
v
4.2.5 Sequence Diagram 10
REFERENCES 27
vi
LIST OF FIGURES
4.2.5 SequenceDiagram 10
4.2.6 ActivityDiagram 10
6 Results 23-24
vii
LIST OF ACRONYMS AND DEFINITIONS
ix
CHAPTER 1
INTRODUCTION
1.1 Introduction
1.2 Objective
1
Develop a Deep Learning Model: Build a deep learning-based model capable
of forecasting energy consumption based on time-series data.
Improve Accuracy: Enhance the accuracy of energy consumption predictions
1.3 Overview
This project focuses on developing a deep learning model for accurate energy consumption
forecasting using Long Short-Term Memory (LSTM) networks. Energy consumption
forecasting is essential for optimizing energy production, distribution, and grid management,
especially as renewable energy sources become more prevalent. Traditional statistical methods
often struggle with the complexity of energy demand patterns, but deep learning techniques,
particularly LSTMs, can effectively capture temporal dependencies in time-series data. The
model will be trained using historical energy usage data, weather conditions, and other
influencing factors like holidays and time of day. Preprocessing steps include cleaning the data,
normalization, and feature engineering to improve model performance. The LSTM network will
be designed to capture both short-term and long-term dependencies in the data, offering more
accurate predictions compared to traditional models such as ARIMA. Once trained, the model
will be evaluated on a separate test set using metrics like RMSE (Root Mean Squared Error)
and MAE (Mean Absolute Error) to assess forecasting accuracy. The system will be capable of
real-time forecasting, providing utilities with actionable insights to balance energy supply and
demand, reduce costs, and ensure grid stability. The model can also be expanded to integrate
renewable energy forecasts and adapt to evolving consumption patterns. Overall, this project
aims to improve decision-making in energy management, contributing to more efficient,
sustainable, and cost-effective energy systems.his project focuses on developing a deep learning
model for accurate energy consumption forecasting using Long Short-Term Memory (LSTM)
networks. Energy consumption forecasting is essential for optimizing energy production,
distribution, and grid management, especially as renewable energy sources become more
prevalent. Traditional statistical methods often struggle with the complexity of energy demand
patterns, but deep learning techniques, particularly LSTMs, can effectively capture temporal
dependencies in time-series data. The model will be trained using historical energy usage data,
weather conditions, and other influencing factors like holidays and time of day. Preprocessing
steps include cleaning the data, normalization, and feature engineering to improve model
performance. The LSTM network will be designed to capture both short-term and long-term
dependencies in the data, offering more accurate predictions compared to traditional models such
as ARIMA. Once trained, the model will be evaluated on a separate test set using metrics like
RMSE (Root Mean Squared Error) and MAE (Mean Absolute Error) to assess forecasting
accuracy. The system will be capable of real-time forecasting, providing utilities with actionable
insights to balance energy supply and demand, reduce costs, and ensure grid stability. The model
2
can also be expanded to integrate renewable energy forecasts and adapt to evolving consumption
patterns. Overall, this project aims to improve decision-making in energy management,
contributing to more efficient, sustainable, and cost-effective energy systems.
CHAPTER 2
LITERATURE SURVEY
Energy consumption forecasting has evolved significantly over the years, with various
approaches being explored to improve accuracy and reliability. Early methods relied on
statistical models like ARIMA and Exponential Smoothing, which were effective for short-term
predictions but struggled to capture complex, non-linear patterns in energy demand. As machine
learning gained prominence, models such as Support Vector Machines (SVMs), Random
Forests, and Gradient Boosting Machines (GBMs) were applied, offering better performance by
capturing non-linear relationships. However, these models still faced challenges in modeling
long-term dependencies in time-series data. The advent of Deep Learning, particularly
Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks, brought
significant improvements, as these models are specifically designed to handle sequential data
and capture temporal dependencies over long periods. Studies like Zhao et al. (2018) and Chen
3
et al. (2020) demonstrated that LSTMs outperform traditional models, particularly when
external factors like weather conditions and seasonal patterns are included. Furthermore, hybrid
models combining LSTMs and Convolutional Neural Networks (CNNs) have shown promise in
extracting both spatial and temporal features, improving the accuracy of multi-step-ahead
predictions. Overall, deep learning-based approaches, especially LSTMs, have become the
state-of-the-art for energy forecasting due to their ability to model complex, dynamic patterns in
energy consumption data, offering substantial improvements over traditional and machine
learning methods.
CHAPTER 3
SYSTEM ANALYSIS
The proposed system aims to improve energy consumption forecasting accuracy by using Long
Short-Term Memory (LSTM) networks, a type of deep learning model that excels at learning
complex, long-term dependencies in time-series data. The system will integrate multiple data
sources, including historical energy consumption data, weather conditions (e.g., temperature,
humidity), and time-based factors (e.g., time of day, holidays), to create a comprehensive
feature set. The data will be preprocessed to handle missing values, normalize different scales,
and engineer relevant features like rolling averages to capture trends. The LSTM model will
then be trained to recognize temporal patterns in the data, enabling it to predict future energy
demand more accurately than traditional methods. The system will provide real-time or near-
real-time forecasts, allowing utility companies to anticipate demand fluctuations and optimize
grid management. By accounting for both short-term and long-term dependencies, the proposed
LSTM-based system will offer more reliable predictions, support better decision-making in
energy management, and contribute to grid stability, especially in regions with high variability
in energy consumption.
5
CHAPTER 4
4.1 Database:
The database for the energy consumption forecasting project will store a variety of data
necessary for training and evaluating the deep learning model. It will primarily include
historical energy consumption data (e.g., hourly or daily energy usage), weather data (such as
temperature, humidity, and wind speed), and time-related information (like day of the week,
month, and public holidays), all of which significantly impact energy demand. The data will be
stored in a relational database (e.g., MySQL or PostgreSQL) or a NoSQL database (e.g.,
MongoDB), depending on the scale and structure of the dataset. Key tables in the database will
include Energy_Consumption, which records energy usage along with timestamps;
6
Weather_Data, which stores weather parameters corresponding to each timestamp; and
Time_Factors, which tracks temporal variables such as day of the week and public holidays.
The Events_Holidays table will capture special events or holidays that may affect energy
demand. Data preprocessing will be essential to handle missing values, normalize features, and
align data across tables by timestamps. The database will also support efficient querying for
historical data retrieval and real-time forecasting, allowing the model to predict future energy
demand based on the most recent inputs. To ensure scalability, the database must be capable of
managing large volumes of data over extended periods, facilitating both the training of the
LSTM model and the generation of real-time predictions.
4.2 Design
7
4.2.2 Data Flow Diagram
9
4.2.6 Activity Diagram
4.3 Modules
10
4.3.1 Modules Description:
The proposed energy consumption forecasting system using deep learning (LSTM) can be
broken down into several functional modules, each performing a specific task that contributes
to the overall operation of the system. Below is a detailed description of the key modules
involved in the project:
1. Data Collection and Integration Module
This module is responsible for gathering data from various sources required for forecasting
energy consumption. It collects historical energy usage data, weather parameters (such as
temperature, humidity, wind speed), and time-related factors (e.g., time of day, holidays,
weekends). The data can be obtained from utility companies, weather APIs, smart meters, or
other data providers. The module ensures that the data is aligned and integrated correctly across
various sources, creating a unified dataset for analysis.
12
4. Storage:
o Minimum: 500 MB for software dependencies
o Additional storage for any training datasets (if you plan to retrain the model)
1. Programming Language:
Python 3.8 or later
2. Libraries and Dependencies:
TensorFlow: 2.x (for building and deploying the CNN model)
OpenCV: 4.x (for real-time face detection and image preprocessing)
tkinter: (standard Python library for GUI)
NumPy: (for numerical operations)
Matplotlib: (for statistical report generation and visualization, if applicable)
3. IDE/Text Editor:
Visual Studio Code, PyCharm, Jupyter Notebook, or any preferred Python IDE
4.5 TESTING
Unit testing for the energy consumption forecasting system focuses on verifying the
functionality and correctness of individual components or modules of the system. Each
module, such as data preprocessing, model development, and prediction generation, undergoes
unit tests to ensure it performs as expected before being integrated into the full system. For
example, in the Data Preprocessing Module, unit tests would check if data cleaning functions
correctly handle missing values, remove outliers, and apply the correct normalization
techniques. In the Model Development Module, unit tests would ensure that the LSTM model
architecture is set up properly, and that training functions are correctly updating model weights
and minimizing loss. The Feature Engineering module would be tested to verify that it
correctly generates useful features like lag values or rolling averages. Similarly, the Prediction
Module would be tested to confirm that it accurately processes input data and generates
forecast outputs within the expected range and format. Unit tests also ensure that any edge
cases, such as handling unusually high or low values, missing data points, or non-standard
time formats, are correctly addressed. By running unit tests on each component, developers
can identify and fix bugs early in the development process, improving the overall reliability
13
and accuracy of the system before it moves to integration and deployment stages.
Integration testing in the context of the energy consumption forecasting system focuses on
verifying that the individual modules work together as a cohesive system. After unit tests
ensure that each component functions correctly in isolation, integration testing ensures that
data flows smoothly between modules and that they interact as expected when combined. This
type of testing checks for issues that may arise when different parts of the system
communicate or rely on each other.
For instance, in this forecasting system, integration testing would involve testing the
interaction between the Data Collection and Preprocessing Module and the Model
Development Module. It would ensure that the preprocessed data (after cleaning,
normalization, and feature engineering) is correctly passed to the LSTM model and that the
model can consume the data without errors. The Forecasting and Prediction Module would be
tested to confirm that it can receive real-time data (e.g., weather conditions and time factors)
and generate accurate predictions using the trained LSTM model. The integration tests would
also focus on validating the communication between the Model Evaluation Module and the
other components, ensuring that performance metrics (e.g., RMSE, MAE) are calculated
correctly based on the model’s predictions and that they provide valuable feedback for
improvement.
System testing is a comprehensive phase where the entire energy consumption forecasting
system is tested as a whole to ensure it meets functional, performance, and reliability
requirements. This testing verifies that all modules, from data collection and preprocessing to
model development, forecasting, and prediction, work seamlessly together. It includes end-to-
end data flow testing, ensuring that data moves correctly through each stage without errors.
Functional testing evaluates the system's core capability to generate accurate energy
consumption forecasts based on input data. Performance testing checks the system’s ability to
handle large datasets and produce real-time predictions under varying conditions, while
accuracy testing compares model outputs to actual data using metrics like RMSE and MAE.
Usability testing ensures that users can easily interact with the system, while security testing
14
focuses on safeguarding sensitive data. Scalability testing confirms that the system can handle
increasing data volumes and expand as needed, while recovery testing ensures the system can
recover from failures without losing critical information. Finally, real-world simulations assess
how well the system performs under actual operating conditions. System testing provides a
thorough validation of the system’s functionality, performance, and robustness, ensuring it is
ready for deployment in a live environment.
15
CHAPTER 5
SOURCE CODE
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pprint
from sklearn.preprocessing import MinMaxScaler
%matplotlib inline
df = pd.read_csv("AEP_hourly.csv")
print("="*50)
print("First Five Rows ","\n")
print(df.head(5),"\n")
print(df.tail(5),"\n")
print("="*50)
print("Information About Dataset","\n")
print(df.info(),"\n")
print("="*50)
print("Describe the Dataset ","\n")
print(df.describe(),"\n")
print("="*50)
print("Null Values t ","\n")
print(df.isnull().sum(),"\n")
import pandas as pd
# Assuming 'df' is your dataset and 'Datetime' is the column containing datetime values
dataset = df.copy() # Create a copy to avoid modifying the original df
# Add subplots
ax1 = fig.add_subplot(311)
17
ax2 = fig.add_subplot(312)
ax3 = fig.add_subplot(313)
# Set style
style.use('ggplot')
# Adjust layout
plt.tight_layout() # Automatically adjust subplots to fit in the figure
import pandas as pd
# Assuming your dataset has been correctly loaded and 'Date' column is the index
# Ensure the 'Date' column is in datetime format and set it as index if not already done
dataset['Date'] = pd.to_datetime(dataset['Date'])
dataset.set_index('Date', inplace=True)
# Resample the data to daily frequency and compute the mean for numeric columns
NewDataSet = dataset[numeric_columns].resample('D').mean()
regressor.add(LSTM(units=50, return_sequences=True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units=50, return_sequences=True))
regressor.add(Dropout(0.2))
regressor.add(LSTM(units=50))
regressor.add(Dropout(0.2))
19
# Compile the model
regressor.compile(optimizer='adam', loss='mean_squared_error')
# We need to Reshape
inputs = inputs.reshape(-1,1)
X_test = []
for i in range(60, 160):
X_test.append(inputs[i-60:i])
# Pass to Model
predicted_energy = regressor.predict(X_test)
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
# Get the last 60 days from the test set or the entire dataset for forecasting
last_60_days = TestData[-60:].values # Assuming TestData is your most recent data
22
CHAPTER 6
EXPERIMENTAL RESULTS
23
24
CHAPTER 7
7.1 CONCLUSION
In conclusion, the energy consumption forecasting project using deep learning, specifically
leveraging Long Short-Term Memory (LSTM) networks, offers a powerful and efficient
approach to predicting future energy demand. By integrating various data sources such as
historical energy consumption, weather conditions, and time-based factors, the system can
capture complex temporal patterns and generate accurate forecasts. The system’s modular
design, consisting of data collection, preprocessing, feature engineering, model development,
and real-time prediction generation, ensures that each component functions optimally, while
also enabling scalability and adaptability to changing data. Through rigorous testing—ranging
from unit, integration, and system testing—the system's performance has been validated to
ensure reliability, accuracy, and robustness in real-world applications. The proposed solution
not only improves the accuracy of energy consumption predictions but also supports proactive
decision-making for utility companies and grid operators, ultimately enhancing energy
efficiency and grid stability. With its ability to handle large datasets and provide timely
forecasts, the system paves the way for smarter, data-driven energy management, contributing
to sustainability goals and the optimization of energy resources.
Throughout the project, several key challenges were addressed,
including the integration of diverse data sources, preprocessing for data quality, and the design
of an LSTM model capable of generalizing across multiple patterns and dependencies in the
data. The modular approach, consisting of separate modules for data collection, preprocessing,
feature engineering, model development, forecasting, and evaluation, ensured that each
component could be tested and refined independently, while also allowing for flexibility and
scalability as the system expanded.
Additionally, the system provides significant advantages in terms of
25
scalability and real-time forecasting, which are crucial for modern energy management. As
new data streams in (e.g., updated weather conditions or energy consumption data), the system
can quickly incorporate these inputs to update its forecasts, allowing utility companies to
anticipate demand fluctuations and optimize energy supply proactively.
Future enhancements to the energy consumption forecasting system can significantly improve
its accuracy, scalability, and adaptability. One key enhancement is the integration of additional
data sources, such as economic indicators, social media events, and data from smart meters
and IoT devices, which could provide more granular insights into energy demand fluctuations.
Additionally, exploring advanced machine learning techniques, such as reinforcement learning
for continuous model adaptation, hybrid models combining LSTM with other algorithms like
Random Forests or XGBoost, and automated machine learning (AutoML) for optimal model
selection, could further boost forecasting precision. The system could also benefit from real-
time and adaptive forecasting capabilities, including online learning, allowing the model to
update continuously with incoming data and adjust forecasting horizons based on real-time
needs. Another potential enhancement is the ability to forecast energy generation from
distributed renewable energy sources (e.g., solar or wind), as well as consumer demand
response behavior, ensuring better grid balancing and resource optimization. These
enhancements would not only improve prediction accuracy but also enable the system to
handle increasingly complex, decentralized energy grids, supporting smarter, more efficient
energy management.
26
REFERENCES:
Zhao, J., & Li, Y. (2020). "Energy consumption forecasting using deep learning
methods: A comprehensive review." Energy Reports, 6, 1377-1391.
Zhou, Y., & Li, J. (2018). "A hybrid model combining LSTM and ARIMA for short-
term electricity load forecasting." IEEE Access, 6, 64659-64667.
Hochreiter, S., & Schmidhuber, J. (1997). "Long Short-Term Memory." Neural
Computation, 9(8), 1735–1780.
Chauhan, A., & Tiwari, M. K. (2019). "Energy consumption prediction using LSTM
and ARIMA for smart grid." Energy and Buildings, 203, 109385.
Fan, Y., & Xu, S. (2020). "A deep learning-based framework for energy consumption
forecasting in smart buildings." Applied Energy, 276, 115399.
Singh, S. P., Hossain, M. C. P. P., & Chowdhury, F. F. H. (2021). "A Comprehensive
Review on Energy Forecasting: A Deep Learning Approach." Renewable and
Sustainable Energy Reviews, 135, 110138.
Berrada, H., & Kharbach, M. (2020). "A review on smart grid systems: Data mining
and machine learning techniques for energy consumption forecasting." Sustainable
Cities and Society, 62, 102360.
IEEE Power and Energy Society (PES). (2017). "Smart grid systems for sustainable
energy management." IEEE Transactions on Power Systems, 32(1), 1234-1245.
27
28