2009 International Conference on Information Technology and Computer Science

Application of Elman Neural Network and MATLAB to Load Forecasting

REN Lina1 LIU Yanxin2 RUI Zhiyuan1 LI Haiyan1 FENG Ruicheng1

(1. Key Laboratory of Digital Manufacturing Technology and Application, The Ministry of Educationˈ
Lanzhou Univ. of Tech. Lanzhou 730050,China; 2.GanSu Tobacco Industrial Company, Lanzhou730050, China)
E-mail:sunnyren214@sina.com

Abstract—In order to improve the load-forecast precision and speciafication.Compared with BP neural network,Elman
availability of power system, a method based on Elman neural neural network is much more appropriate to model for
network and MATLAB is presented to create a load forecast complex and nonlinear system.
model,which according to the Elman neural network model having For load forecast model of power system with dynamic
the characteristics of approach to arbitrary nonlinear functions behavior is generally a nonlinear function,so using Elman
and having the ability of reflecting the dynamic behavior of the neural network to model and analyze is feasible.
system and for the practicability and high efficiency of using However,most load forecast model based on artifitical
neural network tool-box in MATLAB to program.Then using actual
neural network are using C language or FORTRAN language
load data to train the model,the emulation results show that the
model is of quickly convergence speed and high forecasting
to program,which not only makes structure of the model
precision,which can meet the needs of running and scheduling in complex,but also have a long training time. Then using the
power system,and using neural network tool-box in MATLAB to NNTool-Box in MATLAB to program can disembarrass the
program can make the worker won free of elaborate program and users out of elaborate programing[9] and collect their thought
make the working efficiency improved effectively.The example is of to find a method for solving the problem,then working
proof that the method is feasible and effective. efficiency and the quality for solving the problem can be
Keywords- Elman neural network; MATLAB;forecast model; greatly improved.In the paper,we manage to create a load
electric load; forecast model with the feature of facilitation and higher
forecast precision in the situation of using MATLAB
I. INTRODUCTION language to program.
For long years,load forecast of power system is largely
II. ELMAN NEURAL NETWORK MODEL AND USAGE OF IT
and deeply studied by people,and many significant methods
IN MATLAB
are proposed such as regression analysis method[1][2],time
series method[3][4] and grey system method[5] etc. In Elman neural network can be considered as a forward
general,conventional mathematical model use an explicit neural network with a local memory element and a local
mathematical representation to describe,which decides the feedback conjection,which is generally divided into four
limitation of traditional forecast model.In fact,the load of layers,including input layer,hidden layer,association layer
power system is a dynamic system influenced by many and output layer,the basic structure of the network is seen in
factors and having a nonlinear variation,and the variety Figure.1.In which the conjunction among input layer,hidden
regulation of load is difficult to use an explicit mathematical layer and output layer is similar to the forward neural
formula to describe.Then a method based on artifitical neural network,the neurons of input layer play a role of
network[6-8] becomes a great breakthrough in forecast transmission signal,and output layer neurons play a role of
fields,which regards independent variable and dependent linear weighted.The transfer function of hidden layer is still a
variable of traditional explicit function as input-output of the nonlinear function,which is generally using a Sigmoid
network,and inverts the relationship of conventional function function;association layer ia also called state layer,which is a
into a nonlinear mapping with high dimension not an explicit unit delay operator for saving the output state at previous
mathematical representation. Therefore,the method based on moment of the hidden layer to characterize the inner feature
artifitical neural network is largely used in analyzing and of the object.Then the inner feature together with external
forecast the complex and nonlinear system such as load in input signal are used as the input of hidden layer nodes at the
power system,and in which the Elman neural network with current moment,which is correspond to a state feedback.
the characteristics of reflecting the dynamic behavior directly The mathematical model of Elman neural network is
and lively is more and more valued by people.Elman neural analyzed as follows:
network is a typical network with dynamic neuron,which is
on the basis of the structure of BP neural network,a
particular association layer is introduced,and by memorying
the inner state to make it having a function of mapping the
dynamic behavior and then which makes the system having
the ability of adapting time inflecting

978-0-7695-3688-0/09 $25.00 © 2009 IEEE 55

DOI 10.1109/ITCS.2009.20
 y(k) Where K is the learning speed, mc is momentum factor
Output layer and its default is 0.9.So not only the gradient direction at
x(k) current moment is considered when updating,but also the
w3
b2 gradient direction at previous moment is considered,which
Hidden layer x1 ĂĂ xn can reduce the sensibility while using the performance of net
xc1ĂĂxcn Associate layer
w1
to modify the parameter and reject local minimum
2
w effectively.
b1
Input layer xc(k) III. THE PROGRAMMING IN THE SITUATION OF USING
ĂĂ u(k-1) MATLAB LANGUAGE TO PROGRAM
Figure.1 The Structure of Elman Neural Network
A. NNTool-Box in MATLAB
Seen in Figure.1,nonlinear state-place expression of the MATLAB NNTool-Box base on the artifitical neural
Elman neural network are as follows: network theory,which uses MATLAB language to write
y (k ) g ( w 3 x(k )  b2 ) ˄1˅ most subprograms of the theory refered to such as formula
operation,matrix manipulation and equation resolution and
x(k ) f (w1 xc (k )  w 2 (u (k  1))  b1 ) ˄2˅ so on,so that to design and train the neural network.The user
xc ( k ) x(k  1) is only need to call a relevant subprogram according to his
˄3˅
demand,which can help the user do a series of works
Where k represents the moment, y,x,u, xc are separately including designing of the structure,intialization the
representing a dimension vector of output node, m dimension weight,training the net and output the results,and make user
vector of hidden layer nodes, n dimension input vector and m win free of writing complex and big algorithm routine.
3 2 1
dimension feedback state vector. Then w , w , w are
separately representing connection weight matrix from B. Data processing
hidden layer to output layer,from input layer to hidden layer In the medium-term load forecast model,the history load
and from association layer to hidden layer. f (˜) is a transfer data of each month from the year of 2000 to 2006 of some
power system is used as the foundation on study.
function of the neuron of hidden layer,which using
For the data acquisition system having the behavior of not
log sig function,and g (˜) is a transfer function of output exactly reliability and there needing a relative higher load
layer,which using purelin function.Then b1 and b2 are forecast precision in project,so before using history data to
separately as the threshold of input layer and output layer. forecast,sifting out the abnormal data and effectively
In the learning algorithm of the Elman neural network, an processing them have become the first step to
optimization gradient-falling algorithm that is a back- forecast.Therefore,firstly according to the method proposed
propagation algorithm with adaptive learning speed and in the documation[10] to preprocess the load data,and in order
momentum gradient-falling is used,the algorithm is not only to make bigger input data located in the place of with great
can improve the training speed,but also can effectively gradient of transfer function,we should normalize and
inhibit the network to trap into a local minimum.The process the input sample.In the paper, the formula (6) is used
learning purpose is to use the difference between actual to converse input layer data into a value from the area of
ouput and the ouput sample to modify the weight and [0.2,0.9],and then formula (7)is used to converse output layer
threshold,and to make the error sum of squares of output value into load data.
layer minimize.Suppose the actual ouput vector of k step x  xmin
y 0.2  0.8 u
is yd ( k ) ,then in the time slice of (0ˈT),the error function xmax  xmin ˄6˅
is defined as follows: ( y  0.2)( xmax  xmin )
x  xmin
0.8 ˄7˅
1 T
¦ > y d ( k )  y ( k )@
2
E
2k1 Where xmin and xmax represent the minimun and maximum
˄4˅
3 2 of the sample set separately.In general,we often use 0.9
Then taken w , w for example,we use the function E to multiple to minimum and 1.1multiple to maximum on
3 2
have a partial derivative for w , w and can get a formula for project,it is of proof by pilot calculation that which can make
modifying the weight as follows: the sample easily trained and has a high rapidity of
'w13j (k  1) (1  mc)K ( y d (k )  y (k )) convergence.
˜ g c(˜) x j (k )  mc'w13j (k ) C. Creation input matrix and output matrix
j 1,2,, m To rationally set the structure of Elman neural network is
the foundation on improving the forecast performance.In
'w 2jq (k  1) (1  mc)K ( y d (k )  y (k ))
fact,how to set the structure of neural network especially the
˜ f jc(˜)u q (k  1)  mc'w 2jq (k ) neuron number of hidden layer is an empirical
j 1,2,, m, q 1,2,, n problem,which needs a large number of experiment.In the
˄5˅ paper,the processed data used to forecast are divided into two

56
parts,including training sample(which use the data from the DN '
year of 2000 to April 2006) and test sample(which use the 4;5 DN
data from May to August in the year of 2006).And there are 3
/:
,:
seven input value and one output value in the

 
/: DN \
5  6  Q
model.Considering in power system,the load at i moment
  
 
6 5

Q 6 6



influenced not only on external environment and inner


D 6 

6 


6  E

factors at i moment,but also on other factors at i-1,i-2,Ă,i-n


E 
6
moment for the course of formation with time lag of   

 6  
6 6 

5  

load,therefore,the load has dynamic behavior. But for the

data from external factors with indeterminacy in Input layer logarithmic function layer Output layer
themselves,so adding them into the model will influence Figure.3 The emulation structure of Elman neural network
forecast effect of the model.Therefore,in the disscussed
model,how to choose input variable only decide by the basic The emulation structure of Elman neural network is seen
history data from some power system.the output variable is in Fig.3,which includes input layer,hidden layer,association
P(i,j);the input variable u1, u2 are separately asP(i, j-1),P(i,j- layer and output layer. R1 is the number of input layer; S1
2);u3,u4,u5 are separately asP(i-1,j),P(i-1,j-1),P(i-1,j-2) and and S2 are separately as the number of association layer and
u6,u7 are separately as P(i-2,j),P(i-2,j-1),where i represents output layer; P is the input vector; b1 is a neuronic threshold
month and j represents year. vector in some layer such i; and IW1,1 represents the
For MATLAB is appropriate to input-output vector connection weight between input neuron and input
operation in particular,so MATLAB language is used to vector,then LWi,1 represents the connection weight between
program.In the program,the input matrix is a row vector with neuron of some layer such i and input vector; ni is an
P(1h7) and output matrix is a row vector with T(1h1). intermediate operating results among neuron in some layer
D. The programming such i, ai(k) is some output of some layer such i.Input layer
nodes are only transmission of input signal to the hidden
1) The main program flow layer,and the function log sig is used as transfer function of
Figure.2 is the main program flow diagram, which used
to forecast the medium-term load . hidden layer,that is as follows:
1
Start log sig ( x)
1  exp( x) (8)
Association layer is used to memorize the output of
Input the given training data
previous moment,and the linear function purelin is used as
transfer function of output layer,the reason is that all
Correcting and normalization elements of target vector are located in the area from 0.2 to
the abnormal data
0.9 and which can meet the output needs of log sig function
According to the characteristic of network just right.
load ,generating the input matrix For how to give the hidden layer nodes of neural
network.In the paper,according to Kolmogorov
Using "newelm" to establish a network principle[11],we get a theoretical value of hidden layer nodes
and initialization the neural network at first.The principle that is for a any continuous function
Using the optimizational algorithm with
) ,where there is a neural network with three layers can
gradient-falling to tain the network,then realize it perfectly,if input layer of the network has m
to get the weight value of each model units,then 2m  1 units can be given to the hidden layer.then
the method proposed in documentaition[12] (the method
Using Elman neural network model to which is to use the formula N d 4 m(k  3)  1 ,where N
realize the medium-term load forecasting represents the number of hidden layer nodes;m is the number
of input layer nodes and k represents the number of output
Output the forecast results layer nodes)is used and the hidden layer nodes are tried to
set,and we can get the optimum number of hidden layer
nodes for Elman neural network model by emulating and
The end
matching the data in the area of (l  G , l  G )G  N ,where l
Figure.2 the program flow chart of medium-term load forecast using a are the theoretical nodes of the hidden layer.By several
modified back-propagation algorithm training,we come to a conclusion that the neuron number of
each layer are seperately as 7-17-1.
2) Creation neural network Using Elman neural network to model:

57
 % the variable threshold is used to set the maximum and From above results,we can obtain that error of network
minimum of input vector, which sets the maximum as 0.9 answers the requirements by training the network at 127
and the minimum as 0.2. times.Results are shown in Figure.4.
threshold >0.2 0.9;0.2 0.9;0.2 0.9;0.2 0.9;0.2 0.9;0.2 0.9;0.2 0.9@ 0
10
Performance is 6.13087e-005, Goal is 0.0001

% using ' newelm ' function to create a Elman neural network

net newelm(threshold , >17,1@ , ^'log sig '' purelin '` ,' traingdx ') -1
10

When using ' newelm ' function to create neural network,it is

Training-Blue Goal-Black
-2

random to initialize the weight and threshold value.

10

' traingdx ' represents to assume the training function as -3

10

traingdx,which using a back-propagation algorithm with

adaptive learning speed and momentum gradient-falling to -4
10

learn.and to set the initial weight matrix on training are as

follows by choosing initialization function. -5
10
0 20 40 60 80 100 120
net.layers{i}.initFcn='initwb' ˗ 127 Epochs

net.inputWeights{i,j}.initFcn='rands' ˗
Figure.4 the training results of Elman neural network model
net.bias{i,j}.initFcn='rands' ˗
Additionally,we need to test the trained network to judge
net=init(net) ˗ whether to put it into application.In the paper, emulator
3) Train the neural network function sim is used to calculate the output of the network.
% train the network
Out sim(net , P _ test )
> net , tr @ train(net , P, T )
% to setup training parameter for network % P _ test is the test data of network
The running results of the program are as follows:
net .trainParam.epochs 300 % training times
Out =41.336
net.trainParam.goal 0.0001 % the aim to training The relative error is 1.14%(actual load data is 40.87
net.trainParamlr
. _ inc 1.05 % growth scale factor of learning speed HMKWH),so we obtain that the relative error between actual
load and forecast load is little,which can meet the needs of
net.trainParam.lr _ dec 0.7 % decrea sin g scale factor of learning speed scheduling in power system.
Finally, we need to program and train the neural
net.trainParamshow
. 25 %training steps betweentwo display segments network,then to simulate and forecast results can be
net.trainParam.max_ fail 5 % max imum failure times obtained.forecast results of May to August in the year of
% the condition for stopping to train includes: 2006 are shown in Table 1.
Reaching maximum training times,making error function TABLE I. THE FORECAST RESULTS
of the network meet the required value,and making training
time get to maximum,let gradient of performance function f orecast
average
load based
decrease until it reaches the minimum or proving the failure month
actual data of
on Elman
relative absolute
times in exceed of maximum times. load in 2006 error(%) relative
neural
error(%)
% the training results network
For example, train results of load forecasting model of 5 40.87 41.336 1.14
some power system in May are as follows:
6 41.25 40.896 0.8
TRAINGDX, Epoch 0/300, MSE 0.194223/0.0001, 1.375
Gradient 0.16035/1e-006 7 40.18 39.311 2.16
TRAINGDX, Epoch 25/300, MSE 0.184502/0.0001,
8 40.07 39.509 1.4
Gradient 0.169012/1e-006
TRAINGDX, Epoch 50/300, MSE 0.149319/0.0001,
Gradient 0.18223/1e-006 IV. THE RESULTS
TRAINGDX, Epoch 75/300, MSE 0.0760217/0.0001, In association with NNTool-Box of MATLAB
Gradient 0.0813551/1e-006 software,using Elman neural network to have a electric load
TRAINGDX, Epoch 100/300, MSE 0.0223252/0.0001, forecast model,and then we jump to conclusions that are as
Gradient 0.0413701/1e-006 follows by testing for some power system data.
TRAINGDX, Epoch 125/300, MSE 0.000160017/0.0001, 1ǃ It is proved that the way to use Elman neural network
Gradient 0.0027196/1e-006 with dynamic characteristics to identify and forecast
TRAINGDX, Epoch 127/300, MSE 6.13087e-005/0.0001, electric load is feasible.
Gradient 0.00227949/1e-006 2ǃ It is of illustration that the forecast method proposed in
TRAINGDX, Performance goal met. this paper has the characteristics of higher speed and

58
 better effectiveness,and Elman neural network has [4] F.Meslier. New Advances in Short-Term Load Forecasting Using
greater fitting ability and higher forecast precision. Box and Jerkins Approach. Paper No.A78, IEEE Winter Power
Meeting New York.1978.
3ǃUsing MATLAB language to program makes input and
[5] Liu Chenhui. The Way and Theories of Electric Load
output vector quantify,which makes brief of model Forecasting[M].Harbin Industry of technology Press.1987.
greatly. And based on NNTool-Box in MATLAB to [6] Park. D. C, El-Sharkawi, M.A. Electric load forecasting using an
program has much higher efficiency than using other artificial neural network. IEEE Transaction on Power Systems,
languages to program,which only need several seconds 1991,6(2).442-449.
to train the neural network. [7] Alex D. Papalexopoulos. Shangyou Hao. Tie-Mao Peng. An
4ǃForecast results show that Elman neural network model Implement of A Neural Network Based Load Forecasting Model for
The EMS. IEEE Transactions on Power System. Vol.9,No.4.
has the behaviour of brevity and practicality,and the November 1994: pp.1956-1962.
forecast precision of the model can reach percent [8] Wang Xin. Advanced Radial Basis Function Network(RBFN) and its
1.375(it is the value of average absolute relative Application in short-Term Load Forecasting[J].Proceedings of the
error ),which can meet the needs of running and CSEE,1996,16(4):285-287.
scheduling in power system. [9] Cong Shuang.Application and Theories of Neural Network Based on
MATLAB NNTools[M].Hefei:University of Science and Technology
V. REFERENCES of China Press,1998
[10] REN Li-na1 ,RUI Zhi-yuan1,LIU Yan-xin2 ,XU Long-Yun1. Study on
Identification Method of Anomalous Electric Load Data[J].Water
[1] A.D.Papalexopoulos and T.C.Hesterberg, A regression-based
Power, 2008,34(2):43-45.
approach to short term system load forecasting. IEEE Transaction on
Power Systems, vo1.5, no.4, 1990, pp.1535-1547. [11] Wang Kejun,Wang Kecheng.Model,Forecast and Control of Neural
Network.Hei Long-jiang: Haibin Engineering University Press,1996.
[2] T.Haida and S. Muto. Regression Based Peak Load Forecasting Using
A Transformation Technique. IEEE Trans. Power Syst., vol.9, 1994, [12] Zhang Zhiguo. Research on Damage Identification in Bridge Based
pp.1788-1794. on Modal Analysis Theory and Neural Networks[D].Wuhan:Wuhan
University of Technology,2005.3.
[3] Moghram I, Rahman S. Analysis and evaluation of five short-term
load forecasting techniques. IEEE Trans. on PWRS, vol.4, no.4, 1989,
pp:1487-1491.

59