Energy 138 (2017) 306e315

Contents lists available at ScienceDirect

Energy
journal homepage: www.elsevier.com/locate/energy

Model-based predictive maintenance in building automation systems

with user discomfort
Nathalie Cauchi a, *, Karel Macek b, Alessandro Abate a
a
Department of Computer Science, University of Oxford, Oxford, United Kingdom
b
Honeywell Prague Laboratory, Honeywell, S.r.o Prague, Czech Republic

a r t i c l e i n f o a b s t r a c t

Article history: This work presents a new methodology for quantifying the discomfort caused by non-optimal temper-
Received 29 April 2017 ature regulation, in a building automation system, as a result of degraded biomass boiler operation. This
Received in revised form discomfort is incorporated in a model-based dynamic programming algorithm that computes the
14 July 2017
optimal maintenance action for cleaning or replacing the boiler. A non-linear cleaning model is used to
Accepted 15 July 2017
Available online 18 July 2017
represent the different cleaning strategies under taken by contractors. The maintenance strategy mini-
mizes the total operational costs of the boiler, the cleaning costs and the newly defined discomfort costs,
over a long-term prediction horizon that captures the short-term daily thermal comfort within the
Keywords:
Predictive maintenance
heating zone. The approach has been developed based on real data obtained from a biomass boiler at a
Biomass boiler modelling Spanish school and the resulting optimal maintenance strategies are shown to have the potential of
Temperature regulation significant energy and cost savings.
Thermal comfort © 2017 Elsevier Ltd. All rights reserved.
Dynamic programming
Energy savings

1. Background & motivation monitored system subject to degradation due to imperfect main-
tenance is proposed in Ref. [7]. In the context of heating devices for
The European Union aims to increase the contribution of building automation systems (BAS) [8], devised an algorithm that
renewable energy sources by up to 20% by 2020 [1] and it is stip- optimises the frequency of the cleaning process, in order to
ulated that biomass fuels will account for 56% of renewable energy improve boiler performance. Similarly, the work in Ref. [9] has
generation [2]. Many research initiatives are being carried out to applied more accurate first-principle modelling and optimised the
take advantage of biomass potential as a renewable energy source duration and timing of the cleaning process [10]. suggested the
[3]. Biomass boilers provide a cleaner alternative to conventional synthesis of predictive maintenance strategies based on dynamic
gas boiler. However, this comes at a cost of being more subject to programming: this technique is further expanded by including a
soot accumulation due to fouling. Heat transfer surface fouling is a combination of forecasting and optimisation methods in Ref. [11].
continuous issue that negatively impacts heat exchanger operation Predictive maintenance of devices has been widely studied towards
by reducing the boiler efficiency and increasing the cost of opera- potential energy saving goals in BAS, however the effect of
tion [4]. Thus, optimal cleaning strategies are required to minimise discomfort on occupants and its related costs has been broadly
the effect of fouling. This can be achieved by employing predictive ignored [12e14]. Numerous contributions highlighting the effect of
maintenance strategies. discomfort [15,16] have stressed the importance of accounting for
Predictive maintenance is defined as maintenance strategy that the cost of indoor thermal comfort.
is able to forecast the trend of performance degradation [5]. The This contribution studies the problem of quantifying discomfort
work in Ref. [6] addressed this issue by applying mathematical due to non-optimal temperature regulation, in BAS, as a conse-
modelling to deduce the optimal maintenance policy that in- quence of degraded boiler operation, and that of using this cost
corporates periodic inspection and replacements. A reliability- within a predictive maintenance optimisation problem. Predictive
centred predictive maintenance policy for a continuously maintenance strategies are computed based on a total costs
including contributions due to operation, maintenance, and (new
in this work) discomfort. Optimal strategies are computed using
* Corresponding author. Dynamic Programming (DP) and comprise three alternative
E-mail address: nathalie.cauchi@cs.ox.ac.uk (N. Cauchi).

http://dx.doi.org/10.1016/j.energy.2017.07.104
0360-5442/© 2017 Elsevier Ltd. All rights reserved.
 N. Cauchi et al. / Energy 138 (2017) 306e315 307

maintenance actions (No Action, Clean, and Replace). Unlike related the comfort level when the boiler operates using the maximum
studies in literature [11] focusing on long-term horizons (and efficiency hmax (th
thmax ). Thus, to properly capture the effect of
weekly time steps), in order to account for discomfort this study discomfort due to boiler degradation the following rule is applied:
needs to consider also daily horizons and intra-day temperature Account for discomfort only when:
dynamics: we put forward a daily discomfort metric that can be
embedded in the overall longer-term contributions to the cost d
Tz < TSP
and bon ¼ 1; (1)
function. A daily time step further allows for capturing trends that 2
represent the output power consumed by the building. This leads to
devising predictive maintenance strategies over larger horizons where Tz represents the zone temperature, TSP is the required
(for instance over 5 years) that newly account also for discomfort temperature set point, d represents the width of the considered
costs. comfort region, and bon ¼ 1 represents the boiler in the ON state.
This article has the following structure: Section 2 introduces an In order to capture the total time taken to reach the temperature
approach for quantifying discomfort, which is followed by the set point within the zone and to apply the defined rule to compute
synthesis of optimal maintenance policies in Section 3. This in- the associated discomfort, we consider a thermal model repre-
cludes the embedding of discomfort metric within the policy syn- senting the zone dynamics. Whilst this model accounts for the
thesis architecture. The framework is applied to a Spanish school intra-day temperature dynamics, the discomfort metric is a daily
case study which is presented in Section 4. The resulting algorithms value that can be employed in the cost function alongside opera-
are demonstrated under different scenarios in Section 5, which also tional and maintenance costs (as discussed later). In this work, the
discusses the total energy and discomfort savings obtained by the thermal model being used is a first-order model that is given by
scheme.
dðTz Þ Tout
Tz
Cr ¼ þ hbon Pin þ Pocc þ Prad : (2)
dt Row
2. Discomfort as a consequence of degraded temperature
control here Tz is the zone temperature, Tout is the outside air temperature,
Pin is the biomass boiler input power with bon representing the
In this study, we consider a correctly sized biomass boiler with a when the boiler is on, Prad is the heat gained from solar radiation,
hopper, supplying thermal energy to a zone. The boiler has a fixed Pocc is the heat gained due to occupants, Cr is the zone capacitance,
schedule and is turned on during the start of the working day and Row is the thermal resistance of the zone envelope, and h2½0; 1 is
switched off at the end. When in operation, the boiler combusts the daily boiler efficiency, which is the dual of the boiler degrada-
biomass fuel in order to generate heat in the heat exchanger and tion (1
h). The values of the parameters Cr and Row are estimated
consequently supply heat to the building. This combustion process from data using maximum likelihood [18,19] or Kalman filtering
causes soot accumulation in the heat exchange chamber, a process [20].
known as fouling, which degrades the boiler efficiency with time. To compute the total time to reach the temperature set point
During the transient when the boiler is switched on, a lag in the (TSP ) following an OFF/ON boiler transition, (2) is solved analyti-
time taken to reach the building set point is witnessed with cally, considering the initial time to be t0 and bon ¼ 1, (as per (1)):

       Zth  


th th t0 t Tout ðtÞ
Tz th ¼ exp ðTz ðt0 Þ þ hPin Row exp
exp þ exp dt
Cr Row Cr Row Cr Row Cr Row Cr Row
t0

Zth  
t Pocc ðtÞ
þ exp dt (3)
Cr Row Cr
t0

Zth  
t Prad ðtÞ
þ exp dtÞ;
Cr Row Cr
t0

degradation of boiler efficiency. Discomfort can thus be seen as a where th is the final time step and indicates the time at which the
function of the additional time taken to reach a temperature set lower comfort temperature value is reached (Tz ðth Þ ¼ TSP
2d ) and
point due to the degradation of the boiler efficiency, which other- where we assume next for simplicity that t0 ¼ 0. In order to
wise operates properly [17]. has also made use of this notion to reason about longer-term device degradation and the corre-
quantify occupant comfort. sponding maintenance actions, we intend to map the continuous
Consider Fig. 1, where Tz is the zone temperature (namely the temperature dynamics into daily discrete discomfort values that
temperature in the building). When the boiler efficiency h is lower quantify the effect of boiler degradation on thermal comfort. To
than hmax , the zone temperature takes a longer time to reach the perform such a mapping, the following choices and observations
lower bound of the comfort region d. This introduces discomfort are made:
during the switching ON phase of the boiler. This discomfort is
captured in the form of a time delay expressed as the difference 1. Define the daily time steps as n ¼ f1; …; Ng, with N being the
between the time taken for the zone temperature to reach the final time step.
comfort region with the current h level and the time taken to reach
308 N. Cauchi et al. / Energy 138 (2017) 306e315

Fig. 1. The effect of boiler degradation on the zone temperature and the introduced discomfort, during the transient operation of a boiler as it transitions from the OFF to the ON
state. Here Tz ðt0 Þ is the initial temperature, TSP is the zone temperature set-point, d represents the acceptable comfort region and the discomfort region is th
thmax .

 
2. TSP
2d represents the required minimal temperature for 2.1. Computation of discomfort time
comfort (cf. (1)).
3. Cr ; Row are fixed and given parameters based on the character- In order to capture the daily discomfort (for day n), the
istics of the zone heated by the boiler. continuous-time thermal model described above is employed to
4. We represent the term Pin by a new variable Pmn , which is fixed predict the daily temperature evolution, and the total discomfort
during a day. Pmn represents the mean of the total input power time (discTime) is set as the difference between the time taken to
Pin supplied to the boiler over the course of the day and captures the reach set point comfort zone when operating at maximum ef-
the switching on and off of the biomass boiler during the day as ficiency (th½n ¼ thmax ) and the time taken to reach the set point with
a simple daily variable. the efficiency level at day n (th½n ). More precisely, the value of thmax is
5. Tz ðt0 Þ ¼ Tout ½n and Pocc ¼ Prad ¼ 0, which captures a worst-case computed by using (5) with h½n ¼ hmax , while the value of th½n is
discomfort due to the boiler degradation during the heating computed by again applying (5) with the current h½n. The quantity
season. Here Tout ½n represents the worst case equilibrium discTime is then:

8  d
> n o TSP

Tout

 < 2
discTime n ¼ min th½n
thmax Þ; discTimemax h½n >
Pmn Row (6)
>
:
discTimemax ; Otherwise:

ðTSP
dÞ
Tout
temperature in the zone reached during the interval when the The condition h½n > Pmn2Row is introduced to capture
boiler is switched off. discomfort both when the boiler is able to reach the required
6. We have access to the daily demanded output power Qout ½n and comfort level and when it is not. When the comfort level is never
input power Qin ½n from which h½n is computed as, reached, this condition is not met and the maximum possible
discomfort time, discTimemax , which is set to 12 h, is assigned.
Qout ½n Remark 1. Note that when the zone temperature exhibits oscil-
h½n ¼ : (4)
Qin ½n latory dynamics due to either sufficiently high degradation levels in
the boiler (equivalently, sufficiently low boiler efficiency) or when
the boiler has been improperly sized and does not supply enough
7. As the efficiency, h/0 (degradation level is too high) and
heat to the zone, the zone dynamics do not converge to the set-
Tout ¼ Tz , dT
dt
z
/0, hence the boiler will be supplying negligible point as shown in Fig. 1. Consequently, the discomfort computa-
heat gain to the room and Tz will not reach the required minimal tion reasoning would need to be further elaborated using the same
 
line of reasoning. In this case, the boiler is assumed to be properly
temperature for comfort TSP
2d . This leads to a maximum
sized and the constant discTimemax is used to give an adequate
discomfort, which is denoted with value discTimemax . discomfort penalty when the comfort region is never reached.

Solving (3) for th½n (for each daily time step n), the resulting total
time required to reach comfort zone is then given by 3. Synthesis of the optimal maintenance strategy

2 3 We first formalize the fouling process as a dynamic system and

h½nPmn Row select a daily time step such that (i) the transient discomfort metric
th½n ¼ Cr Row ln4 5: (5) can be captured and (ii) the overall longer-term contributions of the
TSP
2d
ðh½nPmn Row þ Tout ½nÞ
heating demand can be embedded. We will represent the daily time
 N. Cauchi et al. / Energy 138 (2017) 306e315 309

steps as n ¼ f1; …; Ng, with N representing the final time horizon. here, pclean and preplace represent the constant monetary costs of
We have access to the daily demanded output power Qout ½n and cleaning and replacement, respectively.
input power Qin ½n from which we also gain access to the aggre-
gated daily biomass boiler efficiency level h½n (cf. (4)). The possible 3. Discomfort costs. This cost is due to degraded thermal regu-
states are ðQout ; hÞ3ℝ2 , and we succinctly represent states as x. The lation due to boiler degradation. It is a function of the monetary
set of all maintenance actions is u½n2U; U ¼ f0; 1; 2g, where: discomfort cost, pdisc , and of discTime½n, which is computed
8 using (6).
<0 No Action;
U¼ 1 Clean; (7) We are interested in the optimal maintenance strategy that
:
2 Replace: minimizes the total costs over a time interval, n ¼ f1; …; Ng. The
optimal maintenance policy is obtained by computing the decision
As suggested in Ref. [11], the following dynamics for Qout ,
rule p½n : ðQout ; hÞ/U for all n ¼ f1; …N
1g, such that the total
Qout ½n þ 1 ¼ gQout ½n þ ð1
gÞh½nr½n; (8) cost

where h½n is a boolean function representing the heating season, X

N
Cðx½n; u½nÞ (13)


 1 Heating season; n¼1
h n ¼
0 Otherwise;
is minimized, subject to the state evolution model (representing
the dynamics of the system) x½n þ 1 ¼ f ðx½n; u½nÞ, and to the
and r½n captures the long-term trend of Qout ,
control action u½n ¼ p½nðx½nÞ, where p½n is the policy. This can be
      solved by applying Dynamic Programming (DP) [21,22]. The DP
2pn 2pn 4pn
r½n ¼ b0 þ b1 cos þ b2 sin þ b3 cos algorithm starts from the final time horizon N and backward
365 365 365
  recursively computes value functions V, which are minimized
4pn
þ b4 sin ; (9) during each time step,
365
V½nðx½nÞ ¼ min Cðx½n; u½nÞ þ V½n þ 1ðx½n þ 1Þ; (14)
where bi , i ¼ f0; …; 4g are parameters: this will be further dis- u½n2U
cussed in Subsection 4.1.
The efficiency model h is set up as The optimal policy p½n is obtained as a look-up table (LUT)
expressing the optimal input action to perform for a given Qout and
8
< h½n þ aQout ½n u½n ¼ 0; h pair. The employed algorithms are further elaborated in


h nþ1 ¼ gðh½nÞ u½n ¼ 1; (10) Subsection 4.3.
:
hmax u½n ¼ 2:

The model depends on the control action performed (u½n).

When no cleaning is performed (u½n ¼ 0), the efficiency degrades 4. Case study
based on the value of Qout ½n and on the previous h½n. In this case,
the parameter (a < 0), related to boiler fouling, is estimated from We consider data from a real building, involving the heating of a
sampled boiler data, representing Qout and h, by applying the school in Spain by a properly-sized biomass boiler [23]. The school
standard lscov Matlab function. If on the other hand cleaning is only operated on weekdays and the heating system is switched
(u½n ¼ 1) is performed, h½n is updated based on a given function on from 08:00 to 15:00, between the first week of November and
gðh½nÞ, which represents the chosen cleaning model. Computation the last week of February (a total of 14 weeks), with the exception
of gðh½nÞ will be further discussed in Subsection 4.1. Finally, when a of the last week in December and the first week in January when
replacement occurs (u½n ¼ 2), the efficiency level is restored to the building is closed. The biomass boiler input ðPin;d Þ1 and output
hmax . power ðPout;d Þ have been logged for one year, with a 15-min sam-
During each period n, the associated costs are a function of the pling time. The daily trend of power demand by the boiler, the daily
dynamical system and action performed and is expressed as supplied power, and the mean daily boiler input power are
computed using the quantities
Cðx½n; u½nÞ ¼ C op ðx½nÞ þ C maint ðu½nÞ þ C disc ðx½nÞ
Qout ½n (11) X
nq X
nq
¼ pin þ pmaint u½n þ pdisc discTime½n: Qout;d ½n¼ Pout;d ½k; Qin;d ½n¼ Pin;d ½k;
h½n k¼ðn
1Þqþ1 k¼ðn
1Þqþ1
(15)
The function is composed of three separate costs, as follows: X
nq
Pin;d ½k
Pmn;d ½n¼ :
q
1. Operational costs. This is the cost associated with the working k¼ðn
1Þqþ1

of the boiler, with pin representing the monetary fuel costs. The
higher the demanded output power and fuel costs, and the here n ¼ f1; …; 365g represents the daily time steps, k is an index
lower the efficiency, the higher the operational costs. representing the aggregation of the 15-min sampled data per day,
2. Maintenance costs. This is due to either the cleaning or to the and q ¼ 4  24 represents the total number of records per day. The
replacement of the boiler with boiler efficiency is defined as the ratio between the output power
demand and the input power, and it can be derived from Qout;d and
8
< pclean u½n ¼ 1; Qin;d as
pmaint ¼ preplace u½n ¼ 2; (12)
:
0 Otherwise:
1
Throughout this paper we denote data with variables of the form xd , while
models variables as x.
310 N. Cauchi et al. / Energy 138 (2017) 306e315

Table 1 heating demand is positive, by means of use of frequency-domain

Values of regressed parameters. bi linear regression [24]. This results in (9) with bi , i ¼ f0; …; 4g
Parameter b0 b1 b2 b3 b4 correspond to the values in Table 1 and are obtained by fitting the

1.079
0.581
model using the lscov Matlab function. The resulting parameters
Value (105 ) 1.664 0.392 0.291
are in the order of 105 due to the magnitude of Qout;d values (104 )
which the function is trying to model. The resulting fit has a stan-
dard deviation equivalent to 17.01% of the mean.
Qout;d ½n The parameter g2½0; 1 is estimated by minimizing the
hd ½n ¼ : (16)
Qin;d ½n following objective function, namely

daily weather measurements of the outside air temperature X

365
2
Tout;d ½n have also been recorded. The zone capacitance Cr and Row g ¼ arg min Qout;d ½n
iQout;d ½n
1
ð1
iÞh½nr½n ;
i
d¼2
are given having a value of 54:56J=K and 0:023K=W, while the
required set-point TSP and comfort region d are given as 21o C and (17)
±1o C, respectively.
We are interested in optimally scheduling the maintenance of over the whole training data set (365 days). The resulting value is
the biomass boiler using the framework presented in Section 3, so g ¼ 3:125  10
10 , which is due to the overall mean difference
that maximal cost savings are attained. Costs relate to energy usage between Qout;d and r½n and leads to a small contribution of the
in regular (possibly degraded) daily operation of the boiler, to de- auto-regressive term.
vice maintenance actions, and to occupant discomfort due to boiler
degradation within the heating zone. We have selected a prediction
4.1.2. Model for efficiency
horizon of N ¼ 5  365 (5 years) and considered daily contributions
The efficiency model h is set up using (10). In our case when
to the overall costs. The case study and all algorithms are imple-
mented in Matlab R2016b. u½n ¼ 0, using hd and Qout;d we get a ¼
1:074  10
7 kWh
1 . The
small order of magnitude of a (10
7 ) is a direct consequence of the
4.1. Model of boiler degradation large magnitude of Qout which is in the order of 104 . This model
corresponds to a 13% efficiency drop over one heating season.
The dynamics for the underlying degradation model are based If cleaning u½n ¼ 1 is performed, h½n is updated based on a
on two variables representing the evolution of demanded output given function gðh½nÞ as in Fig. 2. The trend is computed by fitting
power Qout and efficiency h respectively (cf. Sec 3). The models the curve,

8  2 !  2 !
>
> h½n
b1 h½n
b2
< a1 exp
þ a2 exp
h½n < hmax
gðh½nÞ ¼ c1 c2 (18)
>
>
:
hmax Otherwise;

assume that the zones are not used during the weekends, and are to training data obtained from the rule that the effect of cleaning
built based on one year of training data obtained from the Spanish varies between 7 and 15% depending on the current efficiency level
school data set. and up to the maximum efficiency level (hmax ) [25]. Here,
a1 ¼ 0:654; b1 ¼ 0:681; c1 ¼ 0:428; a2 ¼ 1:402; b2 ¼ 1:723 and
4.1.1. Model for output power demand c2 ¼ 0:661. Fig. 2 shows that for an initial h½n > ¼ 0:5, cleaning
We make use of (8) and create a classifier h½n that for a given n, a results in an approximate improvement of 15% (h½n þ 1z1:15h½n),
value of one is assigned during the heating season and zero whilst for an initial 0:1 < h½n < 0:5, cleaning results in an approxi-
otherwise. Next, we construct the long-term trend r½n by mate 7% of overall improvement (h½n þ 1z1:07h½n). For h½n < 0:1,
regressing Spanish school data of Qout;d corresponding to when the the cleaning effect saturates to 0.08.

Fig. 2. The imperfect cleaning model gðh½nÞ, incorporating the effect that for high initial values of h the effect of cleaning is large, while for low initial values of h the effect of
cleaning is low. The value of h following cleaning saturates at a maximum value of h ¼ 0:8 and at a minimum value of hmin ¼ 0:08.
 N. Cauchi et al. / Energy 138 (2017) 306e315 311

Fig. 3. Resulting model fit between Qout and Tout . The exponential function is seen to have a better fit to the data.

When a total replacement occurs (u½n ¼ 2), the efficiency level the linear relationship between Qout and Pout and then the expo-
is restored to hmax ¼ 0:8. nential relationship between Pout and Tout as can be seen from (3).
For both models, the unknown parameters can be estimated by
solving,
4.2. Regression functions for prediction of the time spent in
discomfort X
L
2
min ðv½n
w½n; qÞ : (21)
q
We are interested in optimally scheduling maintenance based l¼1
on the dynamic fouling model, composed using the states ðQout ; hÞ,
and the total costs (operational, maintenance and discomfort here q are the parameters to be estimated, l is an index representing
costs). Using the state-space models we obtain the operational and the training data having a total length L ¼ 365 and v½n ¼ Tout;d ½n.
For the case of the linear model w½n; q ¼ b Qout ½n

m
maintenance costs directly (cf. (11)). However, to compute the s þ z with
discomfort costs, we need to predict discTime over the required q ¼ ½b; zT , while for the case of the exponential model
time horizon using (6). discTime is a function of the dynamic vari- Q
g out;ds
½n
m

ables h, Tout and Pmn and the fixed variables Cr ; Row ; d and TSP (these w½n; q ¼ e j

a with q ¼ ½g; a; jT and with m ¼ 9:84  103 ; s ¼
are given). Consequently, to predict discTime using the fouling 5:78  103representing the mean and standard deviation of Qout;d
model, we need to construct models which relate the Qout to Tout respectively. The resulting parameter values from this mini-
and Qout to Pmn . The Spanish data set contains the following vari- misation are b ¼
0:3743 and z ¼ 1:847 for the linear model,
ables, Qout;d ; Qin;d ; Tout;d and Pmn;d , which allows us to construct the g ¼
0:4932; a ¼ 0; j ¼ 0:4840 for the exponential model. The
required regression functions to relate Qout to Tout and Qout to Pmn . resulting fit for both candidates is shown in Fig. 3.
Regression models relating Qout and Tout . Two candidate regres- The exponential model is chosen as the best model to compute
sion models that relate Qout and Tout are found by considering the the value of Tout from Qout as it achieved the lowest RMSE (the
recorded Spanish weather data. linear model achieved an RMSE value of 2.378, whereas the expo-
nential model has an RMSE value of 2.278).
 Candidate 1: The linear model Regression models relating Qout and Pmn . A similar approach is
carried out to regress the relationship between Qout and Pmn such
Qout ½n
m that predictions of the total time in discomfort can be made. In this
Tout ½n ¼ b þ z; (19)
s case, as intuitive only a linear model is considered, and the recor-
ded data for the mean input power during a day, Pmn;d is used (cf.
 Candidate 2: The exponential model (15)). The resulting model is given by

Qout ½n
m Pmn ½n ¼ b1 Qout ½n þ z1 (22)

eg s
a
Tout ½n ¼ : (20)
j
where b1 ¼ 4:262  10
2 and z1 ¼ 210. The parameters are esti-
here, m and s are the mean and standard deviation of Qout;d . The mated once again using (21) with q ¼ ðb1 ; z1 ÞT , v½n ¼ Pmn;d ½n and
exponential model is considered in this section due to the expo- w½n; q ¼ b1 Qout;d ½n þ z1 . The resulting fit is shown in Fig. 4. The
nential relationship between Qout and Tout obtained by considering model fitness has a RMSE value of 99.6.

Fig. 4. Resulting model fit between Qout and Pmn .

312 N. Cauchi et al. / Energy 138 (2017) 306e315

4.3. Solution of the dynamic programming problem following prices (i) pin ¼ V0.27/kg2 (ii) pclean ¼ V108.90, preplace ¼
V50003 and (iii) pdisc ¼ V100/day [23]. In all cases, we encode the
Using the defined state evolution models and cost functions the resulting control actions as follows - Dark Blue: Replace boiler;
optimal maintenance problem is solved by employing the DP Light Blue: Clean Boiler; White: No Action.
framework. We consider two modalities for performing the soot
cleaning actions: on-load and off-load [26,27]. The biomass boilers Experiment 1: Presence vs absence of discomfort. Fig. 6 shows
can be cleaned while it is running by being fitted with retractable the resulting look-up tables (LUT) for the optimal control ac-
soot blowers which perform cleaning (on-load). Alternatively and tions as a function of h values. The LUT are generated using
conventionally, soot cleaning can be performed when the boiler is Algorithm 1.
disconnected (off-load).
Two algorithms are proposed each representing a different type
of cleaning modality (on-load and off-load cleaning). Algorithm 1 Remark 2. The resulting optimal maintenance strategy can be
considers the off-load scenario incorporating a total of one time seen to be qualitatively periodic within each heating season due to
delay to perform a cleaning action, while Algorithm 2 assumes that the periodic trend of the demanded output power (cf. (9)). How-
the maintenance actions are carried out instantaneously, which is ever, this periodicity is only approximate, due to the differences in
the case when on-load cleaning is carried out. the initial h and Qout at the beginning of each heating season.

For both algorithms, one must discretise the state space over In order to further clarify the effect of incorporating discomfort
the selected minimum and maximum bounds monetisation zoomed up versions of Fig. 6 showing the resulting
(½hmin ¼ 0:03; hmax ¼ 0:8 and ½Qout
max ¼ 2:5  104 ; Q min ¼ 0). The
out
LUT for the heating season in 2014 and 2015 is presented. From
value function for each of the discretised states is then computed Fig. 5(a), it can be seen that when we consider discomfort in the
at each time step, recursively (cf. (14)). In the case of off-load optimal policy computation, cleaning actions are only performed
cleaning one needs to incorporate the effect that maintenance on weekends when it is expected that the boiler is operated
actions take up to one day to be carried out. This is introduced as throughout the rest of the week. This minimizes the effect of
an additional cost penalty into the value function computation discomfort experiencedhad boiler maintenance been performed
step corresponding to a day where Qout ¼ 0 since the boiler is not when the building is in use. For weeks 2e7 and weeks 14 and 15,
running (cf Algorithm 1 Line 7). when 0:11 < h < 0:18 replacement is carried out during weekends.
For weeks 10e13, this replacement is performed when
5. Case study: results 0:13 < h < 0:19. This is due to the higher expected Qout . When
h  0:13 for weeks 10e13 and h  0:11 for the rest of the weeks
We synthetise optimal maintenance strategies, using the
developed algorithm for both on-load and off-load cleaning, with a
prediction time horizon considered set to N ¼ 5 years. First, we 2
http://www.avebiom.org/es/noticias/News/show/precios-del-pellet-en-
consider two experiments to show the effect of discomfort mone- espana-653.
3
tisation on the resulting policy. For each experiment, we use the http://www.tucalderabarata.es/reparacion-de-calderas/.
 N. Cauchi et al. / Energy 138 (2017) 306e315 313

Fig. 5. Optimal Maintenance strategy that (a) presence vs (b) absence of discomfort costs over one heating season (from Monday of the first week of November 2014 up to Sunday of
the last week of February 2015). The control actions are colour coded using - Dark Blue: Replace boiler; Light Blue: Clean Boiler; White: No Action. (For interpretation of the
references to colour in this figure legend, the reader is referred to the web version of this article.)

Fig. 6. The resulting LUT for the optimal control actions when (a) discomfort costs are included and (b) not included. The control actions are colour coded using - Dark Blue: Replace
boiler; Light Blue: Clean Boiler; White: No Action. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

excluding week 8, the cost of operation is much larger then the cost For these two scenarios, we perform a set of 103 simulation runs
of discomfort and thus maintenance actions are performed on where we obtain an initial efficiency level by sampling a uniform
weekdays. On the other hand, from Fig. 5(b) we deduce that when distribution between the range ½0:03; 0:8 and with Qout fixed to
discomfort is not considered, cleaning actions are performed both 1.183 104 (corresponds to initial value found in Spanish school
on weekdays and weekends for all weeks excluding week 8. No data set), are performed. For each run ((i) using LUT generated using
maintenance actions are performed on Saturday due to the one step pdisc ¼ V100/day; (ii) using LUT generated using pdisc ¼ 30=day), we
ahead prediction of a no power demand Qout since the boiler is further compute the total discomfort cost incurred when the
switched off on Sunday. No maintenance actions are performed in optimal maintenance strategy is employed and the total reduction
week 8 for both scenarios as the boiler is switched off throughout in discomfort cost achieved by the DP that considers the discomfort
the whole week. costs. The resulting total reduction in discomfort is shown in Fig. 7.

Fig. 7. The resulting reduction in total discomfort costs for different initial h values. When 0:11 < h < 0:14, we achieve higher discomfort savings are achieved due to extra discomfort
penalty associated with not replacing the boiler. For h < 0:09 and 0:76 < h, the same control actions are performed in both scenarios resulting in no discomfort savings.
314 N. Cauchi et al. / Energy 138 (2017) 306e315

Fig. 8. The optimal maintenance strategies obtained when employing (a) Algorithm 1 and (b) Algorithm 2. In both figures, the control actions are colour coded using - Dark Blue:
Replace boiler; Light Blue: Clean Boiler; White: No Action.

It can be seen that the total reduction in discomfort is a function of 104 (corresponds to initial value found in Spanish school data set).
the initial efficiency level, h. For 0:11 < h < 0:14, we achieve higher We use a time horizon of N ¼ 5 years and do not include the
discomfort savings, due to extra discomfort penalty associated with discomfort and the control action equivalent to a boiler replace-
not replacing the boiler. We obtain no discomfort savings for the ment in order to be able to make a direct comparison with standard
ranges h < 0:09 and 0:76 < h, as the same control actions are per- practices where replacement actions are only performed after a
formed in both scenarios (cf. Fig. 5). An overall average savings of longer time horizon (typically N ¼ 15 years). A condition-based
15.76% is also deduced. function is extracted from the generated optimal policies

Experiment 2: On-load cleaning (instantaneous) vs Off-load
 Clean 0:76  h½n;
u n ¼ (23)
cleaning (1 day delay). A comparison is made between the No Action Otherwise:
two algorithms (Algorithm 2 and 1). Fig. 8(a) and (b) depict
zoomed up versions of the resulting LUT for 2015e2016 heating here, h½n ¼ 0:76 is chosen as it is the maximum value beyond
season when using Algorithm 1 and 2 respectively. From which no maintenance actions are performed as deduced from the
Fig. 8(a), we deduce that when employing on-load cleaning, all resulting LUTs (Fig. 5).
cleaning actions are carried out during weekdays. An extra We tabulate the results obtained from this analysis in Table 2.
cleaning action is introduced for h  0:14 and is performed on From the outcomes, we confirm that the most effective energy
Saturdays. In this case, the cost of discomfort has minimal effect policy is obtained when the DP algorithm is employed, with com-
on when to perform maintenance actions. On the other hand, parable energy savings between the on-load and off-load cleaning
from Fig. 8(b) it is clear that the cost of discomfort directly limits algorithms. The off-load strategy (introduces an additional cost
when maintenance actions are performed. All cleaning actions during the off-load period which is only due to discomfort and the
are performed during weekends. This is desirable as performing maintenance action being performed as the boiler is not opera-
cleaning during weekdays would result in discomfort as the tional) compared to the on-load strategy requires that an additional
heating zone is in use and would need to be taken off-load. maintenance action is performed. This additional maintenance
strategy keeps the boiler operating at higher efficiency levels and
since the cost of maintenance and discomfort are much lower then
the cost of degraded boiler operation, there is a slight reduction in
5.1. Energy savings
the overall costs due to operation (C op ðx½nÞ) which leads to more
energy savings being gained by the off-load strategy.
We perform a comparison between the developed algorithm
and standard cleaning policies carried out in the industrial practice,
in order to estimate the total energy savings gained by the pro- 6. Conclusions
posed algorithms, using a set of 103 simulations. In all instances, we
obtain the initial degradation level from sampling a uniform dis- The paper presents a methodology for quantifying discomfort
tribution between the range ½0:03; 0:8 and Qout is fixed to 1.183 due to boiler degradation, and a corresponding optimisation

Table 2
Energy savings comparison.

Strategy Total Costs Maintenance Actions Savings compared to Off-load Cleaning DP

Off-load cleaning DP 2.175  106 16 0%

On-load cleaning DP 2.171  106 15 0.15%
Condition based 2.191  106 14 0.75%
Weekly cleaning during heating season 3.105  106 70 42.79%
Six Month cleaning 2.455  106 9 12.89%
Annual cleaning 3.474  106 5 59.73%
 N. Cauchi et al. / Energy 138 (2017) 306e315 315

algorithm based on dynamic programming algorithm that sched- Technol Environ Policy 2015;17(7):1897e906.
[9] Zhao H, Magoule s F. A review on the prediction of building energy con-
ules a strategy for optimal boiler maintenance. The work encom-
sumption. Renew Sustain Energy Rev 2012;16(6):3586e92.
passes dynamics at two different time scales: (i) daily dynamics for [10] Berka J, Macek K. Effective maintenance of stochastic systems via dynamic
the prediction of discomfort, and (ii) long-term dynamics that are programming, In: Proc. technical computing Prague 2011, (Prague: Czech
key for maintenance strategies. The resulting strategy allows for Republic).
[11] Macek K, Endel P, Cauchi N, Abate A. Long-term predictive maintenance: a
different actions based on the cleaning technique employed by study of optimal cleaning of biomass boilers. Energy Build 2017;150:111e7.
contractors (partial cleaning vs replacement). This results in more [12] Valancius R, Jurelionis A, Dorosevas V. Method for cost-benefit analysis of
realistic predictive maintenance strategies, with effective cost and improved indoor climate conditions and reduced energy consumption in of-
fice buildings. Energies 2013;6(9):4591e606.
energy savings. The presented framework can be further enhanced [13] Tehrani MM, Beauregard Y, Rioux M, Kenne JP, Ouellet R. A predictive pref-
with a dynamic discomfort monetisation function, and by incor- erence model for maintenance of a heating ventilating and air conditioning
porating predictive models representing solar heat gain and system. IFAC-PapersOnLine 2015;48(3):130e5.
[14] Nikhil K, Kurian MC. Selection of optimum maintenance approach based on
building occupancy. The dynamic programming algorithm may also analytic hierarchy process (AHP). Int J Eng Dev Res 2016;4:454e9.
be enriched by incorporating time delays in the state evolution €
[15] Maula H, Hongisto V, Ostman L, Haapakangas A, Koskela H, Hyo € n€
a J. The effect
models to represent different time delays introduced by different of slightly warm temperature on work performance and comfort in open-plan
offices-a laboratory study. Indoor air 2016;26(2):286e97.
cleaning and replacement maintenance methods. [16] Cui W, Cao G, Park JH, Ouyang Q, Zhu Y. Influence of indoor air temperature on
human thermal comfort, motivation and performance. Build Environ 2013;68:
Acknowledgements 114e22.
[17] Aswani A, Master N, Taneja J, Krioukov A, Culler D, Tomlin C. Quantitative
methods for comparing different hvac control schemes, In: Performance
This work has been funded by the European Commission in the evaluation methodologies and tools (VALUETOOLS), 2012 6th International
Seventh Framework Programme project AMBI (Grant Agreement Conference on, IEEE, 2012, pp. 326e332.
no. 324432). This work is in part supported by the Alan Turing [18] Maasoumy M, Sangiovanni-Vincentelli A. Total and peak energy consumption
minimization of building HVAC systems using model predictive control. IEEE
Institute, London, UK and Malta's ENDEAVOUR Scholarships Des Test Comput 2012;29(4):26e35.
Scheme. [19] Bacher P, Madsen H. Identifying suitable models for the heat dynamics of
buildings. Energy Build 2011;43(7):1511e22.
[20] Baldi S, Yuan S, Endel P, Holub O. Dual estimation: constructing building
References energy models from data sampled at low rate. Appl Energy 2016;169:81e92.
[21] Bertsekas DP. Dynamic programming and optimal control, vol. 1. MA: Athena
[1] Bo€hringer C, Lo
€ schel A, Moslener U, Rutherford TF. EU climate policy up to Scientific Belmont; 1995.
2020: an economic impact assessment. Energy Econ 2009;31:S295e305. [22] Åkugor B, Deur J. Dynamic programming-based optimisation of charging an
[2] Bentsen NS, Jack MW, Felby C, Thorsen BJ. Allocation of biomass resources for electric vehicle fleet system represented by an aggregate battery model. En-
minimising energy system greenhouse gas emissions. Energy 2014;69: ergy 2015;92(Part 3):456e65 [sustainable Development of Energy, Water and
506e15. Environment Systems].
[3] Tonini D, Vadenbo C, Astrup TF. Priority of domestic biomass resources for [23] Schmidt M, Moreno MV, Schülke A, Macek K, Marík K, Pastor AG, et al.
energy: importance of national environmental targets in a climate perspec- Optimizing legacy building operation: the evolution into data-driven pre-
tive. Energy 2017;124:295e309. dictive cyber-physical systems. Energy Build 2017;148:257e79.
[4] Qureshi BA, Zubair SM. Predicting the impact of heat exchanger fouling in [24] Shumway RH, Stoffer DS. Time series analysis and its applications: with R
power systems. Energy 2016;107:595e602. examples. Springer Science & Business Media; 2010.
[5] Wang L, Chu J, Wu J. Selection of optimum maintenance strategies based on a [25] Morley T. What you need to know about biomass boiler servicing. Green-
fuzzy analytic hierarchy process. Int J Prod Econ 2007;107(1):151e63. power technology online resource, [Accessed December 2016].
[6] Babishin V, Taghipour S. Optimal maintenance policy for multicomponent [26] Valero A, Cortes C. Ash fouling in coal-fired utility boilers. Monitoring and
systems with periodic and opportunistic inspections and preventive re- optimization of on-load cleaning. Prog energy Combust Sci 1996;22(2):
placements. Appl Math Model 2016;40(2324):10480e505. 189e200.
[7] Zhou X, Xi L, Lee J. Reliability-centered predictive maintenance scheduling for [27] Bianchini A, Cento F, Golfera L, Pellegrini M, Saccani C. Performance analysis of
a continuously monitored system subject to degradation. Reliab Eng Syst Saf different scrubber systems for removal of particulate emissions from a small
2007;92(4):530e4. size biomass boiler. Biomass Bioenergy 2016;92:31e9.
[8] Pattanayak L, Ayyagari SPK, Sahu JN. Optimization of sootblowing frequency
to improve boiler performance and reduce combustion pollution. Clean