Roberto García-Manzano, Alfonso P. Ramallo-González, Ramón Sanchez-Iborra Universidad de Murcia, Murcia, Spain
Roberto García-Manzano, Alfonso P. Ramallo-González, Ramón Sanchez-Iborra Universidad de Murcia, Murcia, Spain
Roberto García-Manzano, Alfonso P. Ramallo-González, Ramón Sanchez-Iborra Universidad de Murcia, Murcia, Spain
Energy Representation Tool for Air Conditioning that Enhance Energy Savings and Improve
Energy Literacy on Users
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2707
Rome, Italy, Sept. 2-4, 2019 https://doi.org/10.26868/25222708.2019.211006
________________________________________________________________________________________________
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2708
Rome, Italy, Sept. 2-4, 2019
________________________________________________________________________________________________
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2709
Rome, Italy, Sept. 2-4, 2019
________________________________________________________________________________________________
• System usability test: is assessed using a standardized • It is verified, i.e., it can determine effectively between
test widely used in the industry as it is the System systems usable and not usable. The score of each of the
Usability Scale or its. It consists on a quick questionnaire answers is converted to a new number: added together and
that allows you to measure the usability of a system, even then multiplied by 2.5 to convert the original scale (0-40)
if the number of participants is not too high. Designed by to the end (0-100). Although the scale reaches 100, they
Brooke (1986) and used as a standard in the industry, it are not percentages.
has been referenced in more than 1300 articles and Based on studies (Sauro, 2011), a score above 68 would
publications (Usability.gov, 2013). be considered to be above the average, and any score
• Evaluation of displays: includes statements about the below 68 would be below the average.
impact of each of the displays, the levels of urgency to fix Bangor, Kortum and Miller (2011) in his study
a level of high consumption and comparisons between “Determining individual what your scores mean: Adding
them. an adjective rating scale", established a scale which
The test was performed in a test room that consists of a accounted for scores and their corresponding adjectives or
real room of 35m2 located in the city of Murcia. The test names. They also established the score by which separate
was conducted during the months of July and August. the usability of "good" systems of the "not so good". This
In the test room is available a poster with a replica of the cohort is located in the score of 71.4. Any score above
conditioned air in the different faculties of the University 71.4 is considered a system whose usability is above the
of Murcia. The machine has a QR code with an identifier average, and any score below 71.4 corresponds to a
that will be later used to collect consumption. The reason system whose usability is below the average
for this design is to extend this functionality to more than Results
400 air conditioning machines available in the University
To evaluate the results of the experimentation, a series of
of Murcia.
statistical evaluations were done and are explained in this
In terms of communication of the components of the section.
experiment, orders are sent to an API (Application
Desirability
Programming Interface) testing which changes in the
level of consumption have occurred, being automatically The analysis of the results begins with the intention of use
reflected in the application. In addition, to avoid losing of users depending on their demographic profile and other
the environmental component, also an air conditioning questions. Figure 4 shows that women who underwent the
machine is present in the room, so real temperature assessment have an increased willingness to use the
perceptions are perceiving by the participant. The system than men, although the Mann-Whitney test
interviews are conducted on an individual basis, with the confirms there is no statistical significance with a p-value
room at an ambient temperature of 23 ° C. = 0.581.
The session with the participant starts with the user by 5
filling in the first two sections of the questionnaire:
demographic profile and attitudes to the use of energy. 4 4.182
Once users are ready to begin, they are given a tablet with 3.643
3
the application installed and opened. The only
instructions given to users is that to start the test (not 2
guided) should focus on the poster containing the replica
of the air-conditioning machine. 1
The first display that the user faces is the model based on 0
particles, showing each of the different levels of Men Women
consumption. Then the user will use models based on text, Figure 5: Desirability depending on gender.
surface and sound. When the user has finished the test,
Figure 5 shows as, in general, there is a high willingness
they can fill the usability test and evaluation of each of the
to independently use an augmented reality system despite
display models.
the monthly net income of the family. The willingness to
System usability scale (SUS) use is inversely proportional to the level of the family
This test provides a reliable tool to measure usability. It income, i.e. those families having less income are which
consists of a ten-item questionnaire with five options of require more information to adjust their consumption. It
response in a Likert scale; from "completely agree" to is true that there is a decline in the willingness to use in
"totally in disagreement". the range from 1,500 to €1,999 monthly income, this may
As already said in the presentation of the experiment, the be due to a low of subjects that meet this criterion since
test has become an industry standard. Its benefits are: the sample is not very large. Making the Kruskall-Wallis
test you get a p-value of 0.662 indicating that a
• It is a scale that is easy to use for the participants. relationship between income level and willingness to use
• It can be used with a small population, obtaining reliable in not significant.
results.
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2710
Rome, Italy, Sept. 2-4, 2019
________________________________________________________________________________________________
5 5
4. 5 4 .5 4 .6
4 4.286 4
4 4 4
3 .6
3 3.25 3
3
2 2
1 1
0 0
1.000 € - 1.499 € High Sch. Degree
1.500 € - 1.999 € Prof. Educ. Master Degree
2.000 € - 2.499 €
None Advanced Degree
> 2.500 €
Figure 7: Desirability depending on academic level.
Figure 6: Desirability depending on income.
The willingness of use the tool is also very high regardless
of the academic level. (Figure 6). Although this time the
Kruskall-Wallis test confirms that a correlation exists
between the level of studies and willingness to use the
system (p-value = 0.0215). At the same time, at first
glance there is no significant difference in the willingness
to use depending on who pays the bills, the Mann-
Whitney test yields a p-value of 0.145 by what we can
confirm that this is so.
Figure 7 shows a graph comparing willingness to use with
different questions of the section "attitude towards the use
of energy", it makes reference to "save energy is
something I do frequently", ACT_K is "I would consider
better person if I wore less energy" and ACT_N is "If you
have more information about my energy consumption
would make a more responsible use”.
Figure 8: Desirability depending on attitudes towards
It can be seen that those who said that they saved energy energy.
frequently with one value greater or equal to 3 on 5
present a great willingness to use the system, those who Usability Test
claim a 2 are not interested in using the system, the The evaluated system has an average of 89.6 rating so it
Kruskall-Wallis test confirms that both claims are related can be considered "excellent" on the proposed scale, and
to a 95% confidence level (p-value = 0.002). In the same very close to the category "best imaginable". Figure 8
way as in the previous question, it seems that users who shows the distribution of scores. The majority of scores in
indicated that feel better saving energy giving a score the test have fallen into the bucket, 90-99, which
would be equal to or greater than 3 out of 5 show great demonstrates that the designed augmented reality system
willingness to use the system, but using the above test excels in usability with a fairly high note.
yields a p-value of 0.135 by what this assumption should Results depending on the visualisation model
be discarded.
This section will analyse the results of the different
For the statement that indicates that the user would make display models proposed and evaluated by users. In
a better use of energy if he has more information, also the Figure 9 the results of the first item of the questionnaire
willingness to use the system is correlated. For higher related to models are shown. The statement was related to
scores or equal to 4 out of 5 the test confirms this with a the speed with which the level of consumption was
p-value of 0.006. Curiously, none of the respondents understood when it was high. The text-based model is
marked with 5 out of 5 to the assertion "saving energy is above the rest followed by the model of particles and
something I do frequently". From this first part of the surface. Sound-based model is the worst in this aspect.
analysis, it can be inferred that the will to use the system
by users, in general terms, is quite high.
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2711
Rome, Italy, Sept. 2-4, 2019
________________________________________________________________________________________________
3
3 3 . 04
2 . 64
2 2 . 36
Figure 9: Scores on the SUS. Figure 11: Answers to the statement: I have felt satisfied
when I see a low level.
5
4 . 76 5 4 . 76 12
4 Soun
Surf.
d
3 . 76 1 Text
3 1
0 Partic.
3
2 8
1 6 6
0 2
4 1
Particles Text Surface Sound
3 3 3 3
Figure 10: Answers to the statement: I have understood
fast the consumption when observing a high level.
In Figure 10 the results of the second item in the test 0
20- 30- 40- 50- >65
evaluation of the models is shown. In this case, the Age
question is related to the satisfaction that it is felt when
the consumption is shown to be low. A rather moderate Figure 12: Favourite model depending on age group.
satisfaction is seen when showing low consumption level display as the model based on particles with a more
in general. The highest score was the surface model; it "annoying" model as the sounds to correct high levels of
might be because in this model the display occupies a consumption.
larger percentage of the screen with green colour. As Conclusion
shown in Figure 10 with none of the models of display
users have felt a special urgency to act to correct a In the literature consulted about the topic nothing similar
moderate energy consumption, once again, the model to this study was found, so it is likely that this paper opens
which slightly stands out above the rest is the surface- a new avenue of research. It was found that joining
based model. In Figure 10 is observed clearly as the model augmented reality with a lack on energy literacy allows
is the one that encourages the user to correct a high the user to understand in real time and on demand, the
consumption. energy consumption of certain devices. There are also
other studies that aim to promote energy literacy of users
In Figure 11 t is shown a breakdown of the visualization and efficiency of devices emerging in the industry, so the
method preferred for the age range. Continuing with the interest is justified.
affirmation of the previous paragraph, it comes with no
surprise that users that fit in the lower age ranges do prefer With the tests carried out here it was found that, among
more Visual models and those who are in the upper age users who had the test, there is a great willingness to use
ranges, prefer more direct models as based on text. this type of technology to visualize and thus controlling
energy use in the home. It has been shown that, among the
As a conclusion to the sections of results, it is correct to surveyed population, both men and women have a very
assert that augmented reality system have achieved a high willingness to use the augmented reality system,
score well above the average in terms of usability despite moreover, the results confirm that an inverse correlation
display model chosen, but if any of them outstands by the exists between the monthly income of the family unit and
majority of users this has been the model based on the willingness to use.
particles. It is also true that the model that most urgently
moves users to correct a situation of high energy Statistical tests have also confirmed that the willingness
consumption has been the model based on sounds, so to use the system is directly linked to the educational level
perhaps the model idea is one that combines an attractive of the users, the highest interests are concentrated at the
higher educational levels. In addition, there is no
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2712
Rome, Italy, Sept. 2-4, 2019
________________________________________________________________________________________________
difference at will when determining who is responsible Gardner, G.T., Stern, P.C. (2002) Environmental
for paying the bills. The participants who claimed that problems and human behavior (2nd ed.), Pearson
they were saving energy were more willing to use the Custom Publishing, Boston, MA
augmented reality system. The same applies to persons Jervey, B. (2012). Why Energy Literacy Maners & What
who claim that they feel better when saving energy. Also
to Do About It. [online] TreeHugger. Disponible en:
a high intention to use the system is seen in those who
hnps://www.treehugger.com/energy-policy/ why-
scored above 4 out of 5 to the statement which indicates
energy-literacy-maners-what-to-do-about-it.html.
that the user would make better use of energy if their have
more information about their consumption. Pahl, S., Goodhew, J., Boomsma, C., Sheppard, S. (2016).
According to the results obtained in the analysis of the The Role of Energy.
willingness to use the system, it is possible to conclude Pellicia, L, Klimant, P., Schumann, M., Pürzel, F.,
that, in general terms, it is quite high, and very transversal. Winstock, V., Putz, M. (2015). Energy visualization
In terms of the evaluation of the usability of the developed techniques for machine tools in virtual reality.
system, even considering that it is still at a very early Pérez-Lombard, L., Ortiz, J., Pout, C. (2008). A review
stage, it has shown a usability far above expectations, on buildings energy consumption information, Energy
obtaining a score of 89.6 points on the test and an and Buildings, 40(3),2008,394-398.
"excellent" rating, very close to the "best imaginable".
Ramallo-González, A.P. et al. Deliverable 5.3:
Within the display models proposed for evaluation, users
Performance Evaluation and Lessons Learnt, Project
have chosen the model based on particles as the one that
ENTROPY . http://entropy-project.eu/wp-
more impact generates on them. They were not especially
motivated with specific proposed models to correct a high content/uploads/2018/12/D5.3.pdf.
consumption level, but if they claim that they felt a special Rashed-Ali, H., Quarles, J., Fies, C. and Sanciuc, L.
urgency to correct a high consumption level it was when (2014). Use of augmentedreality in teaching energy
the sound-based model is presented to them. A priori, it efficiency: prototype development and testing.
seems that models based on particles with sound alerts Sauro, J. (2011). Measuring U: Measuring Usability with
when consumption is high, seems to be the best option. the System Usability Scale (SUS). [online]
In conclusion and observing all the results, it can be said Measuringu.com. Disponible en:
that it is possible to move forward on energy literacy using hnps://measuringu.com/sus/ [15 de August 2018].
such a system. Users will have a usable system, economic,
Usability.gov (2013). System Usability Scale (SUS) |
simple and direct that informs in real time of their habits
Usability.gov. [online] Usability.gov.
of consumption, in such a way that they can correct them.
hnps://www.usability.gov/how-to-and-tools/
Acknowledgement methods/system-usability-scale.html
This work has been sponsored by the Spanish Ministry of
Economy and Competitiveness through PERSEIDES (ref.
TIN2017-86885-R) and CHIST-ERA (ref. PCIN-2016-
010) projects and by MINECO grant BES-2015-071956.
Ramallo-González thanks the program Saavedra Fajardo
(grant number220035/SF/16) funded by Consejería de
Educación y Universidades of CARM,Fundación Séneca-
Agencia de Ciencia y Tecnología de la Región de Murcia.
References
Bangor, A. (2009). Determining What Individual SUS
Scores Mean: Adding an Adjective Rating Scale.
[online] Semanticscholar.org.
hnps://www.semanticscholar.org/paper/Determining-
What-Individual-SUS-Scores-Mean-A-an-Bangor.
Barnum, C. (2011). Usability testing essentials.
Burlington, MA: Morgan Kaufmann Publishers.
European Building Performance Directive (recast 2011).
European Commission.
Fiesta-audit.eu. (2018). Métodos y objetivos. [online]
http:// www.gesta-audit.eu/es/el-proyecto/ [18 August
2018].
________________________________________________________________________________________________
Proceedings of the 16th IBPSA Conference 2713
Rome, Italy, Sept. 2-4, 2019