Oliva - A Maturity Model For Enterprise Risk Management

Int. J.
Production Economics 173 (2016) 66–79
Contents lists available at ScienceDirect
Int. J. Production Economics

journal homepage: www.elsevier.com/locate/ijpe
A maturity model for enterprise risk management

Fábio Lotti Oliva n
University of São Paulo Business Administration, Avenida Professor Luciano Gualberto, 908, Office C27, 05508-010 Sao Paulo, SP, Brazil
art ic l e i nf o a b s t r a c t
Article history: The enterprise risk management has been a recurring theme on the agenda of organizations. The
Received 7 June 2015 competition is increasingly established among the supply chains of organizations. In this sense, it is
Accepted 9 December 2015 appropriate that this research aims to analyze the enterprise risk management in the supply chain of
Available online 19 December 2015
Brazilian companies. As a conceptual framework, it has been adopted three theoretical pillars, namely,
Keywords: New Institutional Economics, Supply Chain and Enterprise Risk Management. The research was divided
Enterprise risk analysis into three stages: interviews with experts on the subject, survey with managers of large Brazilian
Enterprise risk management companies, and validation of the proposals from the analysis of the results with the same experts. The
Supply chain analysis of results was established mainly by the use of multivariate statistical techniques such as cor-
Enterprise risk management maturity
respondence analysis, factor analysis, cluster analysis and multinomial logistic regression. As a result of
Supply chain risk management maturity
the study, it was presented a proposal of model for an enterprise risk analysis, as well as a proposal of
model
model for analysis of the level of maturity in enterprise risk management in the supply chain of large
Brazilian companies.
& 2015 Elsevier B.V. All rights reserved.
1. Introduction techniques proliferate aiming to incorporate the risk assessment

element in the organizational culture. Risk management must
In the corporate world, the attention to events that may com- permeate the organizational processes (COSO, 2004; ISO 31000,
promise the completion of corporate actions largely occupies the 2009; RIMS, 2006; Ernest and Young, 2012).
agenda of executives. The fierce competition, the bargaining According to the executives, studies show that the main risks
power of clients, the dependence on suppliers, the constant concentrate on the risks present in the relationships with agents in
demand for innovation, changes in the regulatory environment, the business environment. According to the international survey
the new expectations of society, make the organizational envir- conducted by the AON, a global consulting focused on enterprise
onment more complex. In contrast, companies seek strategies to risk management, present in more than 120 countries, in 2009,
raise their internal complexity in order to combat the external with over 320 executives, showed that the most feared risk by
complexity that increases gradually. In the context of this research, major corporations is the risk of damage to image or to reputation.
it is worth noting the increasingly intense adoption of techniques It is known that reputation is a value built over time with several
relating to enterprise risk management as a way to ensure the important agents within the scope of operation of the organization
achievement of organizational goals (Mintzberg et al., 2005; (AON, 2012).
Espejo, 1993; Espejo et al., 1996; Damodaran, 2008; Louisot, 2010). Another important point of consideration lies in the under-
Given the complexity of the organizational environment, it standing of risk management practices in large organizations. Risk
appears that in addition to the dispute between industry agents, management plays a significant role that only transcends risk
clients, suppliers, competitors and new entrants, it is clear that the analysis (Thun and Hoenig, 2011; Blome and Schoenherr, 2011;
dispute is established as a competition between supply chains, Mikes, 2009; Miller and Lessard, 2001). The communication,
that is, the aggregation of agents with common goals compete for treatment and monitoring of enterprise risks are some of the tasks
resources, markets and professionals with other supply chains inherent to risk management. There has been an effort from the
(Chopra and Sodhi, 2004; Manuj and Mentzer, 2008). organizations in conceiving a common planning to guide risk
The enterprise risk analysis as an essential part of enterprise management, however, the way how these models are applied and
risk management has been intensively developed by universities, the particularity of each organizational context may reveal unique
consulting firms, companies and other organizations. Models and and distinctive practices that are not described in general manuals.
Nevertheless, organizations seek a continuous evolution of their
n
Tel.: þ 55 11 3091 5982. processes, which can also be found in risk management. Thus,
E-mail address: fabiousp@usp.br knowing the best practices and identifying at what stage the
http://dx.doi.org/10.1016/j.ijpe.2015.12.007
0925-5273/& 2015 Elsevier B.V. All rights reserved.
F.L. Oliva / Int. J. Production Economics 173 (2016) 66–79 67
organization is with respect to enterprise risk management have 2. Theoretical framework

been a desire of companies, consulting firms and other organiza-
tions concerned with the risks of their environment of operation The theoretical framework adopted in this research was
(COSO, 2004; ISO 31000, 2009, RIMS, 2006). established on three pillars. The conceptual basis must support the
Therefore, it is important to incorporate in the theoretical development of research tools and the development of the ana-
models the analysis of the relationships with agents in the orga- lysis of results. The administrative theory consists of concepts,
nizational environment. The enterprise risk analysis should con- classifications, relationships and models. Thus, we sought to
sider a scope beyond the boundaries of the organization, that is, develop a conceptual framework based on the New International
the enterprise risk analysis of the organization should include the Economics, Enterprise Risk Management and Supply chain. Jointly,
enterprise risk analysis of its supply chain. it is understood that the three conceptual pillars have contributed
Some authors argue that there are important gaps in supply to appease the conceptual completeness required for the devel-
chain risk management (SCRM) knowledge, for example, clear opment of this research.
definition and delimitation about SCRM (Juttner, 2005; Neiger
et al., 2009; Zsidisin, 2003; Manuj et al., 2014; Aqlan and Lam, 2.1. New institutional economics
2015a). Considering supply chain risk management as an impor-
The Institutional Economics is dedicated to the study of insti-
tant theme emerging from a growing appreciation for supply chain
tutions, organizations and their interactions in the corporate
risk by practitioners and by researchers, a distinctive research
environment. With a systemic approach, it comprises social, cul-
presents three gaps pertinent to future investigation in SCRM
tural, behavioral and historical aspects to explain the economic
management: no clear consensus on the definition of SCRM; lack
relations in society. The classical authors of this section of the
of commensurate research on response to supply chain risk inci-
economy are: Veblen, Commons and Mitchell. As a point in com-
dents; and a shortage of empirical research in the area of SCRM
mon, the authors present studies that focus on institutions as
(Sodhi et al., 2012).
elements that order the economic relations between individuals
Considering supply chain risk management in large company,
and society. A contrast to the Classical Economics, which addresses
the critical problems for firms nowadays consist of analysis, eva- the balances of long-term markets driven by rationality and profit
luation and monitoring of risk in their processes. Poor design and maximization of their agents. Considering the Institutional Eco-
manufacturing decisions, losses in inventory, quality problems in nomics efficient to describe the institutions, but little adherent to
the outsourcing process, cultural management problems are some explain the economic relations between the agents, the New
consequence due to lack of a better supply chain risk management Institutional Economics emerges with the main texts consisting of
(Manuj et al., 2014; Hult et al., 2010; Lee, 2004; Norrman and the academic exponents: Coase, Williamson and North. In order to
Jansson, 2004). In mid of 2012, Deloitte surveyed 600 executives at increment and adapt to the new demands of reality, instead of
manufacturing and retail companies, located in North America, rejecting the precepts of the neoclassical theory as did the former
Europe and China, to understand their perceptions of their risks. institutionalists, the new institutionalists have brought new con-
The main results are: (1) 45% understand that their supply chain ceptual elements to the discussion. Coase presents concepts rela-
risk management programs were somewhat effective or not ted to transaction costs. Williamson discusses the issue concerning
effective, (2) 33% used risk management to provide a proactive and the specificity of assets and the opportunism of agents. North
strategic management of their supply chains (Deloitte, 2013). reveals the important concept that institutions order the relations
Based on the problem described above, the purpose of this study between organizations that seek to obtain marginal gains resulting
is: the analysis of enterprise risk management in the supply chain of from variations in relative prices (Nee, 2003; Barzel, 2002;
large Brazilian companies. Therefore, we intend to identify the main Ménard, 1995, 1997, 2000).
enterprise risks, identify the importance of each enterprise risk in the Thus, the conceptual triad proposed by Coase, Williamson and
relationship between company and agent of its supply chain, identify North are revealing for understanding the current economy. Coase
the practices in enterprise risk management. Therefore, as primary (1937) proposes that the costs of the firm's activity transcend the
contributions this study proposes an enterprise risk analysis model in production costs, going beyond the acquisition costs of inputs,
the supply chain and a maturity level assessment model with respect processing and sales. The costs for coordinating agents of the
to enterprise risk management. economic system to carry out the primary activities of the orga-
The justification for conducting an academic research resides in nization become important in a more complex environment. It is
identifying the importance, contribution, feasibility and originality therefore considered that transaction costs consist of costs of
of the study which reveal its quality (Cooper and Schindler, 2001; information gathering, negotiation and business establishment. In
Denzin, 1970; Hair et al., 1998). In this research, the importance is order to qualify these transactions, Williamson (1985) propose
two important concepts to the NIE. The opportunism of agents,
established by the need that businesses have to identify and
which reveals the importance of business coordination and that
manage its enterprise risks, which go beyond the internal risks
the sizing of its costs can no longer be disregarded, in such a way
and involve other actors of its value chain, considering the most
that they increasingly appear as apparent costs. North (1990)
competitive business environment (Manuj et al., 2014; Hult et al.,
reveals in his own and intense way the importance of institutions
2010; Lee, 2004; Norrman and Jansson, 2004; Deloitte, 2013).
as entities that order the relations between agents in the economic
Considering the lack of empirical research on supply chain risk
environment. Institutions are responsible for the institutional
management (Sodhi et al., 2012; Juttner, 2005;.Manuj et al, 2014),
framework comprised by formal and informal rules that govern
the contribution and originality of this research are declared by
the behavior of the economic agents, usually grouped in organi-
main objetives which propose an enterprise risk analysis model in zations due to common goals.
the supply chain and a maturity level assessment model with
respect to enterprise risk management. The feasibility study 2.2. Enterprise risks
should be ensured by adopted methodological procedures. Trian-
gulation, reliability in quantitative research, the validity of ques- Enterprises transact with customers, suppliers, government
tionnaires and investigations are present in this study and are and other organizations. The process of purchasing raw materials
discussed in the methodology (Cooper and Schindler, 2001). and other inputs, production and distribution to markets follows a
68 F.L. Oliva / Int. J. Production Economics 173 (2016) 66–79
logical sequence within standards set by society, as well as some other entities and other proposals began to emerge to better
non-rational actions of some agents. The environment consists of manage enterprise risks (FERMA, 2003; IFAC, 2001; ISO 31000,
all relevant forces beyond organizational boundaries organizations 2009; IBGC, 2007). It is worth mentioning the publication of the
are subject to legal changes, changes in social behavior, economic international standard ISO 31000 as a second wave of proposals to
changes, which are often patients in this process, ie, little can standardize the practices of enterprise risk management. The
influence. For example, companies and their value chain can have proposal of ISO 31000 (2009) comprises the presentation of a risk
their economic activities affected by a tax change, an exchange management architecture consisting of principles, structure and
rate change or a legal challenge filed by the government. On the process, in such a way that the standard provides a conceptual
other hand, companies can react more vigorously forward manner model and a methodology to be applied in corporations. These
to changes in the relationship with its customers, the competitive principles, structures, processes and models promote a decreasing
design or in the relationship with its suppliers. For example, financial and operating losses, reducing inventory losses, improves
positive actions or negative actions of employees of a company or supplier relations and reducing external capital dependences.
other agent in the value chain can affect the perception of the Futhermore, whereas practices of management, it is understood
company's image for some time and may even in the most acute that the good practices of Enterprise Risk Management imply in
cases change the perception of the company's reputation. Many creating synergies between risk management activities and
can even act more intensely in their environment and design the increasing risk awareness which facilitates better operational,
future with key stakeholders. In this way, environmental forces tactical and strategic decision-making (Beasley et al., 2005;
affect the objectives of organições thus they are sources of busi- McShane et al., 2010; Hoyt and Liebenberg, 2011; Tang, 2006a;
ness risks (Bateman, Snell, 2012; Hitt et al., 2014; AON, 2012; Paape and Speklé, 2012).
Gaultier-Gaillard, Louisot and Rayner, 2009; Kim and Mauborgne, A global survey conducted by Ernst & Young with over 500
2004; COSO, 2004). interviews revealed a close relationship between the maturity
Considering the risks to which organizations are subject, the level in risk management and the financial performance of orga-
literature handles the subject as corporate risks. In this context, nizations. A prominent result among those presented by the sur-
enterprise risk is considered to be a measure of uncertainty and vey showed that, by considering the ranking of companies with a
includes factors that may facilitate or prevent the achievement of higher maturity level in risk management, it can be seen that 20%
the organizational goals (IFAC, 2001). FERMA (2003), Federation of of best rated companies report a financial performance (EBITDA)
European Risk Management Association, European association three times higher the financial performance of worst rated
that promotes and disseminates the importance of enterprise risk
companies in the ranking (Ernest & Young, 2012).
management in the organizations, defines risk as a combination of
Considering practical and conceptual gaps in Enterprise Risk
the probability of an event and its consequences to the organiza-
Management-ERM, some academics and professionals argue that
tions. Being one of the most important international organizations
some important gaps which demanding researches to promote
that studies and promotes the importance of enterprise risk
more evidences are: (1) the benefits from the implementation of
management, the mission of COSO, Committee of Sponsoring
ERM in business; (2) the technical and cultural barriers in the
Organizations of the Treadway Commission, is to ensure the lea-
implementation of ERM in business, (3) evaluate if the ERM
dership of thought on the subject, developing models and guide-
increase Firm Value (Beasley et al., 2005; McShane et al., 2010;
lines on risk management, internal control and fraud prevention to
Hoyt and Liebenberg, 2011; Sydow and Frenkel, 2013).
improve the organizational performance. For COSO (2004), enter-
Similarly, assuming that all companies face organizational risks
prise risk is represented by the possibility of occurrence of an
of some kind and that they increasingly demand a formal structure
event and that it will adversely affect the achievement of
to deal with them, Hillson (1997) proposes a risk maturity model
organizational goals.
based on four stages: naive, novice, normalized and natural. This
Considering an applied interesting research, Deloitte conducted
model proposes the assessment of the organizations' performance
a study from 1994 to 2003, it identified the risks and classified
according to the requirements, culture, process, experience and
them into four categories: strategic risks, operational risks,
application.
financial risks and external risks. And out of these 100 companies,
The risk maturity model proposed by RIMS (2006) consists of
66% experienced some kind of strategic risk, 61% experienced
five stages, ad hoc, initial, repeatable, managed, leadership. The
some kind of operational risk, 37% experienced some kind of
assessment of the level to which the organization belongs consists
financial risk, and 62% experienced some kind of external risk
(Deloitte, 2005). of the identification of the degree of compliance with seven core
Over the past decade, enterprise risk management has assumed attributes established, namely, ERM-based management, ERM
a greater importance. In this period, there was a series of scandals process management, risk appetite management, root cause dis-
that caused financial losses to clients, suppliers and investors cipline, uncovering risks, performance management, business
directly connected to companies that operated outside the ethical resiliency and sustainability.
precepts established, in an indirect manner, the destabilization of
the institutional pillars led to major losses to the entire society. 2.3. Risks in the supply chain
The search for a renewed corporate governance that would
establish new standards, processes, rules and transparency for all Considering the supply chain of a company as the set of all
agents, brought in its essence a demand for models and meth- activities of value to the agents involved in its business, it can be
odologies for enterprise risk management that could become a seen that competition and cooperation can coexist between agents
common benchmark to the organizations and their stakeholders who share a myriad of converging and diverging interests. The
(COSO, 2004, Louisot, 2010; Hoyt and Liebenberg, 2011). alignment of interests of the agents involved can be considered as
Considering the current practices accepted in business man- the optimal point of business management, but one cannot deny
agement, a first attempt of ordering risk management occurred in the possibility of reversal in this management. Agents have free will,
1994 with the publication of the report Enterprise Risk the interests change, or are not clearly stated, the business envir-
Management-Integrated Framework by the Committee of Spon- onment is dependent on various forces that compose and recom-
soring Organizations of the Treadway Commission (COSO, 2004). pose, and thus opportunism may settle in any relationship and the
As a result, given the success of this conceptual model, several optimal point may be lost (Porter, Millar, 1985; Porter, 2004, 2008;
Shank and Govindarajan, 1993; Tang, 2006a; Christopher and company's image or a company's reputation cannot be assessed
Holweg, 2011; Habermann et al., 2015). without considering the risks to the image or the reputations of its
A proposal with this view was formulated by the IMA (1997), suppliers or clients (Gaultier-Gaillard, Louisot and Rayner, 2009).
the institute proposes that the relationship between strategy and Therefore, it is understood that the enterprise risk analysis
value creation can be established when management primarily consists of the risk analysis of the environment of value, that is,
promotes the creation of value for shareholders, and secondly, the analysis of the company's risks in conjunction with its stake-
when management promotes the satisfaction of all other agents holders in a business environment.
involved in the business, provided that it contributes to the crea-
tion of value for shareholders. In this sense, enterprise risk man-
agement may help maximize the chances of the company to 3. Methodology
achieve its goals, considering that the enterprise risk is repre-
sented by the possibility that an event will occur and adversely 3.1. Methodological aspects
affect the achievement of the organizational goals (COSO, 2004).
The academic literature has dedicated effort on the study of the Considering the research objectives, we elaborated a series of
risks involved in the supply chain, extended supply chain, supply methodological procedures seeking to achieve them. Initially,
chain or production chain, which are often treated as equivalent interviews were conducted with experts on enterprise risk man-
terms. Given the concreteness and urgency of the relationships agement in large companies. Based on these information and the
suppliers, company and clients, it was found that these related conceptual model, we developed the research tool to conduct a
studies to assess the risks involved in the operation and logistics quantitative research with the companies' managers. Then, we
processes to generate value to the client through products and elaborated a data analysis seeking to propose an enterprise risk
services arouse great interest in the academic and business analysis model and propose a model for assessing the maturity
environment. The risk analysis of the supply chain seeks to iden- level of the companies with respect to enterprise risk
tify the main risks that may affect the performance of business management.
processes such as: management of relationships with suppliers, The first stage consisted of a survey with experts who validated
managing of the development of products and services, manu- the expectations, assumptions and options on the overall purpose
facturing management, management of customer demand, and of the study, namely, the analysis of enterprise risk management
management of relationships with clients (Ballou, 2004; Chopra in the supply chain of Brazilian companies. The answers guided
and Sodhi, 2004; Christopher and Peck, 2004; Juttner, Peck and the composition of the questionnaire that was applied to man-
Christopher, 2003; Manuj and Mentzer, 2008; Walters and Rain- agers of large companies.
bird, 2004; Walters, 2007; Grey, Shi, 2005; Heckmann et al., 2015). Then, we developed the survey with managers from large
Based on the principles of the New Institutional Economics, the companies. We sought to identify what main risks are assessed on
firm is understood as an instance beyond the set of procedures for their supply chain and what are the main characteristics in the risk
the procurement of goods and services for its production processes management of these companies. To this end, we developed a
to generate products and services to be offered to the market. The questionnaire with closed questions and multiple choice based on
firm is considered a nexus of contracts, which establishes an order the conceptual model and theoretical elements arising from the
to the internal relationships and external relationships with the conceptual framework developed for the subject.
agents in the business environment, focusing on efficiency The third stage consisted of the development of a proposal to
through the reduction of transaction costs and the reduction of systematize the process of enterprise risk analysis in the supply
production costs (Coase, 1937, 1960; Williamson, 1975, 1991, 1996; chain in which the company participates. In addition, based on
North, 1990). Thus, the New Institutional Economics offer a con- data collected in the primary survey, we proposed a model for
ceptual framework for analyzing the risks involved in the assessing the maturity level with respect to enterprise risk
exchange of resources with the organizational environment, that management.
is, the risks in transactions that consist of risks to negotiate, Aiming to validate the proposals, meetings were held with
establish and monitor the contracts established between the firm experts for the presentation of the proposals and collection of
and agents of the organizational environment. On the institutional criticism and suggestions for improvements, which were incor-
environment, arise patterns of controlling relationships where porated after being evaluated.
trust and power are interrelated such that the risk can best be
mitigated (Bachmann, 2001; Aqlan and Lam, 2015b; Olson and 3.2. Conceptual model
Swenseth, 2014; Tang, 2006b; Tang and Tomlin, 2008; Chen et al.,
2013; Li et al., 2015). To meet the stated objectives and answer the research ques-
The risk analysis of the supply chain as an enterprise risk that is tions, we conducted a literature research on the subjects involved,
only concerned with the risks from the relationships with sup- which supported the development of the conceptual models,
pliers, distributors and clients; and likewise, all risks of the com- research tools and the analysis of results. The literature research
pany analyzed in the context of the business environment, but focused on the texts addressing the New Institutional Economics,
without evaluating these same relationships with the agents in the Supply chain and Enterprise Risk Analysis. In this way, developed a
broader context of the business environment, reveals the need to conceptual model presented below, which proposes a systemic
analyze the enterprise risks beyond the boundary of the firm way to identify enterprise risks in the environment of value of the
(Manuj et al., 2014, Nooraie and Parast, 2015; Li et al., 2015; Hahn company.
and Kuhn, 2012). Thus, the company's risks can only be under- The business environment consists of various organizations
stood when analyzed in a systemic manner, that is, the company's with their relationships that follow a predetermined order. It is in
risks are the internal risks and the external risks consist of risks this environment that the company operates. It is divided into
posed by other agents to the company and the risks to which the macroenvironment and microenvironment. The macroenviron-
relationships of the company and the agents are subject in the ment includes the environmental forces that may influence the
business environment. An emblematic example is the inter- company's destination, but that it can barely influence. We high-
dependence relationship between the agents of the automobile light the main forces: economic, social, political, technological and
industry (Thun and Hoenig, 2011). For example, the risks to a environmental. In the microenvironment are the industry forces
Business Environment
Environment
Economic
Environment of Value
Financial Operational
Clients Competitors
Company Environmental
Political Image Suppliers Distributors Sustainability Social
Government Society
Ethical Innovation Strategic
Technological
Fig. 1. Enterprise risks in the environment of value.

Source: author (2014).
that can influence the company's destination, which in this case agents of the business environment, however, to be considered so,
the company can influence more effectively. The main agents are: they should maintain or create value for the company. The main
clients, competitors, suppliers and distributors. The environmental agents of the environment of value are: clients, suppliers, dis-
forces in the microenvironment are manifested by the bargaining tributors, society, government and competitors (Kim and Mau-
power, opportunism, innovation and the strategy of its agents borgne, 2004; Porter, 2008; Shank and Govindarajan, 1993;
(Coase, 1937, 1960, 2014; Williamson, 1975, 1991, 1996; North, Christopher and Holweg, 2011; Habermann et al., 2015).
1990; Ménard, 1995; Porter, 2004, 2008) (Fig. 1). Considering the intention to present a conceptual model for
Fig. 1 shows the environment of value, which consists of the enterprise risk analysis, it is understood that the enterprise risks
main agents that generate value for the company. In this envir- arise from the internal relationships of the corporation and the
onment, the relationships are established with the intent that both company's relationship with the business environment. However,
parties are contemplated with what the other part can best offer. the analysis of the company's risks alone, that is, dissociated from
Governance levels that maximize the common gain should be its links of value becomes incomplete and somehow misleading,
sought. The agents of the environment of value are the same for example, a company may be in a comfortable condition in
relation to the ethical risks and perchance a strategic supplier may 50 largest banks, ranked by net equity, and the 50 largest insur-
be adopting illegal labor practices. Therefore, we advocate that ance companies, ranked by the net premiums written.
enterprise risks should be analyzed in a systematic manner, con- We defined a random representative sample of companies of
sidering the company itself and its relationships of value against a size (n), based on the population size (N), sampling error (e),
business environment with threats and opportunities. Thus, it is confidence interval (z), and estimated standard deviation, as fol-
understood that the main risks arising from the business envir- lows:
onment are originated from the: economic, political, social, tech- s2 NZ 2 3;342 1:1001:962
nological and environmental events. The risks of the environment n¼ ¼ ¼ 149
d2 ðN 1Þ þ s2 Z 2 0;502 ð1:100 1Þ þ 3:342 1:962
of value, where the company is also an agent that influences and is s ¼ estimated standard deviation ¼ 3:34;
influenced are: financial, operational, image, environmental sus- N ¼ population size ¼ 1:100;
tainability, ethical, innovation and strategic (COSO, 2004; ISO Z ¼ abscissa of the standard associated with confidence level
31000, 2009; Ballou, 2004; Chopra and Sodhi, 2004; Christopher
of 95% ¼ 1:96; d ¼ sampling error ¼ 0:50
and Peck, 2004; Manuj and Mentzer, 2008; Hillson, 1997; RIMS,
2006) (Fig. 1). The original sample consisted of 243 large companies. We
The model presented is supported by the theoretical elements managed to conduct the survey in 181 companies, that is, over 25%
of the new institutional economics, supply chain and enterprise of the companies did not agree to participate or we were unable to
risks. The new institutional economics offered a more relational contact a manager capable of doing so. We applied the ques-
view between the agents, in such a way that it expressed aware- tionnaire, to an employee of the company with academic and
ness of the importance of assessing the risks to which the corpo- professional experience appropriate to answer questions about
rate relations are exposed, and consequently, how corporate enterprise risk management. It is understood that the enterprise
objectives are susceptible to this exposure. The supply chain risk management must permeate the core processes of the orga-
offered the concept of value creation to the stakeholders. The nization, therefore, there was no attempt to limit the survey to
theory of enterprise risks offered concepts, classifications, rela- professionals who are experts on the subject. Among the ques-
tionships and important models to identify the main internal risks tionnaires answered, 168 can be considered valid, that is, 13
of the company, business environment risks and risks from the questionnaires were discarded due to the lack of information or
environment of value. error of completion.
Considering the survey instrument, in the analysis of enterprise
3.3. Data collection risks were presented the main risks to which the companies sur-
veyed are submitted in relation to the business environment of the
The primary survey was divided into three field surveys, agents, based on the conceptual model (Fig. 1). Considering the
namely, survey with expert on enterprise risk, survey with data collection, which requires the respondent to award a grade 0-
enterprise risk managers, and expert validation of the proposed 10 to declare the importance of risk in relation to the agent of its
enterprise risk analysis model and the proposed model to assess business environment, joined by multivariate correspondence
the maturity level in risk management. analysis which most important risks for certain agents. The
The initial survey with experts consisted of interviews with research instrument presents risks divided into risks environ-
professionals working in the market, aiming to obtain the key mental risks and risks of the value chain. Note that the assessed
insights on the issue of risk management in major Brazilian risks were declared for each agent involved in the company's value
companies. The interviews were guided by a script with open system. Regarding the assessment of the maturity level in enter-
questions that sought to explore the following topics: main prise risk management of the companies surveyed, developed 18
enterprise risks, main methods to analyze enterprise risks, and questions about corporate risk management, considering the
best practices in enterprise risk management. We interviewed variables on the maturity stages in risk management based on the
three managers of large corporations, a manager in the automotive literature (Hillson, 1997; RIMS 2006; COSO, 2004; ISO 31000,
industry, a manager in the food industry and a manager in the 2009; IBGC, 2007). As a result, we developed a multivariate ana-
electronic equipment industry, who presented their experiences in lysis thread for this purpose, namely, factor analysis, cluster ana-
enterprise risk management in large companies. In addition, we lysis and multinomial logistic regression.
interviewed two consultants in enterprise risk management, who The third field research consisted of the validation of the results
presented their experiences as consultants in several large achieved in the analysis of data with experts. We sought to con-
companies. duct meetings with the same managers and consultants who
With this information, we elaborated the pre-understanding on collaborated with the pre-understanding of the research problem,
which concepts of the theoretical framework defined previously where we presented the enterprise risk analysis and maturity level
could better represent the risk management process of large Bra- assessment models in enterprise risk management. Furthermore,
zilian companies. In other words, we defined a preliminary pro- the results were presented to two professors who research and
posal for the systematization of risk analysis and the assessment of teach on issues related to enterprise risk management. During the
the maturity level of risk management, which guided the devel- meetings it was possible to collect some improvements that have
opment of the quantitative research instrument (see Fig. 1). been incorporated. In general, the contributions were not struc-
As for the survey with managers, in the second stage of the tural, in such way that it can be considered that the proposals are
research, we carried out extensive research with managers from consistent with the understanding of the respondents.
different areas of the selected organizations. Considering the
population with the 1,100 largest Brazilian companies, according
to the ranking published by EXAME magazine (2011), consisting of 4. Analysis of results
1000 companies from the non-financial sector plus 100 companies
from the financial sector. The non-financial sector companies were 4.1. Enterprise risk analysis
ranked by the sales revenue, an indicator of the company's con-
tribution in terms of products and services offered to society in Enterprise risks may be of different nature and may have dif-
2010. In terms of sales volume, the 1000 companies earned more ferent degrees of importance to each agent. A part of the field
than US$ 1.5 trillion. The financial sector companies consist of the research with managers of large companies was intended to reveal
the importance of each risk for each relationship with the agents In other words, where β 43, it is considered that the catego-
of their environment of value. Based on the knowledge of each rical variables, events and agents, are dependent at a risk of 5%.
manager, we asked them to assign a score from 0 to 10 for each The multivariate technique provides a perceptual map in two
enterprise risk and for each agent arranged in a matrix, therefore, dimensions where it is possible to identify the closest events to the
the manager should assign, according to their understanding, for agents, that is, to identify which enterprise risks are considered
example, what is the importance of political risk for their rela- most important to each agent of its environment of value. Graph 1
tionship with the government. Considering the graphic dispersion of events and agents, initi-
Considering the scores assigned by the managers, we devel- ally, it can be seen that the enterprise risks arising from environ-
oped a correspondence analysis two dimensions, namely, events mental sustainability and environment issues are more sensitive to
and agents. The events were divided into two categories, business society, which seems to be consistent with the latest desires of
environment and environmental of value. In the business envir- society; there are various efforts from non-governmental organi-
onment, we presented the events related to the macroenviron- zations engaged in changing the human attitude with regard to the
ment: economic, environmental, political, social and technological. respect for nature (Hofmann et al., 2014; Veiga, 2005). For compe-
In the environment of value we presented the events related to the titors, it appears that the most important risks in the relationship
following aspects: financial, operational, image, environmental between the company and competition are the enterprise risks:
sustainability, ethical, innovation and strategic. The agents are financial, innovation and technological. The difference in prices
organizations that have an important relationship with the com- charged by competitors appeared as an important source of risks.
pany, they are organizations deemed important in the main- Opportunistic relationships are established when the ordering of
tenance or creation of value to the company, that is, the organi- the institutional environment is not fully established or is at least
zations present in the environment of value the company. The acceptable by the main agents (North, 1990; Ménard, 1995, 1997,
agents considered in this research were: the company, clients, 2000). Both the innovation from competition and the technological
suppliers, competitors, distributors, government and society. changes in the business environment are considered relevant
With regard to the validity of using the correspondence ana- enterprise risks to the company, which must be constantly alert to
lysis technique, we rejected the hypothesis that the categorical these types of events that may dramatically change the relationship
variables, events and agents, are independent at a level of sig- of power between the agents of its environment of value (Kim and
nificance of 5%. In addition, we calculated the value of β to confirm Mauborgne, 2004). Regarding the most important risks to the
the dependence of the variables (Hair et al., 1998; Pestana and company, the operational risks and strategic risks stand out. Cor-
Gageiro, 2000). porate risks arising from the lack of coordination of the supply
chain can lead to serious problems in the acquisition of raw mate-
β ¼ χpffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ðl 1Þðc 1Þ
2
ð12 1Þð7
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
ffi 1Þ ¼ 45:54
¼ 435:69
ðl 1Þðc 1Þ ð12 1Þð7 1Þ rial, production and delivery of products and services to customers
X 2 : value of chi square (Li et al., 2015; Habermann et al., 2015). Strategic risks, for example,
l : number of rows of the contingency table arising from the managers' lack of ability to understand the changes
c : number of columns of the contingency table in the business environment, may cause the corruption of the
company's value or the complete failure of its operation. For
Graph 1. Perceptual map of the correspondence analysis.

Source: Author (2014).
instance, bets on certain characteristics of new products that indicates the degree of partial correlation between the variables. In
require large investments and which have the chance of being this sample, it was found that the KMO value is equal to 0.955.
rejected by society may lead the company to an adverse financial According to Pestana and Gageiro (2000), for samples with KMO
situation or damaged image possibly without recovery (Chopra and value lower than 0.60, the factor analysis may not be adequate.
Sodhi, 2004; Louisot, 2010; Nooraie and Parast, 2015). Thus, for the sample of the research, the use of factor analysis is
Given the data collection method, which provides the relativity appropriate. Furthermore, we performed Bartlett's sphericity test,
of the importance assigned by the managers to the company's which consists in testing the hypothesis that the correlation
risks and its relationships with key agents of its environment of matrix of the variables is the identity matrix. By considering that
value, Table 1 below shows the risks in order of importance given the p-value is equal to 0.000, we eliminate the hypothesis that the
the proximity observed in the perceptual map. correlation matrix of the variables is the identity matrix, therefore,
the use of the factor analysis for this sample is appropriate. Table 2
4.2. Evaluation of the level of maturity in enterprise risk below shows the final values of the validity statistics of the use of
management the factor technique.
The factor analysis showed that the risk management practices
The questionnaire was administered to a sample of 243 man- declared by the managers can be classified into four explanatory
agers from the population of the 1,100 largest Brazilian companies factors, namely, organization, technicality, transparency and
according to the ranking of EXAME (2011). With the database involvement (see Fig. 3). The factor analysis provides groupings of
organized, we conducted a factor analysis, data reduction, aiming variables, called factors, which have the same numerical behavior
to identify the important factors for the explanation of the phe- with respect to the responses of the survey, in such a way that
nomenon studied. Then, we conducted a cluster analysis aiming to these variables can be replaced, for the purposes of analysis, by the
identify the possible groups with distinct approaches to enterprise factor, thus facilitating the understanding of the phenomenon.
risk management. Based on the categorization achieved, it was Obviously, reducing information from the model always cause the
possible to develop a multiple logistic analysis, it allows the reduction of its explanatory power. Considering an appropriate
identification of which variables are the most important to assess adoption of the factor analysis technique, it is possible to obtain
whether the company belongs to that cluster. The predictive factors with strong explanatory power that simplify the analysis
power of this analysis can be noted by the use of the equations due to the reduction of the elements to be evaluated. Therefore,
associated with each cluster. In such a way that we come up with a based on the questions of the interview, the variables associated
way to assign the level of maturity of the company with respect to with the numerical factors proposed by the factor analysis, we
the enterprise risk management (Fig. 2). assign a meaning to each one, where the first factor was called
Organization, which represents how much the company dedicates
4.2.1. Analysis of the explanatory factors of enterprise risk efforts to produce a structured risk management. The second
management factor is called Technicality, which seeks to portray how often the
Initially, we performed the analysis of the correlation matrix of company makes use of qualitative or quantitative techniques to
the variables. According to Hair et al. (1998), there should be a support the risk management process in the corporation. The third
large number of correlations higher than 0.30. In this case, it is factor, Transparency, reveals how often the company addresses the
worth noting that out of the 324 positions of the matrix18 18, only subject openly with its collaborators, seeking to involve them in
34 positions indicate value lower than or equal to 0.30, that is, participatory management of enterprise risks. Finally, the last
10.49% of the positions. However, it is worth mentioning that factor is called Involvement, as it is expected to show how much
these positions arise only from variable 2, which have low corre- the company is able to engage other agents of its environment of
lation with other variables. Since this variable was eliminated from value to make its risk management more efficient and effective.
the analysis, there is no position lower than or equal to 0.3, only 12
positions are lower than or equal to 0.4, that is, 3.70% of positions Table 2
is lower than or equal to 0.40. Therefore, considering this aspect, Factor analysis statistics.
the factor analysis appears to be appropriate for use. In addition, Source: Author (2014).
we calculated the Kaiser-Meyer-Olkin (KMO) statistics, which Statistics Values
Table 1 KMO 0.955

Most Relevant Risks in the Environment of Value. Bartlett's sphericity test (p-value) (level of 0.000
Source: Author (2014). significance o 0.05)
Minimum, maximum and average commonality 0.729; 0.919; 0.807
Agents Most relevant risks Minimum, maximum and average MSA 0.925; 0.976; 0.954
Minimum, maximum and average factor loading 0.554; 0.900; 0.729
Organization Strategic, Operational, Ethical and Image Eigenvalues 10.49; 0.98; 0.81; 0.64
Clients Social and Image Number of selected variables 8
Suppliers Strategic, Operational, Ethical and Image Number of aggregating factors (eigenvalues 4
Competitors Financial Innovation, Technological, Ethical and Image over ¼ 0.5)
Distributors Strategic, Operational, Ethical and Image Variance explained by the aggregating factors 22.46%; 32.37%; 15.07%;
Government Economic 10.84%
Society Environmental Sustainability and Environment Total variance explained by the aggregating factors 80.75%
Factor Analysis Cluster Analysis Multiple Logistic

Analysis
Fig. 2. Enterprise risk management maturity level data analysis.

Involvement Factor 4 - Involvement

External Support and Analysis in the
Environment of Value
Transparency Factor 3 - Transparency

Participative Management and Effective
Communication
Technicality Factor 2 - Technicality

Technical Approach and Attributes
Organization Factor 1 – Organization

Planning, Organization, Implementation and
Process Control
Fig. 3. Explanatory factors of enterprise risk management.

Source: author (2014).
Considering the factors presented, based on a numerical analysis Table 3

of the data, it is observed that companies have different degrees of Number of companies by cluster.
organization, technicality, transparency and involvement. A com-
pany can be more organized and less transparent with respect to Cluster Number of Companies
risk management, or may involve more agents of its environment of
value in pursuit of a more systemic risk management and possess a C1 34
C2 57
low degree of technicality in its risk management. However, in C3 16
general, it can be seen that there is a certain hierarchy regarding the C4 22
factors involved in risk management. All companies have some C5 28
degree of organization, technicality, transparency and involvement. C6 11
Total 168
However, for example, the greater the degree of technicality, the
greater the degree of organization. The greater the degree of
transparency, the greater the degree of technicality and organiza- individual distances of each element of a group with the elements
tion. The greater the degree of involvement, the greater the degree of another group. Considering the numerical analysis of the
of transparency, technicality and organization. That is, apparently, Agglomeration Schedule and the visual analysis of the Vertical
the companies have certain hierarchy with respect to the explana- Icicle and Dendrogram, it can be seen that the ideal number of
tory factors that classify the enterprise risk management. In the clusters is 6 groups. The Agglomeration Schedule shows the
following analyzes this issue will be properly detailed. composition of the groups step by step, so that it is possible to
evaluate the distance of the two groups associated with each
4.2.2. Analysis of characteristic groups in enterprise risk iteration. The criterion for defining the optimal number of clusters
management is the verification of the first major difference between the dis-
In this study, we decided to conduct the hierarchical cluster tances of each interaction. In this case, the first major difference
method to identify the ideal number of groups and the initials occurs between interaction 159 and 160, in the value of 1.481,
centroids, and then, based on the identified data, we conducted which consist of the identification of 9 clusters. Considering this
the non-hierarchical cluster method K-means. This composition of large number of groups, it can be seen that the next major varia-
the use of methods of hierarchical and non-hierarchical clusters tion occurs between interactions 162 and 163, in the value of
allows us to obtain the best of each one. The hierarchical method is 1,593, thus providing the optimal identification of six clusters.
advantageous when the ideal number of clusters is unknown, As previously mentioned, following the processing of the
which occurs in the vast majority of studies, on the other hand, the hierarchical cluster analysis, we used the non-hierarchical cluster
method proves to be disadvantageous because its genesis consists analysis K-means. The K-means method adopts as measure of
of the hierarchical composition of clusters, that is, initially, each similarity the Euclidean distance and as the clustering method the
element is a potential cluster and the new clusters are established nearest centroid sorting, which seeks to minimize the internal
from the previous ones, so that once the elements are grouped, variance of the elements of each cluster and maximize the var-
they will remain in the same cluster until the end of the proces- iance between clusters (Pestana and Gageiro, 2000).
sing. However, the non-hierarchical method necessarily needs a After determining the composition of 6 clusters, the processing
suggested number of clusters and preferably the initial centroids, of the non-hierarchical cluster analysis K-means presented the
otherwise, the algorithm will define which information it will use following configuration of segmentation of the companies sur-
as a starting point. On the other hand, the method proves to be veyed (Table 3).
advantageous because the method seeks the choice of the best The analysis of variance table, ANOVA, shows which variables
cluster configuration for the database presented (Hair et al., 1998). contributed most to discriminate the clusters. Initially, it appears
Thus, we started the processing of the hierarchical cluster that the four variables, factors obtained in the factor analysis, were
analysis, establishing as a measure of similarity the Euclidean significant for the formation of the 6 clusters at a significance level
squared distance measure and as the clustering method the of 5%. Consecutively, it is found that the F statistics, present in the
between-groups linkage. This method seeks to create groups with analysis of variance, indicates how much the variable has con-
the smallest distance, which is calculated as the average of the tributed to the composition of the clusters. The higher the value of
the F statistics, the more the variable discriminates the groups. In average values show that the hierarchically superior group main-
this case, by order of greatest contribution in the discrimination of tains higher values for the previous factors and indicates a value
the clusters, we have the following variables: Factor 4 – Involve- explicitly higher to the next factor, for example, cluster C3 has
ment, Factor 1 – Organization, Factor 2 – Technicality and Factor 3 higher average figures for factors 1 and 2 compared to the average
– Transparency. That is, Factor 4 – Involvement was the variable values of clusters C4 and C5, in addition, it indicates a greater
that contributed the most to discriminate the clusters, F¼ 42.238, average value for factor 3, when compared to the average values of
in such a way that the external variability is greater, 18.901, and clusters C4 and C5 (see Table 5 and Chart 1).
the internal variability was the lowest, 0.447. Table 4 It should be mentioned that cluster C6, which gathers 11 ele-
By refining the cluster analysis obtained, we sought to calculate ments, was eliminated because it indicates atypical average values
the average values of the original variables of the factors for the for the consolidation of the behavior intended to be shown. The
elements of each cluster, for example, companies in cluster C1 have values of factors 1, 2, 3 and 4 are 4.3030; 3.3523; 4.0303; 6.7727,
average values of the original variables of factor 1-organization respectively, values lower than 0.5, except for factor 4, moreover,
equal to 4.5294, or companies in cluster C2 have average values of there is no increasing behavior of the values compared to the
the original variables of factor 4 – involvement equal to 7.4298. It is factors as presented by the other clusters.
worth noting that the original variables vary from 0 to 10. There is a The hierarchical behavior will be more explored with the pre-
relationship between the factors and groups obtained. Cluster C1 sentation of the results obtained with the application of the
shows low values by factors, all lower than 5, in such a way that it technique of multivariate multinomial logistic regression analysis
was considered a group with insufficient characteristics for a clas- for the categories represented by the clusters.
sification of the enterprise risk management. On the other hand,
cluster C4 indicates value 5.8182 for factor 1-organization and 4.2.3. Analysis of the level of maturity in enterprise risk management
lower values the subsequent factors. Cluster C5 keeps the average The multiple logistic analysis is a multivariate technique that
value for factor 1 above 5 and indicates value 6.4063 for factor 2- associates a categorical variable Y of p instances with a set of q
technicality. Cluster C3 keeps the average values for factors 1 and independent variables X. Among its main statistical attractiveness,
2 above 5, and indicates value 5.7917 for Factor 3-transparency. we highlight the greater flexibility regarding the prerequisites of
With the same behavior, cluster C2 keeps the average values for its application. The normality and homogeneity of variance of
factors 1, 2 and 3 above 5, and indicates value 7.4298 for factor 4- residuals are not required. The technique and the multiple
involvement. Overall, there is a hierarchy between the clusters, the regression allow us to identify what are the key independent
variables associated with the categories of the dependent variable
Table 4 Y. In addition, as a distinctive offer, the multiple logistic technique
Evaluation of clusters.
allows the estimation of the probability of occurrence of a specific
event (Hosmer and Lemshow, 2000).
Variable Cluster mean Error mean F Sig.
square square Y p ¼ X 1 þ X 2 þ X 3 þ ::: þ X q
Factor 1 – Organization 18.448 0.461 39.974 .000 Y is a categorical independent variable that can assume p distinct values
Factor 2 – Technicality 17.290 0.497 34.772 .000 Xi are metric or non-metric dependent explanatory variables
Factor 3 – Transparency 11.346 0.681 16.668 .000
Factor 4 – Involvement 18.901 0.447 42.238 .000 The formulas for calculating the probabilities of a
company to belong to a certain level of maturity in knowledge
Table 5
Factors and clusters.
Clusters
Factor C1 – Insufficient C4 – Contingency C5 – Structured C3 – Participative C2 – Systemic
F1 – Organization 4.5294 5.8182 5.3214 8.1875 7.8187

F2 – Technicality 3.5588 3.3807 6.4063 7.6563 7.4583
F3 – Transparency 4.0882 2.1515 4.5238 5.7917 6.9181
F4 – Involvement 2.9853 3.9773 5.9643 4.1875 7.4298
Average 3.7728 3.7118 5.7465 6.8693 7.4181
Chart 1. Factors and clusters.

management are: As for the adhesion of the model, it is found that the significance
oddsðy ¼ 1Þ test of the coefficients of the final model indicates value 0.000,
gð1Þ ¼ RORy ¼ 1;y ¼ 1 ¼ oddsðy ¼ 1Þ
¼1
lower than 5%, that is, the model was significant at the level of 5%.
X
8 With regard to the variability captured by the model, that is, the
α2 þ β2i X 2i explanatory power of the model, it is considered high because all the
oddsðy ¼ 2Þ
gð2Þ ¼ RORy ¼ 2;y ¼ 1 ¼ ¼e i¼1 ¼ eZ 2 pseudo R² are higher than 94%. The Cox and Snell R² is equal to 94.46%,
oddsðy ¼ 1Þ
Nagelkerke R² is equal to 99.26% and McFadden R² is equal to 95.49%.
X
8
α3 þ β3i X 3i Further evidence regarding the adhesion of the model is repre-
oddsðy ¼ 3Þ sented by the Likelihood Ratio Test of the model, and it can be seen
gð3Þ ¼ RORy ¼ 3;y ¼ 0 ¼ ¼e i¼1 ¼ eZ 3
oddsðy ¼ 1Þ that out of the 18 variables collected, 8 are significant for the
X
8 composition of the model equations. The coefficients of the equa-
α4 þ β4i X 4i tion are significant at the level of 5%. The test results are lower than
oddsðy ¼ 4Þ
gð4Þ ¼ RORy ¼ 4;y ¼ 0 ¼ ¼e i¼1 ¼ eZ 4 0.000, except for the coefficient of variable Q4 whose value is 0.039.
oddsðy ¼ 1Þ
In addition, with respect to the adhesion of the model, the
X
8
α5 þ β5i X 5i Goodness-of-fit hypothesis tests appear to be favorable. The null
odds ðy ¼ 5Þ hypothesis Ho that there are no significant differences between
gð5Þ ¼ ROR y ¼ 5; y ¼ 0 ¼ ¼ e i¼1 ¼ eZ 5
odds ðy ¼ 1Þ the values observed and predicted by the model is not rejected.
gð1Þ
The p-values of Pearson and Deviance hypothesis test are greater
Pðy ¼ 1Þ ¼ gð1Þ þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ ¼ 1 þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ
1
than 0.05. Thus, the multinomial logistic model presented appears
gð2Þ gð2Þ to be duly in line with the survey data.
Pðy ¼ 2Þ ¼ ¼
gð1Þ þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ 1 þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ On the other hand, the analysis of the estimated parameters
does not indicate that the coefficients of the equations are sig-
gð3Þ gð3Þ
Pðy ¼ 3Þ ¼ ¼ nificant at the level of 0.05. Likewise, the Wald test, where the null
gð1Þ þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ 1 þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ
hypothesis Ho indicates that the coefficients are equal to zero is
gð4Þ gð4Þ not rejected at the level of 0.05. However, some studies show that
Pðy ¼ 4Þ ¼ ¼
gð1Þ þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ 1 þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ the Wald test has become unstable in certain situations, suggest-
ing that the adhesion of the model should not only be analyzed by
gð5Þ gð5Þ
Pðy ¼ 5Þ ¼ ¼ this statistic, but by a set of statistic of adhesion pertaining to the
gð1Þ þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ 1 þ gð2Þ þ gð3Þ þ gð4Þ þ gð5Þ
multinomial logistic regression (Hauck and Donner, 1977).
Considering category 1 as reference, that is, the calculations are As far as the measures already presented, the percentage of
based on level 1 of maturity, we can apply the formulas to obtain accuracy of the model is another important evidence to evaluate the
the probability of a company X to be at a certain level of maturity adhesion of the model presented. It was found that, in general, for all
(see Table 6). the five categories, the level of accuracy is greater than 92%, which
Considering the variables of the research, the components of X sets a high predictive power of the model. The average percentage of
assume the respective values of the original variables of the accuracy is 97.5%, that is, out of the 157 companies surveyed, the
research, therefore, X¼(Q1, Q3, Q4, Q7, Q10, Q12, Q13, Q17). Table 7 model predicted the clusters correctly for 153 companies. Table 8
Note that the original variables Q1, Q3 and Q4 are part of factor Thus, given that most adjustment statistics reveals that the
1-Organization, Q7 and Q10 of factor 2-Technicality, Q12 and Q13 model has high level of adhesion and the level of overall accuracy
of factor 3-Transparency, Q17 of factor 4 – Involvement, are pre-
sent in the equations of the multinomial logistic model, that is, the Table 7
four factors deemed important to define the clusters are also Original variables comprising the multinomial logistic model.
present in the equations of the logistic model, through the original Source: Author (2014).
variables that comprise them, thus confirming that all four factors
Q1 What is the LEVEL of CONTROL of the organization's risk?
are discriminant of the companies surveyed. Therefore, consider- Q3 Are the processes relating to risk Planned, Organized, Executed and
ing the predictive power of the model, the answers to these Managed rationally?
questions will be important to determine in which cluster a Q4 Are the processes relating to risk dealt with adequate FREQUENCY?
Q7 Are the risks MEASURED QUANTITATIVELY?
company can be classified, that is, what is its level of maturity in
Q10 Is there a CULTURE OF RISK ASSESSMENT in the management of the
risk management. In addition, the model provides the probability organization?
of this company to belong to each of the possible levels of Q12 Are there OPEN EVENTS, with ideal frequency, to discuss the manage-
maturity. Thus, considering the 18 original variables, it can be seen ment risks of the organization?
Q13 Is the risk management in the organization DECENTRALIZED?
that the 8 variables Q1, Q3, Q4, Q7, Q10, Q12, Q13 and Q17 are the
Q17 What is the LEVEL OF EXTERNAL SUPPORT (consultancy, university,
most important variables for defining the level of maturity in the partners or other entities) to the enterprise risk management?
risk management of a company.
Table 6
Equations by clusters.
Cluster Equation
C1 (y ¼1) Z1 ¼ 0
C2 (y ¼2) Z2 ¼ 79; 82 þ 5; 44 X1 þ 3; 23 X2 þ 0; 64 X3 þ 3; 29X4 þ 1; 23X5 þ 1; 17X6 3; 78X7 þ 3; 67X8
C3 (y ¼3) Z3 ¼ 316; 52 þ 7; 13 X1 þ 29; 02 X2 þ 8; 24 X3 þ 21; 96X4 þ 0; 97X5 8; 46X6 4; 51X7 29; 82X8
C4 (y ¼4) Z4 ¼ 46; 82 þ 15; 57 X1 þ 6; 62X 2 0; 91X3 þ 0; 83X 4 11; 42X 5 3; 03X6 10; 76X7 þ 4; 29X 8
C5 (y ¼5) Z5 ¼ 29; 33 þ 0; 71 X1 0; 12X 2 1; 42X 3 þ 5; 28X 4 þ 4; 27X 5 0; 49X 6 5; 22X7 þ 3; 32X 8
is 97.5%, it is understood that the multinomial logistic model It can be seen that the company 32 belongs to cluster 4, level
presented is in line with the data collected and given the repre- 1 of maturity, that is, it is a company, according to its manager,
sentativeness of the sample, the model can be extrapolated to the who has a good level of organization, however, it has a low level of
population of companies surveyed. technicality, transparency and involvement in enterprise risk
Considering the distinctive characteristic of the logistic models in management.
providing a calculation of probability for the occurrence of an event, Therefore, based on the above, we propose a model for asses-
duly represented by the categories of the dependent variable, below sing the level of maturity of enterprise risk management with
5 categories, which are revealed by the enterprise risk manage-
we present one example of calculation of such probabilities Table 9.
ment characteristics assumed by the companies. In Fig. 4, the
Table 8 categories are presented and right after each one is described, that
Evaluation of the level of accuracy of the prediction model.
is, it shows the characteristics that an organization must take to be
considered in this category of maturity in enterprise risk
Clusters Predicted management.
Clusters observed 1 2 3 4 5 Percentage of accuracy Level 1, called insufficient enterprise risk management, includes
1 34 0 0 0 0 100%
2 0 55 0 0 2 96.5%
companies that have little awareness of the enterprise risks.
3 0 0 16 0 0 100% There is no physical or conceptual structure dedicated to
4 0 0 0 22 0 100% enterprise risks. The adoption of risk management practices
5 0 2 0 0 26 92.9% occurs on a non-structured manner.
Average 97.5%
Level 2, called contingency enterprise risk management, involves
companies that are aware of the risks to which they are subject.
To illustrate the calculations of probability, we assumed vari- Risk management techniques, tools and methods are roughly
able X in the following instance X32 ¼ (7,8,8,6,2,4,5, and 3), which used. Risk management is centralized and is characterized by
sets the following values: the low involvement of employees in general.
Level 3, called structured enterprise risk management, involves
Table 9 companies with a higher degree of organization of processes
Example 1 of application of the multinomial logistic model. related to enterprise risk management. There is a more intense
use of risk management techniques, tools and methods.
Cluster Zj P(y¼j) Level 4, called participative enterprise risk management,
involves companies with high level of awareness and organi-
C1 (y¼ 1) Z1 ¼ 0 P(y¼1) ¼ 00.00% zation with regard to the processes related to enterprise risk
C2 (y¼ 2) Z2 ¼ 8:17 P(y¼2) ¼ 00.00% management. The risk management is more decentralized.
C3 (y¼ 3) Z3 ¼ 19:26 P(y¼3) ¼ 00.00%
Communication is an integral and important part in risk man-
C4 (y¼ 4) Z4 ¼ 36:95 P(y¼4) ¼ 99.99%
C5 (y¼ 5) Z5 ¼ 14:60 P(y¼5) ¼ 00.00% agement. The enterprise risk management is guided by the
participation of most employees.
Level 5 - Systemic Enterprise Risk Management

Systemic
ERM
Level 4 – Participative Enterprise Risk Management

Participative
ERM
Level 3 – Structured Enterprise Risk Management

Structured
ERM
Level 2 – Contingency Enterprise Risk Management

Contingency
ERM
Level 1 - Insufficient Enterprise Risk Management

Insufficient
ERM
Fig. 4. Level of maturity in enterprise risk management.

Level 5, called systemic enterprise risk management, is the The model proposed for the analysis of maturity in enterprise
highest level of the classification. In this level, companies have a risk management stands out for the originality of the subject itself.
conscious, organized and transparent enterprise risk manage- Few maturity models focused on enterprise risk management
ment. These companies use external support from consulting were found in the literature. In addition, the severity and the
firms, partners and research institutes to improve risk man- voracity with which the subject is posed have shown the lack of
agement. In addition, the risk management of company adherence on certain aspects of current models. Some aspects of
increasingly includes the assessment of the risks of its envir-
distinction of the model proposed should be highlighted. First, it
onment of value, considering that the risks do not respect the
can be seen that the level of risk management assumes higher
organizational boundaries, they are sovereign with respect to
levels, in such a way that it is no longer a specialty within the
property boundaries. Thus, it is expected that modern enterprise
risk management transcends its practices beyond the bound- organization, but it permeates the key processes of the organiza-
aries of the organization. tion. It was found that companies in the intermediate level of
maturity have an enterprise risk management with a high degree
of organization, use of methods and techniques, and greater
5. Conclusions decentralization, that is, distinctive characteristics in the past lit-
erature, today they are common to the companies. In addition, the
The development of this study was dedicated to the analysis of research revealed important new characteristics to assess the level
enterprise risk management in the supply chain of large Brazilian of maturity, including transparency in the communication of
companies. The importance of this subject can be assessed by its potential risks, participation of external agents in risk manage-
constant presence on the agenda of managers. Modern society is a ment, risk assessment in the environment of value.
source of risks, but also a source to repair them. Humankind
dominates the risk, transforming the passive fear by managing it.
Seeking to meet the objectives of the study and answer the Acknowledgments
research questions, we developed a survey with over 150 com-
panies ranked as the largest Brazilian companies, according to a Faculdade de Economia, Administração e Contabilidade-FEA.
publication specialized in economics. The statistical analysis was Universidade de São Paulo-USP.
performed through the descriptive analysis of data and then by the Fundação Instituto de Administração-FIA.
multivariate analysis, which allowed us to further deepen the
study to propose a solution to the research questions and meet the
objectives of the study.
The model proposed for the analysis of enterprise risks pre- References
sents some points of distinction with respect to the current
models. First, we should highlight that the model proposed Aqlan, F., Lam, S.S., 2015a. A fuzzy-based integrated framework for supply chain risk
assesses the enterprise risks by three levels: organization, envir- assessment. Int. J. Prod. Econ. 161, 54–63.
onment of value and business environment. The organization is Aqlan, F., Lam, S.S., 2015b. Supply Chain risk modeling and mitigation. Int. J. Prod.
Res. 53 (18), 5640–5656.
exposed to forces and events arising out of these three environ- AON, 2012. Aon Risk Management, available at: 〈http://www.aon.com〉, accessed:
ments, which have distinctive characteristics, in such a way that 03/2012.
the segmented analysis enables higher accuracy. Another impor- Bachmann, R., 2001. Trust, power and control in trans-organizational relations.
Organ. Stud. 22 (2), 337–365.
tant aspect is that the analysis of enterprise risks should transcend Ballou, R.H., 2004. Business Logistics Management: Planning, Organizing, and
the organizational boundaries, that is, in addition to assessing the Controlling the Supply Chain. Pearson Prentice Hall, New York.
analysis of the organization's risks in the three levels, it should Barzel, Y., 2002. Organizational forms and measurement costs. J. Inst. Theor. Econ.
161 (3), 357–373.
assess the risks to which the organization's relationships with Bateman, T.S., Snell, S.A., 2012. Management: Leading & Collaborating in the
other agents of the business environment are subject. In other Competitive World Management. McGraw-Hill Higher Education, New York.
words, the enterprise risk analysis should be a relational analysis, Beasley, M.S., Clune, R., Hermanson, D.R., 2005. Enterprise risk management: an
empirical analysis of factors associated with the extent of implementation. J.
not only of the positioning of the organization, but above all, the
Account. Public Policy 24, 521–531.
enterprise risk analysis must be established in the context of its Blome, C., Schoenherr, T., 2011. Supply risk management in financial crises: a
environment of value, a concept coined in this thesis, which allows multiple case study approach. Int. J. Prod. Econ. 134 (1), 43–57.
the assessment not only of the risks of the main agent, but the Chen, J., Sohal, A.S., Prajogo, D.I., 2013. Supply chain operational risk mitigation: a
collaborative approach. Int. J. Prod. Res. 51 (7), 2186–2199.
organization and the risks arising from the transactions with other Chopra, S., Sodhi, M.S., 2004. Managing Risk to Avoid Supply-Chain Breakdown.
agents that generate value. It is worth noting that the enterprise MIT Sloan Management Review. 46 (1), 53–61.
risks do not respect the boundaries of the organization. Con- Christopher, M., Peck, H., 2004. Building the resilient supply chain. Int. J. Logist.
Manag. 15 (2), 1–14.
centrating the enterprise risk analysis within the organizational Christopher, M., Holweg, M., 2011. Supply chain 2.0: managing supply chains in the
scope might at first strengthen the aspects of efficiency, however, era of turbulence. Int. J. Phys. Distrib. Logist. Manag. 41 (1), 63–82.
in the medium and long-term, the organization may lose its Coase, R.H., 1937. The nature of the firm. Econ. New Ser. 4 (16), 386–405.
Coase, R.H., 1960. The problem of social cost. J. Law Econ. 3, 01–44.
effectiveness in achieving its strategic objectives. Coase, 2014. available at: 〈http://www.coase.org〉, accessed: 03/2014.
As a specific goal, which was to identify the enterprise risk Cooper, D.C., Schindler, P.S., 2001. Business Research Methods. McGraw-Hill, New
management practices, the research has shown the form, fre- York.
COSO, 2004. Committee of Sponsoring Organizations of the Treadway Commission.
quency, technicality, structure, transparency, participation, com- Enterprise Risk Management-Integrated Framework.
munication and the involvement of third parties as elements of Damodaran, A., 2008. Strategic Risk Taking: A Framework for Risk Management.
the enterprise risk management of large Brazilian companies. Pearson Prentice Hall, New York.
Deloitte, 2005. Destruidores de valor. Rev. Mundo Corp. 3 (10), 7–9.
Understanding these issues allowed the identification of the main
Deloitte, 2013. The Ripple Effect: How Manufacturing and Retail Executives View
characteristics of the risk management of companies, as well as the Growing Challenge of Supply Chain Risk, available at: 〈http://deloitte.com〉,
the segmentation of companies, according to the way they conduct accessed: 07/2014.
their risk management. In such a way that it is possible to develop Denzin, N., 1970. Strategies of multiple triangulation. In: Denzin, N. (Ed.), The
Research Act in Sociology: A Theoretical Introduction to Sociological Method.
a structured way of assessing how large companies deal with Butterworth, London.
enterprise risk management. Ernest and Young, 2012. available at: 〈http://ey.com〉, accessed: 05/2012.
Espejo, R., 1993. Management of Complexity in Problem Solving, in: Raul Espejo e Ménard, C., 1997. Le pilotage des formes organisationnelles hybrids. Rev. Econ. 48
Markus Schwaninger, Organisational Fitness: Corporate Effectiveness Through (3), 741–750.
Management Cybernetics. Campus Verlag, New York. Ménard, C., 2000. Enforcement Procedures and Governance Structures: What
Espejo, R., et al., 1996. Giving Requisite Variety To Management: A Discussion Based Relationship?. In: Ménard (Ed.), Institutions Contracts and Organizations.
on the Viable System Model, In: Organizational Transformation And Learning. Edward Elgar, Cheltenham.
Wiley & Sons, Ontario. Mikes, A., 2009. Risk management and calculative cultures. Manag. Account. Res.
EXAME, 2011. Melhores & Maiores: As 1.000 Maiores Empresas do Brasil. Revista 20, 18–40.
Exame, Edição Especial, São Pailo. Miller, R., Lessard, D., 2001. Understanding and managing risks in large engineering
FERMA, 2003. Federation of European Risk Management Association. Normas de projects. Int. J. Proj. Manag. 19, 437–443.
gestão de riscos, available at: 〈http://www.ferma.eu〉, accessed: 03/2012. Mintzberg, H., Ahlstrand, B., Lampel, J., 2005. Strategy Safari: A Guided Tour
Gaultier-Gaillard, S., Louisot, J.P., Rayner, J., 2009. Managing reputational risk – Through The Wilds of Strategic Management. Simon and Schuster, New York.
From theory to practice, in: Reputation Capital: Building and Maintaining Trust Nee, V., 2003. New Institutionalism, Economic and Sociological. Handbook for
in the 21st Century. Springer Berlin Heidelberg, Berlin. Economic Sociology. Princeton University Press, Princenton.
Grey, W., Shi, D., 2005. Enterprise risk management: a value chain perspective. Neiger, D., Rotaru, K., Churilov, L., 2009. Supply chain risk identification with value-
handbook of integrated risk management for e-business. J. Ross Publishing, focused process engineering. J. Oper. Manag. 27 (2), 154–168.
Florida. Nooraie, S.V., Parast, M.M., 2015. A multi-objective approach to supply chain risk
Habermann, M., Blackhurst, J., Metcalf, A.Y., 2015. Keep your friends close? supply management: Integrating visibility with supply and demand risk. Int. J. Prod.
chain design and disruption risk. Decis. Sci. 46 (3), 491–526. Econ. 161, 192–200.
Hahn, G.J., Kuhn, H., 2012. Value-based performance and risk management in Norrman, A., Jansson, U., 2004. Ericsson's proactive supply chain risk management
supply chains: a robust optimization approach. Int. J. Prod. Econ. 139, 135–144. approach after a serious sub-supplier accident. Int. J. Phys. Distrib. Logist.
Hair Jr., J.F., Anderson, R.E., Tatham, R.L., Black, W.C., 1998. Multivariate Data Ana- Manag. 34 (5), 434–456.
lysis.. Prentice Hall, Upper Saddle River, New Jersey. North, D.C., 1990. Institutions, Institutional Change and Economic Performance.
Hauck Jr., W.W., Donner, A., 1977. Wald's test as applied to hypotheses in logit Cambridge University Press, Cambridge.
analysis. J. Am. Stat. Assoc. 72 (360), 851–853. Olson, D.L., Swenseth, S.R., 2014. Trade-offs in supply chain system risk mitigation.
Heckmann, I., Comes, T., Nickel, S., 2015. A critical review on supply chain risk- Syst. Res. Behav. Sci. 31, 565–579.
Definition, measure and modeling. Omega 52 (C), 119–132. Paape, L., Speklé, R., 2012. The adoption and design of enterprise risk management
Hillson, D.A., 1997. Towards a Risk Maturity Model. Int. J. Proj. Bus. Risk Manag. 1 practices: an empirical study. Eur. Account. Rev. 21 (3), 533–564.
(1), 35–45. Pestana, M.H., Gageiro, J.N., 2000. Análise de Dados para Ciências Sociais: a com-
Hitt, M., Ireland, R.D., Hoskisson, R., 2014. Strategic Management: Concepts and plementaridade do SPSS. Edições Sílabo, São Paulo.
Cases: Competitiveness and Globalization. Cengage Learning, Boston. Porter, M., 2008. Competitive Advantage: Creating and Sustaining Superior Per-
Hofmann, H., Busse, C., Bode, C., Henke, M., 2014. Sustainability-Related Supply formance. Free Press, New York.
Chain Risks: Conceptualization and Management. Bus. Strategy Environ. 23, Porter, M., 2004. Competitive Strategy: Techniques for Analyzing Industries and
160–174. Competitors. Free Press, New York.
Hosmer, D.W., Lemshow, S., 2000. Applied Logistic Regression. Wiley, New York. Porter, M., Millar, V.E., 1985. How information gives you competitive advantage.
Hoyt, R.E., Liebenberg, A.F., 2011. The value of enterprise risk management. J. Risk Harv. Bus. Rev., 149–174.
Insur. 78 (4), 795–822. RIMS, 2006. Risk and Insurance Management Society. RIMS Risk Maturity Model
Hult, G.T.M., Craighead, C.W., Ketchen, D.J., 2010. Risk uncertainty and supply chain (RMM) for Enterprise Risk Management, available at: 〈http://www.rims.org〉,
decisions: a real options perspective. Decis. Sci. 41 (3), 435–458. accessed: 04/2012.
IBGC, 2007. Instituto Brasileiro de Governança Corporativa. Guia de orientação para Shank, J.K., Govindarajan, V., 1993. Strategic Cost Management: The New Tool For
o gerenciamento de riscos corporativos, available at: 〈http://www.ibgc.org.br〉, Competitive Advantage. The Free Press, New York.
accessed: 06/2012. Sodhi, M.S., Son, B., Tang, C.S., 2012. Researchers' perspectives on supply chain risk
IFAC, 2001. International Federation of Accountants. Governance in the Public management. Prod. Oper. Manag. 21 (1), 1–13.
Sector: A Governing Body Perspective, available at: 〈http://www.ifac.org〉, Sydow, J., Frenkel, S.J., 2013. Labor, risk, and uncertainty in global supply networks-
accessed: 09/2012. exploratory insights. J. Bus. Logist. 34 (3), 236–247.
IMA, 1997. Institute of Management Accountants. Measuring and Managing Tang, C.S., 2006a. Perspectives in supply chain risk management. Int. J. Prod. Econ.
Shareholder Value Creation, available at: 〈http://www.imanet.org〉, accessed: 103 (2), 451–488.
10/2012. Tang, C.S., 2006b. Robust strategies for mitigating supply chain disruptions. Int. J.
ISO 31000, 2009. International Organization for Standardization. ISO 31000: Risk Logist.: Res. Appl. 9 (1), 33–45.
Management – Principles and Guidelines on Implementation. Tang, C.S., Tomlin, B., 2008. The power of flexibility for mitigating supply chain
Juttner, U., 2005. Supply chain risk management. Int. J. Logist. Manag. 16, 120–141. risks. Int. J. Prod. Econ. 116 (1), 12–27.
Juttner, U., Peck, H., Christopher, M., 2003. Supply chain risk management: out- Thun, J.H., Hoenig, D., 2011. An empirical analysis of supply chain risk management
lining and agenda for future research. Int. J. Logist.: Res. Appl. 6 (4), 17–32. in the German automotive industry. Int. J. Prod. Econ. 131 (1), 242–249.
Kim, C., Mauborgne, R., 2004. Blue Ocean Strategy. Harv. Bus. Rev., 76–85. Veiga, J.E. da, 2005. Do Global ao Local. Armazém do Ipê, Campinas.
Lee, H.L., 2004. The triple-a supply chain. Harv. Bus. Rev. 83, 102–112. Walters, D., Rainbird, M., 2004. The demand chain as an integral component of the
Li, G., Fan, H., Lee, P.K.C., Cheng, T.C.E., 2015. Joint supply chain risk management: an value chain. J. Consum. Mark. 21 (7), 465–475.
agency and collaboration perspective. Int. J. Prod. Econ. 164, 83–94. Walters, D., 2007. Supply chain Risk Management: Vulnerability and Resilience in
Louisot, J.P., 2010. Gestion des risques. Afnor, Paris. Logistics. Ed. Kogan-page, London.
Manuj, I., Mentzer, J.T., 2008. Global supply chain risk management. J. Bus. Logist. Williamson, O.E., 1975. Markets and Hierarchies: Analysis and Antitrust Implica-
29 (1), 133–156. tions. The Free Press, New York.
Manuj, I., Esper, T.L., Stank, T.P., 2014. Supply chain risk management approaches Williamson, O.E., 1985. The Economic Institutions of Capitalism: Firms, Markets,
under different conditions of risk. J. Bus. Logist. 35 (3), 241–258. Relational Contracting. The Free Press, New York.
McShane, M.K., Nair, A., Rustambekov, E., 2010. Does Enterprise Risk Management Zsidisin, G.A., 2003. Managerial perceptions of supply risk. J. Supply Chain Manag.
increase Firm Value? J. Account. Audit. Financ. 26 (4), 641–658. 39 (1), 14–25.
Ménard, C., 1995. Markets as institutions versus organizations as markets? Disen-
tangling some fundamental concepts. J. Econ. Behav. Organ. 28 (2), 161–182.

Oliva - A Maturity Model For Enterprise Risk Management

Uploaded by

Copyright:

Available Formats

Oliva - A Maturity Model For Enterprise Risk Management

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Oliva - A Maturity Model For Enterprise Risk Management

Uploaded by

Copyright:

Available Formats

Int. J.

Production Economics 173 (2016) 66–79

Contents lists available at ScienceDirect

Int. J. Production Economics

A maturity model for enterprise risk management

1. Introduction techniques proliferate aiming to incorporate the risk assessment

organization is with respect to enterprise risk management have 2. Theoretical framework

Ethical Innovation Strategic

Fig. 1. Enterprise risks in the environment of value.

Graph 1. Perceptual map of the correspondence analysis.

we calculated the Kaiser-Meyer-Olkin (KMO) statistics, which Statistics Values

Table 1 KMO 0.955

Factor Analysis Cluster Analysis Multiple Logistic

Fig. 2. Enterprise risk management maturity level data analysis.

Involvement Factor 4 - Involvement

Transparency Factor 3 - Transparency

Technicality Factor 2 - Technicality

Organization Factor 1 – Organization

Fig. 3. Explanatory factors of enterprise risk management.

Considering the factors presented, based on a numerical analysis Table 3

Factor C1 – Insufﬁcient C4 – Contingency C5 – Structured C3 – Participative C2 – Systemic

F1 – Organization 4.5294 5.8182 5.3214 8.1875 7.8187

Chart 1. Factors and clusters.

Level 5 - Systemic Enterprise Risk Management

Level 4 – Participative Enterprise Risk Management

Level 3 – Structured Enterprise Risk Management

Level 2 – Contingency Enterprise Risk Management

Level 1 - Insufficient Enterprise Risk Management

Fig. 4. Level of maturity in enterprise risk management.

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.