Impact of Supplier Evaluation On Product Quality
Impact of Supplier Evaluation On Product Quality
Impact of Supplier Evaluation On Product Quality
net/publication/318501298
CITATIONS READS
3 1,728
5 authors, including:
Kim Needy
University of Arkansas
103 PUBLICATIONS 1,430 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
CII RT-308 Achieving Zero Rework through Effective Supplier Quality Practices View project
Achieving Zero Rework through Effective Supplier Evaluation and Selection - Phase II View project
All content following this page was uploaded by Thais Alves on 01 August 2017.
Abstract: Supplier evaluation and engagement are regular topics in the Lean
literature at large. It is well known that Toyota extensively works with suppliers to
develop their work and capabilities, and provides challenges and feedback to
suppliers so that they can achieve the quality requirements defined by the company.
In the construction industry, companies conduct evaluations of suppliers, but
practitioners cite numerous reasons why supplier evaluations are not constantly
shared with suppliers. Moreover, the topic of supplier evaluation and how suppliers’
ratings affect the delivery of products free of defects to construction projects has not
received much attention by the IGLC community. This paper presents results of a
study, which underscores the importance of conducting supplier evaluation as a
means to assure quality products are delivered to construction projects. Findings
show that suppliers with low ratings, or who are not evaluated, are assigned more
hours of inspection, as are suppliers who subcontract portions of their work. This
practice translates not only into additional budgets required to inspect suppliers, but
also additional management-related costs.
1 I NTRODUCTION
The topic of supplier evaluation is usually discussed as part of materials management
and procurement evaluations (Construction Industry Institute [CII] 1999), as well as
studies related to risk analysis in construction projects (Monckza et al. 2016). Suppliers
are usually evaluated prior to being hired to deliver a purchase order (PO) through the
use of “formal supplier evaluation systems,” and contractors are more likely to have
these systems than owners (CII 1999). Despite having formal systems to evaluate
suppliers, a recent study by CII RT308 revealed that contractors are evenly split (54.4%
yes vs. 45.6% no) when it comes to the use of prior supplier evaluation to make decisions
about future purchases. Owners who participated in the same study were even less likely
to use prior performance evaluations to base their decisions to award POs (Alves et al.
2016b). This current practice goes against Lean principles that promote the use of
indicators and information to support a transparent management of value streams and
1
Associate Professor, J.R. Filanc Construction Engineering and Management Program, Dept. of Civil,
Constr., and Env. Engineering, San Diego State University, USA, talves@mail.sdsu.edu
2
Graduate Research Assistant, Dept. of Civil, Constr., and Env. Engineering, San Diego State
University, USA, panthildesai@gmail.com
3
Dean, Graduate School and International Education, University of Arkansas, Fayetteville, AR, USA,
kneedy@uark.edu
4
Industrial Engineer, Former University of Arkansas student, Fayetteville, USA,
ashleigh.hegwood@gmail.com
5
Director, Performance Excellence, MEI Technologies, Inc. (MEIT), Houston, TX, USA
seanmusick@gmail.com
2 L ITERATURE R EVIEW
Suppliers play an important role in the literature on Lean practices throughout multiple
phases of a product life-cycle, from the development of suppliers, through product design,
manufacturing, and distribution. Liker's Toyota Way (2003) describes at least three
principles that directly relate to the discussion presented in this paper: Principle 5: Build
a culture of stopping to fix problems, to get quality right at the first time; Principle 17:
Respect your extended network of partners and suppliers by challenging them and
helping them improve; and, Principle 18: Go and See for yourself to thoroughly
understand the situation (Genchi Genbutsu).
Liker's (2003) principles 5 and 18 are also supported by the discussion presented by
Rother (2010, p.181), who stresses the importance of process improvement at Toyota and
how it responds to process abnormalities: the responses to process abnormalities should
be immediate; and, the response to process abnormalities should come from someone
other than the production operators. The former statement underscores the importance
of dealing with problems as soon as they happen so that adjustments can be made before
any targets are missed and before the problems grow bigger. This suggests that a team
leader should be appointed to help teams proactively resolve these problems.
Before becoming a preferred partner at Toyota, suppliers often work closely with the
company for many years before they are ever awarded a PO. Toyota has high
expectations for its suppliers and, during the years before the award of a PO, suppliers
learn about these expectations and how they will work in practice (Liker 2003). The
literature on supplier evaluation shows that suppliers who are frequently evaluated,
receive feedback about their performance, and know what is expected from them
perform better and are more committed to their clients as they can adjust their processes
to match clients’ requirements (Kannan and Tan 2002; Prahinski and Fan 2007).
Moreover, Walsh et al.'s (2015) study in the EPC industry revealed that companies that
communicate more often with suppliers, use supplier ratings, track indicators like cost
and time spent on POs, and use suppliers with quality management systems tend to find
nonconformances (NCs) early in the life of a PO, preventing these NCs from reaching
construction projects.
In addition to these principles that can be used to support a healthy environment to
respect and grow the capabilities of an extended network of suppliers, another principle
outlined by Liker (2003) supports the discussion presented herein: Principle 1: Base your
long-term management decisions on a long-term philosophy, even at the expense of
short-term financial goals. The study presented in this paper illustrates how suppliers
with low ratings are hired at the expense of product quality. This often happens because
of low PO costs offered by these suppliers, and the practice adopted by some companies
to compensate this deficiency is to assign additional inspection hours to a PO instead of
working with suppliers to improve their capabilities.
3 R ESEARCH M ETHOD
This study was developed as part of the extension work of the research team 308 (RT308)
funded by the Construction Industry Institute (CII) from 2015-2016. RT308 membership
included academics (co-authors of this paper) and industry practitioners (owners, EPC
contractors [co-author], and suppliers) referred to subject matter experts (SMEs) in this
paper. As part of the team’s goal to define ways for companies to determine the
fabrication and inspection capabilities of suppliers, the team developed a survey (PO
instrument), which was initially tested within the team, revised, and later distributed to
the CII membership. The PO instrument was developed based on RT308’s previous work
(Walsh et al. 2015) and adapted to address one single material: shop fabricated piping.
The SMEs suggested that the team should be able to collect data on this product which is
present in virtually any project they developed, and in future efforts the PO instrument
could be adapted to collect data for other materials so that fabrication and inspection
capabilities can be defined for them as well.
The PO instrument, fully available at Alves et al. (2016b), contained 28 questions
organized into four main parts: contact data from the respondent (to clarify any
questions about the responses) and project demographics (name, type, dollar value, role
of the respondent); basic data about the single PO being used in each survey (reference
number, number of spools dollar value of the PO, inspection hours budgeted, location of
supplier facility, and criticality level of the PO); pre-award evaluation (level of inspection
assigned, existence of supplier evaluation/reasons for doing or not doing so, pre-award
supplier evaluation using criteria discussed later in this paper); and post-execution
evaluation (inspection hours utilized, existence of subcontracting, final level of
inspection, comparison between planned vs. actual hours of inspection, total number of
pipe spools received for the PO, number of unplanned quality events, post –award
supplier evaluation using criteria discussed later in this paper). The institutional review
boards (IRBs) at San Diego State University and the University of Arkansas, Fayetteville
approved the survey distribution and handling of data obtained, which also complied
with CII guidelines for data collection and analysis.
3.1 Hypotheses Testing
RT308 developed the hypotheses outlined in Table 1 to investigate the relationship
between certain practices and characteristics of the PO indicated by respondents. Only
hypotheses related to supplier ratings and product quality are discussed in this paper. A
full discussion of all hypotheses tested can be found in Alves et al. (2016b).
The Mann-Whitney (M-W) test, also known as Wilcox-Mann-Whitney test, was
used to evaluate the hypotheses given that the data was not normally distributed. The M-
W test is considered a statistically powerful test among all non-parametric tests (Field
2012).
Data entries submitted through Qualtrics (an online survey platform) were reviewed
and questions were cleared with respondents. The analysis was developed using IBM
SPSS and data was organized and analyzed using Field's (2012) recommendations. The
questions used to conduct the analysis are also shown in Table 1.
4 A NALYSIS OF R ESULTS
A total of 41 POs were received by RT308, however, not all submissions had answers to
all questions, e.g., number of nonconformances at the site were not indicated, pre-award
evaluation not available. This resulted in different numbers of POs being used to
evaluate different questions. Only a few responses regarding pre-award supplier
evaluation were submitted, however, most respondents had post-execution evaluations,
which were used to analyze the hypotheses discussed in this paper. The team had
originally hoped to evaluate differences between pre-award and post-execution, but due
to the low number of data points for pre-award that was not possible. During both
phases of RT308’s project, respondents to both PO surveys indicated multiple reasons for
not evaluating suppliers (Alves et al. 2016a, b) including: supplier had been evaluated by
owner of the project, evaluations had been updated with post-execution, existence of
approved list of suppliers did not require additional evaluation, the client had issued the
PO, and previous experience with the supplier.
4.1 Demographics
The data came from 10 companies representing 37 different projects. Out of that sample,
owner companies submitted 10 POs (24%) and contractors submitted 31 (76%). One
single company submitted 15 POs, and the authors understand that this is a limitation of
this data set. The POs dollar values ranged from $255,000 to $50,660,888. The total dollar
value of all submitted POs was $281,091,967. About 63% of the POs submitted had a
dollar value between $1 million and $10 million, 20% were between $100,000 and $1
million, and 17% of the POs were over $10 million. The minimum number of spools in a
single PO was 16 spools and the maximum number was 37,500 spools.
Among the 41 POs, about 90% of the POs were delivered by suppliers located in
developed countries and 10% by suppliers in developing countries. For confidentiality
reasons, the instrument did not collect data about the names of specific suppliers, only
the location of the facility that manufactured the order was recorded. However,
respondents indicated 19 specific cities, some of them appearing more than once, and
others indicated only the country where the supplier’s facility was located (e.g., USA).
Additionally, in 32% of the 41 POs, the supplier had subcontracted a portion of the PO.
Data reveal that subcontracting is a common practice even for commodities such as shop
fabricated piping. The analysis of the POs submitted revealed that 47.5% of the POs
ended up with an inspection level higher than originally budgeted, 35% had a lower
inspection level at the end, and only 17.5% had the inspection level to be equal to the
budgeted level. The analysis revealed that inspection work was under planned for 14% of
the POs submitted.
This goes against the idea exposed in The Toyota Way’s Principle 1: Base your long-
term management decisions on a long-term philosophy, even at the expense of short-
term financial goals (Liker 2003). Also, suppliers who subcontract parts of their POs
might display a low price tag at first, but costs add up when problems arise with their
extended supply chain. The problem is, according to RT308’s SMEs, they do not always
know about subcontracting practices ahead of time.
Suppliers with low Yes More inspection hours budgeted and more NCs are
ratings in evaluations. found at the shop. This suggests that additional
money ends up being spent to assure quality when
these suppliers are selected.
Subcontracting, even Yes During the pre-award evaluations, subcontracting
for portions of a should be factored in as something that will increase
commodity like shop the costs of surveillance and quality assurance.
fabricated piping,
result in more hours of
inspection budgeted.
Pre-award evaluation Yes More budgeted inspection hours are used when the
of suppliers. supplier is NOT evaluated. This finding adds to
RT308’s previous findings in that both pre-award
and post-award evaluation matter and should be
used in continuous improvement efforts.
When suppliers are not evaluated, organizations are unclear regarding what to expect
and this results in additional budgeted hours to assure quality. Previous RT308’s findings
indicated that companies often do not share potential feedback and ratings because of a
fear of lawsuits (Walsh et al. 2015). Due to adversarial relationships found in the
industry, often times suppliers and contractors do not receive the feedback they might
have to share with one another, and lessons learned are not shared (Caldas et al. 2009).
These attitudes do not help to build respect in these organizations extended networks of
suppliers and it certainly does not help them improve.
Additionally, data on supplier evaluations and NCs are collected but not used to
make decisions on new purchase orders or are erased as new POs are evaluated and
included in companies’ databases (Alves et al. 2016a). Companies could use this data to
develop predictive analytics, based on machine learning algorithms, to inform their
supplier selection processes and improve the quality of products they use in projects.
5 C ONCLUSIONS
This paper presented the results of a study that investigated the link between supplier
ratings and current industry practices, inspection hours, and nonconformances. EPC
companies are missing opportunities to resolve problems as they appear on the shop
floor, understanding the situation, and working to prevent new incidents. Data suggests
that suppliers with low performance ratings and those that subcontract work likely
require more inspection hours when they are awarded a PO due to the likelihood that
more NCs are found at the shop. Instead of working with suppliers ahead of the PO
execution to prevent these problems, organizations are just increasing their inspection
effort, without necessarily addressing the root cause of problems. The number of POs
assessed and the specific material considered limits this study’s conclusions. Future
studies might consider using a larger dataset, impact to safety performance, and different
materials to validate and expand the results presented. Additional analyses of the
current dataset are underway and will be reported by the authors in future publications.
6 A CKNOWLEDGMENTS
The authors are thankful for the financial support provided by the Construction Industry
Institute (CII) and the countless hours of work spent on this project by RT308’s team
members. The paper reflects the opinions of the authors and not those of the CII.
7 R EFERENCES
Alves, T.C.L., Needy, K.L., and Desai, P. (2016a). Achieving Zero Rework through
Effective Supplier Quality Practices. The University of Texas at Austin. Research
Report 308-1?.
Alves, T.C.L., Ravagui, K., Needy, K.L. (2016b). “Supplier Selection in EPC Projects: An
Overview of the Process and its Main Activities.” ASCE, 2016 Construction Research
Congress, University of Puerto Rico, Mayagüez, 10pp.
Caldas, C.H., Gibson Jr, G.E., Weerasooriya, R., and Yohe, A.M. (2009).
“Identification of Effective Management Practices and Technologies for Lessons
Learned Programs in the Construction Industry.” ASCE, J. of Const. Engrg. and
Mgmt., 135(6), 531-539
Construction Industry Institute (1999). Procurement and Materials Management: A
Guide to Effective Project Execution.
Field, A. (2012). Discovering Statistics Using SPSS, 3rd Ed., Sage Publications, London.
Hegwood, A. (2016). Effective Supplier Quality Practices in the Construction Industry.
Undergraduate honors thesis. University of Arkansas, College of Engineering. 59pp.
Kannan, V. R. and Tan, K.C. (2002). “Supplier Selection and Assessment: Their Impact
on Business Performance.” The Journal of Supply Chain Mgmt., Fall 2002, 11-21
Liker, J. (2003). The Toyota Way: 14 Management Principles from the World's Greatest
Manufacturer, McGraw-Hill, New York, 350pp.
Neuman, Y., Alves, T.C.L., Walsh, K.D., and Needy, K.L. (2015) “Quantitative Analysis of
Supplier Quality Surveillance Practices in EPC Projects.” Journal of Construction
Engineering and Management, 141(11).
Monczka, R. M. Handfield, R. B., Giunipero, L. C. and Patterson, J.L. (2016). Purchasing
and Supply Chain Management. 6th ed. 858pp.
Prahinski, C. and Fan, Y. (2007). “Supplier Evaluations: The Role of Communication
Quality.” The Journal of Supply Chain Management, Summer 2007, 16-28
Rother, M. (2010). Toyota Kata. McGraw-Hill, New York, NY, 306pp.
Walsh, K.D., Needy, K.L., Alves, T.C.L., Neuman, Y., can, R.Y., Ahmad, S. (2015).
Achieving Zero Rework through Effective Supplier Quality Practices. Construction
Industry Institute, The University of Texas at Austin. Resesearch Report 308-11.
Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.
Alternative Proxies: