Impact of Supplier Evaluation On Product Quality

Download as pdf or txt
Download as pdf or txt
You are on page 1of 9

See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/318501298

Impact of Supplier Evaluation on Product Quality

Conference Paper · July 2017


DOI: 10.24928/2017/0268

CITATIONS READS

3 1,728

5 authors, including:

Thais Alves Panthil Desai


San Diego State University Atkins Global
76 PUBLICATIONS   792 CITATIONS    1 PUBLICATION   3 CITATIONS   

SEE PROFILE SEE PROFILE

Kim Needy
University of Arkansas
103 PUBLICATIONS   1,430 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

CII RT-308 Achieving Zero Rework through Effective Supplier Quality Practices View project

Achieving Zero Rework through Effective Supplier Evaluation and Selection - Phase II View project

All content following this page was uploaded by Thais Alves on 01 August 2017.

The user has requested enhancement of the downloaded file.


Alves T.C.L., Desai P., Needy K.L., Hegwood A., and Musick, S. (2017). “Impact of Supplier Evaluation on
Product Quality” In: LC3 2017 Volume II – Proceedings of the 25th Annual Conference of the
International Group for Lean Construction (IGLC), Heraklion, Greece, pp. 11–18. DOI:
https://doi.org/10.24928/2017/0268

IMPACT OF SUPPLIER EVALUATION ON PRODUCT


QUALITY
1 2 3 4
Thais da C. L. Alves , Panthil Desai , Kim L. Needy , Ashleigh Hegwood , and Sean
5
Musick

Abstract: Supplier evaluation and engagement are regular topics in the Lean
literature at large. It is well known that Toyota extensively works with suppliers to
develop their work and capabilities, and provides challenges and feedback to
suppliers so that they can achieve the quality requirements defined by the company.
In the construction industry, companies conduct evaluations of suppliers, but
practitioners cite numerous reasons why supplier evaluations are not constantly
shared with suppliers. Moreover, the topic of supplier evaluation and how suppliers’
ratings affect the delivery of products free of defects to construction projects has not
received much attention by the IGLC community. This paper presents results of a
study, which underscores the importance of conducting supplier evaluation as a
means to assure quality products are delivered to construction projects. Findings
show that suppliers with low ratings, or who are not evaluated, are assigned more
hours of inspection, as are suppliers who subcontract portions of their work. This
practice translates not only into additional budgets required to inspect suppliers, but
also additional management-related costs.

Keywords: Lean construction, supplier evaluation, nonconformances, quality.

1 I NTRODUCTION
The topic of supplier evaluation is usually discussed as part of materials management
and procurement evaluations (Construction Industry Institute [CII] 1999), as well as
studies related to risk analysis in construction projects (Monckza et al. 2016). Suppliers
are usually evaluated prior to being hired to deliver a purchase order (PO) through the
use of “formal supplier evaluation systems,” and contractors are more likely to have
these systems than owners (CII 1999). Despite having formal systems to evaluate
suppliers, a recent study by CII RT308 revealed that contractors are evenly split (54.4%
yes vs. 45.6% no) when it comes to the use of prior supplier evaluation to make decisions
about future purchases. Owners who participated in the same study were even less likely
to use prior performance evaluations to base their decisions to award POs (Alves et al.
2016b). This current practice goes against Lean principles that promote the use of
indicators and information to support a transparent management of value streams and
1
Associate Professor, J.R. Filanc Construction Engineering and Management Program, Dept. of Civil,
Constr., and Env. Engineering, San Diego State University, USA, talves@mail.sdsu.edu
2
Graduate Research Assistant, Dept. of Civil, Constr., and Env. Engineering, San Diego State
University, USA, panthildesai@gmail.com
3
Dean, Graduate School and International Education, University of Arkansas, Fayetteville, AR, USA,
kneedy@uark.edu
4
Industrial Engineer, Former University of Arkansas student, Fayetteville, USA,
ashleigh.hegwood@gmail.com
5
Director, Performance Excellence, MEI Technologies, Inc. (MEIT), Houston, TX, USA
seanmusick@gmail.com

11 | Proceedings IGLC | July 2017 | Heraklion, Greece


Impact of Supplier Evaluation on Product Quality

working with suppliers to promote continuous improvement. Supplier ratings are


collected but, unfortunately, not much is done with them. This paper discusses results of
a recent survey with construction companies in the Engineering Procurement and
Construction (EPC) industry. Hypotheses are developed and tested to verify the
relationship between supplier ratings, inspection hours, and final product quality.

2 L ITERATURE R EVIEW
Suppliers play an important role in the literature on Lean practices throughout multiple
phases of a product life-cycle, from the development of suppliers, through product design,
manufacturing, and distribution. Liker's Toyota Way (2003) describes at least three
principles that directly relate to the discussion presented in this paper: Principle 5: Build
a culture of stopping to fix problems, to get quality right at the first time; Principle 17:
Respect your extended network of partners and suppliers by challenging them and
helping them improve; and, Principle 18: Go and See for yourself to thoroughly
understand the situation (Genchi Genbutsu).
Liker's (2003) principles 5 and 18 are also supported by the discussion presented by
Rother (2010, p.181), who stresses the importance of process improvement at Toyota and
how it responds to process abnormalities: the responses to process abnormalities should
be immediate; and, the response to process abnormalities should come from someone
other than the production operators. The former statement underscores the importance
of dealing with problems as soon as they happen so that adjustments can be made before
any targets are missed and before the problems grow bigger. This suggests that a team
leader should be appointed to help teams proactively resolve these problems.
Before becoming a preferred partner at Toyota, suppliers often work closely with the
company for many years before they are ever awarded a PO. Toyota has high
expectations for its suppliers and, during the years before the award of a PO, suppliers
learn about these expectations and how they will work in practice (Liker 2003). The
literature on supplier evaluation shows that suppliers who are frequently evaluated,
receive feedback about their performance, and know what is expected from them
perform better and are more committed to their clients as they can adjust their processes
to match clients’ requirements (Kannan and Tan 2002; Prahinski and Fan 2007).
Moreover, Walsh et al.'s (2015) study in the EPC industry revealed that companies that
communicate more often with suppliers, use supplier ratings, track indicators like cost
and time spent on POs, and use suppliers with quality management systems tend to find
nonconformances (NCs) early in the life of a PO, preventing these NCs from reaching
construction projects.
In addition to these principles that can be used to support a healthy environment to
respect and grow the capabilities of an extended network of suppliers, another principle
outlined by Liker (2003) supports the discussion presented herein: Principle 1: Base your
long-term management decisions on a long-term philosophy, even at the expense of
short-term financial goals. The study presented in this paper illustrates how suppliers
with low ratings are hired at the expense of product quality. This often happens because
of low PO costs offered by these suppliers, and the practice adopted by some companies
to compensate this deficiency is to assign additional inspection hours to a PO instead of
working with suppliers to improve their capabilities.

12 | Proceedings IGLC | July 2017 | Heraklion, Greece


Thais da C. L. Alves, Panthil Desai, Kim L. Needy, Ashleigh Hegwood, and Sean Musick

3 R ESEARCH M ETHOD
This study was developed as part of the extension work of the research team 308 (RT308)
funded by the Construction Industry Institute (CII) from 2015-2016. RT308 membership
included academics (co-authors of this paper) and industry practitioners (owners, EPC
contractors [co-author], and suppliers) referred to subject matter experts (SMEs) in this
paper. As part of the team’s goal to define ways for companies to determine the
fabrication and inspection capabilities of suppliers, the team developed a survey (PO
instrument), which was initially tested within the team, revised, and later distributed to
the CII membership. The PO instrument was developed based on RT308’s previous work
(Walsh et al. 2015) and adapted to address one single material: shop fabricated piping.
The SMEs suggested that the team should be able to collect data on this product which is
present in virtually any project they developed, and in future efforts the PO instrument
could be adapted to collect data for other materials so that fabrication and inspection
capabilities can be defined for them as well.
The PO instrument, fully available at Alves et al. (2016b), contained 28 questions
organized into four main parts: contact data from the respondent (to clarify any
questions about the responses) and project demographics (name, type, dollar value, role
of the respondent); basic data about the single PO being used in each survey (reference
number, number of spools dollar value of the PO, inspection hours budgeted, location of
supplier facility, and criticality level of the PO); pre-award evaluation (level of inspection
assigned, existence of supplier evaluation/reasons for doing or not doing so, pre-award
supplier evaluation using criteria discussed later in this paper); and post-execution
evaluation (inspection hours utilized, existence of subcontracting, final level of
inspection, comparison between planned vs. actual hours of inspection, total number of
pipe spools received for the PO, number of unplanned quality events, post –award
supplier evaluation using criteria discussed later in this paper). The institutional review
boards (IRBs) at San Diego State University and the University of Arkansas, Fayetteville
approved the survey distribution and handling of data obtained, which also complied
with CII guidelines for data collection and analysis.
3.1 Hypotheses Testing
RT308 developed the hypotheses outlined in Table 1 to investigate the relationship
between certain practices and characteristics of the PO indicated by respondents. Only
hypotheses related to supplier ratings and product quality are discussed in this paper. A
full discussion of all hypotheses tested can be found in Alves et al. (2016b).
The Mann-Whitney (M-W) test, also known as Wilcox-Mann-Whitney test, was
used to evaluate the hypotheses given that the data was not normally distributed. The M-
W test is considered a statistically powerful test among all non-parametric tests (Field
2012).
Data entries submitted through Qualtrics (an online survey platform) were reviewed
and questions were cleared with respondents. The analysis was developed using IBM
SPSS and data was organized and analyzed using Field's (2012) recommendations. The
questions used to conduct the analysis are also shown in Table 1.

13 | Proceedings IGLC | July 2017 | Heraklion, Greece


Impact of Supplier Evaluation on Product Quality

Table 1: Hypotheses related to supplier evaluation and number of NCs

Hypothesis Source of date in new PO instrument

Hypothesis 2: A supplier with a higher Q-10 Inspection hours budgeted; Q-24


rating will have a higher Pfab (fabrication Post-execution evaluation.
capability) and fewer inspection hours
budgeted.
Hypothesis 4: More sub-suppliers Q-21 Supplier subcontracting a portion of
associated with a PO (Q-21) leads to a the work for PO; Q-19 Inspection hours
lower Pfab and more inspection hours utilized
Hypothesis 6: Higher supplier ratings Q-24 Post-execution evaluation; Q-27
(≥4) should result in lower number of Number of unplanned quality events at
NCs at shop. the shop.
Hypothesis 8: The number of inspection Q-10 Inspection hours budgeted; Q-
hours will be higher on POs for which 14 Did you perform a supplier
the supplier was not evaluated. evaluation before the PO was issued?
Hypothesis 9: More inspection hours are Q-19 Inspection hours utilized; Q-24 Post-
spent when the supplier’s rating is low. execution evaluation.

3.2 Supplier Evaluation Criteria


After an analysis of supplier evaluation procedures provided by RT308 members, a list of
items evaluated was compiled and presented for discussion in one of the team’s meetings.
The subject matter experts (SMEs) categorized the items identified into nine categories,
which were later used in the PO instrument to assess supplier pre- and post-execution on
a Likert scale from 1 to 5 (Alves et al. 2016a):
• Plant operations / Building and infrastructure – storage and shipping, structure
for QA/QC personnel and records, examination and testing, housekeeping,
security.
• Manufacturing capability – machining capability, fabrication capability
(including welding, coatings, etc.), capability to manufacture per user
requirements, calibration records and procedures, non-destructive testing and
inspection capability and quality, shop capacity.
• Experience and qualifications – geographical areas in which supplier is qualified
to work, familiarity with codes specific to certain geographic regions,
certifications (ASME, API, etc.), work history.
• Workforce – documented training of the workforce, craft types available at
location, qualifications of the workforce, source of backup workers.
• Engineering capability – professional/technical specialties held in house,
compliance with documents and specifications, cooperation, responsiveness of
requests, and turnaround documents, robust document management system.
• Material control – QA/QC manual for material control, process for material
substitution, material verification, prevention of counterfeit materials, material
certification.
• Adherence to procedures and standards – adherence to codes and specifications,
quality, legibility and completeness of documentation, non-conformance control,

14 | Proceedings IGLC | July 2017 | Heraklion, Greece


Thais da C. L. Alves, Panthil Desai, Kim L. Needy, Ashleigh Hegwood, and Sean Musick

proper notification of inspection and hold points, Identification of parts –


traceability.
• Safety adherence and record – written safety, health and environmental program,
compliance to local/governmental safety and environmental requirements,
experience modification Ratio (EMR), total recordable incidence rate (TRIR).
• Handling of subcontractors/sub-suppliers – outsourced/subcontracted
product/service control, procedure to check subcontractor compliance with
quality requirements, documented evidence of compliance, auditing
subcontractors, ability to schedule and expedite subcontractors/sub-suppliers.

4 A NALYSIS OF R ESULTS
A total of 41 POs were received by RT308, however, not all submissions had answers to
all questions, e.g., number of nonconformances at the site were not indicated, pre-award
evaluation not available. This resulted in different numbers of POs being used to
evaluate different questions. Only a few responses regarding pre-award supplier
evaluation were submitted, however, most respondents had post-execution evaluations,
which were used to analyze the hypotheses discussed in this paper. The team had
originally hoped to evaluate differences between pre-award and post-execution, but due
to the low number of data points for pre-award that was not possible. During both
phases of RT308’s project, respondents to both PO surveys indicated multiple reasons for
not evaluating suppliers (Alves et al. 2016a, b) including: supplier had been evaluated by
owner of the project, evaluations had been updated with post-execution, existence of
approved list of suppliers did not require additional evaluation, the client had issued the
PO, and previous experience with the supplier.
4.1 Demographics
The data came from 10 companies representing 37 different projects. Out of that sample,
owner companies submitted 10 POs (24%) and contractors submitted 31 (76%). One
single company submitted 15 POs, and the authors understand that this is a limitation of
this data set. The POs dollar values ranged from $255,000 to $50,660,888. The total dollar
value of all submitted POs was $281,091,967. About 63% of the POs submitted had a
dollar value between $1 million and $10 million, 20% were between $100,000 and $1
million, and 17% of the POs were over $10 million. The minimum number of spools in a
single PO was 16 spools and the maximum number was 37,500 spools.
Among the 41 POs, about 90% of the POs were delivered by suppliers located in
developed countries and 10% by suppliers in developing countries. For confidentiality
reasons, the instrument did not collect data about the names of specific suppliers, only
the location of the facility that manufactured the order was recorded. However,
respondents indicated 19 specific cities, some of them appearing more than once, and
others indicated only the country where the supplier’s facility was located (e.g., USA).
Additionally, in 32% of the 41 POs, the supplier had subcontracted a portion of the PO.
Data reveal that subcontracting is a common practice even for commodities such as shop
fabricated piping. The analysis of the POs submitted revealed that 47.5% of the POs
ended up with an inspection level higher than originally budgeted, 35% had a lower
inspection level at the end, and only 17.5% had the inspection level to be equal to the
budgeted level. The analysis revealed that inspection work was under planned for 14% of
the POs submitted.

15 | Proceedings IGLC | July 2017 | Heraklion, Greece


Impact of Supplier Evaluation on Product Quality

4.2 Supplier rating cut-off: high vs. low ratings


Once the team started the analysis of the PO instrument data, a cut-off value of the 5-
point Likert scale had to be selected to categorize high versus low supplier evaluations.
At first, the cut-off value of 4 was chosen for high rating, and suppliers with a rating
equal or higher than 4 would be considered highly rated, whereas anything below that
would be treated as low rating. To confirm this decision, a sensitivity analysis was
carried out to investigate if a different cut-off number should be used in the analysis.
The analysis used the supplier ratings for post-execution evaluation and the number
of NCs found for a same PO. Cut-off values of 3, 3.5, 4, and 4.5 were evaluated and
outliers were addressed. To see this detailed sensitivity analysis refer to Hegwood (2016).
The analysis revealed that moving the cut-off to a supplier rating of 3 or 3.5 would allow
too many data points where the specific PO had a high number of NCs associated with
the product. Conversely, a cut-off rating of 4.5 left only a few suppliers with a low
number of NCs. The lower the cut-off rate, the higher the number of discrepancies
linking the numbers of NCs and the supplier ratings, and the higher the number of
suppliers who would receive a good rating with a high number of NCs when compared
to other suppliers in the sample. Finally, the analysis suggested that the cut-off rate of 4
was appropriate as suppliers with ratings of 4 and above did not consistently display a
high number of NCs when compared to suppliers getting lower scores.
4.3 Analysis of Hypotheses
Table 2 shows the findings related to the hypotheses testing conducted by the team.
Results are organized to indicate what matters when supplier ratings are considered. It is
worth noting that the data used in this discussion was not normalized by the cost of the
PO or the number of spools in a PO. When the data is normalized, results change in
some cases (that is, results might not be significant). Results were discussed with the
team’s SMEs and the non-normalized numbers presented herein where considered
representative for this analysis. A full analysis can be found at Alves et al. (2016b).
The testing of Hypothesis 2 (a supplier with a higher rating will have a higher Pfab
(fabrication capability) and fewer inspection hours budgeted) indicated that more
inspection hours are budgeted when supplier ratings are low, that is lower than 4
(p=0.001, highly significant). Similarly, Hypothesis 4 (more sub-suppliers associated with
a PO leads to lower Pfab and more inspection hours) was also confirmed (p=0.01,
significant) as more inspection hours are used when portions of a PO are subcontracted.
Hypothesis 6 (higher supplier ratings should results in lower number of NCs at the
shop) was confirmed as well (p=0.000, highly significant) as more NCs are found at the
shop when the supplier rating, as measured using the criteria presented above, is low
(below 4). Hypothesis 8 (the number of inspection hours will be higher on POs for which
the supplier was not evaluated) was confirmed (p=0.018) in that more inspection hours
are budgeted when the supplier is not evaluated. Finally, Hypothesis 9 (more inspection
hours are spent when the supplier’s rating is low) was confirmed (p=0.000) as the
analysis revealed that more inspection hours are spent when the supplier’s rating is low
(below 4).
The analysis presented underscores the importance of selecting suppliers with good
evaluations, and, more importantly, developing suppliers to achieve higher ratings and
meeting clients’ demands and expectations. The analysis developed with the support of
the team’s SMEs indicated that suppliers with low ratings usually have the lowest cost
and are perceived by procurement professionals to provide the best deal for their projects.

16 | Proceedings IGLC | July 2017 | Heraklion, Greece


Thais da C. L. Alves, Panthil Desai, Kim L. Needy, Ashleigh Hegwood, and Sean Musick

This goes against the idea exposed in The Toyota Way’s Principle 1: Base your long-
term management decisions on a long-term philosophy, even at the expense of short-
term financial goals (Liker 2003). Also, suppliers who subcontract parts of their POs
might display a low price tag at first, but costs add up when problems arise with their
extended supply chain. The problem is, according to RT308’s SMEs, they do not always
know about subcontracting practices ahead of time.

Table 2: Findings RT308’s project extension and practical implications

Description Does it make How? Why? When?


a difference?

Suppliers with low Yes More inspection hours budgeted and more NCs are
ratings in evaluations. found at the shop. This suggests that additional
money ends up being spent to assure quality when
these suppliers are selected.
Subcontracting, even Yes During the pre-award evaluations, subcontracting
for portions of a should be factored in as something that will increase
commodity like shop the costs of surveillance and quality assurance.
fabricated piping,
result in more hours of
inspection budgeted.
Pre-award evaluation Yes More budgeted inspection hours are used when the
of suppliers. supplier is NOT evaluated. This finding adds to
RT308’s previous findings in that both pre-award
and post-award evaluation matter and should be
used in continuous improvement efforts.

When suppliers are not evaluated, organizations are unclear regarding what to expect
and this results in additional budgeted hours to assure quality. Previous RT308’s findings
indicated that companies often do not share potential feedback and ratings because of a
fear of lawsuits (Walsh et al. 2015). Due to adversarial relationships found in the
industry, often times suppliers and contractors do not receive the feedback they might
have to share with one another, and lessons learned are not shared (Caldas et al. 2009).
These attitudes do not help to build respect in these organizations extended networks of
suppliers and it certainly does not help them improve.
Additionally, data on supplier evaluations and NCs are collected but not used to
make decisions on new purchase orders or are erased as new POs are evaluated and
included in companies’ databases (Alves et al. 2016a). Companies could use this data to
develop predictive analytics, based on machine learning algorithms, to inform their
supplier selection processes and improve the quality of products they use in projects.

5 C ONCLUSIONS
This paper presented the results of a study that investigated the link between supplier
ratings and current industry practices, inspection hours, and nonconformances. EPC
companies are missing opportunities to resolve problems as they appear on the shop
floor, understanding the situation, and working to prevent new incidents. Data suggests
that suppliers with low performance ratings and those that subcontract work likely
require more inspection hours when they are awarded a PO due to the likelihood that

17 | Proceedings IGLC | July 2017 | Heraklion, Greece


Impact of Supplier Evaluation on Product Quality

more NCs are found at the shop. Instead of working with suppliers ahead of the PO
execution to prevent these problems, organizations are just increasing their inspection
effort, without necessarily addressing the root cause of problems. The number of POs
assessed and the specific material considered limits this study’s conclusions. Future
studies might consider using a larger dataset, impact to safety performance, and different
materials to validate and expand the results presented. Additional analyses of the
current dataset are underway and will be reported by the authors in future publications.

6 A CKNOWLEDGMENTS
The authors are thankful for the financial support provided by the Construction Industry
Institute (CII) and the countless hours of work spent on this project by RT308’s team
members. The paper reflects the opinions of the authors and not those of the CII.

7 R EFERENCES
Alves, T.C.L., Needy, K.L., and Desai, P. (2016a). Achieving Zero Rework through
Effective Supplier Quality Practices. The University of Texas at Austin. Research
Report 308-1?.
Alves, T.C.L., Ravagui, K., Needy, K.L. (2016b). “Supplier Selection in EPC Projects: An
Overview of the Process and its Main Activities.” ASCE, 2016 Construction Research
Congress, University of Puerto Rico, Mayagüez, 10pp.
Caldas, C.H., Gibson Jr, G.E., Weerasooriya, R., and Yohe, A.M. (2009).
“Identification of Effective Management Practices and Technologies for Lessons
Learned Programs in the Construction Industry.” ASCE, J. of Const. Engrg. and
Mgmt., 135(6), 531-539
Construction Industry Institute (1999). Procurement and Materials Management: A
Guide to Effective Project Execution.
Field, A. (2012). Discovering Statistics Using SPSS, 3rd Ed., Sage Publications, London.
Hegwood, A. (2016). Effective Supplier Quality Practices in the Construction Industry.
Undergraduate honors thesis. University of Arkansas, College of Engineering. 59pp.
Kannan, V. R. and Tan, K.C. (2002). “Supplier Selection and Assessment: Their Impact
on Business Performance.” The Journal of Supply Chain Mgmt., Fall 2002, 11-21
Liker, J. (2003). The Toyota Way: 14 Management Principles from the World's Greatest
Manufacturer, McGraw-Hill, New York, 350pp.
Neuman, Y., Alves, T.C.L., Walsh, K.D., and Needy, K.L. (2015) “Quantitative Analysis of
Supplier Quality Surveillance Practices in EPC Projects.” Journal of Construction
Engineering and Management, 141(11).
Monczka, R. M. Handfield, R. B., Giunipero, L. C. and Patterson, J.L. (2016). Purchasing
and Supply Chain Management. 6th ed. 858pp.
Prahinski, C. and Fan, Y. (2007). “Supplier Evaluations: The Role of Communication
Quality.” The Journal of Supply Chain Management, Summer 2007, 16-28
Rother, M. (2010). Toyota Kata. McGraw-Hill, New York, NY, 306pp.
Walsh, K.D., Needy, K.L., Alves, T.C.L., Neuman, Y., can, R.Y., Ahmad, S. (2015).
Achieving Zero Rework through Effective Supplier Quality Practices. Construction
Industry Institute, The University of Texas at Austin. Resesearch Report 308-11.

18 | Proceedings IGLC | July 2017 | Heraklion, Greece

View publication stats

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy