2011 NDIA GROUND VEHICLE SYSTEMS ENGINEERING AND TECHNOLOGY

SYMPOSIUM
SYSTEMS ENGINEERING (SE) MINI-SYMPOSIUM
AUGUST 9-11, 2011 DEARBORN, MICHIGAN

PRODUCIBILITY AND CONFIDENCE INDICES DURING DEFENSE

ACQUISITION

James R. Hadley and

Daniel J. McCarthy
Munro and Associates, Inc.
Troy, MI

ABSTRACT

Given the complex nature of systems today, systems engineering’s primary focus is typically consumed with
optimizing function and performance. This condition often causes producibility and cost to become an after-thought,
leading to late, over budget production. Therefore an objective and relevant method is required to provide real-time
feedback to system engineers relative to producibility and confidence that facilitates better systems design and
programmatic decisions.

This paper will discuss the use of producibility model metrics to score several key design elements for the creation
of a single standardized producibility index (PI) to encourage engineers to improve their designs for producibility
earlier in the development life-cycle. Additionally monitoring certain analysis activities to gauge the level of
accuracy in the producibility model will provide metrics to create a single standardized producibility confidence
index (PCI) that can be used to mitigate risk in programmatic decision making. Lastly, the On-Board Vehicle Power
(OBVP) system will be used to demonstrate the PI and PCI.

INTRODUCTION need for block upgrades to provide capabilities and

readiness that were initially expected. The NDIA on
Ever evolving needs of the warfighter require defense multiple occasions has referenced this issue of
product development programs to be delivered on limited focus on producibility in the product
time and on budget. GAO studies in 2010 stated that development process and the need for a standardized
current practices do not adequately identify and producibility processes and metrics to improve
objectively quantify producibility and cost issues in design activities (2; 3).
the early phases of development (1). These
oversights lead to excessive delays, cost overruns, Over the last few decades corporations have begun
low operational availability and unacceptable establishing product development processes to
sustainment costs. This trend must be reversed in address producibility concerns. These development
order for the warfighter to benefit from more processes create design guidelines that provide a
advanced technology, faster, at lower costs. One way standard set of activities to follow in order to
to make this possible is to establish a metric to guide optimize producibility. These process definitions
the generation of more elegant and producible delineate what activities should be done, but it is
defense product designs that will improve soldier uncommon that there is a standardized metric
effectiveness and reduce logistics lifecycle costs associated with these activities to qualify how
through identification of issues at the early stages. optimized the design has become as result of the
This early identification of issues will reduce the activities. A study conducted at the Defense Systems

Page 1 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Management College (DSMC) by Commander David index is an especially important development for
Brown, U.S. Navy, has shown that the use of a design evaluation indices, as it expands the score’s
producibility index can improve product design (4). focus to the overall system design. The index does
This study showed a significant increase in this by using communication between components to
optimization of simple designs due to the use a determine the general complexity of the system.
simple producibility metric. One may extrapolate that However, the issue with this index is that even
a more comprehensive producibility index could be though it does look at the overall system design, the
applied to more complex development programs to score does not focus well on the producibility issues
drive optimization in their designs. of the design. Recently this concept of a complexity
index has been expanded upon by DARPA’s Meta
Several different producibility indices have been Project (9). The expanded complexity index does add
generated focusing on different key characteristics of terms focusing on part count and part interactions, as
the product development process. One of these a component of the overall complexity of the system.
indices is the Boothroyd and Dewhurst design However, in the opinion of our team these terms still
efficiency index (5). This index uses the Boothroyd provide a rather limited focus of producibility in the
Dewhurst Design for Assembly (DFA) scoring design evaluation of this index. Another approach to
system to determine the theoretical current assembly a design evaluation is the creation of a full
time and optimal assembly time for a design. The producibility model for the product design using
ratio of optimal to actual is used as the design commercially available software like Design Profit®.
efficiency index. The importance of this index is that Design Profit® creates a graphical map of the product
it serves as a goal for engineers to improve their structure based on the assembly of the product. This
design to reach the optimal assembly time for their model includes assembly time, labor costs, part costs,
product. The next progression of producibility indices quality costs, etc. for the creation of a total accounted
was completed by John Priest and Jose Sanchez in cost of the product based on the design (10).
their article An Empirical Methodology for However, while this model creates several metrics
Measuring Producibility Early in Product that are indicators of producibility, such as part
Development (6). Their definition of a producibility count, assembly score, quality costs, etc., and it does
index is simply defined as a summation of the key not provide direct index values to show a definitive
evaluation categories, consisting of each category’s increase in the overall producibility of the product’s
difficulty value multiplied by a weighting factor for design. For this reason the producibility methodology
emphasizing the impact of the category. This method of Munro & Associates, Inc. is often combined with
was somewhat limited as it did not directly the producibility model. The producibility model
standardize what design categories should always be collects the raw producibility metrics of a design, but
evaluated and it only focused on the fabrication of a it is the application of the Munro method that refines
single part. The method was expanded by a team of these raw metrics into a comprehensive producibility
system engineers at the Florida International analysis to provide detailed direction to future
University, redefining the producibility index as a improvement activities to optimize the product
summation of four standard design focus areas (7). design and manufacturing process. The Munro
These four areas included stock material selection, method does this by first identifying all the
part size, part weight, and cumulative form feature inefficiency present in the current design and process.
effects. However, this index still has the limitation of Once this inefficiency is indentified the method
being focused on the fabrication of a single machined generates redesigns that quantify exactly what parts
part. In contrast another approach to a design and processes can be eliminated, what will be
evaluation index is the complexity index created by replacing them, and how much this improves
Jones, Hardin, and Irvine in their article, Simple producibility and reduces cost of the overall product
Parametric Model for Estimating Development (11).
(RDT&E) Cost on Large-Scale Systems (8). This

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 2 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

The first goal of this paper is to build upon the Material Solution Analysis (MSA) phase (12). The PI
aforementioned research to create a new producibility creates a standardized method for this assessment and
index that provides a comprehensive and provides the capability to initiate assessments earlier
standardized producibility score for the overall during the MSA phase. In order to be comprehensive
system design that can be used throughout the in the analysis of producibility, but maintain
product development process. The index’s math simplicity, four main design focus categories have
structure will build off the techniques pioneered in been determined for the PI.
the previous adaptations of producibility indices. The
Munro methodology will provide an established Main Design Categories
producibility perspective and the theory required for
1. Architecture Elegance
this index. Using the Munro method as a basis for the
2. Value Optimization
producibility index allows for direct use of the
3. Assembly Elegance
metrics generated in a producibility model of a
4. Quality Improvement
product. This capability will then allow for automatic
calculation of the new producibility index in the Each of the main design categories serves as a tool to
model as the analysis of the design is completed. focus engineers to key areas of the design known to
be leading factors for the development of a
The second goal of this paper is to build on the
producible product. All of the categories further
5000.02 and MRL standards to create a producibility
define their focus by the use of two to four sub-
confidence index that provides a risk assessment of
categories allowing for the breakdown of a score to
the knowledge being used to develop the
facilitate problem resolution in the design. This
producibility index score. The MRL process has
allows the index to be used as a metric to monitor
pioneered the establishment of standardized guidance
larger trends of the product design, as well as
to drive a focus of producibility and
determining what specific assemblies or components
manufacturability into the 5000.02 process (12).
are creating the problems and why. Architecture
Specifically the MRL process provides a clear
Elegance scores the interaction of major assemblies
understanding of the exit criteria needed in each
in the product design through two sub-categories,
phase of the design process, directing engineers to
subassembly interaction and nesting structure of
focus on nine key threads to drive producibility.
subassemblies. Value Optimization scores the design
Therefore, since the MRL process has established the
of components and their assemblies through two sub-
importance of the use of this knowledge to create a
categories, system design and fastener/connector
more producible design, these design threads serve as
usage. Assembly Elegance scores the assembly of the
a foundation for the producibility confidence index
product through four subcategories, simplicity of the
developed in this paper.
overall assembly process, assembly ease of individual
parts and subassemblies, and dwell time. Quality
Improvement scores the quality of the assembly
PRODUCIBILITY INDEX THEORY process through two sub-categories, cost of quality
and variation control.
The producibility index (PI) provides a single
standardized score for the evaluation of the The score generated for each of the sub-categories is
producibility of a design using metrics already a percentage value indicating the relative
developed in a producibility model. The PI can be optimization of the design in that specific area. To
broken into several sub-scores of key characteristics assign a percentage increase in the producibility
and tracked through the producibility model in order value for each sub-category a reference is needed for
to better focus engineering effort on critical issues in the calculations. For some sub-categories the design
the product. The DOD MRL process indicates that goal is to eliminate the value being tracked, and
producibility assessments should begin during the therefore the calculation can be set so the score

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 3 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

increases to 100%, as the limit of the tracked value Value Optimization

goes to zero. However, for other sub-category scores
a standard design goal is required to serve as a The first sub-category in Value Optimization assigns
numeric reference for the calculation. It is understood a design score to the optimization of non-fastener and
that design goals for each sub-category will vary non-connector parts in the product design. Good parts
based on product industry and volume levels, but are those parts which are absolutely necessary to
until a full sensitivity analysis can be established to deliver the functional requirements of the customer.
correlate these factors, our team will provide general A part can be considered a good part if it satisfies one
standard design goals. These standard design goals of two key characteristics. The first key characteristic
are taken from best-in-class designs across multiple is that the part must fundamentally be a different
industries observed during Munro & Associates’ 25 material than the rest of the product in order to
year history in product development. This section achieve the customer requirements for the product’s
will discuss the theory for PI scoring, whereas the function. The second key characteristic is that the
math structure for the PI equation will be established part must move relative to the rest of the product in
in a later section. order to achieve the customer requirements for the
product’s function. The theory for the creation of
Architecture Elegance these key characteristics is a refinement of the value
analysis theory discussed by Miles and Gage (13).
The first sub-category in Architecture Elegance Therefore the perfect design would only consist of
assigns a design score to underutilization of good parts providing exactly the functionality needed
subassemblies. Subassemblies can be productive to fulfill the customer’s need. However, the realities
assembly design tools, as they allow for independent of a design often cause the generation of additional
build-up and testing of several components that can components to mount or interface good parts.
be later installed in the overall product. However, an Therefore the use of three non-fastener/connector
issue with subassemblies is that they generate waste parts per good part, including the good part, serves as
in the design by requiring additional features, parts, a good standard design goal.
and assembly processes to allow for the interface of
the subassembly with the overall product assembly. The second sub-category in Value Optimization
Therefore in the producibility index a goal will be set assigns a design score to the minimization of
for the minimal number of parts required in a fasteners and fluid or electrical connectors in the
subassembly to justify the cost associated with the product design. It is important to eliminate fasteners
interface features and components. The use of a and connectors in a design, as they are a source of
minimum of 25 parts per subassembly serves as a inherent waste and poor quality in the product. While
good standard goal to guide designs. fastener piece cost may be relatively low, each
fastener added to the design increases assembly
The second sub-category in Architecture Elegance complexity variation and fabrication labor costs.
assigns a design score to the optimization of the Additionally, fastened joints are typically the top
assembly hierarchy structure. As more subassembly driver of mechanical quality issues. Lastly, in order
levels are added to the product, added delays and to even use a fastener in a design it will require an
handling issues are incurred due to dependency on engineering analysis of loads on the fastener and the
lower level builds. This typically results in added parts to which it interacts. This engineering analysis
facility costs as well as requiring expanded supply time could be better used to design features into the
chain logistics. In order to minimize this added build interacting parts, allowing for direct mounting of the
complexity, a design goal will be set for the optimal components without the need of additional fastener
number of nested assembly levels. The use of 3 parts or ideally combining the fastened parts into a
subassembly levels as a maximum serves as a good single integrated part. Furthermore, in fluid or
standard goal for designs. electrical systems, interface issues created from

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 4 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

connectors for related tubes or cables play a high role or subassembly is handled more than once in the
in manufacturing and test issues and more assembly of the product. Once again the perfect
importantly, operational availability. Additionally, a assembly process should only include placement of
significant source of quality issues related to fluid or parts to assemble the product. Therefore if a part is
electrical system design often resides at connection required to be touched multiple times, assembly labor
points. Therefore if the system is designed to remove time is being wasted. Whenever the operator or the
connectors it will increase reliability and part is required to be manipulated, there is waste in
producibility of the system. the assembly process. Every change in direction of
insertion (CDI) in the assembly process requires
Assembly Elegance labor or machine time to move the tool or part into
position. Designs which minimize these
The first sub-category in Assembly Elegance assigns
reorientations are commonly referred to as “top-down
a design score to the optimization of the overall
assemblies” where all parts are added strait down
assembly process in terms of assembly steps. The
from above. Top down assemblies tend to be
perfect assembly process would only require the
extremely elegant designs, with minimal waste.
placement of parts to assemble the product. However,
Efforts to achieve “top-down assemblies” will have
the realities of a design often cause additional
profound impact on the design elegance by leading
assembly steps in the process to attach components.
the team to suggest new ideas to eliminate parts and
To drive optimization the use of nine assembly steps
secondary operations. Additionally if the part
per good part, including addition of parts, serves as a
requires a manipulation (flip, rotate, etc.) this
good standard design goal. Specifically nine
requires labor or machine time and possible tooling
assembly steps are chosen for the standard design
costs. This time and cost could be more effectively
goal because another standard design indicates three
used to assemble the subassembly faster and less
parts are allowed per good part, and each part is
costly if the subassembly was designed to assembly
allowed to have three associated operations.
from the same direction. Therefore if the system is
The second sub-category in Assembly Elegance designed to remove manipulations in the process it
assigns an ease of handling and placement score will increase producibility of the system.
based on the assembly scores of each subassembly,
The fourth sub-category in Assembly Elegance
part, pre-processed part, and multi-touch. The
assigns a design score to the minimization of dwell
assembly score is the degree of difficulty assembling
time in the assembly process. It is important to
parts and subassemblies often referred to as the
eliminate dwell time in an assembly process as it is a
design for assembly (DFA) or Munro score (10). For
source of inherent waste in the product. Operations
the course of this producibility analysis we suggest
with dwell time add cost by adding time to the
the use of the Munro method for calculating
process, and by requiring an inventory of parts in
assembly score. In the Munro method two seconds
process. Additionally if a quality concern does
per part is the absolute optimum design, however,
develop, it can require the scrapping of an entire
three seconds is a reasonable goal. As the time per
batch, causing a large cost hit due to the loss of
part increases it is an indicator of how sub-optimally
inventory and time.
the part is designed, as it takes longer to collect,
orient, and mount the part. Therefore the use of an Quality Improvement
assembly score of three seconds serves as a good
standard design goal for each subassembly, part, pre- The first sub-category in Quality Improvement
processed part, and multi-touch. assigns a design score to the minimization of Quality
Burden (Q-burden). Q-burden is an existing metric
The third sub-category in Assembly Elegance assigns based on issue occurrence rate and cost of the
a design score to the minimization of manipulations occurrence that refers to the cost each completed unit
in the assembly process. A multi-touch is when a part bears to account for scrap, rework, and warrantee. In

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 5 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

an ideal product Q-burden should not be a significant The producibility confidence index (PCI) provides a
contributor to the total cost. Therefore it is important single standardized score evaluating the amount of
to minimize Q-burden by eliminating quality issues at production knowledge used in the design and the
their source, the design, as much as possible. related confidence in the assessment. This metric acts
as a companion evaluation to the PI, as it provides a
The second sub-category in Quality Improvement risk assessment of uncertainty associated with the
assigns a design score to the minimization of design producibility stated by the PI. In order to be
inspection operations. The perfect design requires no comprehensive in the tracking of knowledge used in
inspections and has no poka yoke issues, since each the PI, but maintain simplicity, four main design
part has been designed such that it is not possible to focus categories have been determined for the PCI.
assemble the product incorrectly. Excessive
inspections indicate uncontrolled processes that Main Knowledge Scoring Fields
require constant checking to ensure the product is
properly assembled. Additionally, excessive 1. Specification Capture
inspections lead to the generation of wasted cost, as 2. Assembly Knowledge
the operation requires a significant amount of time to 3. Part Knowledge
complete and often requires expensive fixtures and 4. Infrastructure Knowledge
equipment. However, even with the creation of these
Each of the main knowledge categories serves as a
inspection tools, escaping defects are a well
tool to direct engineers to key analyses that need to
established issue. Often subjective criteria contribute
be completed in order to have a more accurate
to this phenomenon and drive significant delays to
producibility index. This allows the index to be used
achieve problem resolution consensus. It is important
as a quick reference to monitor overall completeness
to note that often not all inspection can be removed
of the producibility analysis, as well as determining
from an assembly process as some will still be
what specific assemblies or components still need to
required for validation/functional testing of the final
be designed and analyzed. This section will discuss
assembly and occasional multi-subassembly modules.
the theory for PCI scoring, whereas the math
Poka yoke issues are included in this term as they are
structure for the PCI equation will be established in a
another source of unnecessary inspections in the
later section.
assembly process due to improper part design for the
product assembly. When a poka yoke issue is present Specification Capture
in a part it will require the operator to do an informal
inspection every time that part is placed resulting in The Specification Capture category of the PCI
an addition of excessive time to the assembly assigns a knowledge score associated to
process. Additionally due to the nature of poka yoke specifications captured in the design. This category
issues they could cause the part to be assembled evaluates the completeness of the overall design
incorrectly in which case it drives more waste in the structure to ensure the development team has
assembly process by requiring scrapping or successfully provided for all of the customer
reworking the associated parts. Therefore if the specification requirements. If some of the
system is designed with zero poka yoke issues and specification requirements have not been captured,
the processes are controlled to eliminate inspection then this indicates the design is not complete and will
then producibility of the system will increase. require the addition of subassemblies and
components to complete the design.

Assembly Knowledge
PRODUCIBILITY CONFIDENCE INDEX
THEORY The Assembly Knowledge category of the PCI
assigns a knowledge score associated to assembly
process knowledge captured in the design. This

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 6 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

category evaluates the completeness of the assembly,

to ensure the development team has successfully
analyzed the build process for the assembly of all PRODUCIBILITY INDEX EQUATION
parts in the final product. If some of the
The equation for the producibility index is the
subassemblies have not been analyzed (and are not
weighted average of the ten sub-category design
COTS), then this indicates the design is not complete
scores in the format shown below:
and will require the addition of assembly steps and
components to complete the design. 𝑁
1
𝑃𝐼 = �(𝑊𝑖 × 𝑃𝑖 )
Part Knowledge ∑𝑁
𝑖=1 𝑊𝑖 𝑖=1

The Part Knowledge category of the PCI assigns a Each sub-category design score (𝑃𝑖 ) is calculated
knowledge score associated to subassembly and part using a ratio of the key design factors, that will be
knowledge captured in the design. This category explained in the following section. The sub-category
evaluates the completeness of the assembly score for scores are based on a 0 to 1 ranking system for easy
each subassembly and part, to ensure the comparison between sub-categories. To accomplish
development team has successfully analyzed the this 0 to 1 rank, the standard design goal for the sub-
handling or re-handling of each subassembly and part category discussed previously is often used. The
in the final product. If some of the subassemblies, calculation the producibility score is completed for
pre-processed parts, parts, or multi-touches have not each of the 10 sub-categories, defining N as equal to
been analyzed, then this indicates the design is not 10. Associated with each of the sub-category scores
complete and will require the additional analysis of is a coefficient (𝑊𝑖 ) that serves as a priority ranking
these components to complete the design. system for each category. These priority ranks are
Additionally completing this analysis often results in established to direct more development activity in
finding added complexity issues that will need to be higher value added areas of each design category. For
resolved in the design. This allows time for the example in the Value Optimization category it is
generation of several design solutions early in more important to integrate several non-fastener
development, as opposed to limited time forcing the components into one, as opposed to eliminating
generation of only one solution due to late discovery several fasteners, as there will be higher fabrication
of the problem. Then since these solutions can be issues, engineering development required, and cost
implemented early in the product development associated with non-fastener components. Therefore
process the solution is more likely to have a to provide priority to the term scoring optimization of
significant positive impact on producibility. non-fastener/connector parts, it is considered the
primary term for the category and the term scoring
Infrastructure Knowledge
minimization of fasteners/connectors is considered
The Infrastructure Knowledge category of the PCI the secondary term. The primary term in each
assigns a knowledge score associated to category will have its score multiplied by the
infrastructural knowledge captured in the design coefficient 𝑊𝑖 equal to one. The secondary term in
through the MRL process. This category evaluates each category will have its score multiplied by the
the completeness of the infrastructure analysis, to coefficient 𝑊𝑖 equal to one half. The auxiliary term in
ensure the development team has successfully each category will have its score multiplied by the
analyzed infrastructure required for the assembly of coefficient 𝑊𝑖 equal to one fourth.
all key critical components in the final product. A
Architecture Elegance
low MRL score indicates the infrastructure analysis
needed for the design is not complete and will require The primary sub-category in Architecture Elegance
completion of the MRL process. assigns a design score to underutilization of
subassemblies. As this is the primary term for this

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 7 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

sub-category, the priority coefficient 𝑊1 is equal to design in this area. This equation for scoring system
one. Utilizing the standard design goal of a minimum design relative to the reduction of secondary parts is
of 25 parts per subassembly needed to determine the shown below:
subassembly as necessary, a ratio of subassemblies
above 25 parts relative to the total subassemblies can 𝑉𝐺 × 𝐺
𝑃3 = � �
be used to measure the design in this area. This 𝑃−𝐹
equation for scoring this subassembly interface cost Where 𝑉𝐺 is defined as three non-fastener/connector
is shown below: parts per good part, G is the total good parts, P is the
𝐴−𝐵 total parts in the product, and F is the total fasteners
𝑃1 = � � in the product.
𝐴

Where A is defined as total subassemblies in the The secondary sub-category in Value Optimization
product, 𝑉𝐴 (not shown) is the standard design goal of assigns a design score to the minimization of
a minimum of 25 parts in a subassembly, and B is the fasteners and connectors in the product design. As
total subassemblies with less than 𝑉𝐴 . this is the secondary term for this sub-category, the
priority coefficient 𝑊4 is equal to one half. As
The secondary sub-category in Architecture Elegance discussed previously it is important to eliminate
assigns a design score to the optimization of the fasteners and connectors in a design, therefore a ratio
assembly hierarchy structure. As this is the secondary of non-fastener/connector parts to total parts is
term for this sub-category, the priority coefficient 𝑊2 established such that as the number of fasteners and
is equal to one half. If a the standard design goal of a connectors drives to zero, the scoring ratio drives to
maximum of three subassembly levels is referenced, one. This equation for scoring minimization of
then a ratio of subassemblies with greater than three fastener and connector usage is shown below:
assembly levels relative to the total subassemblies
can be used to measure the design in this area. This 𝑃−𝐹
𝑃4 = � �
equation for scoring this nested subassembly cost is 𝑃
shown below: Where P is defined as the total parts in the product,
𝐴−𝐿 and F is the total fasteners and connectors in the
𝑃2 = � � product.
𝐴

Where A is defined as total subassemblies in the Assembly Elegance

product, 𝑉𝐿 (not shown) is the standard design goal of
The primary sub-category in Assembly Elegance
a maximum of three levels of assembly in a
assigns a design score to the optimization of the
subassembly, and L is the total subassemblies with
overall assembly process. As this is the primary term
less than 𝑉𝐿 .
for this sub-category, the priority coefficient 𝑊5 is
Value Optimization equal to one. Utilizing the standard design goal of
nine assembly steps per good part, the ratio of good
The primary sub-category in Value Optimization parts relative to assembly steps can be used to
assigns a design score to the optimization of parts measure the design in this area. This equation for
that are not fasteners or connectors in the product scoring the manufacturing process is shown below:
design. As this is the primary term for this sub-
𝑉𝑆 × 𝐺
category, the priority coefficient 𝑊3 is equal to one. 𝑃5 = � �
Using the standard design goal of three non- 𝑆
fastener/connector parts per good part as a guide, a Where 𝑉𝑆 is defined as the standard design goal of
ratio of the good parts to total non-fastener/connector nine assembly steps per good part, G is the total good
parts in the product can be used to measure the parts in the product, and S is the total assembly steps.

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 8 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

The secondary sub-category in Assembly Elegance assembly relative to total throughput assembly time is
assigns a design score to the optimization of the established such that as the amount of dwell time
assembly score in the design for ease of handling and drives to zero, the scoring ratio drives to one. This
placement. As this is the secondary term for this sub- equation for scoring dwell time usage is shown
category, the priority coefficient 𝑊6 is equal to one below:
half. This term is the average of the ratio between the
standard Munro assembly score design goal relative 𝑇−𝐷
𝑃8 = � �
to actual Munro assembly score for each 𝑇
subassembly, pre-processed part, part, and multi- Where T is defined as the total throughput assembly
touch in the product design. This equation for scoring time and D is the total throughput dwell time.
part design for assembly is shown below:
Quality Improvement
𝑍
1 𝑉𝑅
𝑃6 = � � � The primary sub-category in Quality Improvement
𝑍 𝑅
𝑖=1 assigns a design score to the minimization of Q-
Where Z is defined as the total of all the burden. As this is the primary term for this sub-
subassemblies, pre-processed parts, parts, and multi- category, the priority coefficient 𝑊9 is equal to one.
touches in the design, 𝑉𝑅 is the standard design goal As discussed previously it is important to eliminate
of a Munro assembly score of three seconds, and R is Q-Burden in a design, therefore a ratio of non-Q-
the Munro assembly score for each subassembly, pre- burden cost relative to the total accounted cost of the
processed part, part, or multi-touch. product is established such that as the amount of Q-
burden drives to zero, the scoring ratio drives to one.
The first auxiliary sub-category in Assembly This equation for scoring the reduction of Q-burden
Elegance assigns a design score to the minimization is shown below:
of manipulations in the assembly process. As this is
𝐶−𝑄
an auxiliary term for this sub-category, the priority 𝑃9 = � �
coefficient 𝑊7 is equal to one fourth. As discussed 𝐶
previously it is important to eliminate manipulations Where C is the total accounted cost of the product
in the assembly process, therefore a ratio of and Q is the total Q-burden.
manipulations to total assembly steps is established
such that as the number of manipulations drives to The secondary sub-category in Quality Improvement
zero, the scoring ratio drives to one. This equation for assigns a design score to the minimization of
manipulation efficiency is shown below: inspection operations and poka yoke issues. As this is
the secondary term for this sub-category, the priority
𝑆−𝑀 coefficient 𝑊10 is equal to one half. As discussed
𝑃7 = � �
𝑆 previously it is important to eliminate inspections and
Where S is defined as the total assembly steps in the poka yoke issues in the design, therefore a ratio of
process and M is the total manipulation used in inspections and poka yoke issues to total assembly
assembly. steps is established such that as the number of
inspections and poka yoke issues drives to zero, the
The second auxiliary sub-category in Assembly scoring ratio drives to one. This equation for scoring
Elegance assigns a design score to the minimization inspection usage and poka yoke issues is shown
of dwell time in the assembly process. As this is an below:
auxiliary term for this sub-category, the priority
𝑆−𝑌
coefficient 𝑊8 is equal to one fourth. As discussed 𝑃10 = � �
previously it is important to eliminate dwell time in a 𝑆
design, therefore a ratio of non-dwell throughput

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 9 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Where S is defined as the total assembly steps in the Where 𝑅𝐶 is defined as the design specification
process and Y is the total number of inspections and captured and R T is the total number of design
poka yoke issues. specifications.

Assembly Knowledge

PRODUCIBILLITY CONFIDENCE INDEX The Assembly Knowledge category of the PCI

EQUATION assigns a knowledge score associated to assembly
process knowledge captured in the design. Therefore
The equation for the producibility confidence index is the ratio of subassemblies with completed analyzed
the average of the four category knowledge scores in manufacturing process is set relative to the total
the format shown below: subassemblies in the product such that as the amount
𝑁 of subassemblies analyzed in the design increase, the
1 scoring ratio drives to one. This equation for scoring
𝑃𝐶𝐼 = � 𝑃𝐶𝑖
𝑁 assembly knowledge is shown below:
𝑖=1

Each of the category knowledge scores are calculated 𝑆𝐴

𝑃𝐶2 = � �
using a ratio of the key design knowledge factors 𝐴
controlling each specific focus area. The category
Where 𝑆𝐴 is defined as the total analyzed
knowledge are created such that each of the scores
subassemblies and A is the total subassemblies in the
will fall on a 0 to 1 scale allowing for easy
product.
comparison between categories and the proper
calculation of the overall PCI using comparable Part Knowledge
indexed scores. In this calculation the knowledge
score is completed for each of the 4 sub-categories, The Part Knowledge category of the PCI assigns a
defining N as equal to 4. Additionally as the score knowledge score associated to subassembly and part
increases toward a value of 1 it shows the increase of knowledge captured in the design. Therefore the ratio
knowledge about the design and can provide a rough of subassemblies, pre-processed parts, parts, and
percentage score as to relative progress toward the multi-touches with Munro assembly scores is set
completion of the design. relative to the total subassemblies, pre-processed
parts, parts, and multi-touches in the product such
Specification Capture that as the amount of subassemblies, pre-processed
parts, parts, and multi-touches analyzed in the design
The Specification Capture category of the PCI
increase, the scoring ratio drives to one. This
assigns a knowledge score associated to
equation for scoring part knowledge is shown below:
specifications captured in the design. Therefore the
ratio of design specification captured in the design is 𝑃𝑍
set relative to the total design specifications needing 𝑃𝐶3 = � �
𝑍
to be addressed in the product such that as the
amount of specifications captured in the design Where 𝑃𝑍 is defined as the total analyzed
increase, the scoring ratio drives to one. This subassemblies, pre-processed parts, parts, and multi-
equation for scoring specification capture is shown touches and Z is the total number of subassemblies,
below: pre-processed parts, parts, and multi-touches in the
product.
𝑅𝐶
𝑃𝐶1 = � �
RT Infrastructure Knowledge

The Infrastructure Knowledge category of the PCI

assigns a knowledge score associated to

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 10 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

infrastructural knowledge captured in the design Assembly Steps 100% 68.5%

through the MRL process. Therefore the average of Total Assembly Score 100% 62.4%
the ratios between the MRL ranks for each key Subassemblies, Parts, &
100% 67.2%
critical component set relative to the MRL rank of 9, Multi-Touches
established at full rate production, is used as a Multi-Touches, CDIs, &
100% 55.2%
Manipulations
measurement for this knowledge area of the design.
Throughput Assembly Time 100% 72.0%
This equation for scoring infrastructure knowledge is
Assembly Dwell Time 100% 100.0%
shown below:
Total Cost 100% 112.1%
𝐶
1 𝑀𝑅𝐿 Q-Burden 100% 67.0%
𝑃𝐶4 = �� � ��
𝐶 9 Inspections and Poka Yoke
100% 68.7%
𝑖=1 Issues
Table 1: Producibility modeling metrics used in PI
Where C is defined as the total number of key critical
equation
components in the product and MRL is the
Manufacturing Readiness Level of each key critical The metrics shown in Table 1 indicate that the GSC
component. Note that if there is no component or Phase II design is a significant improvement over the
process deemed to be key critical requiring a MRL GSC Phase I design. This is thanks to the integration
assessment, then this term should be set to a value of of producibility modeling into the system engineering
one. process when they were creating the GSC Phase II
design. However, these metrics by themselves to not
directly state what area of producibility the design
APPLICATION OF PRODUCIBILITY INDICES has truly improved and where is there the possibility
for further improvement. Therefore to clarify what
In the past the On-Board Vehicle Power (OBVP) makes the GSC Phase II more producible than the
system and specifically the Generator System GSC Phase I, these metrics were input into the PI.
Controller (GSC) was used to demonstrate how The producibility sub-category scores and overall PI
producibility modeling can improve a system’s score are shown in Table 2.
design (14). This study will build on that process by
using metrics from the original producibility model PI Term GSC Phase I GSC Phase II % Change
of the GSC Phase I design, and compare them with P1 42.37% 30.19% -28.75%
metrics from a new producibility model of the current
P2 94.92% 94.34% -0.61%
GSC Phase II design. These metrics from the Design
Profit® producibility models will provide all the P3 5.08% 7.53% 48.07%
required inputs into the PI and PCI equations. Table 1
P4 62.21% 60.37% -2.95%
shows these metrics set as percentages relative to the
values in the original GSC Phase I design baseline. P5 2.23% 3.25% 46.02%

P6 26.86% 29.43% 9.59%

GSC Phase I GSC Phase II
P7 70.96% 76.59% 7.93%
Total Subassemblies 100% 89.8%
P8 100.00% 100.00% 0.00%
Subassemblies < 25 Parts 100% 108.8%
P9 63.41% 78.13% 23.22%
Subassemblies > 3 Levels 100% 100.0%
P10 94.40% 94.38% -0.02%
Good Parts 100% 100.0%

Parts 100% 69.6%

PI 45.39% 46.54% 2.54%
Fasteners and Connectors 100% 73.0%
Table 2: PI sub-category and overall scores

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 11 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

There are two ways to view the results of a PI Specifically P3 and P5 refer respectively to the overall
calculation. The first is to review the overall PI score optimization of parts and overall optimization of the
to see if it is trending in the positive direction, which assembly process relative to the theoretical good
is the case for the GSC Phase II. This confirms that parts of the design. For most industries the score for
the overall producibility of the product has increased these terms will often indicate opportunity for
and the new design is better than the previous design. improvement. This is because the best way to drive
The second view, and possibly the more important producibility into a design is to try and design a
usage of the PI, is to review each of the PI sub- product with the least number of parts and least
category scores to see why PI has changed as much number of assembly process steps. The product with
as it did. This review can then be used a tool to direct fewer parts requires less part design and fabrication,
future development work on the design to specific and the assembly process with fewer steps is easier to
areas with the most opportunity to improve control and takes less time to complete.
producibility. For example the GSC Phase II has
positive increases in most of the sub-category scores, However, if the PI scores of the two designs are
however, the scores for P1 and P4 are showing coming from different producibility models with
decreases. P1 is the term referring to the different levels of detail, then the trends being
underutilization of subassemblies, and a decrease in reviewed in the analysis of the PI could be
this score would indicate that several smaller misleading. Therefore in order to confirm that PI
subassemblies have been generated in the new design scores are comparable with similar levels of detail
relative to previous design. Therefore future design being analyzed, the PCI should always be stated
work would include added effort to integrate alongside the PI score. The category scores and
subassemblies to better utilize fewer subassemblies overall score for the PCI of the GSC Phase I and
and reduce costs. P4 is the term referring to Phase II designs can be seen in Table 3.
fastener/connector usage, and a decrease in this score
would indicate that when several non-fastener parts PCI Term GSC Phase I GSC Phase II
were integrated in the design, the associated fasteners PC1 100.00% 100.00%
were not reduced as well. This could be the result of
PC2 77.97% 81.13%
several possibilities. One possibility is that the
increase in subassemblies has led to the use of more PC3 100.00% 100.00%
fasteners and connectors for attachment and PC4 100.00% 100.00%
subassembly interfaces. Another possibility is that the
new integrated parts may be using more fasteners
than required. Therefore the future design work PCI 94.49% 95.28%
would be to review the design for opportunities to Table 3: PCI category and overall scores
reduce the number of fasteners and connectors.
However, while showing positive increases does First, it is important to note that this example
indicate improvement in a sub-category score, it does compares the GSC phase I and II, which are
not necessarily mean that additional engineering completed designs. Therefore the PC1, which refers
support is not required in a specific area. For example to specification capture, is at 100% as the completed
the GSC Phase II shows significant increases in P3 designs captured all customer requirements. PC2
and P5 scores, but when the main score is reviewed it which refers to analyzed subassemblies is not at
shows the design is rather low in those select areas. 100% as a few of the subassemblies within the model
What this means is that the design team did an were not considered in scope at the time of the
excellent job to improve these areas over the previous analysis, and so were not analyzed. Additionally due
design, however, focus needs to be continually to the fact that the designs were completed, PC3
directed to these areas as there is still a great which refers to subassembly, part, and multi-touch
opportunity to improve producibility in that area. scoring, was at 100% as all parts had been analyzed

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 12 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

for the design. Lastly, PC4 which refers to MRL is at

100% as the GSC is considered to use common
components and manufacturing process, therefore CONCLUSION
nothing is indentified as key critical so the term is set
It is the goal of all engineers to find the optimal
to 100%. If this index is used earlier in the
solution to a problem. The hope of our team is that
development process it would show lower category
the producibility index (PI) and the producibility
scores from a reduced amount of detail in the
confidence index (PCI) detailed in this paper will
producibility model. What these scores indicate is
serve as a powerful tool to help engineers to optimize
that both of the GSC Phase I and Phase II
their designs. Specifically the 10 sub-categories of
producibility models have comparable levels of
the PI should provide indications to engineers where
detail. This means that PI scores of both designs can
opportunities exist in their design to improve
be compared without concern.
producibility. This analysis of opportunities will also
It is important to note that even if two designs do reduce risk thanks to the PCI providing an indication
have similar PCI scores it does not necessarily mean as to the level of detail being used in the analysis.
their PI scores can be compared. We expect that This theory has been demonstrated effectively with
individual standard design goals within the sub- the On-Board Vehicle Power (OBVP) design team
category scores of the PI will differ across industries, for the Generator System Controller (GSC).
product classes, sizes, and volumes. Also different However, there is significant room for improvement
designs will have different levels of requirement for in the producibility indices. Specifically the
design performance that might inherently reduce continuation of refinement in the standard design
producibility. Therefore at this point in our goals and priority coefficients will allow the scores in
development of the PI and PCI, only designs for the the PI and PCI to better indicate when changes in
same application with similar PCI scores can have producibility have occurred and properly weight the
their PI scores compared for analysis. importance of that change to show the impact on the
overall product design. Additionally the PI and PCI
have been applied to the completed design analyses
of the GSC Phase I and II to show common trends in
FUTURE WORK a product that has been redesigned to become more
producible. However, the real strength of the theory
It is the hope of our team to continue to expand upon
behind the PI and PCI is that it can be applied early
the producibility indices with a full sensitivity
in product development and continue to be used
analysis that will better refine the values used for the
throughout the process to guide design decisions.
design goals of each sub-category. This sensitivity
Therefore the best way to demonstrate the true value
analysis would investigate product industries, classes,
and effectiveness of the PI and PCI would be to apply
volume levels, and size to determine how these affect
the indices at the beginning of development and show
the expected design goals. At the completion of this
the use throughout the development process.
future study we would integrate weighting factors to
these established design goals in order to provide Hopefully the producibility indices theory discussed
improved PI and PCI scales that can provide a metric in this paper will be able to expand to provide the
across defense programs. The weighting of these foundation for the establishment of the PI and PCI as
factors should provide universal PI and PCI scales a standard design evaluation allowing for a common
regardless of product industries, classes, volume technique to be applied across the system engineering
levels, and size. This would expand the benefit of community. This will then enable system engineering
these indices to provide standard global grading for to use a common language across industries to
any defense program and allow these indices to be discuss the improvement of producibility in their
applied to the 5000.02 and MRL timeline and grading products, thereby providing the DOD a way to
scales. achieve their goals to improve asset uptime while

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 13 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

reducing the initial costs and eliminating the ACKNOWLEDGEMENTS

possibility of cost overruns. The PI and PCI will
enable the systems engineering community to This work was funded in part by the Defense-wide
establish a new trend that allows the warfighter to Manufacturing Science and Technology Program
benefit from more advanced technology, faster, at (FA-8650-10-C-5702), led through the Office of the
lower costs. Secretary of Defense, Manufacturing Technology.
The authors would like to thank Jennifer Fielding and
Brench Boden of the Air Force Research Laboratory,
Manufacturing Technology Division, for their
technical assistance and guidance on this program.

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 14 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

WORKS CITED 8. Simple Parametric Model for Estimating

Development (RDT&E) Cost on Large-Scale
1. Government Accountability Office (GAO). Best Systems. Jones, R., Hardin, P., and Irvine, A. St.
Practices: DOD Can Achieve Better Outcomes by Louis, MO : ISPA/SCEA Joint Conference, 2009.
Standardizing the Way Manufacturing Risks Are
Managed. Washington, DC : Government 9. Design System for Managing Complexity in
Accountability Office (GAO), 2010. Aerospace Systems. Becz, Sandor, Pinto,
Alessandro, Zeidner, Lawrence E., et al. Fort
2. Sanders, Al, Boden, Brench, et al. 21st Century Worth, TX : 2010 AIAA ATIO/ISSMO Conference,
Manufacturing Modeling & Simulation Research and 2010.
Investment Needs. Arlington, VA : National Defense
Industrial Association, 2011. 10. Design Profit, Inc. Design Profit Training
Manual. Williamston, MI : Design Profit, Inc., 2010.
3. Sanders, Al, Belie, Gary, et al. Modeling &
Simulation Investment Needs for Producible Designs 11. Munro & Associates, Inc. Lean Design. Troy,
and Affordable Manufacturing. Arlington, VA : MI : Munro & Associates, Inc., 2001.
National Defense Industrial Association, 2010.
12. OSD Manufacturing Technology Program.
4. Brown, David P. Simulation Based Acquisition Manufacturing Readiness Level Deskbook.
Can It Live Up to Its Promise? Modeling & Washington, DC : OSD Manufacturing Technology
Simulation. January-February, 1999. Program, 2010.

5. Boothroyd, G. and Dewhurst, P. Design For 13. Miles, Lawrence D. Techniques of Value
Assembly. Amherst, Massachusetts : University of Analysis and Engineering. York, PA : McGraw-Hill
Massachusetts, 1983. Book Company, Inc., 1961.

6. An Empirical Methodology for Measuring 14. Embedding Affordability and Producibility (AP)
Producibility Early in Product Development. Priest, in Systems Engineering: Cost, Complexity and
John W. and Sanchez, Jose M. 2, Arlington, TX : Readiness as Prime Drivers for Integrated Design.
International Journal of Computer Integrated Marcel, Mike, Kelly, Thomas, Donoghue, Mike,
Manufacturing, 1991, Vol. 4. and Feord, Joe. Dearborn, MI : Ground Vehicle
Systems Engineering and Technology Symposium
7. A Feature-Based Approach to Producibility (GVSETS), 2010.
Evaluation of Machined Component Designs. Chen,
Chen-Sheng, Sagarsee, Samual, Chow, Joe G., et
al. Memphis, TN : MEASC 2004 Conference, 2004.

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 15 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

APPENDIX

Expanded Producibility Index Equation:

𝑍
2 𝐴−𝐵 1 𝐴−𝐿 𝑉𝐺 × 𝐺 1 𝑃−𝐹 𝑉𝑆 × 𝐺 1 1 𝑉𝑅 1 𝑆−𝑀 1 𝑇−𝐷
𝑃𝐼 = �� �+ � �+� �+ � �+� � + � � � �� + � �+ � �
13 𝐴 2 𝐴 𝑃−𝐹 2 𝑃 𝑆 2 𝑍 𝑅 4 𝑆 4 𝑇
𝑖=1

𝐶−𝑄 1 𝑆−𝑌
+� �+ � ��
𝐶 2 𝑆

Standard Design Goal Parameters:

1. VA = Standard Minimum Parts per Subassembly = 25

2. V L = Standard Assembly Levels = 3
3. VG = Standard Non-Fastener/Connector Parts per Good Part = 3
4. VS = Standard Assembly Steps per Good Part = 9
5. VR = Standard Munro Assembly Score = 3
6. N = Number of Scoring Terms = 10

Variables:

1. A = Total Subassemblies
2. B = Number of Subassemblies with Less than VA Parts
3. L = Number of Subassemblies with Less than VL Assembly Levels
4. G = Total Good Parts
5. P = Total Parts
6. F = Total Fasteners and Connectors
7. S = Total Assembly Step Count
8. R = Assembly Score Subassemblies, Pre-Processed Parts, Parts, or Multi-Touches
9. Z = Total Subassemblies, Pre-Processed Parts, Parts, and Multi-Touches
10. M = Total Multi-Touches, Changes in Direction (CDI), and Part Manipulations
11. T = Total Throughput Assembly Time
12. D = Total Assembly Dwell Time
13. C = Total Cost
14. Q = Total Q-Burden
15. Y = Total Inspections and Poka Yoke Issues

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 16 of 17
 Proceedings of the 2011 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS)

Expanded Producibility Confidence Index Equation:

𝐶
1 𝑅𝐶 𝑆𝐴 𝑃𝑍 1 𝑀𝑅𝐿
𝑃𝐶𝐼 = �� � + � � + � � + �� � ���
4 RT 𝐴 𝑍 𝐶 9
𝑖=1

Standard Knowledge Parameters:

1. N = Number of Scoring Terms = 4

Variables:

2. RT = Total Specification Requirements

3. RC = Spec. Requirements Captured with Analyzed Parts
4. A = Total Subassemblies
5. SA = Total Analyzed Subassemblies
6. Z = Total Subassemblies, Pre-Processed Parts, Parts, and Multi-Touches
7. PZ = Total Analyzed Subassemblies, Pre-Processed Parts, Parts, and Multi-Touches
8. C = Total Critical Parts Requiring MRL Scoring
9. MRL = Manufacturing Readiness Level

Producibility and Confidence Indices During Defense Acquisition, Hadley, et al.

Page 17 of 17