Systems Analysis & Improvement

Experimental Design and Analysis

UCD School of Mechanical and Materials Engineering

1
 Learning Objectives
1. Explain how designed experiments can be used to improve product
design and improve process performance
2. Explain how designed experiments can be used to reduce the cycle
time required to develop new products and processes
3. Understand how main effects and interactions of factors can be
estimated
4. Understand the factorial design concept
5. Know how to use the analysis of variance (ANOVA) to analyze data
from factorial designs
6. Know how residuals are used for model adequacy checking for
factorial designs
7. Know how to use the 2k system of factorial designs
8. Understand the concept of blocking and fractional factorial designs.

2
 What is Experimental Design?
• Definition
• A designed experiment is a test or series of tests in which
purposeful changes are made to the input variables of a
process so that we may observe and identify corresponding
changes in the output response.

3
One-Factor-at-a-Time (OFAT) Experiments

Yield versus Yield versus Optimization

reaction time with temperature with experiment using
temperature reaction time the OFAT method.
constant at 155F constant at 1.7 h.

4
 What is Experimental Design?
• Applications to improve a manufacturing process
– Improved yield
– Reduced variability
– Reduced development time
– Reduced overall costs
• Applications to engineering design
– Evaluation and comparison of basic design configs
– Evaluation of material alternatives
– Determination of key product design parameters

5
 Examples in Process and Product
Improvement
• Polishing processes in the semiconductor industry
– Objective: flat wafers (degree of uniformity) to maintain high yield
rates.
– Six process parameters were considered:
– Down Force, Back Force, Carrier Velocity and Table, Type of Pad and
Oscillations

6
 Examples in Product Design _ I
• Designing a new pump for the intravenous
delivery of a drug.
• Objective: The pump should deliver a constant quantity or
dose of the drug over a specified period of time.
– Nine design parameters were considered:
• Diameter and length of the cylinder,
• the fit between the cylinder and the plunger length,
• the diameter and wall thickness of the tube connecting the pump
and the needle inserted into the patients’ vein,
• the material to use for fabricating both the cylinder and the tube,
• the nominal pressure at which the system must operate.

7
 Examples in Product Design _ II
• Designing an aircraft engine (commercial
turbofan).
• Objective: specific fuel consumption and engine thrust.
• Prohibited to build prototypes or actual test articles. Therefore,
an engineer can use a computer simulation model.
• Using the model, Designed experiments can be employed to
determine the most important design parameters and their
optimal settings.
• Using computer models can include Finite Element Models
for structural and mechanical design, Circuit Simulators for
integrated circuit design, Factory or Enterprise-level Models
for scheduling and capacity planning
8
 Examples in Producing a Product
• Producing a chemical product in a pressure
vessel.

• Objective: Filtration rate (gal/h) of this product.

• Four factors were considered:

– Temperature
– Pressure
– Concentration of Formaldehyde.
– Stirring rate
9
 Example in Health Care
• Satisfaction at Emergency Rooms
• Objective: decreasing patient dissatisfaction in the emergency
room
– Patient average length of stay
– Labor cost per patient
• Three process parameters were considered:
– Number of ERs
– Patient arrivals per day
– Number of doctors

10
Guidelines for Designing an Experiment

1. Recognition of objectives and statement of the problem.

– Inputs from all concerned parties to be involved.
2. Choice of factors and levels.
– Process knowledge
– Number of factor levels

11
 Guidelines for Designing an Experiment
3. Selection of the response variable.
– provides useful information about the process
under study

4. Choice of experimental design.

– consideration of sample size
– selection of a suitable run order for the
experimental trials
– Blocking
5. Performing the experiment
– Monitor the process.
– replication
6. Data analysis
– Statistical methods to analyze the data
– Validity check
7. Conclusions and recommendations
– Practical conclusions about the results
– Recommend a course of action 12
 DoE – Plastic Molding Example
• The objective of the experiment is to achieve uniform
dimensions for a part at a particular target value to reduce
variations.

13
Components of DoE in the Molding Example

Output
Factors Levels Interactions
Response

▪ Part hardness that is measured as a result of the experiment and is used

to judge the effects of factors.

14
Components of DoE in the Molding Example

Output
Factors Levels Interactions
Response

▪ Cycle time, mold temperature, holding pressure, holding time and

material type

15
Components of DoE in the Molding Example

Output
Factors Levels Interactions
Response

▪ Factors can be varied and are called levels.

▪ Mold temperature (600o and 700o Fahrenheit), Plastic type (fillers and
no fillers), and the material type has two levels of nylon or acetal

16
Components of DoE in the Molding Example

Output
Factors Levels Interactions
Response

▪ The degree to which factors depend on one another.

▪ Time*Temp (the best level for time depends on the set temperature).
▪ If the temperature level is higher, the cycle time may have to be decreased to
achieve the same response from the experiment.

17
Factorial Experiment – An Example

• Definition
– Contains all combinations of all levels of all factors;
– Ensures no possible treatment combinations get omitted; and
– is preferred over other designs.

Factor B: Oven Factor A: Draw Temperature

Time (X2) (X1)
700o 900o
30 Mins Y1 Y5
Y2 Y6
60 Mins Y3 Y7
Y4 Y8

The table shown here is for a two-way heat treatment experiment,

where Y = Part Hardness

18
Factorial Experiment – An Example
• Analysis of the Mean Effect:
– Based on the example, an analysis of the means helps in
understanding how:
– A change in temperature at which the material is drawn creates a
difference in the average part hardness (Main Effect);
– A change in oven time creates a difference in the average part
hardness (Main Effect); and
– Interaction between temperature and time affects the average part
hardness (Interaction Effect).
Factor B: Oven Factor A: Draw Temperature
Time (X2) (X1)
700o 900o
30 Mins Y1 Y5
Y2 Y6
60 Mins Y3 Y7
Y4 Y8
19
Design of Experiments - Runs
• The number of experiments in a DoE setting is known as
Runs. The number of runs in a:

– Full factorial experiment without replication on 5 factors and 2

levels is: 25 = 32.
– Full factorial experiment with 1 replication on 5 factors and 2
levels is: 32 + 32 = 64.
– Half fractional factorial experiment without replication on 5
factors and 2 levels is: 25-1 = 16.
– Half fractional factorial experiment with 1 replication on 5
factors and 2 levels is: 16 + 16 = 32.

• The difference between a full factorial and half fractional factorial

experiment can be seen through the number of runs.

20
 Factorial Experiments
• Definition
– In each replicate of the experiment, all possible
combinations of the levels of the factors are investigated.
• For instance: two factors A and B with a levels of factor A and b
levels of factor B, each replicate contains all ab possible
combinations.
• Main effect
– The main effect of factor A is the difference between the average response at
the high level of A and the average response at the low level of A

21
 Factorial Experiments

• Interaction between factors

– In some experiments, the difference in response between the levels of one factor is not
the same at all levels of the other factors. When this occurs, there is an interaction
between the factors.

At Low level of factor B:

At High level of factor B:

Since the effect of A depends on the level

chosen for factor B, there is interaction between
A and B.

• A significant interaction can mask the significance of main

effects.
22
Graphical Display of Interaction

a) Factorial experiment with no b) Factorial experiment with

interaction. interaction.

23
 An Example of a Factorial Experiments
• Aircraft primer paints are applied to aluminum surfaces by two methods -
dipping and spraying. The purpose of the primer is to improve paint
adhesion; some parts can be primed using either application method. A
team has identified three different primers that can be used with both
application methods. Three specimens were painted with each primer using
each application method, a finish paint was applied, and the adhesion force
was measured. The 18 runs from this experiment were run in random order.
The resulting data are shown below:

24
 An Example of a Factorial Experiments
• The objective of the experiment was to determine which
combination of primer paint and application method
produced the highest adhesion force.

• It would be desirable if at least one of the primers produced

high adhesion force regardless of application method, as this
would add some flexibility to the manufacturing process.

25
 The general two-factor factorial design

• The analysis of variance (ANOVA) can be extended to handle

the two factor factorial experiment.
• The two-factor factorial is a completely randomized design.

26
The Statistical Model
• The observations from a two-factor factorial experiment may be
described by below model.

• We are interested in testing the hypotheses of no significant factor A

effect, no significant factor B effect, and no significant AB interaction.

• Using Analysis of variance, we can decompose the above model

symbolically as:

27
The Decomposed ANOVA table

28
Computing the sums of squares

yi.. denote the total of the observations at the ith level of factor A,
y.j. denote the total of the observations at the jth level of factor B,
yij. denote the total of the observations in the ijth cell
y... denote the grand total of all the observations. 29
Performing the ANOVA for the Example

The sums of squares required are

30
31
Interaction graph
Since a large response indicates greater adhesion force, we
conclude that spraying is a superior application method and that
primer type 2 is most effective. Therefore, if we wish to operate
the process so as to attain maximum adhesion force, we should use
primer type 2 and spray all parts.

Graph of average adhesion force versus primer types

32
Residual Analysis

• The use of partitioning to test formally for no differences in

treatment means requires that certain assumptions be satisfied.
• Violations of the basic assumptions of ANOVA and model adequacy
can be investigated using RA.
• The residuals are simply the difference between the observations
and the corresponding cell averages. If the model is adequate, the
residuals should be structureless.

• The residuals from a two factor factorial are:

33
Residual Analysis

34
The 2k Factorial Design

• Definition
– A factorial design with k factors, each at two levels.
– Each complete replicate of the design has 2k runs.

• The 22 Design
The “+ and –” notation

35
Contrasts in Signs Table or Standard Order

36
Factor Effects

Contrast

37
 Sums of Squares for the ANOVA

• Therefore, the sums of squares for A, B, and AB are

38
 Example - The Router Experiment

• A router is used to cut registration notches in printed circuit

boards. The average notch dimension is satisfactory, and the
process is in statistical control, but there is too much variability
in the process.
• This excess variability leads to problems in board assembly. The
components are inserted into the board using automatic
equipment, and the variability in notch dimension causes
improper board registration. As a result, the auto-insertion
equipment does not work properly. How would you improve this
process?
• Since the process is in statistical control, the quality
improvement team assigned to this project decided to use a
designed experiment to study the process. The team considered
two factors: bit size (A) and speed (B).
– Two levels were chosen for each factor (bit size A at 1/16” and 1/8”
and speed B at 40 rpm and 80 rpm)

39
 Example - The Router Experiment Response

• Sixteen test boards were instrumented with accelerometers that

allowed vibration on the (X, Y, Z) coordinate axes to be measured.
The resultant vector of these three components was used as the
response variable.

• Objective: To reduce vibration levels resulting in the reducing of

the variability in notch dimension.

40
Example - The Router Experiment Factor effects

• Both bit size and speed are

important, and there is
interaction between two
variables.

41
Example - The Router Experiment:
Regression Model and Residual Analysis
• Obtain the residuals from a 2k design by fitting a regression model to the
data. between two variables.

• The estimate of the intercept B0 is the grand average of all 16 observations.

• The estimate of other Bj are one-half of the effect estimate for the
corresponding factor.

42
Example - The Router Experiment: Practical
Interpretation

• If we run with speed high and use the small bit, the production rate will be
satisfactory.

43
The 2k Design for k ≥ 3 Factors

44
Geometric Presentations

Interaction effects are also estimated using contrasts

45
46
 Effect estimates (main effects)

Chapter 13 47
Effect estimates (Interaction effects)

48
Effect estimates and sums of squares

• The estimate of any main effect or interaction

• The sum of squares for any effect is

49
50
51
52
53
 Further Reading Materials
• Douglas C. Montgomery (2019). Introduction to
Statistical Quality Control, 8th Edition. Wiley.
– Chapter 13

54