UNIT III

SELECTING THE RESEARCH DESIGN

3.1. The nature and significance of a research design

A research design is the detailed blueprint used to guide a research study toward its objectives.
The process of designing a research study involves many interrelated decisions. The most
significant decision is the choice of research approach, because it determines how the
information will be obtained. Typical questions at this stage are: should we rely on secondary
sources such as the Census? Which is more appropriate, an exploratory approach with group
discussions or a survey? Is a mail, telephone, fax, or personal interview survey better for this
problem?

A research design is the specification of procedures for collecting and analyzing the data
necessary to help identify or react to a problem or opportunity, such that the difference between
the cost of obtaining various levels of accuracy and the expected value of the information
associated with each level of accuracy is maximized.

Several aspects of this definition deserve emphasis.

 First, research design requires the specification of procedures. These procedures involve
decisions on what information to generate, the data collection method, the measurement
approach, the object to be measured, and the way in which the data are to be analyzed,

 Second, the data are to be collected to help identify or react to a problem or opportunity.
All data collected should eventually relate to decision faced by management. Obviously,
the efficient collection of data relevant to a decision requires a clear definition of the
problem/opportunity.

 The fourth major implication is that varying levels of accuracy of information can be
generated in response to the same problem. Information accuracy is affected by the
occurrence of a number of potential errors.
  Finally, the goal of applied research design is not to generate the most accurate
information possible. Rather, the objective is to generate the most valuable information in
relation to the cost of generating the information.

To design something also means to ensure that the pieces fit together. The achievement of this fit
among objectives, research approach, and research tactics is inherently an interactive process in
which earlier decisions are constantly reconsidered in light of subsequent decision. This may
mean a revision of the research objectives as new insights are gained into the complexities of the
population to be samples, or a reassessment of the research approach in light or realistic cost
estimates. Consequently, few researches find that they have designed their research studies in the
neat and linear fashion.

Research methods vary from focus groups to simulated test markets, to large, nationally
representative sample surveys. Some research problems require only library research. Whereas
others may require thousands of personal interviews. Each method has certain advantages and
disadvantages, and one method may be more appropriate for a given research problem than
another. After thoroughly considering the research problem, researchers select a research design,
which is a set of decisions that make up the master plan specifying the methods and procedures
for collecting and analyzing the needed

3.2. Types of Research Design

All research approaches can be classified into one of the four general categories of research
Design:

1. Exploratory research design

2. Descriptive research design

3. Causal Research design

4. Experimental research design

The choice of the most appropriate design depends largely upon the objectives of the research
and the nature of the research question. In most cases research projects involves any of the
following three objectives:
  To develop hypotheses,

 To measure the state of a variable of interest (that is, level of brand loyalty), or

 To set hypotheses that specifies the relationships between two or more variables. The
table below shows the three types of research designs and the basic research objective
that would prescribe a given design.

The four types of research design differ significantly in terms of;

a. The research purpose

b. Research questions

c. The precision of the hypotheses that are formed and

d. The data collection methods that are sued

Table 1.1

Research Objective Appropriate Design

To gain background information, to define terms, to Exploratory

clarify problems and hypotheses, to establish research
priorities

To describe and measure a phenomena at a point in time Descriptive

To determined causality, to make ”if-then” statements Casual

Research Design: A Caution

Some may think that it is implied in this discussion that the order in which the designs are
presented that is, exploratory, descriptive, and causal, is the order in which these designs should
be carried out. This is incorrect. Three points should be made relative to the interdependency of
research designs.

 First, in some cases, it may be perfectly legitimate to begin with any one of the three
designs and to use only one design.

 Second, research is an “iterative” process: by conducting one research project, we learn

that we may need additional research, and soon. This may mean that we need to utilize
multiple research designs. We could very well find, for example, that after conducing
descriptive research, we need to go back and conduct exploratory research.

 Third, if multiple designs are slued in any particular order (if there is an order), it makes
sense to first conduct exploratory research, then descriptive research, and finally casual
research. The only reason fro this order pattern is that each subsequent design requires
greater knowledge about the research problem on the part of the researcher. Therefore,
exploratory may give one the information needed to conduct a descriptive study, which in
turn, may provide the information necessary to design a causal experiment. Now that you
have been cautioned regarding thinking of research design solely in a step-by-stem
fashion, we begin our discussion of the three types of research design.

3.2.1. Exploratory Research Design

Exploratory research is unstructured, informal research that is undertaken to gain background
information about the general nature of the research problem. By unstructured, we mean that
exploratory research does not have a formalized set of objectives, sample plan, or questionnaire.
It is usually conducted when the researcher does not know much about the problem and needs
additional information or desires new or more recent information.

Often, exploratory research is conducted at the outset of a research projects. Because exploratory
research is aimed at gaining additional information about a topic and generating possible
hypotheses to test, it is described as informal. Such research may consists of going to the library
and reading published secondary data; asking customers, salespersons, and acquaintances of
their opinions about a company, its products, services, and prices; or of simply observing
everyday company practices.

Exploratory research is systematic, but it is very flexible in that it allows the researcher to
investigate whatever sources he or she desires and to the extent he or she feels is necessary in
order to gain a good feel for the problem at hand. In the following sections, we discuss the
specific uses of exploratory research as well as the different methods of conducing exploratory
research.

Uses of Exploratory Research

Exploratory research is used in a number of situations: such as to gain background information,
to define research terms, to clarify problems and hypotheses, and to establish research priorities.

Gain Background Information

When very little is known about the problem or when the problem ahs not been clearly
formulated, exploratory research may be sued to gain much-needed background information.
This is easily accomplished in firms having information system in which a review of internal
information tracked over time can provide useful insights into the background of the firm, brand,
sales territories, and so no. even for very experienced researchers it is rear that some exploratory
research is not undertaken to gain current, relevant background information. There is fat too
much to be gained to ignore exploratory information.

Define Research Terms

Expletory research helps to define terms and concepts. By conducing exploratory research to
define a question such us, “what is image” The researcher quickly leans that “image” is
composed of several components perceived convenience of location, friendliness of employees,
and so on. Not only would exploratory research identify the components of bank image but it
could also demonstrate how these components may be measured.

Clarify Problems and Hypotheses

Exploratory research allows the researcher to define the problem more precisely and to generate
hypotheses of the upcoming study. For example, exploratory research on measuring bank image
reveals the issue of different groups of bank customers. Banks have three types of customers:
retail customers, commercial customers, and other banks for which services are performed for
fees. This information is useful in clarifying the problem of the measurement of bank image
because it raises the issue of which customer group bank image should be measured on.

Exploratory research can also be beneficial in the formulation of hypotheses, which are
statements describing the speculated relationships among two or more variables. Formally stating
hypotheses prior to conducting a research study is very important to ensure that the proper
variables are measured. Once a study has been completed, it may be too late to state which
hypotheses are desirable to test.

Establish Research Priorities

Exploratory research can help a firm prioritize research topics in order of importance, especially
when it is faced with conducting several research studies. A review of customer complaint letters
for example, may indicate which product or services are most in need of management’s attention.
For example, one furniture store chain owner decided to conduct research on the feasibility of
carrying office furniture after some or exploratory interviews with salespeople revealed that their
customers often asked for direction to stores carrying office future.

Methods of Conducting Exploratory Research

A variety of methods are available to conduct exploratory research. These include:

 Secondary data analysis

 Experience surveys

 Case analysis

 Focus groups and

 Projective techniques

Secondary Data Analysis

By secondary data analysis, we refer to the process of searching for and interpreting existing
information relevant to the research problem. Secondary data are data that have been collected
for other purpose. Libraries and the Internet are full of secondary data, which includes
information found in books, journals, magazines, special reports, bulletins, newsletter, and so on.
An analysis of secondary data is often the “care” of exploratory research. This is because there
are many benefits to the examining secondary data and the cost is typically minimal.
Furthermore, the search time for such data are begin reduced every day as more and more
computerized databases become available.

Experience Surveys
Experience surveys refer to gathering information from those thought to be knowledgeable on
the issues relevant to the research problem. Experience surveys differ from surveys conducted as
part of a descriptive research in the sense that there is usually no formal attempt to ensure that
the survey result are representative of any defined group of subjects. Nevertheless, useful
information can be gathered by this method of exploratory research.

Case Analysis
By case analysis, we refer to a review of available information about a former situation (s) that
has some similarities to the present research problem. Usually, there are few research problems
that do not have some similarities to some situation in the past. Even when the research deals
with a radically new product, there are often some similar past experience that may be observed.

Focus Groups
An increasingly popular method of conducting exploratory research is through focus groups
discussions, which are small groups of people brought together and guided by a moderator
through an unstructured, spontaneous discussion for the purpose of gaining information relevant
to the research problem. Although focus groups should encourage openness on the part of the
participants, the moderator’s task is to ensure the discussion is “focused” on some general area of
interest.

3.2.2. Descriptive Research Design

When we wish to know how many customers we have, what brands they buy and in what
quantities, which advertisements they recall, and what their attitudes are toward our company
and our competitors, we turn to descriptive research, which provides answers to questions such
as who, what, where, when, and how as they are related to the research problem. Typical,
answers to these questions, are found in secondary data or by conducing surveys.

Decision makers often need answers to some basic questions before they can formulate
effective business strategies. Consider the following examples. In order to develop appropriate
business strategies a descriptive research is needed to answer questions of the following nature,
who may be defined as the firm’s competitors, customers? What may be defined as the products,
brands, size, and so on that are being purchased? Where may be defined as the places the
customers are buying these products. When refers to the time or the frequency with which
purchases are made. How may mean the ways in which customers are using the products.

Classification of Descriptive Research design

There are two basic descriptive research studies available to the business researcher:

1. Cross-sectional

2. Longitudinal

Cross-sectional studies
Cross-sectional studies measure a population at only one point in time. Cross-sectional studies
are very prevalent in business research, outnumbering longitudinal studies and casual studies.
Because cross-sectional studies are one time measurements, they are often described as
“snapshots” of the population. As an example, many magazines survey a sample of their
subscribers and ask them questions such as their age, occupation, income, educational level, and
so on. This sample data taken at one point in time is used to describe the readership of the
magazine in terms of demographics. cross-sectional studies normally employ fairly large sample
sizes; so many cross-sectional studies are referred to as sample surveys. Sample surveys are
cross sectional studies whose sample is drawn in such a way as to be representative of a specific
population.

Longitudinal Studies
Longitudinal Studies repeatedly measure the same population over a period of time. Because
longitudinal studies involve multiple measurements, they are often describes as “movies” of the
population. Longitudinal studies are employed by almost 50 percent of business using business
research. To ensure the success of the longitudinal study, researchers, must have access to the
same members of the sample, called a panel, so as to take repeated measurements. Maintaining a
representative panel of respondents is a major undertaking.

‘Longitudinal’ is a broad term. It can be defined as research in which:

 data are collected for each item or variable for two or more distinct periods;

 the subjects or cases analyzed are the same, or at least comparable, from one period to the
next; and

 The analysis involves some comparison of data between or among periods (Menard
1991:4).

There are a number of different designs for the construction of longitudinal evidence: repeated
cross-sectional studies; prospective studies, such as household panel surveys or cohort panels;
and retrospective studies, such as oral histories and life and work histories.

Repeated cross-sectional studies

In the social sciences, cross-sectional observations are the form of data most commonly used for
assessing the determinants of behavior (Coleman 1981; Davies 1994; Blossfeld and Rohwer
1995). However, the cross-sectional survey, because it is conducted at just one point in time, is
not suited for the study of social change. It is therefore common for cross-sectional data to be
recorded in a succession of surveys at two or more points in time, with a new sample on each
occasion. These samples either contain entirely different sets of cases for each period, or the
overlap is so small as to be considered negligible. Where cross-sectional data are repeated over
time with a high level of consistency between questions, it is possible to incorporate a time trend
into the analysis. Examples of repeated cross-sectional social surveys are: the UK’s General
Household Survey and Family Expenditure Survey, and the EU’s Eurobarometer Surveys.

Prospective designs

The temporal data most often available to social researchers are panel data, in which the same
individuals are interviewed repeatedly across time. Variations of this design (Buck et al. 1994:
21-22) include:

Representative Panels with a random sample of respondents and repeated data collections at
fixed intervals (typically from 2-3 months to a year). Thus panel surveys trace individuals at
regular discrete points in time. The fundamental feature they offer is that they make it possible to
detect and establish the nature of individual change. For this reason, they are well-suited to the
statistical analysis of both social change and dynamic behaviour. Among the best known
prospective panel studies are the US Panel Study of Income Dynamics (PSID), the British
Household Panel Study (BHPS) and the German Socio-Economic Panel (SOEP).

Cohort Panels can be considered as a specific form of panel study that takes the process of
generation replacement explicitly into account. A cohort is defined as those people within a
geographically or otherwise delineated population who experienced the same significant life
event within a given period of time. Researchers select an age group, or some subset of an age
group, and then administer a questionnaire to a sample or to the whole group. Thus, one or more
generations are followed over their life course. The interest is usually in the study of long term
change and in individual development processes: such studies typically re-interview every five
years. If, in each particular generation the same people are investigated, a cohort study amounts
to a series of panel studies; if, in each generation, at each period of observation, a new sample is
drawn, a cohort study consists of a series of trend studies (Hagenaars 1990). Examples are the
UK National Child Development Study and the German Life History Study.

Concerning panel data, the main operational problems with prospective studies (other than linked
panels) are:

Panel attrition

If the same set of cases is used in each period, there may be some variation from one period to
another as a result of missing data (due to refusals, changes of residence or death of the
respondent). Such systematic differences between waves cause biased estimates. For example, a
major problem in most surveys on poverty is the under-sampling of poor people: they are hard to
contact (and therefore usually under sampled in the first wave of data) and hard to retain for
successive annual interviews. Even though weight variables could be used to mitigate under-
representation, it is difficult to assess the real efficiency of such weights.

Course of events

Since there is only information on the states of the units at predetermined survey points (discrete
time points), the course of the events between the discrete points in time remains unknown;

Panel conditioning

Precisely because in a panel survey the same subjects are repeatedly interviewed, it is possible
that responses given in one wave will be influenced by those given in the previous waves
(Trivellato 1999). Unwillingness to participate in the study may also result from continued study
and may result in attrition. Yet another possibility is that respondents will change as a result of
participation in the survey (Menard 1991).

Consequently, the potential of panel data can only be fully realised if such data meet high quality
standards (Duncan 1992; Ghellini and Trivellato 1996). In particular, Trivellato (1999) stated
that for a panel survey to be successful, the key ingredients are a good initial sample and
appropriate following rules, that is, a set of rules that permit mimicing the population that almost
always changes in composition over time. Taking the BHPS as an example, because the BHPS is
a household panel study which tracks household formation and dissolution, individuals may join
and leave the sample. Thus, the study has a number of following rules determining who is
eligible to be interviewed at each wave. New eligibility for sample inclusion could occur
between waves in the following ways: 1) A baby is born to an Original Sample Member (OSM);
2) An OSM moves into a household with one or more new people; 3) One or more new people
move in with an OSM (Freed Taylor et al. 1995).
The drawback of linked panels is that they can only provide a very limited range of information
and often on a highly discontinuous temporal basis (as in the case of a Census). Moreover, such
panels suffer from problems of confidentiality and of data protection legislation, so there is often
only very limited access (Buck et al. 1994).

Even if retrospective studies have the advantage of usually being cheaper to collect than panel
data, they suffer from several limitations that are increasingly being acknowledged (Davies and
Dale 1994; Blossfeld and Rohwer 1995):

1. recall bias: retrospective questions concerning motivational, attitudinal, cognitive or

affective states are particularly problematic because respondents find it hard to accurately
recall the timing of changes in these states;

2. there is a limit to respondents’ tolerance for the amount of data that can be collected on
one occasion;

3. retrospective studies must be based on survivors. Those subjects who have died or
migrated will, necessarily, be omitted and biases may arise; retrospective studies can also
misrepresent specific populations.

Table 1.2

Classification of Descriptive Research Studies

Types of Study Features of Study

Cross Sectional One time measurement, including a sample

survey where the emphasis is placed on a
large, representative samples

Longitudinal Repeated measurements on the same

sample, including a traditional panel
(questions remain the same) and an
omnibus panel (questions differ)
There are two types of panels:

1. Traditional panels

2. omnibus panels

Traditional panels ask panel members the same questions on each panel measurement. Omnibus
panels vary questions from one panel measurement to the next. How longitudinal data is applied,
depends on the type of panel used to collets the data. Essentially, the omnibus panel’s primary
usefulness is that it represents a large group- of people, stores, or some other entity- that is
agreeable to providing business research information. An omnibus panel, like traditional panels,
to some extent requires larger entity, implying representatives as well.

Firms are interested in using data from traditional panels because they can gain insights into
changes in consumers’ purchases, attitudes, and so on. For example, data from traditional panels
can show how members of the panel switched brands form one time period to the next. Studies
examining how many consumers switched brands are know as brand-switching studies.

Another use of longitudinal data is that of market tracking. Market tracking studies are those that
measure some variable(s) of interest, that is, market share or unit sales over time. By having
representative data on brand market shares, for example, a business manger can “track” how his
or her brand is doing relative to a competitive brand’s performance.

3.2.3. Casual Research Design

Causality may be thought of as understanding a phenomenon in terms of conditional statements
of the form “if x, then y.” These “if –then” statements become our way of manipulating variables
of interest. For example, if the thermostat is lowered, then the air will get cooler. If I drive may
automobile at lower speeds,, then my gasoline mileage will increase. If I spend more on
adverting then sales will rise. As humans, we are constantly trying to understand the world in
which we live. Fortunately for mankind there is an inborn tendency to determine casual
relationships. This tendency is ever present in our thinking and our actions. Likewise,
commercial managers are always trying to determine what will cause a change in consumer
satisfaction, a gain in market share, or an increase in sales. Unfortunately, our desire to
understand our world in terms of causality, if then statements is very difficult, if not impossible
because there are formal conditions that must be in place before a researcher can attest to
causality. Table 1.3 describes the formal requirements for causality. Consumers are bombarded
on daily and sometimes even hourly basis by a vast multitude of factors, all of which could cause
them to act in one way or another. Nevertheless, there is a high “reward” in the marketplace for
even partially understanding casual relationships. How are causal relationships determined? As
Table 1.3 indicates, they are determined through the use of experiments.

Condition Description

Co variation It must be demonstrated that the casual variable occurs with

 the caused variable and that the two variables have an orderly
relationship (for example, as price goes down, sales go up
Time Sequence
It must be demonstrated that the casual variable changed prior
to or simultaneous with observed changes in the caused
variable (for example, prices were lowered on Monday, and
sales go up for Monday and all other days when prices where
lower).
Systematic elimination
It must be demonstrated that all other possible causal variables
are eliminated from candidacy (for example if an advertising
campaign began on the day we lowered prices, we could not
eliminate the ad campaign as a cause of sales going up)

Experimental design It must be demonstrated that a valid experiment has been

conducted in order to state that the variable is unequivocally
causal (for example, a formal market test would be designed
and conducted in order to determined the effect of a price
reduction on sales).

Casual research design and Internal Validity

Internal validity is the approximate truth about inferences regarding cause-effect or causal
relationships. Thus, internal validity is relevant only in studies that try to establish a causal
relationship. It's not relevant in most observational or descriptive studies,. However, for studies
that assess the effects of social programs or interventions, internal validity is perhaps the primary
consideration. In such contexts, you want to be able to conclude that your program or treatment
made a difference—it improved test scores or reduced symptoms, as shown in the Figure below.

However, there may be reasons, other than your program, that explain why test scores improve
or symptoms are reduced. The key question of internal validity is whether observed changes can
be attributed to your program or intervention (the cause) and not to other possible causes
(sometimes described as alternative explanations for the outcome).

A schematic view of the conceptual context for internal validity

One of the things that are most difficult to grasp about internal validity is that it is relevant only
to the specific study in question. That is, you can think of internal validity as a zero-
generalizability concern. All that internal validity means is that you have evidence that what you
did in the study (for example, the program) caused what you observed (the outcome) to happen.
It doesn't tell you whether what you did for the program was what you wanted to do or whether
what you observed was what you wanted to observe; those are construct validity concerns. It is
possible to have internal validity in a study and not have construct validity. For instance, imagine
a study in which you are looking at the effects of a new computerized tutoring program on math
performance in first-grade students. Imagine that the tutoring is unique in that it has a heavy
computer-game component and you think that will really improve math performance. Finally,
imagine that you were wrong. (Hard, isn't it?) It turns out that math performance did improve
and that it was because of something you did, but it had nothing to do with the computer
program. What caused the improvement was the individual attention that the adult tutor gave to
the child; the computer program didn't make any difference. This study would have internal
validity because something you did affected something that you observed. (You did cause
something to happen.) The study would not have construct validity because the label "computer-
math program" does not accurately describe the actual cause. A more accurate label might be
"personal adult attention."

Since the key issue in internal validity is the causal one, I'll begin by discussing the conditions
that need to be met to establish a causal relationship in a research project. Then I'll discuss the
different threats to internal validity—the kinds of criticisms your critics will raise when you try
to conclude that your program caused the outcome. For convenience, I divide the threats to
validity into three categories. The first involves the single-group threats—criticisms that apply
when you are studying only a single group that receives your program. The second consists of
the multiple-group threat—criticisms that are likely to be raised when you have several groups in
your study (such as a program and a comparison group). Finally, I'll discuss what I call the social
threats to internal validity—threats that arise because social research is conducted in real-world
human contexts where people will react to not only what affects them but also what is happening
to others around them.

3.2.4. EXPERIMENTAL RESEARCH DESIGN

Experiments, if conducted correctly can enable a better understanding of the relationship
between a causal hypothesis and a particular phenomenon of theoretical or practical interest.
True experimental design / random experimental design), provides the highest degree of control
over an experiment, enabling the researcher the ability to draw causal inferences with a high
degree of confidence.

The researcher attempt to control or manipulate variables appropriate to determine how one
variable affect the other and by so doing test a hypothesis and provide explanation. The
researcher needs to identify the independent variable, i.e. the variable under his/her control and
can be changed to measure the effect on the dependent variable. Set the experimental setting and
define the experimental and control group and conduct pre and post testing to measure and test
the variables relationship. Experiments, if conducted correctly can enable a better understanding
of the relationship between a causal hypothesis and a particular phenomenon of theoretical or
practical interest.

True experimental design

A design in which subjects are randomly assigned to program and control groups. With this
technique, every member of the target population has an equal chance of being selected for the
sample. This design is the strongest method for establishing equivalence between a program and
control group because of the random assignment of sample element as indicated above. True
experimental design provides the highest degree of control over an experiment, enabling the
researcher the ability to draw causal inferences with a high degree of confidence.

Quasi-experimental group design

Quasi-experimental group design differs from true experimental group design by the omission of
random assignment of subjects to a program and control group. As a result, you can not be sure
that the program and the control group are equivalent.

EXPOST FACTO DESIGN

In this case the researcher cannot control the variables or manipulate them. Therefore the
researcher can only report what has happened or what is happening through careful sample
selection and statistical analysis of the findings

Experimental Designs and Internal Validity

Experimental designs are usually considered the strongest of all designs in internal validity.
Why? Recall that internal validity is at the center of all causal or cause-effect inferences. When
you want to determine whether some program or treatment causes some outcome or outcomes to
occur, you are interested in having strong internal validity. Essentially, you want to assess the
proposition: If X, then Y. Or, in more colloquial terms:   If the program is given, then the
outcome occurs.

Unfortunately, it's not enough to show that when the program or treatment occurs, the expected
outcome also happens because many reasons, other than the program, might account for why you
observed the outcome. To show that there is a causal relationship, you have to simultaneously
address the two propositions:   

If X, then Y.
and
If not X, then not Y.
Or, once again more colloquially:   

If the program is given, then the outcome occurs.

and
If the program is not given, then the outcome does not occur.

If you are able to provide evidence for both of these propositions, you've in effect isolated the
program from all of the other potential causes of the outcome. You've shown that when the
program is present, the outcome occurs, and when it's not present, the outcome doesn't occur.
That points to the causal effectiveness of the program.

Think of all this like a fork in the road. Down one path, you implement the program and observe
the outcome. Down the other path, you don't implement the program and the outcome doesn't
occur. But, can you take both paths in the road in the same study? How can you be in two places
at once? Ideally, what you want is to have the same conditions—the same people, context, time,
and so on—and see whether when the program is given you get the outcome and when the
program is not given you don't. Obviously, you can never achieve this hypothetical situation. If
you give the program to a group of people, you can't simultaneously not give it! So, how do you
get out of this apparent dilemma?

Perhaps you just need to think about the problem a little differently. What if you could create two
groups or contexts that are as similar as you can possibly make them? If you could be confident
that the two situations are comparable, you could administer your program in one (and see
whether the outcome occurs) and not give the program in the other (and see whether the outcome
doesn't occur). If the two contexts are comparable, this is like taking both forks in the road
simultaneously. You can have your cake and eat it too, so to speak.

That's exactly what an experimental design tries to achieve. In the simplest type of experiment,
you create two groups that are equivalent to each other. One group (the program or treatment
group) gets the program, and the other group (the comparison or control group) does not. In all
other respects, the groups are treated the same. They have similar people, live in similar contexts,
have similar backgrounds, and so on. Now, if you observe differences in outcomes between these
two groups, the differences must be due to the only thing that differs between them—that one
received the program and the other didn't.

so how do you create two equivalent groups? The approach used in experimental design is to
assign people randomly from a common pool of people into the two groups. The experiment
relies on this idea of random assignment to groups as the basis for obtaining two similar groups.
Then, you give one the program or treatment and you don't give it to the other. You observe the
same outcomes in both groups.
The key to the success of the experiment is in the random assignment. In fact, even with random
assignment, you never expect the groups you create to be exactly the same. How could they be,
when they are made up of different people? You rely on the idea of probability and assume that
the two groups are probabilistically equivalent, or equivalent within known probabilistic ranges.

If you randomly assign people to two groups, and you have enough people in your study to
achieve the desired probabilistic equivalence, you can consider the experiment strong in internal
validity and you probably have a good shot at assessing whether the program causes the
outcome(s).

However, many things can go wrong. You may not have a large enough sample. Some people
might refuse to participate in your study or drop out part way through. You might be challenged
successfully on ethical grounds. (After all, to use this approach you have to deny the program to
some people who might be equally deserving of it as others.) You might meet resistance from the
staff members in your study who would like some of their favorite people to get the program.

The bottom line here is that experimental design is intrusive and difficult to carry out in most
real-world contexts, and because an experiment is often an intrusion, you are setting up an
artificial situation so that you can assess your causal relationship with high internal validity. As a
result, you are limiting the degree to which you can generalize your results to real contexts where
you haven't set up an experiment. That is, you have reduced your external validity to achieve
greater internal validity.