Math 146_Spring 2020

_____________________________________________________________________________________

Section 1.3: Introduction to Experimental Design

Planning and Conducting a Study
Scientific studies are made to answer questions about our world such as:
a) Is a new flu vaccine effective in preventing illness?
b) Is the use of bicycles helmets on the rise? Are injuries resulting from bicycle accidents less
severe for riders who wear helmets that for those who do not?
c) Do engineering students pay more for textbooks than psychology students?
d) Are hands-free cell phones safer and less likely to divert driver’s attention?
Data collection and analysis allow researchers to answer such questions. The data collection step is
critical in obtaining reliable information. The type of analysis that is appropriate and the conclusions that
can be drawn depend on how the data are collected.

The data analysis process includes the following steps:

1. Understand the nature of the problem: We must know the goal of the research and what questions we
hope to answer. It is important to have a clear direction before gathering data to avoid being unable to
answer the questions of interest using the data collected.

2. Deciding what to measure, how to measure it and determine sources of data:

 What kind of data needs to be collected for the study? Carefully define the variables to be studied
and decide what information is needed to answer the questions of interest.
 Identify individuals of the study (people or objects) in which information can be obtained from.
 Determine if you need to use an entire population or a representative sample. If using a sample,
decide on a viable sampling method.
Examples:
a) In a study of relationship between the weight of Division I football players and position
played would need to collect data on player’s weight and position.
b) If you want answers to the question “do engineering students pay more for textbooks than
psychology students?”, then you would have to collect data on:
_____________________________________________________
_____________________________________________________

3. Data collection. This step is crucial.

 The researcher must first decide whether an existing data source is adequate or whether new data
must be collected.
 If a decision is made to use existing data, it is important to understand how the data were collected
and for what purpose so that any resulting limitations are fully understood and judged to be
acceptable.
 If new data are to be collected, a careful plan must be developed. The type of statistical analysis
and subsequent conclusions that can be drawn depend on the data collected. In your data
collection plan, address issues of ethics, subject confidentially and privacy.

4. Summarizing Data and Data Analysis: After the data are collected, the next step involves a
preliminary analysis that includes summarizing the data graphically and numerically. This initial analysis
provides insight into important characteristics of the data and can provide guidance in selecting
appropriate methods for further analysis. Then apply the appropriate statistical methods.
Finally note any concerns you might have about your data collection methods and list any
recommendations for future studies.
 Math 146_Spring 2020
_____________________________________________________________________________________

Types of Statistical Studies

1. Observational Study– investigators merely observe subjects and measure variables of interest without
assigning treatments to the subjects. The researcher observes what is happening or what has happened in
the past and tries to draw conclusions based on these observations. When collecting the data, nothing is
altered that may affect the data.
Example:
a) If one wants to study the behavior patterns of autistic children, one has to observe these children,
find out how they interact with their parents and with other children, and then draw conclusions based
on the observations. Here, the researcher simply observes and measures specific characteristics but
does not modify the subjects being studied (subjects are NOT given any specific treatment/instruction
that would modify the results). Observational studies involve using variables that cannot be
manipulated by the researcher such as studying the driving patterns of teenagers, observing drug users
versus non-drug users, right-handedness versus left-handedness, etc.
Disadvantages:
 Since the researcher does not control variables, a definite cause-and effect situation cannot be
shown since other factors may have had an effect on the results.
 Can be expensive and time-consuming (for example, if one wants to study the habitat of lions in
Africa, one would need a lot of time and money).

2. Experimental Study
 In an experimental study, the researcher manipulates one of the variable and tries to determine how
the manipulation influences other variables.

 In an experimental study, investigators apply treatments to individuals or experimental units (people,

animals, plots of land, etc) and then proceed to observe the effect of the treatment. The group that
receives the treatment is called the ________________ group while the group that does not receive
treatment is called the ________________group. In certain cases, the control group is given a
______________ (a pill that contains none of the drug that is being tested). After responses from
both groups are observed, results are compared.

 Common examples of experimental studies are studies made in the medical field such as clinical trials
of drugs. To determine the effectiveness of a newly discovered drug, a group of subjects are given the
drug (treatment group) and a second group are NOT given the drug (control group).

Examples of experimental studies:

a) An excellent example of an experimental study was the 1954 field trial of the Salk vaccine (vaccine
against polio) in which a large group of children were given the actual Salk vaccine and another large
group were given a placebo that contained NO medicine or drug. In a well-designed experiment, effects
in the treatment group are compared to the effects in a group NOT given the treatment. The use of two
groups (treatment group and placebo/control group) will make it possible to evaluate the effectiveness of
the vaccine in preventing polio.

b) A pharmaceutical company discovered a new drug for lowering cholesterol levels. Before this
medicine is released by the Food and Drug Administration for public use, it undergoes a “clinical trial”.
Researchers and doctors have to do a study to determine whether this drug actually works. To find out,
they will give this drug to a group of people with high cholesterol levels for a certain amount of time and
then measure the effect and document the results. Then researchers will also use a second group who
 Math 146_Spring 2020
_____________________________________________________________________________________

WILL NOT receive the drug. Then researchers will compare the results of the two groups. If the
medication did significantly lower cholesterol levels in the treatment group, then this will be approved by
the FDA for public use.

When performing experiments that make use of a placebo group, sometimes a placebo effect occurs.
Placebo effect is when a subject who does not receive the treatment shows similar results as the subjects
receiving treatment. This is primarily the result of the patient just believing in the treatment.

Statistical Terms Associated with Experimental Studies:

a) blinding technique = technique in which the subject doesn’t know whether he or she is receiving a
treatment of a placebo. This is used to counter the placebo effect. Blinding is used so that investigators
can determine whether the treatment effect is significantly greater than the placebo effect. The polio
vaccine experiment was double blind. Blinding occurs at two levels:
 The individuals in the study (children being injected) didn’t know whether they were getting the Salk
vaccine or a placebo.
 The doctors who gave the injections and evaluated the results didn’t know either
 Double-blind experiments help control biases that a doctor might pass on to a patient.

b) randomized two treatment experiment:

It is difficult to account for all variables that might influence a patient’s response to a treatment.
Sometimes patients assigned to a treatment group and a control group are carefully matched by age,
gender, level of medical condition, etc. A common way to assign patients to the treatment and control
group is by using a random selection process.

c) replication = refers to the repetition/duplication of an experiment so that results can be confirmed or

verified.

d) confounding: When conducting experiments, results are sometimes ruined because of confounding.
A confounding variable is an extraneous variable that is not included in the study but has a significant
effect on the variables of the study. Its effect on the response variable (or dependent variable) cannot
be distinguished from the variables that are included in the study. Or it is a variable that was not taken
into consideration by the researcher and has an effect on the results of a study – the effect of this
extraneous variable can’t be distinguished from the variables that are included in the study.
Example:
 A soccer coach wanted to improve the team’s playing ability, so he had them run two miles a day. At
the same time the players decided to take vitamins. In two weeks the team was playing noticeably
better, but the coach and players did not know whether it was from the running or the vitamins.

Quasi-Experimental Study
In a true experimental study, subjects should be assigned to groups randomly. When random assignment
is not possible, researchers use intact groups. These are done quite often in education where intact groups
are available in the form of existing classrooms. When these groups are used, the study is said to be
quasi-experimental study.

Survey
A common way to gather data about people is to ask them questions. Sometimes possible responses are
simply YES or NO. Other times, the respondents choose a number on the scale that represents their
feelings such as “strongly agree” or “strongly disagree” (Likert scale). In the case of an open-ended
 Math 146_Spring 2020
_____________________________________________________________________________________

response or discussion-type response, the researcher must determine a way to convert the response into a
category or number.
a) telephone survey
 less costly, people maybe more candid in their opinions since there’s no face-to-face contact.
 drawback: some people might not be home when calls are made or might not have phones,
so not all people have a chance of being surveyed.
b) mailed questionnaire
 used to cover a wider geographic area than the phone surveys or personal interview because
they are less expensive to conduct.
 respondents can remain anonymous if they wish.
 disadvantage: low number of responses, inappropriate answers to questions, some people
may have difficulty reading or understanding the question
c) personal interview
 researcher can obtain in-depth responses
 disadvantages: costly because the interviewer must be trained in asking questions and
recording responses, interviewer may be biased in selecting respondents
Potential Drawbacks:
 Nonresponse – when the selected respondent cannot be contacted or refuses to respond. If the
nonresponse is sufficiently high, the study may be biased and some sort of adjustment may need to be
made.
 Voluntary response samples – occur when people are invited to respond to a question/survey and all
who respond are counted. This method is used by call-in shows, and internet polls. In this case,
samples are considered self-selected. Only those who have strong opinions about an issue will likely
to participate, therefore creating bias.

Taking a census
A census is a count or measure of an entire population. Taking a census provides complete
information but it is often costly and difficult to perform.

Every 10 years, the U.S. Department of Commerce Census Bureau is required to conduct a census of the
United States in order to learn the demographic make-up of the U.S. Everyone whose usual residence is
the U.S. is required to fill out questionnaire and provide information such as number of household
members, number of years at present address, household income, etc. Contacting some members of the
population such as homeless is almost impossible and some members of the population will not respond
at all. In such cases, statistical estimates for the missing responses are often supplied.