Running Header: M4 Research Design

M4 Research Design Critique Report

Texas A&M University – Commerce
Kelly Luza
Running Header: M4 Research Design

Assignment Instructions: Using the information in the instructions below, add your inputs directly into the empty
textbox below the instructions.

Scenario
Provide a scenario when it would be advantageous for researchers to use each type of design (experimental,
correlational, causal comparative) and summarize the similarities and differences between each of these designs.

Discuss the steps that should be taken to ensure quality survey design and necessary measures to be taken to
increase participant response rates.

Your Scenario Here

Experimental research would be an excellent tool to evaluate weather students in a third-grade class were divided
into two groups where one group was permitted to use their textbooks during an exam and the other group only had
the exam. Data would need to be drawn on participants on both groups to look at their test scores regarding reading
comprehension and test taking strategies. Once the exam was administered data could be used to compare time
management on the exam, note taking skills prior to the exam, overall comprehension of the objectives being tested.
Grades on identical previous exams could be reviewed to see if any major changes in student performance changed
for those who could use the textbook and those who couldn’t.

For further breakdown of data, a student could be chosen from the group that was permitted to use the textbook to
compare his test scores to a previous test. You could also allow him to take another exam using a textbook to see if a
correlation exists between his achievement based on the use of external factors.

You can use casual comparative research to see if certain types of questions had more correct answers in the group
that could use their textbook or the group that had no external aides. This would build an association between the
two groups as to whether the book was a help or a hinderance.

Experimental research differs from casual comparative in that the participants are chosen at random whereas the
other the students are chosen on a given variable. In experimental the researcher is trying to prove the cause and
effect relationship between the independent and dependent variables. This can be proven due to having a control
group. In casual comparative there isn’t a control group and therefore cause and effect can’t be established. This
results in researchers not being able to manipulate different conditions to get a more desirable result. It is also less
rigorous then an experiment since the researcher is only establishing an association between the groups and the
outcome.
Correlational research uses statistical based data to compare multiple variables as associated with scores. It employs
research based questions which will yield predictable outcomes based on non-manipulated statistics. Of the three it
breaks down difficult, complex factors into more useable information.

To warrant that you are producing a well-designed survey, certain guidelines need to be built into its design. The first
major undertaking is to procure a random sample of participants based on number and type of study being
researched. A nom-biased survey needs to be developed that will yield validity and reliability of the study and the
survey instrument. Once the instrument is completed distribution and return need to be established. Follow up and
analysis of the data need to be completed prior to the release of information.

All participants need to consent in writing that they will fill out the survey and be allowed to be interviewed when
requested. The researcher needs to enclose an informational letter along with guidelines of when it needs to be
returned by mail to him/her. Participation on this study is based on the interest level of the participants and will yield
a higher return level. The survey needs to be brief and no longer then three pages. If any surveys are not returned a
reminder needs to be sent after two weeks to request that it be returned so the study can be completed. Incentives
can also be used to entice the participant to complete and return the survey.
Running Header: M4 Research Design

First Article Summary

Provide a paragraph, which highlights the purpose of study and research questions that the author(s) attempted to
address. When you refer to this particular study, simply identify this research by citing the author(s) and date. I would
then like for you to provide the full APA reference at the conclusion of this document.

Your First Article Summary Here

The purpose of this study was to describe which research methods have been utilized in engineering education. The
authors, Borrego, Douglas, and Amelink (2009) focused on the three methods of quantitative, qualitative, and mixed.
The goal of the study was to see which method of research was preferred among the students and how it was being
utilized in the classroom and in the field. The research questions they attempted to address were:
1. Are the three methods used equally?
2. Are they collaborating with other sciences for research purposes?
3. Are there alternate methods being used to prevent engineering from being a “low consensus field”?
4. Why do engineering students prefer to use quantitative methods over the other methods?

Summation of Research Literature

This section should summarize in two or three paragraphs previous research that the authors found that led to their
particular study.

Your Summation Here

The references that the authors used were a combination of methods texts from social sciences and examples from
the “Journal of Engineering Education”. When looking at quantitative methods the authors focused on a review article
that was posted in the JEE 2005 special edition issue. This issue focused on assessment methods relevant to
engineering education and focused on analyzing statistics on a previous unknown situation. Next it focused on using
statistical analysis to determine if there were differences among groups on various assessments. Lastly it focused on
studying theory in relation to testing hypotheses.
Researchers began to realize that a mixed method needed to be used because there was a lack of different indicators
that could be used to make future predictions of students being successful in engineering education. Previous studies
also manipulated part of the data which yielded false information during the analysis part of the outcomes. The need
for statistical analysis proved to be a factor for conducting further quantitative research.
Several outside research studies regarding this topic pointed out the danger in assuming qualitative research was
easier and less rigorous than quantitative research. Unless mixed research was used many who studied the data felt
that the findings weren’t always trustworthy. All agreed that mixed methodologies involved the integration of the
data at different stages thus improving the overall research process. It was further concluded that validity and
reliability concerning all phases of the research yielded more comprehensive results by using the mixed method.
Running Header: M4 Research Design

Method for Collecting and Analyzing Data

This section needs to highlight in two or three paragraphs the method that was used to collect data and how data
were analyzed.

Your Data Collection/Analysis Method Here

When focusing on the different methods the authors provided a definition, research questions, evaluation criteria,
and examples from the “Journal of Engineering Education”. In the journal they focused on pulling articles that tested
out hypotheses as opposed to looking at assessment methods. They then took their findings and tested them out at
an international conference on engineering education research.
It was concluded that the collection of data is not useful or meaningful unless the instrument has been designed to
cover the purpose and goal of the study. The instrument needed to use both exploratory and explanatory areas so
that results could be obtained from both the survey and personal interviews. Unless the instrument meets objectivity,
reliability, and validity the entire study can be found to be inconclusive regardless of research method.

Results and Conclusion

This section needs to summarize in two or three paragraphs the results of this study and address conclusions and
recommendations made by the authors

Your Results and Conclusion Here

Upon completion of their study they concluded that no one method was giving preferential treatment over the other.
Rather that the method that was chosen was based on which research questions the researcher sought to answer. A
lot of engineering research focuses on how outcomes happened by lowering the root of the problem to a small list of
direct variables. Quantitative research methods, especially classroom based experiments, were used more frequently
with engineers. A high trend among students was “transforming qualitative behaviors or work products into
quantitative data for statistical analysis (et al, pg. 59).
When deciding which method to use engineering students focused on three main criteria, “(1) the research problem,
(2) the personal experiences of the researcher, and (3) the audience (et al, pg. 63) If the researcher wanted to test
out his findings on a larger population using the assessments of “surveys, meta-analysis, and experimental designs (et
al, pg. 54) then he would choose a quantitative research method. A qualitative method approach is only used when
the researcher was looking to use “surveys, interviews, focus groups, conversational analysis, observation,
ethnographics (et al, pg.56) Mixed, was seldom used, since it involved collecting data chronologically and then using
that data in one or more stages in the research process. The participants did express that they would like to increase
the use of qualitative methods in the future.
Some of the recommendations that the authors conveyed were they would like the students and faculty to be more
open to describing the actual methods associated with the research process and what challenges the researcher has
faced when picking a method. They challenge readers to investigate alternate research methods and collaborating
with the other science majors who have more tenure in the educational field.

Second Article Summary

Your Purpose of Study Here
The purpose of the study was to investigate why several different research techniques were not utilized by a school
to see if teachers from three different academic levels were qualified to decide which students were to continue or
Running Header: M4 Research Design

enter a gifted and talented language arts program in elementary school. One group of teachers taught in a gifted
and talent program, a second group worked with regular education students, while a third group of teachers taught
students who were identified as having learning disabilities or emotional and behavioral disorders. The team studying
the prior research was looking to see if one group of teachers referred a higher number than others and what criteria
was employed with their decisions. The result was that the teachers of gifted and talented referred the most while
the teachers whose students had educational disabilities referred the least. The new research team decided to take
the earlier study and prove that quantitative methods were used scantily in the placement decisions along with
quantitative evidence not producing the “why” factor to a survey that had not been developed to draw more input
prior to decisions being made.

Your Summation of Research Literature Here

The research previously done on gifted and talented placement relied heavily on non-specific data from the teachers.
The main criteria employed by the teachers was if the students had “gifted characteristics” and if they felt these
types of students needed more academic challenges. It was found that no test scores were available to the teachers
prior to their decisions of placement. The lack of quantitative research was not utilized due to their not being a
survey that could provide the why or the how factors that should be studied. Qualitative research was not also
utilized due to a random control group had not been established with the teachers being the sole selectors of which
students were to be considered. It was further found that little to no data collection was involved which led to the
lack of analysis and inference techniques that could have been employed.
The original study failed to also use a formulation stage that would help to focus on the goal of the study. The
purpose was placement but little to no data could support the decision. The purpose of the original study was
placement based primarily on teacher observation and recommendation. The purpose of the study did not yield
significant evidence that should have been incorporated into a final decision. Little to no planning was also used with
no random selection along with no instrument that could be completed by students and parents regarding the gifted
and talented program. Finally, no implementation stage was put into place prior to the final decision since there was
no data to be considered.

Your Method of Collecting and Analyzing Data Here

The new research team decided that the previous study was inconclusive due to lack of research objective, lack of
data, type of analysis, and type of inference. The need for data was established and collected in the following
categories: interview, responses to survey, categorization of data using quantitative, qualitative, and mixed research
techniques. The new study also looked at the reading background of the students. Time was also a factor that had
not been previously used and the team decided to have the students read two different articles to see how many
fully or only partially completed the reading and questions associated with it. After all data was collected, the team
took all the analysis to see if the previous study had taken enough factors into account prior to the placement or
continuance of students in a gifted and talented program and realized that more subjective than objective criteria
was used, and the placements were not valid.
It was observed by the new research that gifted education had some areas that were never addressed in the original
placement of the students. Standardized test data was not used, and comparison groups were not in place which
invalidated performance levels. There was also no information regarding the attrition of students with little or no
data to assess their abilities and performance levels. Another factor that had not been addressed is a specific
definition of giftedness and instruments that should be used to collect data that would be valid.

Your Results and Conclusions Here

Using mixed research allowed the new team to obtain credible data and work with a random sample of students
prior to assessing placement. They further felt that mixed research design gave them predictability of what might
emerge in the future placements of students into the gifted and talented program. The first study also allowed each
Running Header: M4 Research Design

teacher to have equal status and similar weight in their selections regardless of their background or awareness of the
criteria that should have been used in their placement choices. They felt that using mixed research techniques in the
field of gifted and talented could yield better understanding of the rate of students placed or allowed to continue in
gifted and talented education.

References
Please list your two references in APA format. Don’t forget to list in alphabetical order based upon first author.

Your References Here – APA Format

Borrego, Maura, Elliot P. Douglas, and Catherine T. Amelink. "Quantitative, Qualitative, and Mixed Research
Methods in Engineering Education." Journal of Engineering Education 98.1 (2009): 53-66. ProQuest. 26 Nov. 2017.

Leech, Nancy L., et al. "Mixed Research in Gifted Education: A Mixed Research Investigation of Trends in the
Literature." Journal for the Education of the Gifted 12 2011: 860-75. ProQuest. 26 Nov. 2017.