CHARACTERISTICS OF RESEARCH TOOL Lalrinchhani ,Roll No-11,1st Yr Msc.

N
INTRODUCTION:
The foundation of good research and of good decision making in evidence-based practice (EBP) is the
trustworthiness of the data used to make decisions.
Working in an EBP setting requires the nurse to have the best data available to aid in the decision-making process.
How can an individual make a decision if the results being used as the foundation of that process cannot be trusted?
Put simply, a person cannot make a decision unless the results are trustworthy and correct.
Reliability and validity are the most important qualities of research tools which are measured.
I.RELIABILITY:
The reliability of an instrument denotes the consistency of the measures obtained of an attribute, concept, or
situation in a study or in clinical practice.
DEFINITIONS:
1. Reliability is defined as the ability of an instrument to create a reproducible results,because without the
ability to reproduce results no truth can be known. An instrument’s reliability is the consistency with which it
measures the target attribute.
2. Reliability is the degree of consistency and accuracy with which an instrument measures the attribute for
which it is designed to measure.
3. Reliability is defined as the ability of an instrument to create reproducible results. Therefore, reliability is
concerned with consistency of the measurement tools. A tool can be considered reliable if it measures an
attribute with similar results on repeated use.
4. Reliability is defined as the consistency or repeatability of test results.
5. Reliability is the extent to which scores for people who have not changed are the same for repeated
measurements,under several situations,including repetition on different occasions, by different versions of a
measure, or in the form of different items on a multi-item instrument(internal consistency).
Reliability Testing
Reliability testing examines the amount of measurement error in the instrument being used in a study. Reliability
exists in degrees and is usually expressed as a correlation coefficient (ranging from -1.00 through .00 to +1.00), with
1.00 indicating perfect reliability and 0.00 indicating no reliability.
The most widely used correlation coefficient is Karl Pearson's correlation coefficient, called Pearson’s r. Karl
Pearson’s formula for estimation of reliability:

In the above formula r = correlation coefficient, n = number of pairs of scores or sample, ∑xy = sum of the products
of paired scores, ∑x = sum of x scores, ∑y= sum of y scores, ∑x2 = sum of squared x scores and ∑y2 = sum of squared y
scores.
Reliability coefficients of 0.80 or higher would indicate strong reliability for a psychosocial scale such as the State-
Trait Anxiety Inventory by Spielberger et al. (1970).

COMPONENTS / ASPECTS OF RELIABILITY:

1. Stability Reliability or Test-Retest Reliability:
The stability aspect of reliability means research instrument provides same results when used when used
consecutively for two or more times..
Test-retest reliability: Replication takes the form of administering a measure to the same people on two occasions.It
is conducted to examine instrument stability, which reflects the reproducibility of a scale’s scores on repeated
administration over time when a subject’s condition has not changed.
EQUIVALENCE Reliability or Inter-Rater Reliability, Intra-Rater Reliability:
This aspect of reliability is estimated when a researcher is testing the reliability of a tool which is used by two
different observers to observe a single phenomenon simultaneously and independently, or two presumably parallel
instruments are -administered to an individual at about the same time.
Inter-rater Reliability: Comparison of the equivalence of the judging or rating of two observers independantly on the
same people is referred to as interrater reliability.
Intra-rater Reliability: Assessment in which the same rater makes the measurements on two or morevoccasions,
blinded to the ratings assigned previously. It is an index of self-consistency.
Parallel-Forms Reliability: Comparison of two paper-and-pencil instruments to determine their equivalence in
measuring a concept is referred to as alternate forms reliability or parallel-forms reliability.
The calculation of inter-rater and intra-rater reliability is performed using the following calculation:
r= Number of agreements
Number of possible agreements

Example: Out of 3 judges and 5 criteria, number of agreements =9

9/15= 0.6
There is no absolute value below which inter or intra-rater reliability is unacceptable. However, any values less than
0.80(80%) raises concern about the reliability of the data because there is 20% chance of error.
3.Internal Consistency or Split- Half Reliability:
It is also called homogeneity, used primarily with paper-and-pencil tests or scales. Internal consistency ensures
that all the subparts of a research instrument measure the same characteristics
Statistical calculation (split-half method)
Procedure of calculating split-half reliability of research instrument involves following steps:
 Divide items of a research instrument in two equal parts f through grouping either in odd number questions
and even number questions or first-half and second-half item groups.
 Administer two subparts of the tool simultaneously, score them independently and compute the correlation
coefficient on the two separate scores by using the following formula.This formula is an alternative formula
of calculating Karl Pearson's correlation coefficient used for unpaired observations .Formula 1 :

 In split-half, to overcome the underestimation of reliability of entire scale, as the formula given above has
estimated reliability of only half items, the following formula is used to estimate the reliability of entire test.
Formula 2

where r = the correlation coefficient computed on the split-halves with Formula and r 1 = the estimated
reliability of the entire test.
 The split-half technique is frequently used to estimate the internal consistency; however, a more preferred
method is Cronbach's alpha or coefficient alpha that may be calculated by using the following formula 3:

where r = the estimated reliability k = the total number of items in the test, σ =¿ the variance of each
individual item σ y2= the variance of the total test scores ∑=the sum of
VALIDITY:
The validity of an instrument indicates the extent to which it actually reflects or is able to measure the construct
being examined.
1. Content Validity
The discussion of content validity also includes face validity and the content validity index. Face validity is a
subjective assessment that might be made by researchers, expert clinicians, or even potential subjects. Because
this is a subjective judgment with no clear guidelines for making the judgment, face validity is considered the
weakest form of validity (De Von et al.,2007).
Content validation of an instrument should be done with a minimum of three experts. For postgraduate studies,
a panel of six experts is considered adequate,however, the panel should include at least one statiscian.
For example: When developing a depression scale, researchers must establish whether the scale covers the full
range of dimensions related to the construct of depression, or only parts of it. If, for instance, a proposed
depression scale only covers the behavioral aspects of depression and neglects to include affective ones, it lacks
content validity and is at risk for research bias.
2. Content Validity Ratio and Index
In developing content validity for an instrument, researchers can calculate a content validity ratio (CVR) for each
item on a scale by rating it O (not necessary), 1 (useful), or 3 (essential).
Example: An item that is rated as quite relevant by four out of five experts would have CVI of 0.8 by the formula
CVI= Number of expert in agreement/number of expert
4/5=0.8
Researchers recommend that a scale with excellent content validity should composed of item level CVIs of 0.78(at
least 9 experts) or higher.
3. Readability of an Instrument
Readability is an essential element of the validity and reliability of an instrument.
Construct Validity
Construct validity focuses on determining whether the instrument actually measures the theoretical construct that it
purports to measure, which involves examining the fit between the conceptual and operational definitions of a
variable.
Validity From Factor Analysis
Factor analysis is a valuable approach for determining evidence of an instrument’s construct validity. This analysis
technique is used to determine the various dimensions subcomponents of a phenomenon of interest.
b). Convergent Validity
In examining the construct validity of a new instrument, it is important to determine how closely an existing
instrument measures the same construct as a newly developed instrument (convergent validity
c). Divergent Validity
Divergent validity can be examined when an instrument is available that measures the construct opposite to the
construct measured by the newly developed instrument.
d). Validity From Contrasting (or Known) Groups
To test the validity of an instrument, identify groups that are expected (or known) to have contrasting scores on the
instrument and generate hypotheses about the expected response of each of these known groups to the construct.
Next, select samples from at least two groups that are expected to have opposing responses to the items in the
instrument.
e). Evidence of Validity From Discriminant Analysis
Instruments sometimes have been developed to measure constructs closely related to the construct measured by a
newly developed instrument.
f). Successive Verification of Validity
After the initial development of an instrument, it is hoped that other researchers would begin using the instrument
in additional studies.
Criterion-Related Validity
Criterion related validity is strengthened when a study participants score on an instrument can be used to infer his
or her performance on another variable or criterion. The two types of criterion-related validity are predictive validity
and concurrent validity. Predictive validity is the extent to which an individual's score on a scale or instrument can be
used to predict future performance or behavior on a criterion (Waltz et al., 2010).
Concurrent validity focuses on the extent to which an individual's score on an instrument or scale can be used to
estimate his or her present or concurrent performance on another variable or criterion. Thus, the difference
between concurrent validity and predictive validity is the timing of the measurement of the other criterion.
ACCURACY, PRECISION, AND ERROR OF PHYSIOLOGICAL MEASURES
Accuracy and precision of physiological and biochemical measures tend not to be reported in published studies.
Standards for most biophysical measures are defined by national and international organizations such as the
International Organization for Standardization (ISO, 2015a) and the Clinical Laboratory Standards Institute ( CLSI,
2015).