Journal of Personality Assessment, 94(2), 186–195, 2012

Copyright
C Taylor & Francis Group, LLC

ISSN: 0022-3891 print / 1532-7752 online
DOI: 10.1080/00223891.2011.627967

The Importance of Modeling Method Effects: Resolving the

(Uni)Dimensionality of the Loneliness Questionnaire
CHAD EBESUTANI,1 CHRISTOPHER F. DRESCHER,2 STEVEN P. REISE,3 LAURIE HEIDEN,4 TERRY L. HIGHT,4
JOHN D. DAMON,4 AND JOHN YOUNG2
1
Department of Psychiatry and Human Behavior, University of Mississippi Medical Center
2
Psychology Department, University of Mississippi
3
Psychology Department, University of California at Los Angeles
4
Mississippi Children’s Home Services and CARES School, Jackson, Mississippi

This study sought to resolve the dimensionality of the Loneliness Questionnaire (LQ; Asher, Hymel, & Renshaw, 1984) by applying recommended
confirmatory factor analytic procedures that control for method effects (Brown, 2003). This study was needed given that inconsistent findings have
been reported recently regarding the structure of this instrument (Bagner, Storch, & Roberti, 2004) and all models to date have not accounted for
method effects due to the non-reversed-worded and reversed-worded items of this instrument. Using a large sample of youth in Grades 2 through
12 (N = 11,725), we compared the previously reported 1- and 2-factor models with a newly posited 1-factor model that incorporated correlated
error terms to account for method effects. We found that the 1-factor model that included correlated error terms fit the data best, and that this factor
structure evidenced measurement invariance across boys and girls in childhood, but not in adolescence. The meaning of the LQ indicators was also
consistent for boys across development, but evidenced differences for girls in childhood versus adolescence. More generally, it was demonstrated
that modeling method effects is vital to accurately understanding the dimensionality of loneliness when reversed-worded and non-reversed-worded
items are used as indicators. The measurement and clinical implications of these findings are discussed.

Loneliness is a universal experience that has been associated (e.g., Avery, 1982; Davis & Franzoi, 1986). Page and colleagues
with the human condition since antiquity. Contemporary re- (2006) more recently administered the revised UCLA Loneli-
searchers have defined loneliness as “the cognitive awareness ness Scale to 2,624 high school students from Taiwan, 2,519
of a deficiency in one’s social and personal relationships, and the high school students from Thailand, and 3,320 high school stu-
ensuing affective reactions of sadness, emptiness, or longing” dents from the Philippines, and found that boys scored signifi-
(Asher & Paquette, 2003, p. 75). A recent review (Heinrich & cantly higher on loneliness compared to girls in their Thailand
Gullone, 2006) has highlighted the clinical significance of lone- sample but not in their Philippines or Taiwan samples. Other
liness, including its relationship to several indexes of poor psy- recent studies have been conducted to understand gender dif-
chological and physical health. For example, Nolen-Hoeksema ferences among younger children. For example, Coplan, Clos-
and Ahrens (2002) found that loneliness was significantly re- son, and Arbeau (2007) recently examined loneliness in 139
lated to depression (Pearson’s r = .36–.53) in young (25–35 Canadian kindergarteners and found that loneliness was more
years), middle-aged (45–55 years), and older adults (65–75 closely related to social withdrawal among boys compared to
years). Loneliness is also related to psychological difficulties girls, and was more closely related to overt aggression among
specifically within youth populations, such as social anxiety girls compared to boys. Crick and Ladd (1993) also found that
(Crick & Ladd, 1993). Loneliness has also been found to be as- loneliness evidenced a stronger association with social anxiety
sociated closely with sociometric measures and to be relatively among girls compared to boys in the third and fifth grade in their
stable in youth (Renshaw & Brown, 1993). American (Midwestern) sample. Given the clinical significance
Researchers have also been interested in the ways and ex- of loneliness among both males and females, several assessment
tent to which the experience and correlates of loneliness differ instruments have been developed to assess loneliness in youth.
by gender. Although more is known about gender differences
among adult and college student samples (e.g., Borys & Perl- THE LONELINESS QUESTIONNAIRE
man, 1985; Cramer & Neyedley, 1998), several studies have When assessing loneliness in youth, the Loneliness Question-
been conducted on youth samples across several different coun- naire (LQ; Asher, Hymel, & Renshaw, 1984; also referred to as
tries to understand how loneliness relates to gender. Some ear- the Loneliness and Social Dissatisfaction Scale) is one of the
lier studies among adolescent samples have found that adoles- most well-known and widely used methods of assessing lone-
cent boys tend to report more loneliness than adolescent girls liness in youth (Goossens & Beyers, 2002; Hoza, Bukowski,
& Beery, 2000). The original LQ (Asher et al., 1984) includes
24 items: 16 items related to feelings of loneliness and 8 filler
Received February 16, 2011; Revised April 3, 2011.
Chad Ebesutani is now in the Department of Psychology at Yonsei University,
items (e.g., “I like to read”). The eight filler items are not scored
Seoul, South Korea. and were included “to help children feel more open and relaxed
Address correspondence to Chad Ebesutani, Department of Psychology, about indicating their attitudes about various topics” (Asher
Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, South Korea 120-749; et al., 1984, p. 1457). The instructions encourage youth to in-
Email: ebesutani@yonsei.ac.kr dicate how true each item is for them on a 5-point Likert-type
186
DIMENSIONALITY OF THE LQ 187

scale ranging from 1 (always true) to 5 (not true at all). Asher searchers noted that more studies are needed to confirm this
and Wheeler (1985) later modified the LQ for use in school finding given the unexpected demonstration of a two-factor
settings, adjusting the nonfiller items to be narrowly focused on model. As discussed further later, however, finding two factors
the school environment. For both versions, high internal con- when reversed-worded and non-reversed-worded items make
sistency (Cronbach’s α = .90) and factor-analytic findings sup- up a scale is rather common (e.g., Bagozzi & Heatherton, 1994)
ported a single factor of loneliness. Cassidy and Asher (1992) and potentially misleading given that it should be expected that
also modified the LQ by changing the wording to questions a two-factor model will almost always “fit” the data better than
(e.g., “I’m lonely at school” to “Are you lonely at school?”) and a standard one-factor model when method effects are not ac-
they reduced the number of response options from five to three counted for (e.g., no correlated error terms included).
(“yes,” “no,” “sometimes”). This modification also displayed ac- These researchers were the first to touch on this two-factor
ceptable internal consistency (Cronbach’s α = .77–.79). Despite versus one-factor issue related to the LQ; however, such a
these periodic modifications to the LQ, the original LQ version debate is not unique to this instrument, nor has it been ade-
(Asher et al., 1984) remains the most popular version of the quately resolved in the field. For example, although the Penn
LQ and continues to be used in research to measure loneliness State Worry Questionnaire (PSWQ) was designed to mea-
in youth. For example, Frankel and colleagues (2010) recently sure the unidimensional construct of worry, the measure in-
used the LQ to measure the efficacy of a manualized parent- cludes both reversed-worded and non-reversed-worded items
based intervention to improve social skills among children with and factor-analytic studies continue to argue for a two-factor
autism spectrum disorders in their randomized controlled trial. model with the reversed-worded and non-reversed-worded items
Bauminger, Shulman, and Agam (2003) also examined the de- loading on separate factors (Fresco, Heimberg, Mennin, & Turk,
gree to which children with autism understand the relationship 2002). This pattern of results has also been raised in other re-
between loneliness and social interaction compared to typically cent studies examining other similarly constructed instruments
developing, age-, IQ-, and gender-matched children. Coplan (e.g., the Rosenberg Self-Esteem Scale [Corwyn, 2000; Marsh,
and colleagues (2007) also administered the LQ to 139 Cana- Scalas, & Nagengast, 2010], the adult UCLA Loneliness Scale
dian kindergarten school children and found loneliness to be [Dussault, Fernet, Austin, & Leroux, 2009; Hawkley, Browne,
significantly related to anxiety, aggression, and peer exclusion, & Cacioppo, 2005; Nielson, 2002]). Unfortunately, there con-
concluding it could thus be an important early predictor of inter- tinues to be disagreement in the field regarding the factor struc-
nalizing and externalizing problems. In addition to the LQ also ture of several different instruments that include both reversed-
recently being used to examine the relationship between loneli- worded and non-reversed-worded items. The complexity (and
ness and various problems among U.S. clinical samples, such as often omission) of accurately modeling method effects has led
youth social anxiety (Storch, Brassard, & Masia-Warner, 2003) researchers to question the dimensionality of instruments and
and obesity (Storch et al., 2006), the LQ is also still widely assign labels to apparent “factors” in an attempt to ascribe dif-
used in several other countries. For example, Chen and col- ferential meaning to each factor. Through this process, it has
leagues (2004) administered the LQ to 2,263 children in Grades thus been common for factors to be reified as they are assigned
3 through 6 (ages 9–12) in Brazil, Canada, China, and Italy. names, which could be problematic when they are not substan-
In addition to finding that loneliness is relatively stable over tively meaningful. This issue has not been limited to the LQ;
time, they also found that loneliness was significantly related nearly every measure including both reversed-worded and non-
(albeit with some variability) to depression, suicidality, poor reversed-worded items has undergone such debate. Fresco and
self-concept, and psychosomatic problems across all four na- colleagues (2002), for example, labeled the purported two fac-
tional samples, supporting the robustness of loneliness as an tors of the PSWQ Worry and Absence of Worry, but Brown
important predictor of various problems across various ethnic (2003) later demonstrated these two factors were rendered non-
groups and cultural contexts. Given the current interest among interpretable once controlling for method effects. Both Hawkley
researchers in using the LQ to measure loneliness in both child et al. (2005) and Dussault et al. (2009) also argued that the Re-
and adolescent samples across several different countries, it is vised UCLA Loneliness Scale does not represent its intended
important that researchers understand well the dimensionality unidimensional construct of loneliness, but instead, “a three-
of this measure and, for example, how this measure should be facet mental representation of social connection” (Dussault
best interpreted. et al., 2009, p. 855), labeled Isolation, Relational Connected-
ness, and Collective Connectedness. Regarding the LQ—the
focus of this study—Bagner and colleagues (2004) recently
The Dimensionality of the Loneliness Questionnaire argued for a two-factor structure of the LQ and labeled the pur-
The method factor. Despite the initial development stud- ported two factors Loneliness and Social Satisfaction. Method
ies positing a one-factor model of the LQ (Asher et al., 1984; effects were not controlled for or included in their models, call-
Asher & Wheeler, 1985), a recent study found that non- ing into question the meaningfulness of these two purported
reversed-worded and reversed-worded items loaded on two factors.
separate factors, and that this two-factor model was associ- Although further delineating the dimensionality of mea-
ated with significantly better model fit than the one-factor sures and identifying nuances of constructs are useful when
model (Bagner, Storch, & Roberti, 2004). These researchers warranted, it is important for researchers to exercise cau-
were the first to raise the issue of the potential multidi- tion before positing new factor structures when scales in-
mensionality of the LQ, reporting that the non-reversed- clude both reversed-worded and non-reversed-worded items
worded items clustered together to make up the first fac- and method factors have not been controlled for in factor-
tor (labeled Loneliness) and the reversed-worded items make analytic investigations. Fortunately, substantial progress has
up the second factor (labeled Social Satisfaction). These re- been made in the area of confirmatory factor analysis (CFA)
188 EBESUTANI ET AL.

of multitrait–multimethod (MTMM) data that now allows re- the importance of examining the dimensionality of measures
searchers to control for such method effects when exam- while controlling for method effect. Resolving dimensional-
ining scale dimensionality. In general, two main supported ity issues is important and should be pursued for several rea-
CFA approaches for modeling method effects have been docu- sons, including informing researchers and clinicians regarding
mented: (a) the correlated traits–correlated uniqueness (CTCU) how to score and interpret scale information and thus appro-
modeling approach—which involves specifying correlated priately identify individuals who are high or low on a given
error terms among the reversed-worded or non-reversed-worded construct.
indicators (see Corwyn, 2000; Marsh, 1996; Marsh et al.,
2010)—and (b) the latent method factor (LMF) approach, THIS STUDY
also referred to as the correlated traits–correlated method The aim of this study was thus to examine the dimensionality
approach—which involves specifying a latent reversed-worded of the LQ given (a) mixed reports regarding the nature of the
item method factor, a latent non-reversed-worded item method LQ factor structure and (b) recent recommendations involving
factor, or both to account for the variance between the items and specific CFA techniques (cf. Brown, 2003) to resolve ambiguity
the reversed-worded and non-reversed-worded item method fac- regarding the dimensionality of such measures. Specifically, this
tors (Bagozzi, 1993; Marsh & Grayson, 1995). Notably, each study utilized the CTCU approach and compared a one-factor
approach is associated with unique advantages and disadvan- model (accounting for method effects due to the shared method
tages (see Bagozzi, 1993; Brown, 2006, for a more detailed variance of the non-reversed-worded and reversed-worded items
account of the strengths and weaknesses of both approaches), following the procedures performed by Brown, 2003) versus a
and not surprisingly, the CTCU approach has been found to two-factor model of the LQ as reported by Bagner and col-
perform better in some contexts, whereas the LMF approach leagues (2004). We hypothesized that the one-factor model ac-
is better in other contexts (cf. Tomas, Hontangas, & Oliver, counting for the method effects would evidence better model fit
2000). over a two-factor model and would support the LQ as a unidi-
Different ways to model CTCU and LMF approaches have mensional measure of loneliness, consistent with earlier exami-
been proposed, including at least three approaches for model- nations of its psychometric properties (Asher et al., 1984; Asher
ing correlated uniqueness: (a) correlating all error terms among & Wheeler, 1985). To explore the robust nature and generaliz-
all non-reversed-worded items, (b) correlating all error terms ability of the one-factor model, we also examined measurement
among all reversed-worded items, and (c) correlating all er- invariance of the one-factor model (i.e., equal factor structure,
ror terms among all non-reversed-worded items and simulta- equal factor loadings, equal indicator intercepts) across boys
neously correlating all error terms among all reversed-worded and girls (cf. Brown, 2003). We also examined measurement
items. Importantly, Marsh (1996) noted that “the model with invariance across children and adolescent subgroups given that
correlated uniquenesses among all reversed-worded and all non- the structure of other internalizing-related constructs (e.g., anx-
reversed-worded items is not identified” (p. 815), rendering this iety and depression) has been found to vary across development
approach inadvisable (see also Reilly, 1995). Some authors also from childhood to adolescents (Cole, Truglio, & Peeke, 1997).
found that specifying correlated uniqueness among the reversed-
worded items seems to better account for method effects com- METHOD
pared to specifying correlated uniqueness among non-reversed-
worded items (Corwyn, 2000; Marsh, 1996; Tomas & Oliver, Participants
1999). Three different LMF models have also been explored The sample for this study was derived from children and ado-
(cf. Corwyn, 2000; Marsh et al., 2010), which included the lescents in Grades 2 through 12 in public schools across the
following: specifying a latent factor to represent (a) the non- state of Mississippi (median grade = 7) who participated in the
reversed-worded items method effect, (b) the reversed-worded Behavioral Vital Signs Project (BVS) and completed a battery
items method effect, and (c) the non-reversed-worded items and of questionnaires, including the LQ. Inclusion in this study re-
reversed-worded items method effects simultaneously. It is also quired that there were no missing LQ data. We only included
worth noting that Geiser, Eid, and Nussbeck (2008) recently pro- youths with no missing data because the large majority of youth
posed a new method for modeling response styles via a random had no missing LQ data. Using cases with no missing data also
intercept factor approach. Less is known about this relatively obviated complications related to having to impute categorical
new procedure and future studies are likely to clarify how well (ordinal) data. Of the 12,722 cases with any LQ data, 11,725
this approach accounts for method effects in relation to both (92.2%) had no missing data and were included in this study.
CTCU and LMF strategies. Of these youths, 5,346 (48.4%) were boys and 5,694 (51.6%)
Despite the lack of a clear consensus as to which modeling were girls. Youths’ ethnicities were as follows: 5,825 (52.8%)
method is superior, the dimensionality of measures has often White, 4,219 (38.2%) African American, 276 (2.5%) Latino or
been clarified when controlling for the presence of method ef- Hispanic, 203 (1.8%) Asian, and 477 (4.3%) other. Forty-four
fects through any of these supported approaches. For exam- (0.4%) youths did not provide ethnicity data. Regarding family
ple, Brown (2003) used the CTCU approach (i.e., correlated composition, 5,197 (44.3%) youths reported living with both
residuals to account for the shared method variance of the biological parents, 4,646 (39.6%) reported living with biolog-
reversed-worded items) and confirmed the unidimensionality ical mother only, 747 (6.4%) reported living with biological
of the PSWQ after controlling for method effects. Marsh and father only, and 1,066 (9.1%) reported living with neither bi-
colleagues (2010) also recently used the LTM approach and ological mother nor father. Sixty-nine youths (0.6%) did not
found support for the unidimensionality of the Rosenberg Self- report this information. Taking a weighted average of the data
Esteem Scale, despite previous studies positing multiple fac- from the most recent and available U.S. Census Bureau (2010)
tors. These demonstrations have provided clear illustrations of data based on the school districts surveyed in this study (i.e.,
DIMENSIONALITY OF THE LQ 189

Bolivar, Coahoma, Hancock, Harrison, Hinds, Jackson, Jeffer- To examine the dimensionality of the LQ while accounting
son, Madison, Pontotoc, and Simpson Counties in Mississippi), for method variance due to the wording of the LQ items, we used
approximately 22% of families from the districts surveyed in the following CFA procedures recommended by Brown (2003):
this study fell below the poverty line, with the average family (a) We examined the original one-factor LQ solution not incor-
income of $40,500. porating an error term (i.e., no residual covariance among the
reversed-worded items) as originally posited by Asher and col-
Procedure leagues (1984) to serve as a comparison baseline model; (b) we
Data for this study were collected as part of a broader school- then examined a two-factor solution not incorporating an error
based mental health screening initiative in Mississippi (the term to replicate the findings of Bagner and colleagues (2004)
BVS). The BVS administers scientifically supported mental whereby this two-factor model demonstrated significantly bet-
health screenings to youth in Grades 2 through 12 and pro- ter model fit relative to the original one-factor model noted ear-
vides feedback to schools concerning students’ mental health, lier; and (c) finally, we examined a one-factor unidimensional
such as information pertaining to internalizing and externaliz- model of loneliness including correlated error terms (i.e., resid-
ing behavior, including levels of loneliness, given its association ual covariances among the reversed-worded items, based on a
with various forms of psychopathology and functional impair- correlated uniqueness model; cf. Brown, 2003; Marsh, 1996)
ment. The study utilized a passive consent procedure that was and compared the fit of this one-factor model to the fit of the
approved by the Mississippi Department of Education, as well as two-factor model previously described.
each school involved in the BVS, and the University of Missis- We conducted these CFAs using two random samples (split
sippi Institutional Review Board (IRB). Members of the project random sample 1: n = 5,863; split random sample 2: n = 5,862)
staff distributed assessment instruments to each classroom, and as a cross-validation strategy. We also used CFA instead of
teachers were informed of study procedures. Teachers read a exploratory factor analysis to explore the dimensionality of the
brief set of instructions to the students in their respective class- LQ based on the recommended procedures posited by Brown
rooms and handed out assessment packets to their students. (2003), as CFA has shown to offer more power and flexibility
Students provided answers via Scantron brand optical forms. to specifically examine method effects due to the wording of
items (Joreskog, 1969, 1971; Marsh, 1996). To evaluate model
Measure fit, we used the Comparative Fit Index (CFI), Tucker–Lewis
The LQ (Asher et al., 1984) is a 24-item self-report ques- Index (TLI) and the root mean square error of approximation
tionnaire that measures loneliness in youth. The LQ contains (RMSEA). CFI/TLI values of .90 or greater (Bentler, 1990) and
16 items that are summed to obtain a total loneliness score more recently, values of .95 or greater (Hu & Bentler, 1999),
(ranging from 16–80, with higher scores reflecting higher levels have been deemed cutoffs for good model fit. RMSEA values
of loneliness). The 16 items include 10 non-reversed-worded in the range of .05 to .08 and .05 or lower suggest adequate and
items that ask about loneliness directly (e.g., “I’m lonely”) and good fit, respectively (Browne & Cudeck, 1993).
6 reversed-worded items that assess loneliness indirectly (e.g.,
“I have lots of friends”) and need to be reverse-scored. The LQ Chi-square difference tests. We compared model fit be-
a 5-point Likert scale ranging from 1 (always true) to 5 (not tween competing models (e.g., two- vs. one-factor models) us-
true at all). The 16-item scale has been shown to meet bench- ing the χ 2 difference test. With limited information estimators,
marks for adequate internal consistency (α = .90) and stability such as the WLSMV estimator, however, the nested χ 2 differ-
over a 12-month period (Asher & Wheeler, 1985; Cassidy & ence tests must be conducted using a special “difftest” command
Asher, 1992). Although published norms for the measure are not in Mplus given that (a) degrees of freedom are estimated un-
available, previous sociometric examinations have established der such conditions, and (b) the differences between χ 2 values
approximate cutoffs for scores on the measure predicting mem- are not distributed as the traditional χ 2 distribution. The Mplus
bership in an alienated social group (i.e., rejected or neglected; “difftest” makes the necessary adjustments and conducts the
Asher et al., 1984; Asher & Wheeler, 1985). appropriate χ 2 difference test in these situations. This “difftest”
procedure as well as the method employed by Mplus to estimate
Data Analytic Approach degrees of freedom are described in the Mplus Technical Ap-
We conducted CFA using Mplus version 4.21 (B. Muthén & pendices (www.statmodel.com) and in the Mplus User’s Guide
Muthén, 2007) to compare the two-factor solution of the LQ (L. K. Muthén & Muthén, 2006).
posited by Bagner and colleagues (2004) relative to a more par- Multisample solutions. To evaluate measurement invari-
simonious one-factor solution of loneliness incorporating corre- ance (i.e., equal form, equal factor loadings, indicator intercepts)
lated error terms to control for method effects, as suggested by of the best fitting model across meaningful subgroups, we con-
Brown (2003). Due to the LQ response choices being categorical ducted multisample CFA across boys and girls as well as across
(ordinal), we based these CFAs on polychoric correlation matri- younger and older youth. We first examined equal form (struc-
ces (Holgado-Tello, Chacón-Moscoso, Barbero-Garcı́a, & Vila- ture invariance) of the best fitting model across children and
Abad, 2010) and used the robust weighted least-squares with
mean and variance adjustment (WLSMV) estimation method,
given that this estimator has been found to be most appropriate ically, WLSMV estimates parameters using robust standard errors, a mean- and
for handling categorical (ordinal) data (Flora & Curran, 2004).1 variance-adjusted χ 2 test statistic, and also a diagonal weight matrix (which is
not inverted during parameter estimation, thereby not requiring data matrices to
be positive definite, unlike WLS, which does require positive definite matrices).
1The WLSMV has some noted advantages over other estimators for categor- WLSMV also does not require sample sizes as large as WLS to obtain reliable
ical (ordinal) data, such as the weighted least squares (WLS) estimator. Specif- and unbiased parameter estimates (Flora & Curran, 2004).
190 EBESUTANI ET AL.

adolescents. If equal form was supported (via fit statistics meet- RESULTS
ing benchmarks for good fit), we then examined factor loading Initial Item Diagnostics
invariance across groups via the χ 2 difference test (cf. Brown,
2003). A nonsignificant χ 2 difference test statistic means that None of the LQ items was associated with significant de-
the equal factor loadings constraint across samples did not sig- partures from normality. Skew ranged from –1.60 to 1.25 and
nificantly degrade model fit, thereby supporting the invariance kurtosis ranged from –1.09 to 1.05. The means of the absolute
of factor loadings across groups. If factor loadings were found values for skew and kurtosis were 0.87 and 0.61, respectively.
to be invariant across groups, we then tested the equality of
Confirmatory Factor Analysis
indicator intercepts across groups. In general, the ability to ex-
amine the invariance of any parameter requires that all previous Original one-factor model. We fit the original one-factor
parameters evidence invariance in the order specified earlier. solution (with no correlated error terms included to account
Therefore, noninvariance of factor loadings, for example, would for the method effects related to the reversed-worded and non-
preclude further tests of equal indicator intercepts. We allowed reversed-worded items) to both split random samples (as well
error variances to be freely estimated in all models (cf. Brown, as the full sample). This one-factor model served as a baseline
2003; Byrne, Shavelson, & Muthén, 1989). comparison model that was originally posited by the instrument
Put in more colloquial terms, support for equal form across developers (Asher et al., 1984). The fit indexes associated with
boys and girls would indicate that the general structure of the this model appear in Table 1. Across both random split samples,
one-factor model of loneliness represented the data well across overall model fit was not good (e.g., RMSEA = .116, .164;
these groups. Support for equal factor loadings across groups CFI = .78, .77), which is consistent with previous findings
would indicate that the LQ indicators evidenced similar rela- reported by Bagner and colleagues (2004).
tionships to the latent construct of loneliness across boys and
girls and that the LQ items thus have the same meaning to both Two-factor model. A two-factor solution (with no corre-
boys and girls.2 Support for equal indicator intercepts would lated error terms included to account for the method effects
indicate that individuals from both groups who are at the same related to the reversed-worded and non-reversed-worded items)
level of the latent trait evidenced the same observed scores on was fit to both split random samples (as well as the full sample).
the various LQ indicators. Unequal indicator intercepts, on the The fit indexes associated with this two-factor model appear in
other hand, would suggest that boys and girls, for example, ev- Table 1. Across both random split samples, overall model fit
idenced different observed indicator scores despite being at the met benchmarks for good model fit (e.g., RMSEA = .071, .075;
same level of the latent trait. CFI = .94, .94). Replicating Bagner and colleagues’ (2004)
Given the possibility that the construct of loneliness might findings, this two-factor model also fit the data significantly bet-
have different meanings to boys and girls at different stages ter than the original one-factor model in split random sample 1,
in development, we examined the measurement invariance of χ 2diff (1) = 1434.40, p < .001, and split random sample 2, χ 2diff
the parameters noted previously across boys and girls in sep- (1) = 1491.73, p < .001.
arate youth subsamples: children (Grades 2–7) and adolescent
One-factor model with correlated error terms. The fit
(Grades 8–12).3 Lastly, given that (a) the χ 2 difference tests for
statistics of the one-factor model that included an error term of
these multigroup CFAs were based on the χ 2 statistic (which is
correlated residuals to account for the method effects related
known to be sensitive to sample size), and (b) our sample size
to the reversed-worded and non-reversed-worded items appear
was very large (increasing the likelihood of obtaining signifi-
in Table 1. Model fit indexes across all samples were good
cant findings with the χ 2 difference test due to trivial differences;
(e.g., RMSEA = .076, .075; CFI = .95, .94). Importantly, this
Brown, 2006)—thereby being overly sensitive to finding nonin-
one-factor model (which included an error term of correlated
variance of any given parameter subjected to a χ 2 difference test
residuals to account for the method effects) fit the data of both
(Kelloway, 1995)—we conducted these χ 2 difference tests on
random split samples significantly better than the two-factor
random subsamples of 500 youths per group (cf. Cabrero-Garcia
model noted previously: split random sample 1, χ 2diff (10) =
& Lopez-Pina, 2008; Wännström, Peterson, Asberg, Nygren, &
256.64, p < .001; split random sample 2, χ 2diff (10) = 31.96, p
Gustavsson, 2009) given that such a sample size is still suffi-
< .001.4 This one-factor model also fit significantly better than
ciently large, yet offers a fairer test of invariant measurement
the alternate two-factor model posited by Bagner and colleagues
responses across these noted subsamples.
(2004) whereby the residuals between two items (Items 4 and 8)
were allowed to correlate and be freely estimated due to being
highly similar in item content: split random sample 1, χ 2diff (10)
= 271.20, p < .001; split random sample 2, χ 2diff (10) = 380.19,
2Stated yet another way, factor loading invariance is similar to having parallel p < .001. These results support the one-factor model (with
regression slopes in a regression framework; factor loading invariance across
correlated residuals due to item-wording effects) as the best
boys and girls would mean that a one-unit change in the latent construct of fitting model. We also reported model fit based on the full sample
loneliness is associated with the same statistically significant changes in the
observed factor indicators across boys and girls.
3We divided our sample based on grade, as opposed to age, given that the 4Given the possibility that these significant χ 2 difference test results were

anonymous nature of our IRB-approved data collection procedures prevented us due to the large sample size of this study, we reran these analyses based on a
from collecting information related to actual age. We included second graders random subsample of 500 youths. Results were consistent, χ 2diff (9) = 37.12, p
in the child cohort given that children ages 5 and 6 years old have evidenced < .001, demonstrating that the improved model fit was related to significantly
at least a basic understanding of the concept of loneliness (Cassidy & Asher, improved fit due to the one-factor model with correlated errors accounting for
1992). the noted method factor, as opposed to the large sample size.
DIMENSIONALITY OF THE LQ 191

TABLE 1.—Fit statistics for the confirmatory factor analytic models of the Loneliness Questionnaire.

Model CFI TLI RMSEA χ2 df χ 2diff df p χ 2diff

Split (random) sample 1 (n = 5,863)

Model A: One factor .78 .91 .166 8135.68 50e
Model B: Two factorsa .94 .98 .071 2107.55 69e 1434.40e 1e <.001
Model C: One factor with method effectsd .95 .98 .073 2078.57 65e 256.64e 10e <.001
Split (random) sample 2 (n = 5,862)
Model A: One factor .77 .91 .164 8427.31 53e
Model B: Two factors b .94 .98 .075 2371.33 69e 1491.73e 1e <.001
Model C: One factor with method effectsd .94 .98 .075 2166.46 64e 361.96e 10e <.001
Full sample (N = 11,725)
Model A: One factor .77 .91 .165 16652.55 52e
Model B: Two factors c .94 .98 .073 4432.25 70e 2917.97e 1e <.001
Model C: One factor with method effectsd .94 .98 .073 4243.19 66e 599.34e 10e <.001

Note. CFI = Comparative Fit Index; TLI = Tucker–Lewis Index; RMSEA = root mean square error of approximation; χ 2diff = χ 2 difference test.
a
Correlation between factors was –.54. bCorrelation between factors was –.53. cCorrelation between factors was –.54. dCovariances among the errors of the reversed-worded items.
e
Special estimation procedures were employed by Mplus when conducting the nested χ 2 difference tests because (a) degrees of freedom need to be estimated and (b) the differences
between χ 2 values are not distributed as chi-square when the weighted least-squares with mean and variance adjustment estimator is used. The “difftest” procedure employed as well
as the method used by Mplus to estimate degrees of freedom are described in the Mplus Technical Appendices (www.statmodel.com) and in the Mplus User’s Guide (L. K. Muthén &
Muthén, 2006). Because of these special estimation procedures, however, degrees of freedom and χ 2 need to be estimated and thus cannot simply be derived via subtraction.

(appearing in Table 1), which also revealed good model fit. then examined measurement invariance of the LQ across ado-
All correlations between residuals were statistically significant, lescent boys and girls. We fit the one-factor model (including an
ranging from .24 to .36 (average = .30). error term of correlated residuals) across our random subsample
of adolescent boys (n = 500) and girls (n = 500). Single-sample
Invariance of the Loneliness Questionnaire (One-Factor solutions for each of these adolescent subsamples suggested
Model) Across Boys and Girls good fit for the one-factor model, supporting configural invari-
To specifically examine the psychometric properties and ro- ance among adolescent boys (RMSEA = .08, CFI = .96, TLI =
bust nature of the one-factor model (with modeled method ef- .99), and adolescent girls (RMSEA = .076, CFI = .97, TLI =
fects) across meaningful subgroups, we first examined the mea- .99). The test of equal form across (adolescent) boys and (ado-
surement invariance of the LQ across boys and girls. We exam- lescent) girls was also supported, as evidenced by supportive
ined measurement invariance across boys and girls within our fit indexes (RMSEA = .075, CFI = .96, TLI = .99).5 We then
child subsample and adolescent subsample separately, given that constrained factor loadings to be equivalent across (adolescent)
the structure of other related internalizing problems has been boys and (adolescent) girls. Interestingly, this model constraint
found to differ across development (e.g., Cole et al., 1997). significantly degraded goodness of fit, χ 2diff (7) = 34.38, p <
.001, suggesting that the LQ items do not have the same mean-
Child subsample (boys vs. girls). We first fit the one-factor ing across boys and girls in the adolescent sample. Modification
model (including an error term of correlated residuals) across indexes revealed, however, that LQ10 (“I can find a friend when
boys (n = 500) and girls (n = 500) selected randomly from I need one”) and LQ22 (“I am well-liked by kids in my class”)
the child subsample. Single-sample solutions for these child might be particularly responsible for the omnibus test revealing
subsamples suggested good fit for the one-factor structure of factor loading noninvariance. When allowing these two factor
loneliness across boys and girls, supporting configural invari- loadings to be freely estimated across groups (constraining all
ance: child boys (RMSEA = .076, CFI = .95, TLI = .98), child other factor loadings to be equal across groups), model fit was
girls (RMSEA = .08, CFI = .93, TLI = .97). The test of equal not significantly degraded, χ 2diff (6) = 12.68, ns, supporting
form across (child) boys and (child) girls was also supported, as partial metric invariance of factor loadings across adolescent
evidenced by supportive fit indexes (RMSEA = .08, CFI = .94, girls and boys.6 We then constrained indicator intercepts to be
TLI = .98). We then constrained factor loadings to be equivalent
across (child) boys and (child) girls. This did not significantly 5To obtain adequate model fit, we added additional correlated error terms
degrade goodness of fit, χ 2diff (7) = 18.05, ns, suggesting that between LQ9 (“I feel alone”) and LQ21 (“I’m lonely”), LQ16 (“I get along
the LQ items have the same meaning to both child boys and with other kids”) and LQ20 (“I don’t get along with other children”), LQ1
child girls. Given equal factor loadings, we then constrained (“It’s easy for me to make new friends at school”) and LQ6 (“It’s hard for
indicator intercepts to be equal across groups. This also did not me to make friends”), and LQ9 (“I feel alone”) and LQ17 (“I feel left out of
lead to a significant degrading in model fit, χ 2diff (24) = 22.64, things”) based on the largest modification indexes that identified items with
ns, suggesting that both (child) boys and (child) girls who are at highly similar item content, thereby warranting correlated residuals between
the same level of the latent trait of loneliness evidence the same these highly synonymous items.
6Byrne et al. (1989) first introduced the concept of partial measurement in-
observed scores on the LQ indicators. Together, these results
variance to determine, for example, whether a subset of parameters is invariant
support the measurement invariance of the LQ across (child)
across groups (i.e., testing whether some, but not all, parameters are equivalent
boys and girls in our child sample. across groups). Importantly, Byrne et al. noted that invariance evaluations can
continue when an omnibus test of invariance is not supported if partial mea-
Adolescent subsample (boys vs. girls). After finding mea- surement invariance is supported. This is important given that full measurement
surement invariance of the LQ across (child) boys and girls, we invariance is unlikely to hold for all parameters in practice.
192 EBESUTANI ET AL.

TABLE 2.—Latent structure of the Loneliness Questionnaire (LQ): Confirmatory factor analysis using the full sample (N = 11,725) as well as child boys (n =
3,318), child girls (n = 3,466), adolescent boys (n = 2,352), and adolescent girls (n = 2,589).

LQ Item Item Content Full Sample Child Boys Child Girls Adolescent Boys Adolescent Girls

Reversed-worded items
LQ1 It’s easy for me to make new friends at school. −.38 −.29 −.38 −.37 −.53
LQ4 I’m good at working with other children. −.21 −.11 −.28 −.11 −.34
LQ8 I have lots of friends. −.46 −.42 −.51 −.43 −.52
LQ10 I can find a friend when I need one. −.27 −.18 −.28 −.24 −.40
LQ16 I get along with other kids. −.37 −.27 −.37 −.38 −.51
LQ22 I am well–liked by kids in my class. −.34 −.24 −.34 −.33 −.48
Non-reversed-worded items
LQ3 I have nobody to talk to. .70 .65 .65 .78 .77
LQ6 It’s hard for me to make friends. .78 .76 .75 .84 .80
LQ9 I feel alone. .80 .81 .77 .85 .82
LQ12 It’s hard to get other kids to like me. .77 .74 .74 .83 .80
LQ14 I don’t have anyone to play with. .80 .79 .77 .85 .80
LQ17 I feel left out of things. .72 .72 .67 .80 .73
LQ18 There’s nobody I can go to when I need help. .76 .74 .70 .82 .78
LQ20 I don’t get along with other children. .69 .65 .62 .79 .72
LQ21 I’m lonely. .84 .82 .80 .88 .87
LQ24 I don’t have any friends. .81 .79 .78 .86 .82
Scale reliability (Cronbach’s alpha) .81 .82 .78 .85 .79
M 47.87 46.15 47.59 48.28 50.09
SD 9.85 10.49 9.31 10.54 8.49

equal across groups (while maintaining the previous partial fac- TLI = .98).7 The omnibus test for factor loading invariance,
tor loading invariance constraints). This did not significantly however, was not supported across child and adolescent girls,
degrade model fit, χ 2diff (27) = 44.25, ns, supporting indica- as evidenced by a significant degrading in model fit when con-
tor intercept invariance across adolescent boys and girls for the straining factor loadings to be equivalent across these groups,
majority of LQ items. χ 2diff (33) = 32.80, p < .001. Partial factor loading invari-
ance, however, was supported, as evidenced by a nonsignificant
Invariance of the Loneliness Questionnaire One-Factor degrading in model fit once factor loadings for LQ3 (“I have
Model Across Young and Older Youth nobody to talk to”) and LQ22 (“I am well-liked by kids in
my class”) were allowed to be freely estimated across groups,
We then conducted additional analyses to more specifically χ 2diff (38) = 18.29, ns. Invariance of indicator intercepts was
understand the ways in which the meaning of the LQ items then supported across child and adolescent girls, as constraining
might changes across development by gender. We thus con- indicator intercepts to be equivalent across groups did not sig-
ducted multigroup CFAs across child and adolescent boys (to nificantly degrade model fit, χ 2diff (28) = 1.34, ns. These results
examine measurement invariance of the LQ across developmen- suggest that some indicators evidenced different relationships to
tal stages for boys), as well as across child and adolescent girls the latent factor across child and adolescent girls, whereas full
(to examine measurement invariance of the LQ across develop- factor loading invariance was found among child and adolescent
mental stages for girls). boys.

Boys subsample (children vs. adolescents). The test of Factor Loadings

equal form across children and adolescents was supported, as The factor loadings of the 16 LQ items appear in Table 2.
evidenced by supportive fit indexes (RMSEA = .08, CFI = .96, These results were based on the full sample (N = 11, 725) as
TLI = .98). Factor loading invariance was also supported across well as among all child boys (n = 3,318), child girls (n = 3,
child and adolescent boys, as evidenced by a nonsignificant 466), adolescent boys (n = 2,352), and adolescent girls (n =
degrading in model fit when constraining factor loadings to 2,589)—as opposed to the random subsamples of 500 youths
be equivalent across groups, χ 2diff (6) = 15.76, ns. These per group—to maximize the precision of the factor loading es-
results suggest that the meaning of the LQ items as indicator timates. All factor loadings were significant (p < .01), indicat-
of loneliness did not differ across developmental periods for ing that all items were adequate indicators of loneliness. The
boys (i.e., childhood vs. adolescence). Invariance of indicator signs of all factor loadings were also in the expected direction
intercepts was also supported across child and adolescent boys, (i.e., for every unit change in the latent factor, reversed-worded
as constraining indicator intercepts to be equivalent across
groups did not significantly degrade model fit, χ 2diff (24) =
7Again,
5.58, ns. These results support the measurement invariance of to obtain adequate model fit with this female sample, we needed to
the LQ items across child and adolescent boys. add additional correlated error terms between LQ9 (“I feel alone”) and LQ21
(“I’m lonely”), LQ16 (“I get along with other kids”) and LQ20 (“I don’t get
along with other children”), LQ1 (“It’s easy for me to make new friends at
Girls subsample (children vs. adolescents). The test of school”) and LQ6 (“It’s hard for me to make friends”), and LQ9 (“I feel alone”)
equal form across children and adolescents was supported, as and LQ17 (“I feel left out of things”) based on the largest modification indexes
evidenced by strong fit indexes (RMSEA = .076, CFI = .96, that identified items with highly similar item content.
DIMENSIONALITY OF THE LQ 193

items were associated with the indicated change in the opposite dimensional nature of the LQ, future studies could consider
direction; and for every unit change in the latent factor, employing item response theory (IRT) analysis on the LQ, as
non-reversed-worded items were associated with the indicated (a) IRT has evidenced advantages over classical test theory with
change in the same direction). The factor loadings of the non- respect to evaluating psychometric properties and informing
reversed-worded items, however, were substantially larger than the development of future versions of tests (Reise & Henson,
the loadings of the reversed-worded items. 2003), and (b) IRT is most appropriate for use among uni-
dimensional scales. The non-reversed-worded items also evi-
Internal consistency of the one-factor model. Cronbach’s denced substantially greater factor loading estimates relative to
alpha internal consistency reliability estimates based on all 16 the reversed-worded items. Future studies should examine how
LQ items appear in Table 2. Alpha coefficients met benchmark much information non-reversed-worded items contribute toward
for acceptable reliability (α >.70; Nunnally, 1978) based on the the measurement of loneliness in youth relative to reversed-
full sample and child and adolescent (boys and girls) subsam- worded items. These findings also suggest that similar method-
ples, supporting the 16 items as reliably tapping the construct ological CFA procedures should be applied to the UCLA Lone-
of loneliness within each group. liness Scale for Adults (Russell, Peplau, & Cutrona, 1980). This
would be relevant given that Knight, Chishol, March, and God-
DISCUSSION frey (1988) also reported a two-factor structure of the UCLA
In this study, we used CTCU-based CFA procedures recom- Loneliness Scale with non-reversed-worded items loading on
mended by Brown (2003) to examine the dimensionality of the one factor and reversed-worded items loading on the other fac-
latent construct of loneliness as targeted by the LQ. We were tor. Given the findings reported here and prior work by Brown
specifically interested in whether the LQ represents a two-factor (2003) and Marsh (1996) that has demonstrated the importance
model of loneliness and social satisfaction—recently posited by of including method effects when examining such CFA models,
Bagner and colleagues (2004)—or a one-dimensional model it is likely that a one-factor model of loneliness would also be
of loneliness, after controlling for the method effects of the found with the adult UCLA Loneliness Scale.
reversed-worded and non-reversed-worded LQ items. Recently, Although the one-factor model (including correlated residu-
Brown (2003) examined a similar issue with the PSWQ due als to control for method effects) was supported across all our
to Fresco and colleagues (2002) positing that the PSWQ (orig- subsamples (i.e., child boys, child girls, adolescent boys, ado-
inally posited as a one-factor measure of worry) represented lescent girls), the properties (and meaning) of the LQ indicators
a two-factor measure of worry and absence of worry. Brown interestingly varied across adolescent boys and girls (for certain
(2003), however, demonstrated the importance of including an items) as well as to some extent across developmental periods
error term in such CFA models to account for method effects, for girls. First, factor loadings and indicator intercepts were
such as the wording of items (e.g., reversed-worded and non- found to be fully invariant across boys and girls specifically in
reversed-worded items) to more accurately model and portray our child subsample, but not in our adolescent subsample. These
the structure and dimensionality of such latent constructs. Al- results suggest that the loneliness indicators on the LQ have the
though Brown recommended that these procedures be applied to same meaning across boys and girls in childhood, but become
other measures to more accurately model and test the structure differentially meaningful items as indicators of loneliness across
of other psychological latent constructs, these procedures have boys and girls in adolescence. This is an important finding, par-
been slow to be adopted and are often not applied by other test ticularly given that the (albeit limited) research on gender dif-
developers and researchers. ferences in loneliness in youth have been conducted primarily in
In this study, we were the first to apply these CFA modeling adolescent samples (e.g., Avery, 1982; Davis & Franzoi, 1986),
procedures to the LQ to clarify its structure and dimension- with the exception of a few conducted on younger child samples
ality. The findings reported here support a one-factor model (e.g., Coplan et al., 2007), which have found greater reports of
of loneliness (when accounting for the method effects of the loneliness among boys compared to girls. Given that only partial
reversed-worded and non-reversed-worded items) over a two- metric invariance was supported in these studies for the factor
factor model of loneliness and social satisfaction. We also con- loading parameters across girls and boys in adolescence, more
firmed, however, that the two-factor model of loneliness and research is needed to better understand whether their observed
social satisfaction did evidence good model fit when method differences in the previous studies reflect actual differences in
effects were not taken into account. This was an important repli- their true score of loneliness underlying their observed scores.
cation of Bagner and colleagues’ (2004) findings, given the au- Relatedly, one interpretation and potential implication of
thors’ stated concerns about their findings potentially being due these findings is that both younger boys and girls might ex-
to demographic artifacts inherent in their sample. Given that our perience loneliness in the same way (although potentially to
sample was ethnically, geographically, and culturally distinct different degrees), as evidenced by the item indicators showing
(made up of mostly White and African American youth from the same relationship with the latent factor across both child
rural areas in the Deep South), the two-factor model noted earlier boys and girls. As youth enter adolescence, however, the expe-
is not specific to Bagner and colleagues’ (2004) unique sample. rience and meaning of loneliness might begin to differ across
Additionally, interpreted in conjunction with our other findings, boys and girls, whereby boys might begin to experience lone-
it can be conclusively stated that this was also not indicative of liness in one type of way and girls might experience loneliness
two separate loneliness-related factors (i.e., loneliness and so- another way (as evidenced by certain item indicators not show-
cial satisfaction), but rather, an artifact of the reversed-worded ing invariance across adolescent boys and girls). More research
and non-reversed-worded item sets of the LQ. is needed in this area to understand whether this is true, and if so,
These findings also have implications with respect to future what types of indicators could best serve to measure loneliness
development of the LQ. Given that we have confirmed the uni- in adolescent boys and girls. This study, however, did find that
194 EBESUTANI ET AL.

factor loadings were fully invariant in boys across childhood research should be conducted to better clarify which approach
and adolescence, but not in girls across the same developmental is most advantageous.
periods. These results suggest that the experience and construct Despite areas for future research, this study was important
of loneliness might change the most among girls (as opposed to in resolving the dimensionality regarding the LQ. Future uti-
boys) as they age and mature into adolescence. This study, how- lization of the LQ should continue to rely on the interpretation
ever, was based on a cross-sectional data set, and thus cannot of the full 16-item total scale, as opposed to the two reversed-
speak directly to these questions. Future studies would do well worded and non-reversed-worded scales separately. Further, al-
to employ longitudinal research designs to formally examine the though the results reported here support the ability to compare
relationship between the meaning and psychometric properties LQ scores across child boys and girls (based on measurement
of loneliness indicators and how they differ in boys and girls invariance findings), it does not seem advisable to compare LQ
across developmental periods. scores across adolescent boys and girls (due to the lack of full
measurement invariance found in this study). Comparisons of
Limitations and Future Directions scores to those of other individuals in their same reference group
Despite the findings and noted areas for future research, there (specific to gender and developmental period) would be most
were limitations to this study that warrant consideration. First, appropriate and suitable to aid in the interpretation of total scale
although this study was based on a very large sample of chil- scores on the LQ. It is also hoped that this study encourages
dren and adolescents, derived from a school-based sample of future research to be conducted that not only refines the LQ,
Southern children, there are potential difficulties with general- but also seeks to better understand the ways in which loneli-
ization of the results. The large ethnic diversity and low socioe- ness might be differentially experienced and expressed across
conomic conditions apparent in rural Mississippi could present developmental periods by boys and girls. Through such work,
challenges to adapting the findings of this research in other geo- we can become better at identifying loneliness and its different
graphical and cultural settings. As such, more research is needed manifestations among youth, thereby allowing for interventions
in other areas to aid in the generalizability of these findings. In to be employed to aid in the amelioration of this condition.
particular, replication of these findings in an urban area that con-
tains diverse ethnic groups beyond White and African American REFERENCES
participants is likely warranted. Likewise, it will be important Asher, S. R., Hymel, S., & Renshaw, P. (1984). Loneliness in children. Child
to conduct studies on clinical samples to more closely examine Development, 55, 1456–1464.
the effects of loneliness on various forms of psychopathology, Asher, S. R., & Paquette, J. A. (2003). Loneliness and peer relations in child-
and possibly its role in treatment outcome. hood. Current Directions in Psychological Science, 12, 75–78.
Although we found evidence for a single, substantively mean- Asher, S. R., & Wheeler, V. A. (1985). Children’s loneliness: A comparison
ingful factor of loneliness underlying both reversed-worded and of rejected and neglected peer status. Journal of Consulting and Clinical
non-reversed-worded items, more psychometric work could be Psychology, 53, 500–505.
performed in future studies to examine the relative performance Avery, A.,(1982). Escaping loneliness in adolescence: The case for androgyny.
of the 10 non-reversed-worded items compared to the 6 reversed- Journal of Youth and Adolescence, 11, 451–459.
Bagner, D. M., Storch, E. A., & Roberti, J. W. (2004). A factor analytic study of
worded LQ items. This would be important, particularly given
the loneliness and social dissatisfaction scale in a sample of African-American
that there have been several studies showing that reversed-
and Hispanic-American children. Child Psychiatry and Human Development,
worded items perform poorly relative to non-reversed-worded 34, 237–250.
items in general (e.g., Muris, Meesters, & Gobel, 2001; Stans- Bagozzi, R. P. (1993). Assessing construct validity in personality research:
bury, Ried, & Velozo, 2006). Stansbury and colleagues (2006), Applications to measures of self-esteem. Journal of Research in Personality,
for example, recently found that the four reversed-worded items 27, 49–87.
on the Center for Epidemiologic Studies–Depression scale did Bagozzi, R., & Heatherton, T. (1994). A general approach to representing mul-
not factor well with the other non-reversed-worded items and ev- tifaceted personality constructs: Application to state self-esteem. Structural
idenced relatively worse psychometric properties. On the other Equation Modeling, 1, 35–67.
hand, other researchers have argued that reversed-worded items Bauminger, N., Shulman, C., & Agam, G. (2003). Peer interaction and loneliness
should not be used with young children (e.g., Marsh, 1996). in high functioning children with autism. Journal of Autism and Developmen-
tal Disorders, 33, 489–507.
More research is thus needed to compare the performance of the
Bentler, P. M. (1990). Comparative fit indices in structural models. Psycholog-
reversed-worded and non-reversed-worded items as meaning- ical Bulletin, 107, 238–246.
ful and psychometrically sound indicators of loneliness among Borys, S., & Perlman, D. (1985). Gender differences in loneliness. Personality
boys and girls in both childhood and adolescence. Further, al- and Social Psychology Bulletin, 11, 63–74.
though both reversed-worded and non-reversed-worded items Brown, T. (2003). Confirmatory factor analysis of the Penn State Worry Ques-
have been traditionally included in scales to reduce the likeli- tionnaire: Multiple factors or method effects? Behaviour Research and Ther-
hood of biased reporting styles due to acquiescence and nonac- apy, 41, 1411–1426.
quiescence, Marsh (1996) has questioned whether this advan- Brown, T. (2006). Confirmatory factor analysis for applied research. New York,
tage is worth the complications associated with introducing NY: Guilford.
Browne, M., & Cudeck, R. (1993). Alternative ways of assessing model fit. In
method effects into a measure. Due to the complexities associ-
K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp.
ated with method effects, Marsh has in fact recommended scale 136–159). Newbury Park, CA: Sage.
developers to consider including either only reversed-worded or Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence
non-reversed-worded items in a measure—or at a minimum, to of factor covariance and mean structures: The issue of partial measurement
include an equal number of reversed-worded and non-reversed- invariance. Psychological Bulletin, 105, 456–466.
worded items to facilitate future research exploring their rela- Cabrero-Garcia, J., & Lopez-Pina, J. (2008). Aggregated measures of functional
tive contributions to measuring the targeted construct. Future disability in a nationally representative sample of disabled people: Analysis
DIMENSIONALITY OF THE LQ 195

of dimensionality according to gender and severity of disability. Quality of Knight, R. G., Chishol, B. J., March, N. V., & Godfrey, H. P. D. (1988). Some
Life Research, 17, 425–436. normative, reliability, and factor analytic data for the revised UCLA Loneli-
Cassidy, J., & Asher, S. R. (1992). Loneliness and peer relations in young ness scale. Journal of Clinical Psychology, 44, 203–206.
children. Child Development, 63, 350–365. Marsh, H. W. (1996). Positive and negative global self-esteem: A substantively
Chen, X., He, Y., De Oliveira, A. M., Lo Coco, A., Zappulla, C., Kaspar, meaningful distinction or artifactors? Journal of Personality and Social Psy-
V., . . . DeSouza, A. (2004). Loneliness and social adaptation in Brazilian, chology, 70, 810–819.
Canadian, Chinese and Italian children: A multinational comparative study. Marsh, H. W., & Grayson, D. (1995). Latent variable models of
Journal of Child Psychology and Psychiatry and Allied Disciplines, 45, 1373– multitrait–multimethod data. In R. H. Hoyle (Ed.), Structural equation mod-
1384. eling: Concepts, issues and application (pp. 177–198). Thousand Oaks, CA:
Cole, D. A., Truglio, R., & Peeke, L. (1997). Relation between symptoms of Sage.
anxiety and depression in children: A multitrait–multimethod–multigroup Marsh, H. W., Scalas, L. F., & Nagengast, B. (2010). Longitudinal tests of
assessment. Journal of Consulting and Clinical Psychology, 65, 110– competing factor structures for the Rosenberg Self-Esteem Scale: Traits,
119. ephemeral artifacts, and stable response styles. Psychological Assessment,
Coplan, R. J., Closson, L. M., & Arbeau, K. A. (2007). Gender differences in the 22, 366–381.
behavioral associates of loneliness and social dissatisfaction in kindergarten. Muris, P., Meesters, C., & Gobel, M. (2001). Reliability, validity, and normative
Journal of Child Psychology and Psychiatry, 48, 988–995. data of the Penn State Worry Questionnaire in 8–12-yr-old children. Journal
Corwyn, R. F. (2000). The factor structure of global self-esteem among adoles- of Behavior Therapy and Experimental Psychiatry, 32, 63–72.
cents and adults. Journal of Research in Personality, 34, 357–379. Muthén, B., & Muthén, L. (2007). Mplus 4.21. Los Angeles, CA: Muthén &
Cramer, K., & Neyedley, K. (1998). Sex differences in loneliness: The role of Muthén.
masculinity and femininity. Sex Roles, 38, 645–653. Muthén, L. K., & Muthén, B. O. (2006). Mplus user’s guide (4th ed.). Los
Crick, N. R., & Ladd, G. W. (1993). Children’s perceptions of their peer expe- Angeles, CA: Muthén & Muthén.
riences: Attributions, loneliness, social anxiety, and social avoidance. Devel- Nielson, E. R. (2002). Investigating the effects of combining positively and
opmental Psychology, 29, 244–254. negatively oriented items on the dimensionality of Likert scales. Unpublished
Davis, M., & Franzoi, S. (1986). Adolescent loneliness, self-disclosure, and pri- doctoral dissertation, Brigham Young University, Provo, UT.
vate self-consciousness: A longitudinal investigation. Journal of Personality Nolen-Hoeksema, S., & Ahrens, C. (2002). Age differences and similarities in
and Social Psychology, 51, 595–608. the correlates of depressive symptoms. Psychology and Aging, 17, 116–124.
Dussault, M., Fernet, C., Austin, S., & Leroux, M. (2009). Revisiting the factorial Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York, NY: McGraw-
validity of the revised UCLA Loneliness scale: A test of competing models Hill.
in a sample of teachers. Psychological Reports, 105, 849–856. Page, R. M., Yanagishita, J., Suwanteerangkul, J., Zarco, E. P., Mei-Lee, C., &
Flora, D., & Curran, P. (2004). An empirical evaluation of alternative methods of Miao, N. (2006). Hopelessness and loneliness among suicide attempters in
estimation for confirmatory factor analysis with ordinal data. Psychological school-based samples of Taiwanese, Philippine and Thai adolescents. School
Methods, 9, 466–491. Psychology International, 27, 583–598.
Frankel, F., Myatt, R., Sugar, C., Whitham, C., Gorospe, C., & Laugeson, E. Reilly, T. (1995). A necessary and sufficient condition for identification of
(2010). A randomized controlled study of parent-assisted children’s friend- confirmatory factor analysis model of factor complexity one. Sociological
ship training with children having autism spectrum disorders. Journal of Methods & Research, 23, 421–441.
Autism and Developmental Disorders, 40, 827–842. Reise, S., & Henson, J. M. (2003). A discussion of modern versus traditional
Fresco, D. M., Heimberg, R. G., Mennin, D. S., & Turk, C. L. (2002). Con- psychometrics as applied to personality assessment scales. Journal of Per-
firmatory factor analysis of the Penn State Worry Questionnaire. Behaviour sonality Assessment, 81, 93–103.
Research and Therapy, 40, 313–323. Renshaw, P. D., & Brown, P. J. (1993). Loneliness in middle childhood: Con-
Geiser, C., Eid, M., & Nussbeck, F. (2008). On the meaning of the latent current and longitudinal predictors. Child Development, 64, 1271–1284.
variables in the CT-C (M-1) model: A comment on Maydeu-Olivares and Russell, D., Peplau, L. A., & Cutrona, C. E. (1980). The revised UCLA Lone-
Coffman (2006). Psychological Methods, 13, 49–57. liness scale: Concurrent and discriminant validity evidence. Journal of Per-
Goossens, L., & Beyers, W. (2002). Comparing measures of childhood loneli- sonality and Social Psychology, 39, 472–480.
ness: Internal consistency and confirmatory factor analysis. Journal of Clini- Stansbury, J., Ried, D., & Velozo, C. (2006). Unidimensionality and bandwidth
cal Child and Adolescent Psychology, 31, 252–262. in the Center for Epidemiologic Studies Depression (CES–D) scale. Journal
Hawkley, L. C., Browne, M. W., & Cacioppo, J. T. (2005). How can I connect of Personality Assessment, 86, 10–22.
with thee? Let me count the ways. Psychological Science, 16, 798–804. Storch, E. A., Brassard, M. R., & Masia-Warner, C. L. (2003). The relationship
Heinrich, L. M., & Gullone, E. (2006). The clinical significance of loneliness: of peer victimization to social anxiety and loneliness in adolescence. Child
A literature review. Clinical Psychology Review, 26, 695–718. Study Journal, 33, 1–18.
Holgado-Tello, F., Chacón-Moscoso, S., Barbero-Garcı́a, I., & Vila-Abad, E. Storch, E. A., Milson, V., DeBraganza, N., Lewin, A., Geffken, G., & Silver-
(2010). Polychoric versus Pearson correlations in exploratory and confir- stein, J. (2006). Peer victimization, psychosocial adjustment, and physical
matory factor analysis of ordinal variables. Quality and Quantity, 44(1), activity in overweight and at-risk-for-overweight youth. Journal of Pediatric
153–166. Psychology, 32, 80–89.
Hoza, B., Bukowski, W. M., & Beery, S. (2000). Assessing peer network and Tomas, J. M., Hontangas, P. M., & Oliver, A. (2000). Linear confirmatory factor
dyadic loneliness. Journal of Clinical Child and Adolescent Psychology, 29, models to evaluate multitrait-multimethod matrices: The effects of number of
119–128. indicators and correlation among methods. Multivariate Behavioral Research,
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance 35, 469–499.
structure analysis: Conventional criteria versus new alternatives. Structural Tomas, J., & Oliver, A. (1999). Rosenberg’s Self Esteem Scale: Two factors or
Equation Modeling, 6, 1–55. method effects. Structural Equation Modeling, 6, 84–98.
Joreskog, K. G. (1969). A general approach to confirmatory maximum likeli- U.S. Census Bureau. (2010, August 16). State & county quickfacts: Mississippi.
hood factor analysis. Psychometrika, 34, 183–202. Retrieved from http://quickfacts.census.gov
Joreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psy- Wännström, I., Peterson, U., Asberg, M., Nygren, M., & Gustavsson, P. (2009).
chometrika, 36, 109–133. Can scales assessing psychological and social factors at work be used across
Kelloway, E. K. (1995). Structural equation modelling in perspective. Journal different occupations? Work: A Journal of Prevention, Assessment and Reha-
of Organizational Behavior, 16, 215–224. bilitation, 34, 3–11.
Copyright of Journal of Personality Assessment is the property of Taylor & Francis Ltd and its content may not
be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written
permission. However, users may print, download, or email articles for individual use.