1 s2.0 S1096751621000415 Main
1 s2.0 S1096751621000415 Main
1 s2.0 S1096751621000415 Main
A R T I C L E I N F O A B S T R A C T
Keywords: Within the distance education community, the Community of Inquiry (COI) framework is widely accepted as a
learning presence framework to understand and design text-based learning environments. The framework includes three compo
self-regulation nents: Cognitive Presence, Teaching Presence, and Social Presence. Recent work has proposed the addition of a
Community of Inquiry
fourth component, Learning Presence, which reflects students’ self-regulation, and its role within the original
first-order CFA
second-order CFA
framework. This study evaluated alternative structures of the COI framework to explain student perceptions of
learning online. The study participants (n = 256) were graduate students from multiple institutions who had
taken at least one fully online course as part of their degree requirements. Survey data were collected using a
single Likert-scaled survey instrument. Presented herein are the results of the first phase of a two-part study,
which included a series of confirmatory factor analyses to evaluate the measurement models of the four COI
constructs individually, followed by a model including all four constructs simultaneously. Future work on the
second phase of the this two-part study evaluated a series of structural models using path analyses and hierar
chical linear regression analyses. Findings indicated that teaching presence reached a more parimonious model
with two subscales as opposed to the three subscales of the COI survey. A new subscale "peer faciliation" was
proposed for teaching presence, but had better model fit as a subscale of social presence. The three existing
subscales of social presence could also more parsimoniously represented with two subscales, with the new "peer
faciliation" subscale acting as the third. Finally, learning presence was modeled with three subscales, and was the
strongest overall predictor of cognitive presence, compared to teaching and social presence. This work makes
unique contributions to the study of online learning environments through the COI framework by introducing a
comprehensive survey that includes Learning Presence indicators, producing evidence on the multi-
dimensionality of the COI constructs, and the strong relationship between Learning Presence and Cognitive
Presence.
1. Self-regulation in online learning monitoring, however, are also central to the overall construct (Bandura,
1989; Zimmerman, 2002). For example, Bandura states “motivation,
Self-regulated learning refers to “the degree to which students are which is primarily concerned with activation and persistence in
metacognitively, motivationally, and behaviorally active participants in behavior, is also partly rooted in cognition” (1977, p. 193). Within the
their own learning process” (Zimmerman, 2008, p. 167). Due to the conceptual scope of self-efficacy, Bandura (1977) further explains that a
nature of distance education, the degree to which students can take person’s perception of their own effectiveness will affect whether they
responsibility for their own learning is seen as a critical factor of student persist, or even try, to develop coping strategies for given situations.
success (Garrison, 1987; Shea & Bidjerano, 2010). Research shows positive relationships between students who record and
track their own progress and personal satisfaction and self-efficacy
(Schunk, 2015). Of particular interest, self-regulation is both socially
1.1. Dimensions of self-regulation
and contextually situated, meaning the level of self-regulation not only
varies from person to person, but within the same person over time
Generally, the construct of self-regulated learning is tied closely with
(Credé & Phillips, 2011).
goal-oriented behavior (Sitzmann & Ely, 2011). Student motivation and
* Corresponding author.
E-mail address: ruth.wertz@valpo.edu.
https://doi.org/10.1016/j.iheduc.2021.100832
Received 4 April 2019; Received in revised form 17 May 2021; Accepted 13 July 2021
Available online 3 November 2021
1096-7516/© 2021 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
One major critique of self-regulation theories is their focus on indi 1.3. Self-regulation in the Community of Inquiry
vidual learning through individual regulation (Hadwin & Oshige, 2011).
Even theories of self-regulation that explicitly recognize socio-cultural The Community of Inquiry (COI) framework was introduced as a
influence (e.g., Bandura, 1989, 2001; Barnard, Paton, & Lan, 2008; process framework to help guide the design and facilitation of mean
Zimmerman, 2008) focus primarily on individual perceptions and ingful learning experiences through the intersection of three presences
achievement. In recent years, theories of co-regulation and group within the learning environment: cognitive presence (CP), which repre
monitoring have emerged to compensate for this limitation. Hadwin and sents students’ interaction with the course content; teaching presence
Oshige (2011) explain that self-regulation theories tend to be rooted in (TP), which represents students’ interaction with instructional tools and
Deweyan constructivism, in contrast to co-regulation theories, which learning activities; and social presence (SP), which represents students’
tend to be rooted in Vygotskian social-constructivism. The same authors interaction with other learners and cultural aspects of the learning
argue that co-regulation is primarily a scaffold to self-regulation, where environment (Garrison, Anderson, & Archer, 1999). The work presented
students learn to regulate their own learning through engagement in in this study will focus primarily on the COI survey developed by
shared and directed problem-solving. For example, co-regulation often Arbaugh et al. (2008) which I refer to as the classical COI Survey for both
“involved sharing rationales and explanations of plans, goals, and ac clarity and expedience. Early validation analyses were reported by Swan
tivities in the common regulatory space” (Hadwin & Oshige, 2011, p. et al. (2008), and many have followed. For a systematic review of the
249). classical COI survey see (Stenbom, 2018).
Nearly a decade after the introduction of the COI framework, Shea
and Bidjerano (2010) proposed learning presence (LP) as a fourth inter-
1.2. Cognitive emotional regulation related construct. The LP construct emerged from a constellation of
motivational and behavioral traits that drive self-regulation (Shea et al.,
Another aspect of self-regulation that is equally important to the 2012, 2014; Shea & Bidjerano, 2012) and co-regulation (Hayes, Smith,
learning process, but often overlooked in educational literature is the & Shea, 2015). It was among the first of many suggested revisions to the
role of emotion and emotional regulation (Zimmerman & Schunk, framework. As researchers and practitioners continue to extend the
2011). There is vigorous debate among affective scientists on the utility and gain deeper understanding of the COI, there is still no reso
boundaries and interactions between emotion and cognition (Barrett, lution to recent arguments on whether there is a fourth factor, what that
2017; Cavanagh, 2016; Pessoa, 2008). The most important takeaway factor should be, and then what structural model may best represent
being that these systems are best regarded as highly interrelated where relationships among the new factor and its exisiting counterparts (Kozan
communication and interaction occur through vastly complex neural & Caskurlu, 2018). Recent research has, however, continued to circle
networks (Pessoa, 2013). Eyler (2018) notes in his chapter on emotion in around theoretical representations of learner characteristics (Huang,
human learning, that emotion is fundamental to the process of learning Law, & Lee, 2019), particularly in the area of self-regulation and its roots
and provides key ingredients such as social connection, interest, atten in epistemic beliefs. For this project I chose to focus on the initial
tion, and motivation, unless our emotional states are unregulated. When concept of self-regulation with LP as a fourth interrelated factor within
this happens, we cannot adequately assess, express, or modulate our the COI framework.
feelings, which makes it far more likely that they will interrupt and In response to the suggestion of LP as a fourth factor, (Garrison and
impede other cognitive processes through a process Cavanagh (2016) Akyol, 2013) argued against its inclusion, suggesting that “creating a
has coined “emotional hijacking.” Similarly, in a review of neuroscien construct that does not explicitly recognize the importance of co-
tific perspectives on the interactions between emotion and cognition, regulation or reflect the collaborative nature of a community of in
Pessoa (2009) cites cognitive emotional regulation as an area of interest, quiry” would, in essence “violate fundamental assumptions of the COI
particularly where cognition serves to scaffold and improve emotional framework” (p. 85). We argue that explicit acknowledgement of indi
regulation through a processes of reappraisal (Ochsner & Gross, 2008). vidual contributions to the larger collaborative community does not
(Garnefski, Kraaij and Spinhoven, 2001) loosely define cognitive violate assumptions of social constructivist or distributed learning the
emotional regulation as “a wide range of biological, social, behavioral as ories. Social constructivism does not reject the idea that individuals
well as conscious and unconscious cognitive processes” (p. 1312). In this construct knowledge, but instead focuses on the construction of
same study, they developed the Cognitive Emotional Regulation Ques knowledge through social interaction (Noddings, 2007). Likewise,
tionnaire (CERQ) which was designed to measure what participants think distributed learning theory “describes the classroom as a conglomerate
(not what they actually do) when presented with threatening or stressful of lots of different types of knowledge and levels of expertise, such that
life events. The CERQ includes nine dimensions of emotional regulation no one student holds all the information or understanding of a concept.
including: self-blame, blaming others, acceptance, refocus on planning, Rather the class as a whole is the unit of learning with each student
positive refocusing, rumination or focus on thought, positive reap contributing to the overall performance of the class” (Svinicki, 2008, p.
praisal, putting into perspective, and catastrophizing. This, and similar 5). In both cases, learning is done by the individual, and is mediated by
instruments, are useful as therapeutic tools, but also as classroom tools the social context.
to promote self-awareness and/or provide an entry point for regulatory
scaffolding. In another example, Bortoletto and Boruchovitch (2013) 1.4. Community of Inquiry and the Nth presence
also situate self-regulatory constructs to include emotional regulation,
and thus developed the Emotional Regulation Strategies Scale for Adults (Huang, Law and Lee, 2019) note that “while there have been de
which included 92 self-reported Likert-scaled items “grouped in three bates regarding learning presence, it is clear that one cannot fully
subscales designed to assess the perception of the emotions sadness, explain and study online learning without a consideration of online
anger, happiness and the use of emotional regulation strategies” (p. 236) learners’ characteristics such as self-regulation” (p. 1885). In addition to
which were chosen based on their perceived associations with the self-regulation, other researchers have chosen to focus on emotional
educational context. characteristics. (Cleveland-Innes and Campbell, 2012) define Emotional
When it comes to approaching self-regulation from an integrated Presence (EP) as “the outward expression of emotion, affect, and feeling
emotion~cognition perspective, we have only just scratched the surface by individuals and among individuals in a community of inquiry, as they
in our understanding. New perspectives from the interdisciplinary af relate to and interact with the learning technology, course content,
fective sciences will continue to provide interesting juxtapositions to students, and the instructor” (p. 283).
classical beliefs about self-regulatory theories, particularly in how, and Anderson (2014) later wrote a review of the research published by
to what extent emotion plays a significant role. Rienties and Rivers (2014) in which they proposed several
2
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
methodologies, both qualitative and quantitative, to assess EP from freely available French translation (Heilporn & Lakhal, 2020; Nadeau,
student data. In his review, Anderson commented that emotion, or at 2012).
least affective expression, was included to some degree in SP. He also Overall, recent studies that have focused on the validation and
commented on why earlier proposals of LP were met with resistance measurement of COI constructs have yet to bring self-regulatory anal
from the original framing authors. “While not denying the value of ysis, the multi-deminsonality of COI factors, and their causal relation
‘learner presence’ it takes the model into psychological realms that our ships together into one comprehensive study. The purpose of this
more sociological orientation had avoided in the initial formation” research project is to further explore the constitution and operational
(Anderson, 2014). He also cited simple parsimony as a motivator to modeling of LP within the structure of the COI framework, where LP is a
continue to express the COI framework within its original three-circled multi-dimensional representation of student self-regulation. This was
Venn diagram. We are no closer to a consensus today on an “Nth pres accomplished in a two-phase research study. The first phase, presented
ence” than we were in 2014, but he did go on to concede the point, “this herein, was focused on confirming the measurement and dimensionality
review convinced me that something is lost when the emotional aspect of of TP, SP, CP, and adding the measurement of LP to a single inclusive
human experience in education is ignored” (emphasis added). survey instrument. The second phase, which will be presented in future
Anderson’s comments did raise an important point regarding the work, was focused on investigating structural models of the COI
sociological versus psychological perspectives with which the COI was framework suggested in prior literature that were modified to include LP
framed. Perhaps so many scholars have pursued a fourth presence, and as a fourth factor. The model fit and theoretical implications of these
continue to do so, because the psychological perspective that is funda models were analyzed and discussed for the purpose of proposing a new
mental to learning is notably absent. For example, Kozan and Caskurlu structural model for the COI framework with explicit inclusion of stu
(2018) preformed a review of COI literature in which an additional dents’ self-regulatory characteristics.
presence was proposed. In their review, they reported six variations of
an “Nth presence” where all but two (Distributed Teaching Presence and 2. Research methods
Teacher Engagement) were affect-oriented in some way.
Anderson’s claim that emotional expression, at least to some extent, The objective of this study, the initial phase of a two-phase project,
is captured within the SP construct is especially salient when compared was to confirm the interrelation of LP with the other three COI factors.
with the results of factor analysis performed by Cleveland-Innes and This study was conducted using quantitative multivariate linear
Campbell (2012), which included six new EP survey items appended to regression methodologies grounded in a post-positivist paradigm. The
the classical COI Survey. In their analysis, both TP and CP items, for the post-positivist paradigm acknowledges that while it is reasonable to test
most part, behaved as expected. Nine of 13 TP items clustered into one and compare hypotheses using empirical evidence, no method of inquiry
factor, and 11 of 12 CP items clustered together across two factors. Four is absolute, thus understanding is gained and improved over time
of the six EP items clustered into one factor, but only four of nine SP through multiple methods of inquiry (Patton, 2002). In general, the
items clustered together. The remaining five had loaded onto the TP objective of this study was confirmatory in nature, making CFA an
factor (1), EP factor (1), or were among the six items that did not have a appropriate method to test a priori models. Throughout this process,
valid loading (3). Recognizing that multiple causescould be responsible however, results from the model estimations (e.g., estimated parame
for these changes, it is interesting nonetheless that the addition of these ters, model-fit indices, and modification indices) were used to improve
six EP items seemed to change the patterns of SP more so than the other the initial models.
presences. It is also notable that only the outward emotional expression
was of interest in this study, which was reflected in both the stated 2.1. Research design
definition of EP and the associated survey items proposed (Cleveland-
Innes & Campbell, 2012). This outward focus may be more consistent An existing instrument has been developed to measure TP, SP, and
with the sociological perspectives with which the COI was framed, but it CP in a single Likert-scale survey, but LP has yet to be integrated into a
unfortunately does little to close the self-regulatory measurement gap single instrument. Therefore, the driving research question for this study
that plays a critical role in how we understand and engage with learning was sub-divided into the following three objectives:
online in a community of inquiry.
1. Confirm the independent measurement of TP, LP, SP, and CP
1.5. Research gap and statement of purpose 2. Confirm the dimensionality of TP, LP, SP, and CP
3. Confirm the simultaneous measurement and association of the four
The COI framework, its constructs, and its application continue to be COI constructs
area of significant interest. Recent studies have been conducted to re-
examine the construct validity of the TP, SP, and CP as originally pro The analysis was based on a 47-item survey, entitled Web-based
posed with an independent sample EFA and CFA combined analysis Teaching and Learning Link to Social and Cognitive Presence (WebTALK),
(Kozan & Richardson, 2014), as well as the causal relationships between which is presented in Appendix A. The scale included a five-point Likert
these three factors (Dempsey & Zhang, 2019). Several more studies have scale: strongly agree, agree, neutral, disagree, and strongly disagree. The
used validation techniques to validate translations of the classical COI items in the WebTALK were selected from existing Likert-scale survey
Survey in different languages for broader international application. For instruments that were well-developed with strong validity evidence in
example, Ma et al. (2017) performed a validation study on a Chinese the literature. Given the untested nature of the WebTALK instrument,
translation of the classical COI Survey, with appended self-regulatory reliability evidence via Cronbach’s alpha, and validity evidence via
items to represent LP as a COI factor (see, Shea et al., 2014; Traver, comparison to prior literature and CFA results were evaluated at mul
Volchok, Bidjerano, & Shea, 2014). In this study, Ma et al. analyzed both tiple stages throughout the research process. For example, first-order
construct validity and causal relationships. Yu and Richardson (2015) CFAs were used to individually confirm the measurement of the four
and Olpak & Çakmak (2018) also performed validation studies on constructs. In addition, the first-order CFAs were used to confirm the
Korean and Turkish translations of the classical COI Survey, respectively, dimensionality of the four COI constructs. Here, dimensionality refers to
following similar EFA and CFA methodologies as Kozan and Richardson the hypothesized subscales that comprise each of the COI factors. A
(2014). Caskurlu (2018) and Heilporn and Lakhal (2020) performed second-order CFA was conducted to confirm the four-factor structure of
construct validation studies using second-order confirmatory factor the COI framework at large, and the significant association between
analyses to investigate the dimensionality of the COI constructs using each of the four factors. To summarize, the second-order factors repre
the classical COI Survey (without additional self-regulatory items) and a sent the four primary COI constructs, and the first-order factors
3
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
represent the subscales of each second-order factor. selected from the LEP scale (Moore, 1990) which was developed to
assess post-secondary students’ level of intellectual development ac
2.2. Research setting and participants cording to Perry’s seven stages (Perry, 1970/1999). The scale consists of
60 four-point Likert-type questions across five subscales. Cronbach’s
Research participants were recruited from multiple institutions alpha ranged from 0.63 to 0.84 (Moore, 1990).
through professional networks and conferences. Recruitment efforts In the final stage, a complete draft was sent to peers with expertise in
were targeted toward faculty and administrators across five U.S. in survey design and familiarity with the constructs being measured. They
stitutions who then distributed the online WebTALK survey to students were asked to review the instrument for of item clarity, flow, and gen
enrolled in fully online courses within their programs/institutions. The eral effectiveness to measure the target construct. The peer review
recruitment of research participants was implemented in accordance yielded several rounds of revision before the final survey was dissemi
with Protocol #1307013808 reviewed and approved by the Institutional nated to the research participants. The final 47-item WebTALK survey
Review Board (IRB) at Purdue University. Criterion sampling was used instrument is shown in Appendix A (Table A.1, A.2, A.3 and A.4), which
to select participants for the study, based on 1) the participant was includes the item text, item ID (used in analysis), construct subscale, and
currently a graduate student enrolled in one or more online course, and item source.
2) the participant was had previously completed at least one online
course that counted toward their degree requirements. 3. Results
The survey instrument used in this study, entitled Web-based Teach The WebTALK survey instrument was administered via Qualtrics
ing and Learning Link to Social and Cognitive Presence (WebTALK) was online survey software. A total of 372 participants responded to the
constructed for this research, and consists of items from existing scales to survey. The factor analyses in this study were performed using the IBM®
measure the four constructs of TP, LP, SP, and CP. The Classical COI SPSS® AMOS V22.0.0 statistical package. Based on the sample size
Survey, as discussed in Section 1.3 is well-established with sufficient achieved in this study, the maximum likelihood (ML) estimator was
published reliability and validity evidence. It does not, however, have selected for analysis (Brown, 2015), which cannot tolerate missing
any measures of LP. Thus, the instrument was modified to include items values. The survey instrument did not have any forced response items,
to measure LP. In instrument development, it is common practice to and participants were permitted to exit the survey at any point; there
include more items to start with and eliminate poor performing items fore, missing values had to be evaluated.
during analysis. The WebTALK instrument was designed with as few TP, Of the 372 respondents, 126 cases (33.8%) contained one or more
SP, and CP items as possible to accommodate a larger number of LP missing values associated with the 47 WebTALK items measuring the
items. four dimensions of COI. If participants opted not to complete any of the
In general, the WebTALK survey instrument was developed in three demographic items, these were not included in the missing values
stages. The first stage was to reduce the total number of items used in the analysis. Given the relatively large number of cases that had one or more
existing COI instrument. In a preliminary study (Wertz, 2014), CFA missing value, it was impractical to use listwise deletion for all cases
models of each of the three COI constructs measured by the existing COI with missing data. Instead, listwise deletion was performed for all cases
instrument were analyzed individually. The initial model included all with 11 or more missing values, representing 25% of the overall
the indicators assigned to each construct. Modification Indices (MIs) WebTALK survey. After deletion, 256 cases were retained for analysis.
from the CFAs were used to identify candidates for deletion. Final de Across the 256 cases there were a total of 14 missing values, which
cisions for item deletion were made with a combination of evidence represented less than 1% of any given factor, less than 10% of any given
from the MIs and critical evaluation of each item. Overall, the COI items case, and 0.12% of the overall dataset. Missing values were replaced
were reduced from 34 items to 21 items. using the series mean (i.e., the mean of the values reported within the
The second stage was to select items to measure LP. These items were given case for the other indicator variables within the associated factor
selected from three existing scales to measure motivational, behavioral, grouping), as suggested by Byrne (2010).
and developmental aspects of LP.
Motivated Strategies for Learning Questionnaire (MSLQ). Indicators of 3.2. Participant demographics
motivational and behavioral aspects of LP were taken from the MSLQ
developed by Pintrich, Smith, Duncan, and Mckeachie (1991). This The 256 respondents included in the analysis primarily represented
questionnaire consists of 81 Likert-scale items across two primary con five U.S. institutions, Baker College (36.3%), Purdue University
structs, motivation and learning strategies, with several subscales in (31.1%), Boise State University (16.4%), Norwich University (7.0%),
each section. The MSLQ has been used in over 50 studies in the last and University of North Dakota (5.9%), with the remaining 3.1% un
decade with reported Cronbach’s alpha ranging from 0.69 to 0.93 specified. Within the sample population a broad range of institutions,
(Pintrich, Smith, Garcia, & McKeachie, 1993). The validity evidence of including large research universities and small private universities were
the MSLQ includes confirmatory factor analyses of the motivational and represented.
learning strategy scales. The disciplines of study of the respondents were roughly organized
Online Self-Regulated Learning Questionnaire (OSLQ). Several in into four categories, including Education (42.6%), Business (35.9%),
dicators of self-regulation were taken from the Online Self-Regulated Engineering (21.5%), and Other (1.2%). The respondents were 56.6%
Learning Questionnaire (OLSQ) developed by Barnard et al. (2008). female, 42.6% male, with 0.8% unreported.
This questionnaire consists of 24 Likert-scale items across six subscales: Most respondents (83.2%) have taken more than three online cour
goal-setting, environment structuring, task strategizing, time manage ses, with the remainder having taken one to three online courses.
ment, help seeking, and self-evaluation. The Cronbach’s alpha for the Completing at least one online course was a criterion of participation, so
instrument was 0.67 to 0.96 (Barnard et al., 2009). In addition, evidence any participant that indicated they have not taken at least one online
of construct validity was achieved through a second-order confirmatory course was redirected to the end of the survey.
factor analysis. Finally, 60% of the respondents were between the ages of 30 and 49,
Learning Environment Preferences (LEP) Scale. LP, as it was originally 14% were younger than age 30, 18% were ages 50 or older, and 8% did
described, encompasses the learners’ developmental, motivational, and not specify their age. Age does play a significant role in intellectual
behavioral processes (Shea & Bidjerano, 2010). Several items were also development and epistemological values. It should be noted that a
4
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
critical majority (approximately 80%) of the participants were adult 3.4. Measurement of teaching presence
learners over the age of 30.
Teaching presence is generally expressed with three subscales, Design
& Organization, Facilitation, and Direct Instruction. Recent investigations
3.3. Confirmatory analysis procedures and assumptions by Caskurlu (2018) and Heilporn and Lakhal (2020) suggest that TP
could be expressed more parsimoniously as two subscales: Course
This section presents the results of the first-order CFAs of the TP, SP, Design and Facilitation. In addition, I proposed a third dimension rep
LP, and CP constructs, first-order dimensionality testing of the TP, SP, resenting peer facilitation based on the theoretical assumption that the
LP, and CP constructs, and a second-order CFA of all four constructs responsibility of teaching is shared among all course participants,
combined into one model, and Cronbach’s alpha for each factor included including peer-to-peer interactions. Thus, the initial model included
in the WebTALK instrument. three subscales, Course Design, Instructor Facilitation, and Peer Facilitation,
In the SEM figures provided in this section, unobserved variables with a total of 11 indicators (Fig. 1). The results of the initial model
(latent factors) are shown in ovals and observed variables (indicators) showed that correlations between Course Design and Peer Facilitation (r
are shown in rectangles, and the small circles to the right of the indicator = 0.36, p < .01) and even between Instructor Facilitation and Peer
variables represent the residual (error) of each item. The SEM data were Facilitation (r = 0.57, p < .01) were relatively small, particularly for
analyzed across six categories: 1) multivariate normality, 2) model subscales that are supposed to be explaining the same general construct.
summary, 3) estimated parameters, 4) modification indices (MIs), 5) The second and final model (Fig. 2), peer facilitation was removed from
model fit indices, and 6) bootstrapping 95% confidence intervals (if the TP construct. The final model had nearly perfect model fit [χ 2 (13) =
applicable). Initial models were respecified based on a review of model 15.97, p = .251; GFI = 0.98, IFI = 1.00, TLI = 1.00, CFI = 1.00; RMSEA
fit and MIs, and this process was repeated until no further respecification = 0.03)]. Table 1 presents the parameter estimates, the 95% confidence
was necessary. Any respecifications, however, were based on critical intervals and p-values from the bootstrap procedure.
evaluation of the model, and not solely based on numerical output of
each model. Presented in this section are the initial models, a description
of any respecifications, and the parameter estimates and model fit sta 3.5. Measurement of social presence
tistics of the final models. The evaluation criteria for each of the six
categories of analysis used in this study are summarized in Appendix B Social presence generally comprises three subscales, including Af
(Table B.1). fective Expression, Active Communication, and Group Cohesion. Prior work
Mardia’s normalized estimates were evaluated to assess multivariate by Caskurlu (2018) and Heilporn and Lakhal (2020) similarly suggest
normality within each model. Values ranged from 21.92 to 49.33, where that it is possible to reduce the number of subscales in SP as well. The
values greater than 5 are considered multivariate non-normal (Byrne, initial model for the SP construct included two subscales, Group
2010). To deal with violations of multivariate normality common in real Belonging, and Open Communication, with a total of eight indicators
data (versus simulated data), bootstrapping procedures included in the (Fig. 3). The MIs of the initial model showed that survey item WSP37
IBM® SPSS® AMOS V22.0.0 software package were applied to report was cross-loading onto both of the SP subscales, and that there was a
parameter estimates with more confidence. For the parameter estimates strong correlation between the residuals (error) of survey items WSP36
reported in this section, the bootstrap 95% confidence interval, and the and WSP38.
associated p-value is reported in addition to the original estimate. Item WSP37 states “I felt comfortable interacting with some of the other
Fig. 1. Initial TP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).
5
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
The initial model for the CP construct included four subscales, Trig
gering Event, Exploration, Integration and Resolution with a total of eight
indicators (see Fig. 7). The initial model had very good model fit [χ 2
(14) = 22.98, p = .06; GFI = 0.98, IFI = 0.99, TLI = 0.98, CFI = 0.99;
RMSEA = 0.05)], thus no respecification was needed. Refer to Table 4
Fig. 3. Initial SP measurement model with standardized parameter estimates. for a summary of the parameter estimates, and the 95% confidence in
n = 256. (*p < .05, **p < .01). tervals and p-values from the bootstrap procedure.
participants outside of assigned group discussions.” There is an implicit 3.8. Dimensionality of COI constructs
assumption in this item that assigned group discussions were part of the
online course. If this assumption is not true, or not perceived to be true One assumption of factor analysis is that the unobserved latent
by the respondents, then the item would likely produce unreliable re variables represent a single unidimensional construct, versus a collec
sults. In review of the initial internal reliability analysis, Cronbach’s tion of stratified constructs. Prior research has suggested that all four of
alpha for the Open Communication subscale would increase from 0.83 to the COI constructs contain two or more subscales, which violates the
0.86 if item WSP37 is removed, which suggests that the item is less unidimensional assumption. In this section the results of the chi-square
difference tests used to confirm the dimensionality of the TP, SP, LP and
6
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
Fig. 4. Final SP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).
7
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
Fig. 5. Initial LP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).
Fig. 6. Final LP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).
8
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
Table 3 CMIN/DF = 2.25; GFI = 0.83, IFI = 0.92, TLI = 0.91, CFI = 0.92;
Standardized parameter estimates for the first-order LP measurement model. RMSEA = 0.055)], and was accepted as the final model without further
Bootstrap 95% CI specification.
Parameter Estimate Lower Upper p-value
Fig. 7. Initial and final CP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).
9
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
10
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
Fig. 8. Final second-order COI measurement model with standardized parameter estimates. n = 252.
11
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
Table 5 Table 6
Standardized parameter estimates for the second-order COI measurement Internal reliability of final COI scales and subscales (n = 248).
model. Final COI Scales No. of Items Mean SD Alpha
Bootstrap 95% CI
Teaching Presence (TP) 7 29.38 5.14 0.91
Parameter Estimate Lower Upper p-value Course Design (CD) 3 13.22 2.09 0.84
Instructor Facilitation (IF) 4 16.16 3.39 0.89
CD ← TP 0.87 0.81 0.93 0.008** Learning Presence (LP) 9 38.40 4.32 0.79
IF ← TP 0.91 0.83 0.95 0.008** Motivational (MV) 3 13.26 1.71 0.73
MV ← LP 0.96 0.85 1.06 0.003** Behavioral (BH) 3 12.75 1.83 0.63
BH ← LP 0.68 0.54 0.81 0.003** Developmental (DV) 3 12.38 1.94 0.67
DV ← LP 0.78 0.66 0.87 0.007** Social Presence (SP) 10 38.53 7.28 0.92
GB ← SP 0.96 0.93 0.98 0.002** Group Belonging (GB) 3 11.81 2.44 0.81
OC ← SP 0.94 0.89 0.97 0.003** Open Communication (OC) 3 12.71 2.02 0.84
PF ← SP 0.74 0.62 0.82 0.003** Peer Facilitation (PF) 4 14.02 3.73 0.88
TE ← CP 0.91 0.84 0.96 0.004** Cognitive Presence (CP) 8 33.93 4.66 0.87
EX ← CP 0.98 0.89 1.04 0.004** Triggering Event (TE) 2 8.17 1.69 0.74
IN ←CP 0.97 0.87 1.06 0.003** Exploration (EX) 2 8.46 1.34 0.63
RE ← CP 0.88 0.79 0.95 0.006** Integration (IN) 2 8.69 1.20 0.63
WTP1 ← CD 0.88 0.81 0.93 0.009** Resolution (RE) 2 8.62 1.36 0.70
WTP2 ← CD 0.89 0.82 0.93 0.007**
WTP3 ← CD 0.70 0.60 0.80 0.003**
WTP4 ← IF 0.86 0.80 0.90 0.004**
construct, which exemplifies the gap between the theoretical foundation
WTP5 ← IF 0.83 0.77 0.88 0.005**
WTP6 ← IF 0.77 0.69 0.84 0.008** of the COI framework and perceived interactions within the constructed
WTP7 ← IF 0.84 0.78 0.89 0.006** learning environment; 2) the strongest association among the COI con
structs was between LP and CP, which provides further justification of
Bootstrap 95% CI the value added by including LP as a discernible construct within the
Parameter Estimate Lower Upper p-value
COI framework.
This study did, however, utilize a new survey instrument, which
WLP12 ← MV 0.72 0.56 0.83 0.006**
requires additional analysis to further validate inferences made with
WLP15 ← MV 0.68 0.53 0.80 0.005**
WLP16 ← MV 0.67 0.50 0.75 0.010* these data. The population for this study was comprised of students from
WLP18 ← BH 0.65 0.52 0.77 0.004** five U.S. institutions, that range in size and geographic location. A
WLP19 ← BH 0.56 0.41 0.67 0.009** diverse group of disciplines were represented as well. The population
WLP21 ← BH 0.65 0.51 0.75 0.007** included graduate students, primarily over the age of 30, in fully online
WLP28 ← DV 0.65 0.53 0.74 0.006**
WLP29 ← DV 0.56 0.43 0.70 0.003**
courses. This study focused on the alignment of methodology with un
WLP30 ← DV 0.68 0.55 0.78 0.007** derlying theories that support the COI framework, which should make
WSP32 ← GB 0.81 0.73 0.87 0.008** its methods and conclusions of general rather than specific interest.
WSP33 ← GB 0.81 0.73 0.86 0.007** However, generalizability on a global scale of any kind of human
WSP34 ← GB 0.69 0.59 0.76 0.005**
perception and behavior is something to be approached with great
WSP35 ← OC 0.76 0.66 0.84 0.013*
WSP36 ← OC 0.77 0.70 0.83 0.005** skepticism. In using these data or instruments in future work, both in
WSP39 ← OC 0.85 0.79 0.90 0.007** ternal reliability and external validation checks are highly recom
WTP8 ← PF 0.86 0.80 0.90 0.008** mended, as variation is likely to occur in different populations, as noted
WTP9 ← PF 0.79 0.72 0.86 0.005** across several validation studies discussed in Section 1.5.
WTP10 ← PF 0.75 0.66 0.82 0.005**
WTP11 ← PF 0.82 0.76 0.87 0.005**
Another notable limitation of the present study is the extent to which
emotional regulation, as discussed in Section 1.2, is represented within
the LP construct, as I believe it should be. With the onset of the COVID-
Bootstrap 95% CI
19 pandemic, educational researchers and practitioners are becoming
Parameter Estimate Lower Upper p-value
increasingly more aware and interested in the role emotion plays in the
WCP40 ← TE 0.77 0.66 0.83 0.008** human learning experience—and teaching experience for that matter!
WCP41 ← TE 0.76 0.67 0.84 0.005** Both are fundamentally social activities where our affective processes
WCP42 ← EX 0.70 0.58 0.77 0.011*
are essential to our experience. Concurrently, affective science is a new
WCP43 ← EX 0.64 0.55 0.74 0.004**
WCP44 ← IN 0.74 0.63 0.83 0.005** rapidly growing interdisciplinary field, where new perspectives and
WCP45 ← IN 0.69 0.59 0.77 0.007** traditions bring the promise of new insights. I believe that future di
WCP46 ← RE 0.78 0.66 0.85 0.009** rections for those who engage with communities of inquiry at all levels
WCP47 ← RE 0.73 0.62 0.82 0.005**
should be to delve deeper into the psychosocial domains that have been
TP ↔ LP 0.66 0.52 0.78 0.005**
LP ↔ SP 0.68 0.55 0.80 0.006**
subtly (or not so subtly) avoided, particularly within Western cultures
SP ↔ CP 0.72 0.61 0.81 0.003** (Pessoa, 2021).
TP ↔ SP 0.53 0.36 0.68 0.006** The work presented in this report represents the first-phase of a two-
LP ↔ CP 0.88 0.77 0.97 0.005** phase study. The second phase built upon these findings and employed
TP ↔ CP 0.71 0.59 0.80 0.004**
the use of causal path analysis to investigate the operational structures
r2 ↔ r8 0.73 0.51 1.08 0.003**
of the inter-relations between presences within the COI framework. The
Notes. n = 252; CI = confidence interval; (*p < .05, **p < .01). manuscript in which I present the second phase of this project is in
progress. Preliminary results, however, strongly support the addition of
necessary confirmatory measurements to perform second-order causal LP into the overall COI framework, where LP was found to have the
path analysis, which was performed in the second-phase of this two- strongest direct effect on CP, more so than TP or SP. While I can
phase project. The manuscript presenting this work is currently in appreciate arguments for parsimony (Anderson, 2014), I also believe
progress. there is sufficient evidence from this, and dozens of other studies to
Two hallmark findings from this study include: 1) the inclusion of the support the addition of a 4th presence to the COI framework. The
Peer Facilitation subscale, and its eventual relocation from the TP to SP advantage of LP in comparison to other proposals of an Nth presence (as
12
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
discussed previously in Section 1.4), is the explicit inclusion of learners’ Table A.2 (continued )
regulatory beliefs, attitudes, and behaviors in the analysis of Commu Learning presence items Item ID Subscale Item source
nities of Inquiry. I keep up with all the weekly readings Adapted
and assignments for this course from MSLQ
Table A.1 I distribute my time spent on course WLP20 Behaviors Adapted
WebTALK items for teaching presence. work evenly throughout the week from OSLQ
instead of trying to do everything on
one day
Teaching presence items Item ID Subscale Item source
I set goals to help me manage my time WLP21 Behaviors Adapted
My instructor clearly communicated WTP1 Course Design Adapted for my online course from OSLQ
important course goals or learning from COIa I check my grades for feedback on how WLP22 Behaviors Original
objectives well I am doing in the course Item
My instructor provided clear WTP2 Course Design Adapted I rely on comments from my instructor WLP23 Behaviors Original
instructions on how to participate from COI for feedback on how well I am doing Item
in course learning activities in the course
My instructor clearly communicated WTP3 Course Design Adapted I compare my work to my classmate’s WLP24 Behaviors Adapted
important due dates or time frames from COI work to get a sense of how well I am from OSLQ
for learning activities doing in the course
My instructor helped to keep me WTP4 Instructor Adapted I prefer courses that focus more on WLP25R Development Adapted
engaged in productive dialogue Facilitationb from COI having the right answers than on from LEPc
My instructor helped to keep me on WTP5 Instructor Adapted discussing different methods of
task relative to the goals or Facilitation from COI solving a particular problem
objectives of this course [REVERSE CODED]
My instructor encouraged me to WTP6 Instructor Adapted I prefer courses with activities that WLP26 Development Adapted
explore new concepts related to this Facilitation from COI clearly lay out step-by-step from LEP
course procedures
My instructor provided feedback that WTP7 Instructor Adapted I prefer courses where my final grade WLP27R Development Adapted
helped me understand my strengths Facilitation from COI is based mostly on straightforward from LEP
and weaknesses relative to the goals tests and quizzes covering only
or objectives of this course topics directly taught [REVERSE
My classmates helped to keep me WTP8 Peer Adapted CODED]
engaged in productive dialogue Facilitation from COIc,d I prefer courses that focus on WLP28 Development Adapted
My classmates helped to keep me on WTP9 Peer Adapted discussions of personal answers from LEP
task relative to the goals and Facilitation from COIc,d based on evidence rather than just
objectives of this course right and wrong answers
My classmates encouraged me to WTP10 Peer Adapted I prefer courses where my final grade WLP29 Development Adapted
explore new concepts related to this Facilitation from COIc,d is based mostly on how well I can from LEP
course synthesize and practically apply
My classmates provided feedback (e. WTP11 Peer Adapted various course topics in a real-world
g., through discussion or peer Facilitation from COIc,d context that goes beyond what was
review) that helped me understand directly taught in the course
my strengths and weaknesses I prefer instructors who challenge me WLP30 Development Adapted
relative to the goals or objectives of to develop my own opinions that are from LEP
this course supported by evidence
I prefer instructors who allow students WLP31 Development Adapted
a to have a voice in what topics are from LEP
Arbaugh et al., 2008.
b
Instructor Facilitation was changed from the original subscale label Facilita covered and what types of learning
activities will be completed (e.g.,
tion to distinguish it from the new subscale Peer Facilitation.
c research paper, literature review, or
Peer Facilitation is a new dimension of TP tested in this study. Items are a
project)
derivative of Instructor Facilitation with the word “instructor” changed to
a
“classmates.” Pintrich et al., 1991.
d b
Peer Facilitation was initially proposed in this study as a subscale of TP. Barnard et al., 2008.
c
During the analysis, however, it was reassigned as a subscale of SP. Moore, 1990.
Table A.2 Table A.3
WebTALK items for learning presence. WebTALK items for social presence.
Learning presence items Item ID Subscale Item source Social presence items Item Subscale Item source
ID
I will be able to use what I learn in this WLP12 Motivation Adapted
course in my career from MSLQa I felt a sense of belonging in this WSP32 Group Belongingb Adapted
Understanding the subject matter of WLP13 Motivation Adapted course from COIa
this course is important to me from MSLQ Interacting with other participants WSP33 Group Belonging Adapted
I don’t compromise the quality of my WLP14 Motivation Adapted gave me a sense community from COI
work because it is for an online from OSLQb I was able to form distinct WSP34 Group Belonging Adapted
course impressions of some course from COI
I am confident I can do an excellent job WLP15 Motivation MSLQ participants’ personalities
on the assignments and tests in this My contributions to course WSP35 Open Original
course discussions were acknowledged Communicationc Item
I can master the skills being taught in WLP16 Motivation Adapted by other course participants
this class from MSLQ I felt comfortable participating in WSP36 Open COI
I change the way I study in order to fit WLP17 Behaviors Adapted the course discussions Communication
the course requirements or my from MSLQ I felt comfortable interacting with WSP37 Open Adapted
instructor’s teaching style some of the other course Communication from COI
I study in a place where I can WLP18 Behaviors Adapted participants outside of assigned
concentrate on my course work from MSLQ group discussions
WLP19 Behaviors WSP38
(continued on next column) (continued on next page)
13
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
14
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832
Dempsey, P. R., & Zhang, J. (2019). Re-examining the construct validity and causal Pessoa, L. (2013). The cognitive-emotional brain: From interactions to integration. MIT Press.
relationships of teaching, cognitive, and social presence in community of inquiry http://ebookcentral.proquest.com/lib/valpo-ebooks/detail.action?docID=3339689.
framework. Online Learning, 23(1), 62–79. https://eric.ed.gov/?id=EJ1210951. Pessoa, L. (2021, February 20). The brain network [blog]. Emotion & Cognition and the
Eyler, J. R. (2018). How humans learn: The science and stories behind effective college Brain, by Luiz Pessoa https://cognitionemotion.wordpress.com/.
teaching. West Virginia University Press. http://ebookcentral.proquest.com/lib/val Pintrich, P. R., Smith, D. A. F., Duncan, T., & Mckeachie, W. J. (1991). A manual for the
po-ebooks/detail.action?docID=5597811. use of the motivated strategies for learning questionnaire (MSLQ) (pp. 1–80). doi:
Garnefski, N., Kraaij, V., & Spinhoven, P. (2001). Negative life events, cognitive emotion ED338122.
regulation and emotional problems. Personality and Individual Differences, 30(8), Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and
1311–1327. https://doi.org/10.1016/S0191-8869(00)00113-6 predictive validity of the motivated strategies for learning questionnaire (MSLQ).
Garrison, D. R. (1987). Self-directed and distance learning: Facilitating self-directed Educational and Psychological Measurement, 53, 801–813. doi:ED338122.
learning beyond the institutional setting. International Journal of Lifelong Education, 6 Rienties, B., & Rivers, B. A. (2014). Measuring and understanding learner emotions:
(4), 309–318. https://doi.org/10.1080/0260137870060404 Evidence and prospects. Learning Analytics Review, 1(1), 1–27. https://lace.apps.sla
Garrison, D. R., & Akyol, Z. (2013). Toward the development of a metacognition te.uib.no/publications/learning-analytics-and-emotions.pdf.
construct for communities of inquiry. The Internet and Higher Education, 17, 84–89. Schunk, D. H. (2015). Progress self-monitoring: Effects on children’s self-efficacy and
https://doi.org/10.1016/j.iheduc.2012.11.005 achievement. The Journal of Experimental Education, 51(2), 89–93. https://doi.org/
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based 10.1080/00220973.1982.11011845
environment: Computer conferencing in higher education. The Internet and Higher Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy,
Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6 self-regulation, and the development of a communities of inquiry in online and
Hadwin, A., & Oshige, M. (2011). Self-regulation, coregualtion, and socially shared blended learning environments. Computers in Education, 59(2), 316–326. https://doi.
regulation: Exploring perspectives of social in self-regulated learning theory. org/10.1016/j.compedu.2010.07.017
Teachers College Record, 113(2), 204–264. https://doi.org/10.4324/ Shea, P., & Bidjerano, T. (2012). Learning presence as a moderator in the community of
9780203839010.ch5 inquiry model. Computers in Education, 59(2), 316–326. https://doi.org/10.1016/j.
Hayes, S., Smith, S. U., & Shea, P. (2015). Expanding learning presence to account for the compedu.2012.01.011
direction of regulative intent: Self-, co- and shared regulation in online learning. Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., Pickett, A., … Jian, S. (2012).
Online Learning, 19(3), 15–31. https://eric.ed.gov/?id=EJ1067475. Learning presence: Additional research on a new conceptual element within the
Heilporn, G., & Lakhal, S. (2020). Investigating the reliability and validity of the community of inquiry (CoI) framework. Internet and Higher Education, 15(2), 89–95.
community of inquiry framework: An analysis of categories within each presence. https://doi.org/10.1016/j.iheduc.2011.08.002
Computers & Education, 145, 103712. https://doi.org/10.1016/j. Shea, P., Hayes, S., Uzuner-Smith, S., Gozza-Cohen, M., Vickers, J., & Bidjerano, T.
compedu.2019.103712 (2014). Reconceptualizing the community of inquiry framework: An exploratory
Huang, K., Law, V., & Lee, S. J. (2019). The role of learners’ epistemic beliefs in an online analysis. The Internet and Higher Education, 23, 9–17. https://doi.org/10.1016/j.
Community of Inquiry. British Journal of Educational Technology, 50(4), 1882–1895. iheduc.2014.05.002
https://doi.org/10.1111/bjet.12684 Sitzmann, T., & Ely, K. (2011). A meta-analysis of self-regulated learning in work-related
Kozan, K., & Caskurlu, S. (2018). On the nth presence for the community of inquiry training and educational attainment: What we know and where we need to go.
framework. Computers & Education, 122, 104–118. https://doi.org/10.1016/j. Psychological Bulletin, 137(3), 412–442. https://doi.org/10.1037/a0022777
compedu.2018.03.010 Stenbom, S. (2018). A systematic review of the Community of Inquiry survey. The Internet
Kozan, K., & Richardson, J. C. (2014). New exploratory and confirmatory factor analysis and Higher Education, 39, 22–32. https://doi.org/10.1016/j.iheduc.2018.06.001
insights into the community of inquiry survey. The Internet and Higher Education, 23, Svinicki, M. D. (2008). A guidebook on conceptual frameworks for research in engineering
39–47. https://doi.org/10.1016/j.iheduc.2014.06.002 education: Technical report for rigorous research in engineering education project.
Ma, Z., Wang, J., Wang, Q., Kong, L., Wu, Y., & Yang, H. (2017). Verifying causal American Society of Engineering Education, American Educational Research
relationships among the presences of the community of inquiry framework in the Association, Professional and Organizational Development network and National
Chinese context. International Review of Research in Open and Distributed Learning: Science Foundation.
IRRODL, 18(6), 213–230. https://doi.org/10.19173/irrodl.v18i6.3197 Swan, K., Shea, P., Richardson, J., Ice, P., Garrison, D., Cleveland-Innes, M., &
Moore, W. S. (1990). The learning environment preferences: An instrument manual. Center Arbaugh, J. (2008). Validating a measurement tool of presence in online
for the Study of Intellectual Development. communities of inquiry. E-Mentor, 2(24), 1–12.
Nadeau, J. (2012). Expérimentation de la rétroaction audiovisuelle asynchrone dans un Traver, A. E., Volchok, E., Bidjerano, T., & Shea, P. (2014). Correlating community
cours à distance d’informatique dans la perspective de la théorie de la communauté college students’ perceptions of community of inquiry presences with their
d’apprentisssage [Memoire de maîtrise, TELUQ] http://biblio.teluq.ca/LinkClick. completion of blended courses. The Internet and Higher Education, 20, 1–9. https://
aspx?fileticket=ISkXNYulN2xtgpPTmjiMAw%3d%3d&tabid=39743&langua doi.org/10.1016/j.iheduc.2013.09.001
ge=fr-CA. Wertz, R. E. H. (2014). What is learning presence and what can it tell us about success in
Noddings, N. (2007). Philosophy of education (2nd ed.). Westview Press. learning online? Frontiers in Education Conference (FIE), 1–6. https://doi.org/10.1109/
Ochsner, K. N., & Gross, J. J. (2008). Cognitive emotion regulation: Insights from social FIE.2014.7044246.
cognitive and affective neuroscience. Current Directions in Psychological Science, 17 Yu, T., & Richardson, J. C. (2015). Examining reliability and validity of a Korean version
(2), 153–158. https://doi.org/10.1111/j.1467-8721.2008.00566.x of the Community of Inquiry instrument using exploratory and confirmatory factor
Olpak, Y. Z., & Kiliç Çakmak, E. (2018). Examining the Reliability and Validity of a analysis. The Internet and Higher Education, 25, 45–52. https://doi.org/10.1016/j.
Turkish Version of the Community of Inquiry Survey. Online Learning, 22(1), iheduc.2014.12.004
147–161. Zimmerman, B. J. (2002). Becoming a self regulated learner: An overview. Theory Into
Patton, M. Q. (2002). Qualitative research and evaluation methods. In Qualitative inquiry Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102
(3rd ed.). Sage. https://doi.org/10.2307/330063. Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical
Perry, W. G. (1970/1999). Forms of intellectual and ethical development in the college years: background, methodological developments, and future prospects. American
A scheme. Jossey-Bass Higher and Adult Education Series. Jossey-Bass Publishers. Educational Research Journal, 45(1), 166–183. https://doi.org/10.3102/
Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews 0002831207312909
Neuroscience, 9(2), 148–158. https://doi.org/10.1038/nrn2317 Zimmerman, B. J., & Schunk, D. H. (2011). Handbook of self-regulation of learning and
Pessoa, L. (2009). Cognition and emotion. Scholarpedia, 4(1), 4567. https://doi.org/ performance (pp. xiv, 484). Routledge/Taylor & Francis Group.
10.4249/scholarpedia.4567
15