1 s2.0 S1096751621000415 Main

Download as pdf or txt
Download as pdf or txt
You are on page 1of 15

The Internet and Higher Education 52 (2022) 100832

Contents lists available at ScienceDirect

The Internet and Higher Education


journal homepage: www.elsevier.com/locate/iheduc

Learning presence within the Community of Inquiry framework: An


alternative measurement survey for a four-factor model
Ruth E.H. Wertz *
College of Engineering, Valparaiso University, 1900 Chapel Drive, Valparaiso, IN 46383, USA

A R T I C L E I N F O A B S T R A C T

Keywords: Within the distance education community, the Community of Inquiry (COI) framework is widely accepted as a
learning presence framework to understand and design text-based learning environments. The framework includes three compo­
self-regulation nents: Cognitive Presence, Teaching Presence, and Social Presence. Recent work has proposed the addition of a
Community of Inquiry
fourth component, Learning Presence, which reflects students’ self-regulation, and its role within the original
first-order CFA
second-order CFA
framework. This study evaluated alternative structures of the COI framework to explain student perceptions of
learning online. The study participants (n = 256) were graduate students from multiple institutions who had
taken at least one fully online course as part of their degree requirements. Survey data were collected using a
single Likert-scaled survey instrument. Presented herein are the results of the first phase of a two-part study,
which included a series of confirmatory factor analyses to evaluate the measurement models of the four COI
constructs individually, followed by a model including all four constructs simultaneously. Future work on the
second phase of the this two-part study evaluated a series of structural models using path analyses and hierar­
chical linear regression analyses. Findings indicated that teaching presence reached a more parimonious model
with two subscales as opposed to the three subscales of the COI survey. A new subscale "peer faciliation" was
proposed for teaching presence, but had better model fit as a subscale of social presence. The three existing
subscales of social presence could also more parsimoniously represented with two subscales, with the new "peer
faciliation" subscale acting as the third. Finally, learning presence was modeled with three subscales, and was the
strongest overall predictor of cognitive presence, compared to teaching and social presence. This work makes
unique contributions to the study of online learning environments through the COI framework by introducing a
comprehensive survey that includes Learning Presence indicators, producing evidence on the multi-
dimensionality of the COI constructs, and the strong relationship between Learning Presence and Cognitive
Presence.

1. Self-regulation in online learning monitoring, however, are also central to the overall construct (Bandura,
1989; Zimmerman, 2002). For example, Bandura states “motivation,
Self-regulated learning refers to “the degree to which students are which is primarily concerned with activation and persistence in
metacognitively, motivationally, and behaviorally active participants in behavior, is also partly rooted in cognition” (1977, p. 193). Within the
their own learning process” (Zimmerman, 2008, p. 167). Due to the conceptual scope of self-efficacy, Bandura (1977) further explains that a
nature of distance education, the degree to which students can take person’s perception of their own effectiveness will affect whether they
responsibility for their own learning is seen as a critical factor of student persist, or even try, to develop coping strategies for given situations.
success (Garrison, 1987; Shea & Bidjerano, 2010). Research shows positive relationships between students who record and
track their own progress and personal satisfaction and self-efficacy
(Schunk, 2015). Of particular interest, self-regulation is both socially
1.1. Dimensions of self-regulation
and contextually situated, meaning the level of self-regulation not only
varies from person to person, but within the same person over time
Generally, the construct of self-regulated learning is tied closely with
(Credé & Phillips, 2011).
goal-oriented behavior (Sitzmann & Ely, 2011). Student motivation and

* Corresponding author.
E-mail address: ruth.wertz@valpo.edu.

https://doi.org/10.1016/j.iheduc.2021.100832
Received 4 April 2019; Received in revised form 17 May 2021; Accepted 13 July 2021
Available online 3 November 2021
1096-7516/© 2021 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

One major critique of self-regulation theories is their focus on indi­ 1.3. Self-regulation in the Community of Inquiry
vidual learning through individual regulation (Hadwin & Oshige, 2011).
Even theories of self-regulation that explicitly recognize socio-cultural The Community of Inquiry (COI) framework was introduced as a
influence (e.g., Bandura, 1989, 2001; Barnard, Paton, & Lan, 2008; process framework to help guide the design and facilitation of mean­
Zimmerman, 2008) focus primarily on individual perceptions and ingful learning experiences through the intersection of three presences
achievement. In recent years, theories of co-regulation and group within the learning environment: cognitive presence (CP), which repre­
monitoring have emerged to compensate for this limitation. Hadwin and sents students’ interaction with the course content; teaching presence
Oshige (2011) explain that self-regulation theories tend to be rooted in (TP), which represents students’ interaction with instructional tools and
Deweyan constructivism, in contrast to co-regulation theories, which learning activities; and social presence (SP), which represents students’
tend to be rooted in Vygotskian social-constructivism. The same authors interaction with other learners and cultural aspects of the learning
argue that co-regulation is primarily a scaffold to self-regulation, where environment (Garrison, Anderson, & Archer, 1999). The work presented
students learn to regulate their own learning through engagement in in this study will focus primarily on the COI survey developed by
shared and directed problem-solving. For example, co-regulation often Arbaugh et al. (2008) which I refer to as the classical COI Survey for both
“involved sharing rationales and explanations of plans, goals, and ac­ clarity and expedience. Early validation analyses were reported by Swan
tivities in the common regulatory space” (Hadwin & Oshige, 2011, p. et al. (2008), and many have followed. For a systematic review of the
249). classical COI survey see (Stenbom, 2018).
Nearly a decade after the introduction of the COI framework, Shea
and Bidjerano (2010) proposed learning presence (LP) as a fourth inter-
1.2. Cognitive emotional regulation related construct. The LP construct emerged from a constellation of
motivational and behavioral traits that drive self-regulation (Shea et al.,
Another aspect of self-regulation that is equally important to the 2012, 2014; Shea & Bidjerano, 2012) and co-regulation (Hayes, Smith,
learning process, but often overlooked in educational literature is the & Shea, 2015). It was among the first of many suggested revisions to the
role of emotion and emotional regulation (Zimmerman & Schunk, framework. As researchers and practitioners continue to extend the
2011). There is vigorous debate among affective scientists on the utility and gain deeper understanding of the COI, there is still no reso­
boundaries and interactions between emotion and cognition (Barrett, lution to recent arguments on whether there is a fourth factor, what that
2017; Cavanagh, 2016; Pessoa, 2008). The most important takeaway factor should be, and then what structural model may best represent
being that these systems are best regarded as highly interrelated where relationships among the new factor and its exisiting counterparts (Kozan
communication and interaction occur through vastly complex neural & Caskurlu, 2018). Recent research has, however, continued to circle
networks (Pessoa, 2013). Eyler (2018) notes in his chapter on emotion in around theoretical representations of learner characteristics (Huang,
human learning, that emotion is fundamental to the process of learning Law, & Lee, 2019), particularly in the area of self-regulation and its roots
and provides key ingredients such as social connection, interest, atten­ in epistemic beliefs. For this project I chose to focus on the initial
tion, and motivation, unless our emotional states are unregulated. When concept of self-regulation with LP as a fourth interrelated factor within
this happens, we cannot adequately assess, express, or modulate our the COI framework.
feelings, which makes it far more likely that they will interrupt and In response to the suggestion of LP as a fourth factor, (Garrison and
impede other cognitive processes through a process Cavanagh (2016) Akyol, 2013) argued against its inclusion, suggesting that “creating a
has coined “emotional hijacking.” Similarly, in a review of neuroscien­ construct that does not explicitly recognize the importance of co-
tific perspectives on the interactions between emotion and cognition, regulation or reflect the collaborative nature of a community of in­
Pessoa (2009) cites cognitive emotional regulation as an area of interest, quiry” would, in essence “violate fundamental assumptions of the COI
particularly where cognition serves to scaffold and improve emotional framework” (p. 85). We argue that explicit acknowledgement of indi­
regulation through a processes of reappraisal (Ochsner & Gross, 2008). vidual contributions to the larger collaborative community does not
(Garnefski, Kraaij and Spinhoven, 2001) loosely define cognitive violate assumptions of social constructivist or distributed learning the­
emotional regulation as “a wide range of biological, social, behavioral as ories. Social constructivism does not reject the idea that individuals
well as conscious and unconscious cognitive processes” (p. 1312). In this construct knowledge, but instead focuses on the construction of
same study, they developed the Cognitive Emotional Regulation Ques­ knowledge through social interaction (Noddings, 2007). Likewise,
tionnaire (CERQ) which was designed to measure what participants think distributed learning theory “describes the classroom as a conglomerate
(not what they actually do) when presented with threatening or stressful of lots of different types of knowledge and levels of expertise, such that
life events. The CERQ includes nine dimensions of emotional regulation no one student holds all the information or understanding of a concept.
including: self-blame, blaming others, acceptance, refocus on planning, Rather the class as a whole is the unit of learning with each student
positive refocusing, rumination or focus on thought, positive reap­ contributing to the overall performance of the class” (Svinicki, 2008, p.
praisal, putting into perspective, and catastrophizing. This, and similar 5). In both cases, learning is done by the individual, and is mediated by
instruments, are useful as therapeutic tools, but also as classroom tools the social context.
to promote self-awareness and/or provide an entry point for regulatory
scaffolding. In another example, Bortoletto and Boruchovitch (2013) 1.4. Community of Inquiry and the Nth presence
also situate self-regulatory constructs to include emotional regulation,
and thus developed the Emotional Regulation Strategies Scale for Adults (Huang, Law and Lee, 2019) note that “while there have been de­
which included 92 self-reported Likert-scaled items “grouped in three bates regarding learning presence, it is clear that one cannot fully
subscales designed to assess the perception of the emotions sadness, explain and study online learning without a consideration of online
anger, happiness and the use of emotional regulation strategies” (p. 236) learners’ characteristics such as self-regulation” (p. 1885). In addition to
which were chosen based on their perceived associations with the self-regulation, other researchers have chosen to focus on emotional
educational context. characteristics. (Cleveland-Innes and Campbell, 2012) define Emotional
When it comes to approaching self-regulation from an integrated Presence (EP) as “the outward expression of emotion, affect, and feeling
emotion~cognition perspective, we have only just scratched the surface by individuals and among individuals in a community of inquiry, as they
in our understanding. New perspectives from the interdisciplinary af­ relate to and interact with the learning technology, course content,
fective sciences will continue to provide interesting juxtapositions to students, and the instructor” (p. 283).
classical beliefs about self-regulatory theories, particularly in how, and Anderson (2014) later wrote a review of the research published by
to what extent emotion plays a significant role. Rienties and Rivers (2014) in which they proposed several

2
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

methodologies, both qualitative and quantitative, to assess EP from freely available French translation (Heilporn & Lakhal, 2020; Nadeau,
student data. In his review, Anderson commented that emotion, or at 2012).
least affective expression, was included to some degree in SP. He also Overall, recent studies that have focused on the validation and
commented on why earlier proposals of LP were met with resistance measurement of COI constructs have yet to bring self-regulatory anal­
from the original framing authors. “While not denying the value of ysis, the multi-deminsonality of COI factors, and their causal relation­
‘learner presence’ it takes the model into psychological realms that our ships together into one comprehensive study. The purpose of this
more sociological orientation had avoided in the initial formation” research project is to further explore the constitution and operational
(Anderson, 2014). He also cited simple parsimony as a motivator to modeling of LP within the structure of the COI framework, where LP is a
continue to express the COI framework within its original three-circled multi-dimensional representation of student self-regulation. This was
Venn diagram. We are no closer to a consensus today on an “Nth pres­ accomplished in a two-phase research study. The first phase, presented
ence” than we were in 2014, but he did go on to concede the point, “this herein, was focused on confirming the measurement and dimensionality
review convinced me that something is lost when the emotional aspect of of TP, SP, CP, and adding the measurement of LP to a single inclusive
human experience in education is ignored” (emphasis added). survey instrument. The second phase, which will be presented in future
Anderson’s comments did raise an important point regarding the work, was focused on investigating structural models of the COI
sociological versus psychological perspectives with which the COI was framework suggested in prior literature that were modified to include LP
framed. Perhaps so many scholars have pursued a fourth presence, and as a fourth factor. The model fit and theoretical implications of these
continue to do so, because the psychological perspective that is funda­ models were analyzed and discussed for the purpose of proposing a new
mental to learning is notably absent. For example, Kozan and Caskurlu structural model for the COI framework with explicit inclusion of stu­
(2018) preformed a review of COI literature in which an additional dents’ self-regulatory characteristics.
presence was proposed. In their review, they reported six variations of
an “Nth presence” where all but two (Distributed Teaching Presence and 2. Research methods
Teacher Engagement) were affect-oriented in some way.
Anderson’s claim that emotional expression, at least to some extent, The objective of this study, the initial phase of a two-phase project,
is captured within the SP construct is especially salient when compared was to confirm the interrelation of LP with the other three COI factors.
with the results of factor analysis performed by Cleveland-Innes and This study was conducted using quantitative multivariate linear
Campbell (2012), which included six new EP survey items appended to regression methodologies grounded in a post-positivist paradigm. The
the classical COI Survey. In their analysis, both TP and CP items, for the post-positivist paradigm acknowledges that while it is reasonable to test
most part, behaved as expected. Nine of 13 TP items clustered into one and compare hypotheses using empirical evidence, no method of inquiry
factor, and 11 of 12 CP items clustered together across two factors. Four is absolute, thus understanding is gained and improved over time
of the six EP items clustered into one factor, but only four of nine SP through multiple methods of inquiry (Patton, 2002). In general, the
items clustered together. The remaining five had loaded onto the TP objective of this study was confirmatory in nature, making CFA an
factor (1), EP factor (1), or were among the six items that did not have a appropriate method to test a priori models. Throughout this process,
valid loading (3). Recognizing that multiple causescould be responsible however, results from the model estimations (e.g., estimated parame­
for these changes, it is interesting nonetheless that the addition of these ters, model-fit indices, and modification indices) were used to improve
six EP items seemed to change the patterns of SP more so than the other the initial models.
presences. It is also notable that only the outward emotional expression
was of interest in this study, which was reflected in both the stated 2.1. Research design
definition of EP and the associated survey items proposed (Cleveland-
Innes & Campbell, 2012). This outward focus may be more consistent An existing instrument has been developed to measure TP, SP, and
with the sociological perspectives with which the COI was framed, but it CP in a single Likert-scale survey, but LP has yet to be integrated into a
unfortunately does little to close the self-regulatory measurement gap single instrument. Therefore, the driving research question for this study
that plays a critical role in how we understand and engage with learning was sub-divided into the following three objectives:
online in a community of inquiry.
1. Confirm the independent measurement of TP, LP, SP, and CP
1.5. Research gap and statement of purpose 2. Confirm the dimensionality of TP, LP, SP, and CP
3. Confirm the simultaneous measurement and association of the four
The COI framework, its constructs, and its application continue to be COI constructs
area of significant interest. Recent studies have been conducted to re-
examine the construct validity of the TP, SP, and CP as originally pro­ The analysis was based on a 47-item survey, entitled Web-based
posed with an independent sample EFA and CFA combined analysis Teaching and Learning Link to Social and Cognitive Presence (WebTALK),
(Kozan & Richardson, 2014), as well as the causal relationships between which is presented in Appendix A. The scale included a five-point Likert
these three factors (Dempsey & Zhang, 2019). Several more studies have scale: strongly agree, agree, neutral, disagree, and strongly disagree. The
used validation techniques to validate translations of the classical COI items in the WebTALK were selected from existing Likert-scale survey
Survey in different languages for broader international application. For instruments that were well-developed with strong validity evidence in
example, Ma et al. (2017) performed a validation study on a Chinese the literature. Given the untested nature of the WebTALK instrument,
translation of the classical COI Survey, with appended self-regulatory reliability evidence via Cronbach’s alpha, and validity evidence via
items to represent LP as a COI factor (see, Shea et al., 2014; Traver, comparison to prior literature and CFA results were evaluated at mul­
Volchok, Bidjerano, & Shea, 2014). In this study, Ma et al. analyzed both tiple stages throughout the research process. For example, first-order
construct validity and causal relationships. Yu and Richardson (2015) CFAs were used to individually confirm the measurement of the four
and Olpak & Çakmak (2018) also performed validation studies on constructs. In addition, the first-order CFAs were used to confirm the
Korean and Turkish translations of the classical COI Survey, respectively, dimensionality of the four COI constructs. Here, dimensionality refers to
following similar EFA and CFA methodologies as Kozan and Richardson the hypothesized subscales that comprise each of the COI factors. A
(2014). Caskurlu (2018) and Heilporn and Lakhal (2020) performed second-order CFA was conducted to confirm the four-factor structure of
construct validation studies using second-order confirmatory factor the COI framework at large, and the significant association between
analyses to investigate the dimensionality of the COI constructs using each of the four factors. To summarize, the second-order factors repre­
the classical COI Survey (without additional self-regulatory items) and a sent the four primary COI constructs, and the first-order factors

3
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

represent the subscales of each second-order factor. selected from the LEP scale (Moore, 1990) which was developed to
assess post-secondary students’ level of intellectual development ac­
2.2. Research setting and participants cording to Perry’s seven stages (Perry, 1970/1999). The scale consists of
60 four-point Likert-type questions across five subscales. Cronbach’s
Research participants were recruited from multiple institutions alpha ranged from 0.63 to 0.84 (Moore, 1990).
through professional networks and conferences. Recruitment efforts In the final stage, a complete draft was sent to peers with expertise in
were targeted toward faculty and administrators across five U.S. in­ survey design and familiarity with the constructs being measured. They
stitutions who then distributed the online WebTALK survey to students were asked to review the instrument for of item clarity, flow, and gen­
enrolled in fully online courses within their programs/institutions. The eral effectiveness to measure the target construct. The peer review
recruitment of research participants was implemented in accordance yielded several rounds of revision before the final survey was dissemi­
with Protocol #1307013808 reviewed and approved by the Institutional nated to the research participants. The final 47-item WebTALK survey
Review Board (IRB) at Purdue University. Criterion sampling was used instrument is shown in Appendix A (Table A.1, A.2, A.3 and A.4), which
to select participants for the study, based on 1) the participant was includes the item text, item ID (used in analysis), construct subscale, and
currently a graduate student enrolled in one or more online course, and item source.
2) the participant was had previously completed at least one online
course that counted toward their degree requirements. 3. Results

2.3. Research instrumentation 3.1. Treatment of missing values

The survey instrument used in this study, entitled Web-based Teach­ The WebTALK survey instrument was administered via Qualtrics
ing and Learning Link to Social and Cognitive Presence (WebTALK) was online survey software. A total of 372 participants responded to the
constructed for this research, and consists of items from existing scales to survey. The factor analyses in this study were performed using the IBM®
measure the four constructs of TP, LP, SP, and CP. The Classical COI SPSS® AMOS V22.0.0 statistical package. Based on the sample size
Survey, as discussed in Section 1.3 is well-established with sufficient achieved in this study, the maximum likelihood (ML) estimator was
published reliability and validity evidence. It does not, however, have selected for analysis (Brown, 2015), which cannot tolerate missing
any measures of LP. Thus, the instrument was modified to include items values. The survey instrument did not have any forced response items,
to measure LP. In instrument development, it is common practice to and participants were permitted to exit the survey at any point; there­
include more items to start with and eliminate poor performing items fore, missing values had to be evaluated.
during analysis. The WebTALK instrument was designed with as few TP, Of the 372 respondents, 126 cases (33.8%) contained one or more
SP, and CP items as possible to accommodate a larger number of LP missing values associated with the 47 WebTALK items measuring the
items. four dimensions of COI. If participants opted not to complete any of the
In general, the WebTALK survey instrument was developed in three demographic items, these were not included in the missing values
stages. The first stage was to reduce the total number of items used in the analysis. Given the relatively large number of cases that had one or more
existing COI instrument. In a preliminary study (Wertz, 2014), CFA missing value, it was impractical to use listwise deletion for all cases
models of each of the three COI constructs measured by the existing COI with missing data. Instead, listwise deletion was performed for all cases
instrument were analyzed individually. The initial model included all with 11 or more missing values, representing 25% of the overall
the indicators assigned to each construct. Modification Indices (MIs) WebTALK survey. After deletion, 256 cases were retained for analysis.
from the CFAs were used to identify candidates for deletion. Final de­ Across the 256 cases there were a total of 14 missing values, which
cisions for item deletion were made with a combination of evidence represented less than 1% of any given factor, less than 10% of any given
from the MIs and critical evaluation of each item. Overall, the COI items case, and 0.12% of the overall dataset. Missing values were replaced
were reduced from 34 items to 21 items. using the series mean (i.e., the mean of the values reported within the
The second stage was to select items to measure LP. These items were given case for the other indicator variables within the associated factor
selected from three existing scales to measure motivational, behavioral, grouping), as suggested by Byrne (2010).
and developmental aspects of LP.
Motivated Strategies for Learning Questionnaire (MSLQ). Indicators of 3.2. Participant demographics
motivational and behavioral aspects of LP were taken from the MSLQ
developed by Pintrich, Smith, Duncan, and Mckeachie (1991). This The 256 respondents included in the analysis primarily represented
questionnaire consists of 81 Likert-scale items across two primary con­ five U.S. institutions, Baker College (36.3%), Purdue University
structs, motivation and learning strategies, with several subscales in (31.1%), Boise State University (16.4%), Norwich University (7.0%),
each section. The MSLQ has been used in over 50 studies in the last and University of North Dakota (5.9%), with the remaining 3.1% un­
decade with reported Cronbach’s alpha ranging from 0.69 to 0.93 specified. Within the sample population a broad range of institutions,
(Pintrich, Smith, Garcia, & McKeachie, 1993). The validity evidence of including large research universities and small private universities were
the MSLQ includes confirmatory factor analyses of the motivational and represented.
learning strategy scales. The disciplines of study of the respondents were roughly organized
Online Self-Regulated Learning Questionnaire (OSLQ). Several in­ into four categories, including Education (42.6%), Business (35.9%),
dicators of self-regulation were taken from the Online Self-Regulated Engineering (21.5%), and Other (1.2%). The respondents were 56.6%
Learning Questionnaire (OLSQ) developed by Barnard et al. (2008). female, 42.6% male, with 0.8% unreported.
This questionnaire consists of 24 Likert-scale items across six subscales: Most respondents (83.2%) have taken more than three online cour­
goal-setting, environment structuring, task strategizing, time manage­ ses, with the remainder having taken one to three online courses.
ment, help seeking, and self-evaluation. The Cronbach’s alpha for the Completing at least one online course was a criterion of participation, so
instrument was 0.67 to 0.96 (Barnard et al., 2009). In addition, evidence any participant that indicated they have not taken at least one online
of construct validity was achieved through a second-order confirmatory course was redirected to the end of the survey.
factor analysis. Finally, 60% of the respondents were between the ages of 30 and 49,
Learning Environment Preferences (LEP) Scale. LP, as it was originally 14% were younger than age 30, 18% were ages 50 or older, and 8% did
described, encompasses the learners’ developmental, motivational, and not specify their age. Age does play a significant role in intellectual
behavioral processes (Shea & Bidjerano, 2010). Several items were also development and epistemological values. It should be noted that a

4
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

critical majority (approximately 80%) of the participants were adult 3.4. Measurement of teaching presence
learners over the age of 30.
Teaching presence is generally expressed with three subscales, Design
& Organization, Facilitation, and Direct Instruction. Recent investigations
3.3. Confirmatory analysis procedures and assumptions by Caskurlu (2018) and Heilporn and Lakhal (2020) suggest that TP
could be expressed more parsimoniously as two subscales: Course
This section presents the results of the first-order CFAs of the TP, SP, Design and Facilitation. In addition, I proposed a third dimension rep­
LP, and CP constructs, first-order dimensionality testing of the TP, SP, resenting peer facilitation based on the theoretical assumption that the
LP, and CP constructs, and a second-order CFA of all four constructs responsibility of teaching is shared among all course participants,
combined into one model, and Cronbach’s alpha for each factor included including peer-to-peer interactions. Thus, the initial model included
in the WebTALK instrument. three subscales, Course Design, Instructor Facilitation, and Peer Facilitation,
In the SEM figures provided in this section, unobserved variables with a total of 11 indicators (Fig. 1). The results of the initial model
(latent factors) are shown in ovals and observed variables (indicators) showed that correlations between Course Design and Peer Facilitation (r
are shown in rectangles, and the small circles to the right of the indicator = 0.36, p < .01) and even between Instructor Facilitation and Peer
variables represent the residual (error) of each item. The SEM data were Facilitation (r = 0.57, p < .01) were relatively small, particularly for
analyzed across six categories: 1) multivariate normality, 2) model subscales that are supposed to be explaining the same general construct.
summary, 3) estimated parameters, 4) modification indices (MIs), 5) The second and final model (Fig. 2), peer facilitation was removed from
model fit indices, and 6) bootstrapping 95% confidence intervals (if the TP construct. The final model had nearly perfect model fit [χ 2 (13) =
applicable). Initial models were respecified based on a review of model 15.97, p = .251; GFI = 0.98, IFI = 1.00, TLI = 1.00, CFI = 1.00; RMSEA
fit and MIs, and this process was repeated until no further respecification = 0.03)]. Table 1 presents the parameter estimates, the 95% confidence
was necessary. Any respecifications, however, were based on critical intervals and p-values from the bootstrap procedure.
evaluation of the model, and not solely based on numerical output of
each model. Presented in this section are the initial models, a description
of any respecifications, and the parameter estimates and model fit sta­ 3.5. Measurement of social presence
tistics of the final models. The evaluation criteria for each of the six
categories of analysis used in this study are summarized in Appendix B Social presence generally comprises three subscales, including Af­
(Table B.1). fective Expression, Active Communication, and Group Cohesion. Prior work
Mardia’s normalized estimates were evaluated to assess multivariate by Caskurlu (2018) and Heilporn and Lakhal (2020) similarly suggest
normality within each model. Values ranged from 21.92 to 49.33, where that it is possible to reduce the number of subscales in SP as well. The
values greater than 5 are considered multivariate non-normal (Byrne, initial model for the SP construct included two subscales, Group
2010). To deal with violations of multivariate normality common in real Belonging, and Open Communication, with a total of eight indicators
data (versus simulated data), bootstrapping procedures included in the (Fig. 3). The MIs of the initial model showed that survey item WSP37
IBM® SPSS® AMOS V22.0.0 software package were applied to report was cross-loading onto both of the SP subscales, and that there was a
parameter estimates with more confidence. For the parameter estimates strong correlation between the residuals (error) of survey items WSP36
reported in this section, the bootstrap 95% confidence interval, and the and WSP38.
associated p-value is reported in addition to the original estimate. Item WSP37 states “I felt comfortable interacting with some of the other

Fig. 1. Initial TP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).

5
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

reliable than others in this subscale.


Items WSP36 and WSP38 respectively state “I felt comfortable
participating in the course discussions,” and “I felt comfortable expressing
opinions that differed from those expressed by other course participants.”
Both items are, to some extent, assessing comfort level communicating
with other course participants. WSP38 is specific in how the respondent
felt about expressing differing opinions, whereas WSP36 is more general.
It would follow that item WS38 is, to some degree, embedded in item
WSP36. It is not necessary that one of these items be eliminated, but
given the content overlap, it is acceptable to respecify the model to allow
the residual terms (d36 and d38) to correlate (Byrne, 2010).
One final respecification was not based on the results of the model of
the initial SP model, but rather the TP model. The Peer Facilitation items
that were removed from the TP model were generally reliable (α =
0.88), but they did not have strong correlations to the other TP sub­
scales. On the suspicion that this was a result of the differences in how
the participants perceive peer interactions and instructor interactions,
the Peer Facilitation items were moved to SP model.
Fig. 2. Final TP measurement model with standardized parameter estimates. n The second and final model (Fig. 4) of the SP construct included
= 256. (*p < .05, **p < .01). three subscales with a total of 11 items. The final model had very good
model fit [χ 2 (40) = 66.00, p > .005; CMIN/DF = 1.65; GFI = 0.95, IFI =
0.99, TLI = 0.98, CFI = 0.98; RMSEA = 0.05)]. A summary of the
Table 1
Standardized parameter estimates for the first-order TP measurement model.
parameter estimates, and the 95% confidence intervals and p-values
from the bootstrap procedure are presented in Table 2.
Bootstrap 95% CI

Parameter Estimate Lower Upper p-value 3.6. Measurement of learning presence


WTP1 ← CD 0.88 0.79 0.93 0.009**
WTP2 ← CD 0.87 0.80 0.91 0.005** The initial model for the LP construct included three subscales,
WTP3 ← CD 0.69 0.58 0.79 0.004** Motivational, Behavioral, and Developmental, with a total of 20 indicators
WTP4 ← IF 0.87 0.77 0.91 0.011*
(Fig. 5). Given that the LP scale was new in its current form, it was ex­
WTP5 ← IF 0.84 0.81 0.88 0.005**
WTP6 ← IF 0.79 0.72 0.86 0.004** pected that several of the weaker performing items would be eliminated
WTP7 ← IF 0.85 0.79 0.90 0.005** until each subscale had three or four high-performing items remaining.
CD ↔ IF 0.82 0.71 0.90 0.012* The MIs were used as a guide to select items that were good candidates
Notes. n = 256; CI = Confidence Interval, CD = Course Design, IF = Instructor for deletion, and the deletion occurred over three iterations before
Facilitation, WTP1 = WebTALK item ID as defined in Appendix A; (*p < .05, **p reaching the final model on the fourth iteration.
< .01). In summary, a total of 11 items were eliminated from the LP mea­
surement model, based on univariate non-normality (WLP14), non-
significant or small standardized regression weights (WLP13, WLP20,
WLP17, WLP22, WLP23, WLP25R, WLP26, WLP27R, and WLP31), and
cross-loading (WLP24). With the removal of these items, three high-
performing items remain as indicators of each of the three LP subscales.
The fourth and final model (Fig. 6) consisted of three subscales with a
total of nine indicators. The final model had very good model fit [χ 2 (24)
= 36.44, p = .05; CMIN/DF = 1.52; GFI = 0.97, ILI = 0.98, TLI = 0.96,
CFI = 0.98; RMSEA = 0.045)]. Refer to Table 3 for a summary of the
parameter estimates, and the 95% confidence intervals and p-values
from the bootstrap procedure.

3.7. Measurement of cognitive presence

The initial model for the CP construct included four subscales, Trig­
gering Event, Exploration, Integration and Resolution with a total of eight
indicators (see Fig. 7). The initial model had very good model fit [χ 2
(14) = 22.98, p = .06; GFI = 0.98, IFI = 0.99, TLI = 0.98, CFI = 0.99;
RMSEA = 0.05)], thus no respecification was needed. Refer to Table 4
Fig. 3. Initial SP measurement model with standardized parameter estimates. for a summary of the parameter estimates, and the 95% confidence in­
n = 256. (*p < .05, **p < .01). tervals and p-values from the bootstrap procedure.

participants outside of assigned group discussions.” There is an implicit 3.8. Dimensionality of COI constructs
assumption in this item that assigned group discussions were part of the
online course. If this assumption is not true, or not perceived to be true One assumption of factor analysis is that the unobserved latent
by the respondents, then the item would likely produce unreliable re­ variables represent a single unidimensional construct, versus a collec­
sults. In review of the initial internal reliability analysis, Cronbach’s tion of stratified constructs. Prior research has suggested that all four of
alpha for the Open Communication subscale would increase from 0.83 to the COI constructs contain two or more subscales, which violates the
0.86 if item WSP37 is removed, which suggests that the item is less unidimensional assumption. In this section the results of the chi-square
difference tests used to confirm the dimensionality of the TP, SP, LP and

6
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Fig. 4. Final SP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).

was constrained to equal 1, was specified to test the dimensionality of


Table 2
the TP construct. If this added constraint improves the model fit, then
Standardized parameter estimates for the first-order SP measurement model.
that would imply that Course Design and Instructor Facilitation have
Bootstrap 95% CI enough shared variance that they are not discernible constructs. The chi-
Parameter Estimate Lower Upper p-value square difference test was used to compare the less constrained model
WSP32 ← GB 0.78 0.66 0.85 0.008** with the covariance unconstrained to the model with covariance con­
WSP33 ← GB 0.80 0.71 0.86 0.008** strained. A statistically significant result implies the added constraint
WSP34 ← GB 0.71 0.62 0.79 0.003** worsened the model fit, and should then be rejected, (e.g., for TP, χ 2 (1)
WSP35 ← OC 0.79 0.71 0.87 0.006** = 20.33, p < .001). For factors with three or more subscales the
WSP36 ← OC 0.76 0.68 0.83 0.003**
WSP38 ← OC 0.67 0.56 0.77 0.005**
covariance constraint was applied between two latent variables, one
WSP39 ← OC 0.83 0.77 0.89 0.006** model at a time, each compared to the original less constrained model.
WTP8 ← PF 0.86 0.81 0.91 0.005** All chi-square tests yielded significant results with p < .001, thus no
WTP9 ← PF 0.79 0.70 0.86 0.007** additional model specifications were made.
WTP10 ← PF 0.75 0.65 0.83 0.006**
WTP11 ← PF 0.83 0.78 0.88 0.003**
GB ↔ OC 0.92 0.85 0.98 0.002**
OC ↔ PF 0.71 0.60 0.80 0.003** 3.9. Second-order COI measurement model
PF ↔ GB 0.77 0.65 0.87 0.003**
d36 ↔ d38 0.37 0.15 0.53 0.007** Once the measurement models of the TP, LP, SP, and CP constructs
Notes. n = 256; CI = Confidence Interval, GB = Group Belonging, OC = Open were confirmed individually, the next step in the analysis was to
Communication, PF = Peer Facilitation, d36 and d38 = Residual (error) of items combine the four constructs into one measurement model. Since each of
WSP36 and WSP38, WSP32 = WebTALK item ID as defined in Appendix A; (*p < the four constructs were confirmed to be multi-dimensional, the com­
.05, **p < .01). bined measurement model was analyzed as a second-order CFA, which
consisted of TP, LP, SP, and CP as second-order latent variables, and the
CP constructs are presented. subscales of each construct as first-order latent variables.
The dimensionality of each construct was tested by comparing the Prior to analyzing the second-order COI measurement model, the
final measurement model to a nested model that constrained the Mahalanobis distance (D2) was evaluated to identify potential outliers.
covariance between two latent variables (i.e., subscale) to equal 1. The Based on this analysis four cases were eliminated as outliers, leaving n =
nested model suggests that the two constrained subscales could be 252 for the second-order analysis. Evaluation of potential outliers was
combined into one factor to make a more parsimonious model. One excluded from the analyses of the individual measurement models to
nested model was created for each covariance in the final model, and avoid over-deletion of cases that may have presented as an outlier in the
each nested model was compared to the less constrained final model, analysis of one construct, but not in others. In addition, a CDIFF pro­
individually. The chi-square difference test comparing the less con­ cedure, included in the SPSS® AMOS statistical package, was used to
strained model to the more constrained nested model was used to identify residual variance parameters in the model that could be con­
determine if any of the a priori specified subscales could be combined strained to equality without negatively impacting model fit. The purpose
into a single factor. If the chi-square difference test was significant, it of this procedure was to minimize the number of estimated parameters
follows that the added constraint significantly worsened the model, and in the model to accommodate the smaller sample size used in this
the constraint should be rejected. research study. For a review of the CDIFF procedure for the SPSS®
As an example, the final measurement model for the TP construct AMOS program see Byrne (2010).
consisted of two subscales, Course Design and Instructor Facilitation, as The initial model for the second-order COI measurement model
shown in Fig. 2. A nested model, where the latent variable covariance included four second-order latent variables, 12 first-order latent vari­
ables, and 35 observed indicator variables. The second-order model data

7
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Fig. 5. Initial LP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).

Fig. 6. Final LP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).

8
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Table 3 CMIN/DF = 2.25; GFI = 0.83, IFI = 0.92, TLI = 0.91, CFI = 0.92;
Standardized parameter estimates for the first-order LP measurement model. RMSEA = 0.055)], and was accepted as the final model without further
Bootstrap 95% CI specification.
Parameter Estimate Lower Upper p-value

WLP12 ← MV 0.57 0.43 0.72 0.003**


3.10. Reliability and validity evidence
WLP15 ← MV 0.76 0.63 0.86 0.003**
WLP16 ← MV 0.75 0.63 0.83 0.007** The internal reliability of the COI constructs, as measured by the
WLP18 ← BH 0.57 0.43 0.70 0.004** WebTALK instrument, were evaluated based on the final measurement
WLP19 ← BH 0.60 0.46 0.73 0.004**
models confirmed in this study. Reliability was evaluated by computing
WLP21 ← BH 0.64 0.50 0.77 0.007**
WLP28 ← DV 0.65 0.53 0.76 0.005** Cronbach’s alpha coefficient for each primary scale, which ranged from
WLP29 ← DV 0.56 0.40 0.69 0.006** 0.79 to 0.92, and subscale, which ranged from 0.63 to 0.89. The results
WLP30 ← DV 0.65 0.54 0.79 0.005** of the internal reliability for each of the four COI constructs are pre­
MV ↔ BH 0.68 0.54 0.83 0.003** sented in Table 6.
BH ↔ DV 0.49 0.31 0.72 0.002**
DV ↔ MV 0.70 0.57 0.87 0.002**
Construct validity of the inferences made using the WebTALK in­
strument is based on the good model fit of the specified measurement
Notes. n = 256; CI = Confidence Interval, MV = Motivational, BH = Behavioral, models. While this alone is insufficient evidence, it is also taken in the
DV = Developmental, WLP12 = WebTALK item ID as defined in Appendix A; (*p
context that a) the measurement models were specified based on
< .05, **p < .01).
fundamental theoretical assumptions of the COI framework and prior
empirical investigations from the literature, and b) the models per­
were evaluated using the same procedures as the first-order measure­ formed well with minor respecifications. Since this is the first study to
ment models. use the WebTALK survey as a single comprehensive instrument, further
The modification indices of the initial model showed that WSP38 evidence, including cross-validation with larger sample sizes, are
(see Appendix A for a complete list of survey items) was cross-loading required to strengthen the validity claims for this instrument.
onto other factors. In addition, the modification indices indicated a Evidence for the validity of the bootstrap procedure used in this
strong correlation between the residuals for Instructor Facilitation (r2) study to accommodate the CFA of multivariate non-normal data was
and Peer Facilitation (r8). The wording of the Peer Facilitation survey demonstrated by comparing the original COI survey items used in
items is identical to the Instructor Facilitation survey items except that the WebTALK instrument to prior studies that used SEM analyses to inves­
Peer Facilitation items begin with “my classmates” in place of “my tigate COI structural models (see Phase 2 for further discussion).
instructor,” thus it is reasonable that the error between these items
would be related.
4. Discussion of findings
The second-order model (Fig. 8) was respecified with the removal of
item WSP38, which was cross-loading onto LP as well as SP, and with a
4.1. First-order measurement of COI factors
correlation between the residuals of Instructor Facilitation (r2) and Peer
Facilitation (r8). The results of the respecified model showed that all of
The objective of this study was to confirm the measurement of the
the estimated standardized regression weights and correlations were
four COI constructs in the WebTALK instrument. This was accomplished
significant, as shown in Table 5. Several estimated variance parameters,
through a series of first-order CFAs, which represented the measurement
however, were non-significant (p > .05), which is likely due to a limi­
models for each of the four COI constructs. Iterations between confir­
tation of the sample size relative to the complexity of the model. The
matory and exploratory analyses yielded very good to near perfect
second model had reasonable model fit [χ 2 (513) = 900.5, p < .001;
model fit for each of the four measurement models. Of particular interest

Fig. 7. Initial and final CP measurement model with standardized parameter estimates. n = 256. (*p < .05, **p < .01).

9
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Table 4 separation of instructional and learning roles is beneficial for a model of


Standardized parameter estimates for the first-order CP measurement model. online teaching and learning” (p. 14). In their case, this was an argument
Bootstrap 95% CI constructed around the benefits of explicit inclusion of LP within the COI
framework, but it also has relevance here as further evidence that stu­
Parameter Estimate Lower Upper p-value
dents perceive the roles of their instructors differently, even if TP can be
WCP40 ← TE 0.79 0.68 0.86 0.007** exhibited by anyone in a COI.
WCP41 ← TE 0.76 0.66 0.83 0.006**
WCP42 ← EX 0.69 0.56 0.80 0.005**
WCP43 ← EX 0.63 0.50 0.73 0.006** 4.1.2. Measurement of learning presence
WCP44 ← IN 0.72 0.58 0.85 0.006** The measurement of LP as a three-dimensional construct consisting
WCP45 ← IN 0.73 0.63 0.82 0.004** of developmental, motivational, and behavioral indicators was a unique
WCP46 ← RE 0.76 0.66 0.85 0.007**
contribution of this study. LP, as recent addition to the COI framework,
WCP47 ← RE 0.73 0.61 0.82 0.006**
TE ↔ EX 0.85 0.70 1.00 0.004** cannot currently be measured with same the survey instrument used to
EX ↔ IN 0.93 0.75 1.10 0.004** measure TP, SP, and CP.
IN ↔ RE 0.83 0.67 1.02 0.003** Prior studies have used secondary instruments in addition to the
RE ↔ EX 0.94 0.81 1.08 0.002** Classical COI Survey, such as the OSLQ to measure LP (e.g., Shea &
IN ↔ TE 0.92 0.77 1.05 0.005**
TE ↔ RE 0.83 0.72 0.94 0.005**
Bidjerano, 2012). The WebTALK instrument was developed for this
research, but also to provide revision for parsimony to the Classical COI
Notes. n = 256; CI = confidence interval, TE = Triggering Event, EX = Explo­ Survey based on recommendations from more recent validation studies
ration, IN = Integration, RE = Resolution, WCP40 = WebTALK item ID as
(e.g, Heilporn & Lakhal, 2020; Kozan & Richardson, 2014; Ma et al.,
defined in Appendix A; (*p < .05, **p < .01).
2017). In addition, the OSLQ incorporated motivational and behavioral
indicators, but did not explicitly include developmental indicators. This
was the inclusion of peer facilitation items, which were new to the third dimension of LP, although not accounted for in prior research, was
measurement of the COI constructs, as well as the measurement of LP. based on Zimmerman’s (2008, p. 168) definition of self-regulated
During the evaluation of the measurement models, I made two impor­ learning, “the degree to which students are metacognitively, motiva­
tant respecifications based on the results of the initial models, the first tionally, and behaviorally active participants in their own learning
with respect to the Peer Facilitation subscale, and the second with respect process,” where intellectual development is an explicit factor in stu­
to the overall measurement of LP. dent’s ability to self-regulate. The relationships between LP and the
other COI constructs observed in this study, compared to those observed
4.1.1. Including peer facilitation as a subscale of social presence in prior studies, were likely affected by the change in how the LP
A fundamental assumption of the COI framework is that the teaching construct was defined and measured. I believe, however, that the three-
responsibilities are shared among all course participants, thus TP should dimensional approach provides a richer definition of the LP construct,
include co-regulatory processes as well as instructor-focused regulation. and better reflects the contribution of the learner as an individual into
The Classical COI Survey however, only included items that began with the larger social context.
“the instructor” so co-regulation was not being captured in the mea­
surement of TP. To probe this assumption further, the WebTALK in­ 4.2. Second-order measurement of COI factors
strument included four co-regulation items in the Peer Facilitation
subscale that mirrored the Instructor Facilitation items but started with Confirmatory factor analysis was used to confirm the measurement
“my classmates” in place of “my instructor.” During the analysis, the Peer of TP, SP, LP, and CP as four discernible constructs when analyzed as a
Facilitation subscale showed relative low correlations to the other two whole. Given, however, that each factor was found to comprise two to
subscales included in the measurement of TP. Following the possibility four subscales, a second-order SEM was used for analysis. While this
that co-regulation was perceived by participants to be more closely related analysis was successful in reaching good model fit, the sample size may
to other social processes, I moved the Peer Facilitation subscale from the limit applications using the second-order SEM for any further causal
TP construct to the SP construct, where it showed stronger relationships path analyses. This limitation is discussed and further addressed in
with the other specified subscales. future work that is in progress, which presents the second phase of this
The finding that the Peer Facilitation subscale showed better model fit two-phase project. To clarify, however, the sample size of this study is
with the SP construct than with the TP construct does not violate the sufficiently large to test the construct validity using second-order CFA
assumption that teaching responsibilities are shared among the course because only one presence model is tested at a time, and thus the
participants, but it does exemplify the perceptible differences between number of factors remains relatively small. The results of this study were
student–instructor and student–peer interactions. It is also possible that consistent with Caskurlu (2018) and (Heilporn and Lakhal, 2020) where
the perceived difference between the Peer Facilitation and Instructor “the validity of both first-order and second-order models implies that the
Facilitation subscales is a result of unequal authority. For example, in a best representation of the CoI framework should explicitly take cate­
course where the instructor has primary authority over the final gories into account, as students’ perceptions reflect these across two
outcome (e.g., final grade) of the course, which is common in most independent groups” (p.15). Furthermore, by treating the COI presences
formal education settings, it is understandable that students would as unidimensional constructs we dampen our ability to use these models
perceive feedback given by instructors differently than feedback given as tools to visualize and explain more nuanced perspectives of students’
by peers. In a course where peers had equal, or near equal, authority perceptions and experiences in learning online.
over the final grade by means of heavily weighted and frequent peer
review, it is possible that Peer Facilitation would fit more closely with TP 5. Conclusion and future work
as initially posited.
Shea et al. (2014) found similar discontinuities between the theo­ This study makes several unique methodological contributions to the
retical construct of TP where all actors can take on teaching roles, versus COI literature. First, the WebTALK survey included Peer Facilitation as a
the actual perceptions of students who see clear distinctions between the new subscale of SP and included Motivational, Behavioral, and Devel­
responsibilities of their instructors versus other actors within the opmental subscales of LP. Second, the use of second-order CFA to
learning environment. They concluded that “fully conflating teacher and address the issue of construct multi-dimensionality has not been used in
learner roles, as is done in the [original] conceptualization of the previous investigations of causal relationships, or investigations that
construct of TP, obscures as much as it illuminates. Therefore, the also included analysis of self-regulation. This study provides the

10
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Fig. 8. Final second-order COI measurement model with standardized parameter estimates. n = 252.

11
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Table 5 Table 6
Standardized parameter estimates for the second-order COI measurement Internal reliability of final COI scales and subscales (n = 248).
model. Final COI Scales No. of Items Mean SD Alpha
Bootstrap 95% CI
Teaching Presence (TP) 7 29.38 5.14 0.91
Parameter Estimate Lower Upper p-value Course Design (CD) 3 13.22 2.09 0.84
Instructor Facilitation (IF) 4 16.16 3.39 0.89
CD ← TP 0.87 0.81 0.93 0.008** Learning Presence (LP) 9 38.40 4.32 0.79
IF ← TP 0.91 0.83 0.95 0.008** Motivational (MV) 3 13.26 1.71 0.73
MV ← LP 0.96 0.85 1.06 0.003** Behavioral (BH) 3 12.75 1.83 0.63
BH ← LP 0.68 0.54 0.81 0.003** Developmental (DV) 3 12.38 1.94 0.67
DV ← LP 0.78 0.66 0.87 0.007** Social Presence (SP) 10 38.53 7.28 0.92
GB ← SP 0.96 0.93 0.98 0.002** Group Belonging (GB) 3 11.81 2.44 0.81
OC ← SP 0.94 0.89 0.97 0.003** Open Communication (OC) 3 12.71 2.02 0.84
PF ← SP 0.74 0.62 0.82 0.003** Peer Facilitation (PF) 4 14.02 3.73 0.88
TE ← CP 0.91 0.84 0.96 0.004** Cognitive Presence (CP) 8 33.93 4.66 0.87
EX ← CP 0.98 0.89 1.04 0.004** Triggering Event (TE) 2 8.17 1.69 0.74
IN ←CP 0.97 0.87 1.06 0.003** Exploration (EX) 2 8.46 1.34 0.63
RE ← CP 0.88 0.79 0.95 0.006** Integration (IN) 2 8.69 1.20 0.63
WTP1 ← CD 0.88 0.81 0.93 0.009** Resolution (RE) 2 8.62 1.36 0.70
WTP2 ← CD 0.89 0.82 0.93 0.007**
WTP3 ← CD 0.70 0.60 0.80 0.003**
WTP4 ← IF 0.86 0.80 0.90 0.004**
construct, which exemplifies the gap between the theoretical foundation
WTP5 ← IF 0.83 0.77 0.88 0.005**
WTP6 ← IF 0.77 0.69 0.84 0.008** of the COI framework and perceived interactions within the constructed
WTP7 ← IF 0.84 0.78 0.89 0.006** learning environment; 2) the strongest association among the COI con­
structs was between LP and CP, which provides further justification of
Bootstrap 95% CI the value added by including LP as a discernible construct within the
Parameter Estimate Lower Upper p-value
COI framework.
This study did, however, utilize a new survey instrument, which
WLP12 ← MV 0.72 0.56 0.83 0.006**
requires additional analysis to further validate inferences made with
WLP15 ← MV 0.68 0.53 0.80 0.005**
WLP16 ← MV 0.67 0.50 0.75 0.010* these data. The population for this study was comprised of students from
WLP18 ← BH 0.65 0.52 0.77 0.004** five U.S. institutions, that range in size and geographic location. A
WLP19 ← BH 0.56 0.41 0.67 0.009** diverse group of disciplines were represented as well. The population
WLP21 ← BH 0.65 0.51 0.75 0.007** included graduate students, primarily over the age of 30, in fully online
WLP28 ← DV 0.65 0.53 0.74 0.006**
WLP29 ← DV 0.56 0.43 0.70 0.003**
courses. This study focused on the alignment of methodology with un­
WLP30 ← DV 0.68 0.55 0.78 0.007** derlying theories that support the COI framework, which should make
WSP32 ← GB 0.81 0.73 0.87 0.008** its methods and conclusions of general rather than specific interest.
WSP33 ← GB 0.81 0.73 0.86 0.007** However, generalizability on a global scale of any kind of human
WSP34 ← GB 0.69 0.59 0.76 0.005**
perception and behavior is something to be approached with great
WSP35 ← OC 0.76 0.66 0.84 0.013*
WSP36 ← OC 0.77 0.70 0.83 0.005** skepticism. In using these data or instruments in future work, both in­
WSP39 ← OC 0.85 0.79 0.90 0.007** ternal reliability and external validation checks are highly recom­
WTP8 ← PF 0.86 0.80 0.90 0.008** mended, as variation is likely to occur in different populations, as noted
WTP9 ← PF 0.79 0.72 0.86 0.005** across several validation studies discussed in Section 1.5.
WTP10 ← PF 0.75 0.66 0.82 0.005**
WTP11 ← PF 0.82 0.76 0.87 0.005**
Another notable limitation of the present study is the extent to which
emotional regulation, as discussed in Section 1.2, is represented within
the LP construct, as I believe it should be. With the onset of the COVID-
Bootstrap 95% CI
19 pandemic, educational researchers and practitioners are becoming
Parameter Estimate Lower Upper p-value
increasingly more aware and interested in the role emotion plays in the
WCP40 ← TE 0.77 0.66 0.83 0.008** human learning experience—and teaching experience for that matter!
WCP41 ← TE 0.76 0.67 0.84 0.005** Both are fundamentally social activities where our affective processes
WCP42 ← EX 0.70 0.58 0.77 0.011*
are essential to our experience. Concurrently, affective science is a new
WCP43 ← EX 0.64 0.55 0.74 0.004**
WCP44 ← IN 0.74 0.63 0.83 0.005** rapidly growing interdisciplinary field, where new perspectives and
WCP45 ← IN 0.69 0.59 0.77 0.007** traditions bring the promise of new insights. I believe that future di­
WCP46 ← RE 0.78 0.66 0.85 0.009** rections for those who engage with communities of inquiry at all levels
WCP47 ← RE 0.73 0.62 0.82 0.005**
should be to delve deeper into the psychosocial domains that have been
TP ↔ LP 0.66 0.52 0.78 0.005**
LP ↔ SP 0.68 0.55 0.80 0.006**
subtly (or not so subtly) avoided, particularly within Western cultures
SP ↔ CP 0.72 0.61 0.81 0.003** (Pessoa, 2021).
TP ↔ SP 0.53 0.36 0.68 0.006** The work presented in this report represents the first-phase of a two-
LP ↔ CP 0.88 0.77 0.97 0.005** phase study. The second phase built upon these findings and employed
TP ↔ CP 0.71 0.59 0.80 0.004**
the use of causal path analysis to investigate the operational structures
r2 ↔ r8 0.73 0.51 1.08 0.003**
of the inter-relations between presences within the COI framework. The
Notes. n = 252; CI = confidence interval; (*p < .05, **p < .01). manuscript in which I present the second phase of this project is in
progress. Preliminary results, however, strongly support the addition of
necessary confirmatory measurements to perform second-order causal LP into the overall COI framework, where LP was found to have the
path analysis, which was performed in the second-phase of this two- strongest direct effect on CP, more so than TP or SP. While I can
phase project. The manuscript presenting this work is currently in appreciate arguments for parsimony (Anderson, 2014), I also believe
progress. there is sufficient evidence from this, and dozens of other studies to
Two hallmark findings from this study include: 1) the inclusion of the support the addition of a 4th presence to the COI framework. The
Peer Facilitation subscale, and its eventual relocation from the TP to SP advantage of LP in comparison to other proposals of an Nth presence (as

12
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

discussed previously in Section 1.4), is the explicit inclusion of learners’ Table A.2 (continued )
regulatory beliefs, attitudes, and behaviors in the analysis of Commu­ Learning presence items Item ID Subscale Item source
nities of Inquiry. I keep up with all the weekly readings Adapted
and assignments for this course from MSLQ
Table A.1 I distribute my time spent on course WLP20 Behaviors Adapted
WebTALK items for teaching presence. work evenly throughout the week from OSLQ
instead of trying to do everything on
one day
Teaching presence items Item ID Subscale Item source
I set goals to help me manage my time WLP21 Behaviors Adapted
My instructor clearly communicated WTP1 Course Design Adapted for my online course from OSLQ
important course goals or learning from COIa I check my grades for feedback on how WLP22 Behaviors Original
objectives well I am doing in the course Item
My instructor provided clear WTP2 Course Design Adapted I rely on comments from my instructor WLP23 Behaviors Original
instructions on how to participate from COI for feedback on how well I am doing Item
in course learning activities in the course
My instructor clearly communicated WTP3 Course Design Adapted I compare my work to my classmate’s WLP24 Behaviors Adapted
important due dates or time frames from COI work to get a sense of how well I am from OSLQ
for learning activities doing in the course
My instructor helped to keep me WTP4 Instructor Adapted I prefer courses that focus more on WLP25R Development Adapted
engaged in productive dialogue Facilitationb from COI having the right answers than on from LEPc
My instructor helped to keep me on WTP5 Instructor Adapted discussing different methods of
task relative to the goals or Facilitation from COI solving a particular problem
objectives of this course [REVERSE CODED]
My instructor encouraged me to WTP6 Instructor Adapted I prefer courses with activities that WLP26 Development Adapted
explore new concepts related to this Facilitation from COI clearly lay out step-by-step from LEP
course procedures
My instructor provided feedback that WTP7 Instructor Adapted I prefer courses where my final grade WLP27R Development Adapted
helped me understand my strengths Facilitation from COI is based mostly on straightforward from LEP
and weaknesses relative to the goals tests and quizzes covering only
or objectives of this course topics directly taught [REVERSE
My classmates helped to keep me WTP8 Peer Adapted CODED]
engaged in productive dialogue Facilitation from COIc,d I prefer courses that focus on WLP28 Development Adapted
My classmates helped to keep me on WTP9 Peer Adapted discussions of personal answers from LEP
task relative to the goals and Facilitation from COIc,d based on evidence rather than just
objectives of this course right and wrong answers
My classmates encouraged me to WTP10 Peer Adapted I prefer courses where my final grade WLP29 Development Adapted
explore new concepts related to this Facilitation from COIc,d is based mostly on how well I can from LEP
course synthesize and practically apply
My classmates provided feedback (e. WTP11 Peer Adapted various course topics in a real-world
g., through discussion or peer Facilitation from COIc,d context that goes beyond what was
review) that helped me understand directly taught in the course
my strengths and weaknesses I prefer instructors who challenge me WLP30 Development Adapted
relative to the goals or objectives of to develop my own opinions that are from LEP
this course supported by evidence
I prefer instructors who allow students WLP31 Development Adapted
a to have a voice in what topics are from LEP
Arbaugh et al., 2008.
b
Instructor Facilitation was changed from the original subscale label Facilita­ covered and what types of learning
activities will be completed (e.g.,
tion to distinguish it from the new subscale Peer Facilitation.
c research paper, literature review, or
Peer Facilitation is a new dimension of TP tested in this study. Items are a
project)
derivative of Instructor Facilitation with the word “instructor” changed to
a
“classmates.” Pintrich et al., 1991.
d b
Peer Facilitation was initially proposed in this study as a subscale of TP. Barnard et al., 2008.
c
During the analysis, however, it was reassigned as a subscale of SP. Moore, 1990.
Table A.2 Table A.3
WebTALK items for learning presence. WebTALK items for social presence.

Learning presence items Item ID Subscale Item source Social presence items Item Subscale Item source
ID
I will be able to use what I learn in this WLP12 Motivation Adapted
course in my career from MSLQa I felt a sense of belonging in this WSP32 Group Belongingb Adapted
Understanding the subject matter of WLP13 Motivation Adapted course from COIa
this course is important to me from MSLQ Interacting with other participants WSP33 Group Belonging Adapted
I don’t compromise the quality of my WLP14 Motivation Adapted gave me a sense community from COI
work because it is for an online from OSLQb I was able to form distinct WSP34 Group Belonging Adapted
course impressions of some course from COI
I am confident I can do an excellent job WLP15 Motivation MSLQ participants’ personalities
on the assignments and tests in this My contributions to course WSP35 Open Original
course discussions were acknowledged Communicationc Item
I can master the skills being taught in WLP16 Motivation Adapted by other course participants
this class from MSLQ I felt comfortable participating in WSP36 Open COI
I change the way I study in order to fit WLP17 Behaviors Adapted the course discussions Communication
the course requirements or my from MSLQ I felt comfortable interacting with WSP37 Open Adapted
instructor’s teaching style some of the other course Communication from COI
I study in a place where I can WLP18 Behaviors Adapted participants outside of assigned
concentrate on my course work from MSLQ group discussions
WLP19 Behaviors WSP38
(continued on next column) (continued on next page)

13
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Table A.3 (continued ) Table B.1 (continued )


Social presence items Item Subscale Item source Category Parameters or Criteria
ID statistics
Modification Covariance MIs and • Proportionately large compared the
I felt comfortable expressing Open Adapted
indices (MIs) Regression Weight model chi-square (e.g., can indicate
opinions that differed from those Communication from COI
MIs content overlap, measurement error,
expressed by other course
cross-loading or other
participants
misspecifications)
I felt a sense of trust that my WSP39 Open Adapted
Model fit Chi -square • best if non-significant (p > .05)
opinions were respected by other Communication from COI
• CMIN/DF ratio less than 5 generally
course participants
indicates acceptable model fit, even
a if chi-square is significant
Arbaugh et al., 2008.
b
Model fit GFI, IFI, TLI and CFI • 1.0 indicates perfect fit
The Group Belonging subscale is relabeled from the original Affective • 0.95–0.99 very good fit
Expression. • 0.9 generally accepted minimum for
c
Open Communication contains items from the original Group Cohesion good fit
subscale. Model Fit RMSEA • 0 indicates perfect fit
• > 0.05 very good fit
Table A.4 • 0.08–0.10 generally accepted
WebTALK items for cognitive presence. maximum for good fit

Cognitive presence items Item Subscale Item source


ID

Course assignments have increased my WCP1 Triggering Adapted


interest in course topics Event from COIa
Course discussions and interactions have WCP2 Triggering Adapted
made me want to explore a topic Event from COI Acknowledgements
further
I have utilized a variety of information WCP3 Exploring COI The author would like to humbly thank Dr. Şenay Purzer, Dr. Ruth
sources to explore problems posed in Streveler, Dr. Matthew Ohland, and Dr. Jennifer Richardson (Purdue
this course
Brainstorming or finding relevant WCP4 Exploring Adapted
University) for their guidance and support in the development of this
information have helped me resolve from COI project. The author would also like to acknowledge the contributions of
questions related to the content of this her partners at Purdue University, Norwich University, Baker College,
course Boise State University, and University of North Dakota who assisted with
Combining new information with my WCP5 Integrating Adapted
data collection and expert review for this project.
prior knowledge has helped me answer from COI
questions raised in this course
Reflection on course content or course WCP6 Integrating Adapted References
discussions have helped me COI
understand fundamental concepts in Anderson, T. (2014, December 16). New report on emotional presence in online education.
this class Virtual Canuck. https://virtualcanuck.ca/2014/12/16/new-report-on-emotional-pre
I can describe ways to test or apply the WCP7 Resolution Adapted sence-in-online-education/.
knowledge created in this course from COI Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C.,
I have developed solutions to course WCP8 Resolution Adapted & Swan, K. P. (2008). Developing a community of inquiry instrument: Testing a
problems that can be applied in future from COI measure of the community of inquiry framework using a multi-institutional sample.
Internet and Higher Education, 11(3–4), 133–136. https://doi.org/10.1016/j.
practice
iheduc.2008.06.003
a Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change.
Arbaugh et al., 2008. Psychological Review, 84(2), 191–215. https://doi.org/10.1037/0033-295x.84.2.191
Table B.1 Bandura, A. (1989). Human agency in social cognitive theory. The American Psychologist,
44(9), 1175–1184. https://doi.org/10.1037/0003-066x.44.9.1175
Evaluation criteria for SEM models. Bandura, A. (2001). Social cognitive theory: An agentic perspective. Annual Review of
Psychology, 52(1), 1–26. https://doi.org/10.1146/annurev.psych.52.1.1
Category Parameters or statistics Criteria Barnard, L., Lan, W. Y., To, Y. M, Paton, V. O., & Lai, S. L. (2009). Measuring self-
regulation in online and blended learning environments. Internet and Higher
Model • Just identified or over identified Education, 12(1), 1–6. https://doi.org/10.1016/j.iheduc.2008.10.005
identification Barnard, L., Paton, V., & Lan, W. (2008). Online self-regulatory learning behaviors as a
Parameter Standardized • Positive mediator in the relationship between online course perceptions with achievement.
estimates regression weights • Significant (p < .05) The International Review of Research in Open and Distributed Learning. https://doi.org/
• Sufficient size (e.g., > 0.3–0.4) 10.19173/irrodl.v9i2.516
Parameter Correlations • Less than 1 (e.g., large Barrett, L. F. (2017). How emotions are made: The secret life of the brain. Houghton Mifflin
Harcourt.
estimates correlations >1 are an indication
Bortoletto, D., & Boruchovitch, E. (2013). Learning strategies and emotional regulation
of multicolinearity)
of pedagogy students. Paidéia (Ribeirão Preto), 23(55), 235–242. https://doi.org/
• Significant (p < .05) 10.1590/1982-43272355201311
• Sufficient size (e.g., > 0.5 for Brown, T. A. (2015). Confirmatory Factor Analysis for Applied Research. Second Edition:
correlations between latent Guilford Publications.
variable measuring the same Byrne, B. M. (2010). Sturctural equation modeling with AMOS: Basic concepts, applications,
construct) and programming (2nd ed.). Routledge Taylor and Francis.
Parameter Covariance and • Positive definite Caskurlu, S. (2018). Confirming the subdimensions of teaching, social, and cognitive
estimates correlation matrices presences: A construct validity study. Internet and Higher Education, 39. https://doi.
Multivariate Mardia’s normalized • > 5 indicates sample is org/10.1016/j.iheduc.2018.05.002
normality estimate of multivariate non-normal (use Cavanagh, S. R. (2016). The spark of learning: Energizing the college classroom with the
multivariate kurtosis bootstrap procedure) science of emotion. West Virginia University Press.
Bootstrap p-values • p-values for parameter estimates Cleveland-Innes, M., & Campbell, P. (2012). Emotional presence, learning, and the online
learning environment. International Review of Research in Open and Distance Learning,
95% should be significant (p < .05)
13(4). https://search.proquest.com/docview/1634473258/abstract/BF9F154C096
confidence
7465BPQ/1.
intervals
Credé, M., & Phillips, L. A. (2011). A meta-analytic review of the motivated strategies for
learning questionnaire. Learning and Individual Differences, 21(4), 337–346. https://
(continued on next column)
doi.org/10.1016/j.lindif.2011.03.002

14
R.E.H. Wertz The Internet and Higher Education 52 (2022) 100832

Dempsey, P. R., & Zhang, J. (2019). Re-examining the construct validity and causal Pessoa, L. (2013). The cognitive-emotional brain: From interactions to integration. MIT Press.
relationships of teaching, cognitive, and social presence in community of inquiry http://ebookcentral.proquest.com/lib/valpo-ebooks/detail.action?docID=3339689.
framework. Online Learning, 23(1), 62–79. https://eric.ed.gov/?id=EJ1210951. Pessoa, L. (2021, February 20). The brain network [blog]. Emotion & Cognition and the
Eyler, J. R. (2018). How humans learn: The science and stories behind effective college Brain, by Luiz Pessoa https://cognitionemotion.wordpress.com/.
teaching. West Virginia University Press. http://ebookcentral.proquest.com/lib/val Pintrich, P. R., Smith, D. A. F., Duncan, T., & Mckeachie, W. J. (1991). A manual for the
po-ebooks/detail.action?docID=5597811. use of the motivated strategies for learning questionnaire (MSLQ) (pp. 1–80). doi:
Garnefski, N., Kraaij, V., & Spinhoven, P. (2001). Negative life events, cognitive emotion ED338122.
regulation and emotional problems. Personality and Individual Differences, 30(8), Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and
1311–1327. https://doi.org/10.1016/S0191-8869(00)00113-6 predictive validity of the motivated strategies for learning questionnaire (MSLQ).
Garrison, D. R. (1987). Self-directed and distance learning: Facilitating self-directed Educational and Psychological Measurement, 53, 801–813. doi:ED338122.
learning beyond the institutional setting. International Journal of Lifelong Education, 6 Rienties, B., & Rivers, B. A. (2014). Measuring and understanding learner emotions:
(4), 309–318. https://doi.org/10.1080/0260137870060404 Evidence and prospects. Learning Analytics Review, 1(1), 1–27. https://lace.apps.sla
Garrison, D. R., & Akyol, Z. (2013). Toward the development of a metacognition te.uib.no/publications/learning-analytics-and-emotions.pdf.
construct for communities of inquiry. The Internet and Higher Education, 17, 84–89. Schunk, D. H. (2015). Progress self-monitoring: Effects on children’s self-efficacy and
https://doi.org/10.1016/j.iheduc.2012.11.005 achievement. The Journal of Experimental Education, 51(2), 89–93. https://doi.org/
Garrison, D. R., Anderson, T., & Archer, W. (1999). Critical inquiry in a text-based 10.1080/00220973.1982.11011845
environment: Computer conferencing in higher education. The Internet and Higher Shea, P., & Bidjerano, T. (2010). Learning presence: Towards a theory of self-efficacy,
Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6 self-regulation, and the development of a communities of inquiry in online and
Hadwin, A., & Oshige, M. (2011). Self-regulation, coregualtion, and socially shared blended learning environments. Computers in Education, 59(2), 316–326. https://doi.
regulation: Exploring perspectives of social in self-regulated learning theory. org/10.1016/j.compedu.2010.07.017
Teachers College Record, 113(2), 204–264. https://doi.org/10.4324/ Shea, P., & Bidjerano, T. (2012). Learning presence as a moderator in the community of
9780203839010.ch5 inquiry model. Computers in Education, 59(2), 316–326. https://doi.org/10.1016/j.
Hayes, S., Smith, S. U., & Shea, P. (2015). Expanding learning presence to account for the compedu.2012.01.011
direction of regulative intent: Self-, co- and shared regulation in online learning. Shea, P., Hayes, S., Smith, S. U., Vickers, J., Bidjerano, T., Pickett, A., … Jian, S. (2012).
Online Learning, 19(3), 15–31. https://eric.ed.gov/?id=EJ1067475. Learning presence: Additional research on a new conceptual element within the
Heilporn, G., & Lakhal, S. (2020). Investigating the reliability and validity of the community of inquiry (CoI) framework. Internet and Higher Education, 15(2), 89–95.
community of inquiry framework: An analysis of categories within each presence. https://doi.org/10.1016/j.iheduc.2011.08.002
Computers & Education, 145, 103712. https://doi.org/10.1016/j. Shea, P., Hayes, S., Uzuner-Smith, S., Gozza-Cohen, M., Vickers, J., & Bidjerano, T.
compedu.2019.103712 (2014). Reconceptualizing the community of inquiry framework: An exploratory
Huang, K., Law, V., & Lee, S. J. (2019). The role of learners’ epistemic beliefs in an online analysis. The Internet and Higher Education, 23, 9–17. https://doi.org/10.1016/j.
Community of Inquiry. British Journal of Educational Technology, 50(4), 1882–1895. iheduc.2014.05.002
https://doi.org/10.1111/bjet.12684 Sitzmann, T., & Ely, K. (2011). A meta-analysis of self-regulated learning in work-related
Kozan, K., & Caskurlu, S. (2018). On the nth presence for the community of inquiry training and educational attainment: What we know and where we need to go.
framework. Computers & Education, 122, 104–118. https://doi.org/10.1016/j. Psychological Bulletin, 137(3), 412–442. https://doi.org/10.1037/a0022777
compedu.2018.03.010 Stenbom, S. (2018). A systematic review of the Community of Inquiry survey. The Internet
Kozan, K., & Richardson, J. C. (2014). New exploratory and confirmatory factor analysis and Higher Education, 39, 22–32. https://doi.org/10.1016/j.iheduc.2018.06.001
insights into the community of inquiry survey. The Internet and Higher Education, 23, Svinicki, M. D. (2008). A guidebook on conceptual frameworks for research in engineering
39–47. https://doi.org/10.1016/j.iheduc.2014.06.002 education: Technical report for rigorous research in engineering education project.
Ma, Z., Wang, J., Wang, Q., Kong, L., Wu, Y., & Yang, H. (2017). Verifying causal American Society of Engineering Education, American Educational Research
relationships among the presences of the community of inquiry framework in the Association, Professional and Organizational Development network and National
Chinese context. International Review of Research in Open and Distributed Learning: Science Foundation.
IRRODL, 18(6), 213–230. https://doi.org/10.19173/irrodl.v18i6.3197 Swan, K., Shea, P., Richardson, J., Ice, P., Garrison, D., Cleveland-Innes, M., &
Moore, W. S. (1990). The learning environment preferences: An instrument manual. Center Arbaugh, J. (2008). Validating a measurement tool of presence in online
for the Study of Intellectual Development. communities of inquiry. E-Mentor, 2(24), 1–12.
Nadeau, J. (2012). Expérimentation de la rétroaction audiovisuelle asynchrone dans un Traver, A. E., Volchok, E., Bidjerano, T., & Shea, P. (2014). Correlating community
cours à distance d’informatique dans la perspective de la théorie de la communauté college students’ perceptions of community of inquiry presences with their
d’apprentisssage [Memoire de maîtrise, TELUQ] http://biblio.teluq.ca/LinkClick. completion of blended courses. The Internet and Higher Education, 20, 1–9. https://
aspx?fileticket=ISkXNYulN2xtgpPTmjiMAw%3d%3d&tabid=39743&langua doi.org/10.1016/j.iheduc.2013.09.001
ge=fr-CA. Wertz, R. E. H. (2014). What is learning presence and what can it tell us about success in
Noddings, N. (2007). Philosophy of education (2nd ed.). Westview Press. learning online? Frontiers in Education Conference (FIE), 1–6. https://doi.org/10.1109/
Ochsner, K. N., & Gross, J. J. (2008). Cognitive emotion regulation: Insights from social FIE.2014.7044246.
cognitive and affective neuroscience. Current Directions in Psychological Science, 17 Yu, T., & Richardson, J. C. (2015). Examining reliability and validity of a Korean version
(2), 153–158. https://doi.org/10.1111/j.1467-8721.2008.00566.x of the Community of Inquiry instrument using exploratory and confirmatory factor
Olpak, Y. Z., & Kiliç Çakmak, E. (2018). Examining the Reliability and Validity of a analysis. The Internet and Higher Education, 25, 45–52. https://doi.org/10.1016/j.
Turkish Version of the Community of Inquiry Survey. Online Learning, 22(1), iheduc.2014.12.004
147–161. Zimmerman, B. J. (2002). Becoming a self regulated learner: An overview. Theory Into
Patton, M. Q. (2002). Qualitative research and evaluation methods. In Qualitative inquiry Practice, 41(2), 64–70. https://doi.org/10.1207/s15430421tip4102
(3rd ed.). Sage. https://doi.org/10.2307/330063. Zimmerman, B. J. (2008). Investigating self-regulation and motivation: Historical
Perry, W. G. (1970/1999). Forms of intellectual and ethical development in the college years: background, methodological developments, and future prospects. American
A scheme. Jossey-Bass Higher and Adult Education Series. Jossey-Bass Publishers. Educational Research Journal, 45(1), 166–183. https://doi.org/10.3102/
Pessoa, L. (2008). On the relationship between emotion and cognition. Nature Reviews 0002831207312909
Neuroscience, 9(2), 148–158. https://doi.org/10.1038/nrn2317 Zimmerman, B. J., & Schunk, D. H. (2011). Handbook of self-regulation of learning and
Pessoa, L. (2009). Cognition and emotion. Scholarpedia, 4(1), 4567. https://doi.org/ performance (pp. xiv, 484). Routledge/Taylor & Francis Group.
10.4249/scholarpedia.4567

15

You might also like

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


Alternative Proxies:

Alternative Proxy

pFad Proxy

pFad v3 Proxy

pFad v4 Proxy