545073

research-article2014
NMS0010.1177/1461444814545073New Media & SocietyKsiazek et al.

Article

new media & society

User engagement with

2016, Vol. 18(3) 502­–520
© The Author(s) 2014
Reprints and permissions:
online news: Conceptualizing sagepub.co.uk/journalsPermissions.nav
DOI: 10.1177/1461444814545073
interactivity and exploring nms.sagepub.com

the relationship between

online news videos and user
comments

Thomas B Ksiazek
Villanova University, USA

Limor Peer
Yale University, USA

Kevin Lessard
Villanova University, USA

Abstract
With the emergence and rapid acceptance of online news come new and varied
opportunities for user engagement with content, along with alternative metrics
for capturing those behaviors. This study focuses on interactive engagement with
online news videos. We propose a theoretical framework for conceptualizing user
engagement on a continuum from exposure to interactivity. Furthermore, we make
a distinction between user–content (e.g. commenting) and user–user (e.g. replying to
another user’s comment) modes of interaction. We then explore publicly available
measures of these concepts and test a series of hypotheses to explore commenting
and conversational behaviors in response to YouTube news videos. We conclude by
discussing the theoretical and practical implications for advancing our understanding of
user engagement with online news.

Corresponding author:
Thomas B Ksiazek, Department of Communication, Villanova University, Garey Hall B2a, 800 Lancaster
Avenue, Villanova, PA 18940, USA.
Email: thomas.ksiazek@villanova.edu
Ksiazek et al. 503

Keywords
Digital journalism, engagement, interactivity, news videos, online news, user
comments, YouTube

The news media landscape has undergone drastic changes over the past decade, as
digital platforms are creating unprecedented competition for all legacy, or traditional,
media (i.e. newspapers and magazines, as well as broadcast TV and radio news). All of
this is part of a broader trend toward convergence in media industries, defined by fea-
tures such as cross-platform availability of content, less organizational control over
production and distribution, greater user empowerment, lower barriers to entry, and
increased participation and interactivity among consumers and producers of content.
In terms of sustainability and audience building, some (Hayes, 2008; Webster,
2011) describe this industry transition as shifting from a “push” to “pull” model of
mediated communication, especially for video content. The suggestion is that produc-
ers in the past relied on a linear model and would simply push content through a lim-
ited number of distribution channels with the expectation that users would pay
attention. Now, producers operate in a highly competitive and nonlinear environment,
characterized by features such as time-shifting technologies and “on demand” access
that enable greater user control over when, where, and how they access media. In this
context, news organizations must do more to pull users in, offering something not only
more useful, informative, compelling, and gratifying than their competition, but also
more engaging.
Interactivity is one of the unique defining features of engagement with online news
(Deuze, 2003). MacGregor (2007) argues, “To engage is a new value, but not really a
news value. This focus on the quality of interaction heralds change” (p. 293). As such,
indicators of interactive engagement are complementing more traditional measures of
exposure (circulation, ratings, etc.) in understanding the behavior of news audiences.
The notion of engaging users by encouraging them to interact with both content and
other users has had substantial traction in marketing and advertising circles for some
time, not least because these activities can now be captured and examined via social
media tools (Napoli, 2011; Peterson and Carrabis, 2008). With the challenges facing
many news organizations, the ability of journalists to offer engaging content to attract, or
“pull,” users would seem to be more important than ever.
This study aims to better understand indicators of engagement with online news, spe-
cifically the commenting and conversational practices of users in response to online
news videos. In what follows, we clarify the concept of engagement and propose a
framework for conceptualizing interactivity and its centrality to the understanding of
engagement with online news. Next, we explore various indicators of user behavior to
illustrate the usefulness of interactivity as an organizing principle. This involves disen-
tangling popularity and exposure metrics from measures of interactivity, each of which
constitutes a differing level of engagement. We draw on publicly available online behav-
ioral data to explore the relationships among these metrics, and we integrate these data
with a content analysis of news videos to analyze the relationships between content
504 new media & society 18(3)

features and interactivity. We contrast commenting practices between “hard” and “soft”
news, “objective” and “biased” news, and professional and citizen journalism. The cen-
tral goal of this study is to further our understanding of the diverse ways that users expe-
rience and interact with the news.

User engagement and interactivity

Engagement is a broad phenomenon that describes all sorts of user attention and involve-
ment with media (see Napoli, 2011: 97–98). At its most basic level, it begins with expo-
sure, but most perceive engagement as constituted of both psychological and behavioral
experiences (Brodie et al., 2011; Calder et al., 2009; Malthouse and Calder, 2011; Napoli,
2011; Oh et al., 2010), and it can be a property of the users, the medium, or both (O’Brien
and Toms, 2008).
A common thread in the research on engagement is the involvement, real or perceived,
of the user in either producing, consuming, or disseminating information (Deuze, 2003;
Hargittai and Walejko, 2008; Jenkins, 2006; Napoli, 2011; Oh et al., 2010). This involve-
ment is often expressed as interactivity, which is understood as a fundamental component
of the broader phenomenon of engagement. Research on interactivity often highlights the
multi-directional flow of information. The interactive nature of online media enables the
audience to not only receive information but also disseminate and remake it (Jenkins,
2006; Thurman, 2008). Consistent with this line of thinking, Schultz (1999) defines inter-
activity as a “chain of interrelated messages” in a two-way or reactive system of commu-
nication that allows for input from individuals on all sides of the message.
Recent research has applied a uses and gratifications perspective to understand the
drivers of interactivity. Ruggiero (2000) notes the potential for interactive media to fulfill
certain psychological needs. Subsequent empirical research finds three primary motiva-
tions for interactivity: information seeking, socialization or social interaction, and enter-
tainment. This work further distinguishes motives for user–content and user–user
interactivity (discussed below), where the former is driven by the need for information
and the latter is motivated by the need for social interaction (Chung and Yoo, 2008; Ko
et al., 2005; Yoo, 2011). Additionally, a need for entertainment appears to drive both
forms of interactivity (Chung and Yoo, 2008; Yoo, 2011). This theme is also central to
the work of Clay Shirky (2010), who argues that the Internet enables new forms of col-
laboration as it affords people an easy entrée into cultural production, as opposed to mere
consumption.
Regarding online journalism specifically, Schmitz Weiss and De Macedo Higgins
Joyce (2009) suggest the Internet offers unique opportunities for interactivity because it
“allows for a closer relationship with the audience; a shortened social space” between
media producer and consumer (p. 593). Related, immediacy is a primary characteristic of
interactivity in online news media, one that allows users to publicly register their response
to a news article or video (Lange, 2007). Commenting on news stories is a way users can
engage with content that they did not have available to them until recently. We argue that
these comments, as well as replies from other users to already posted comments, can be
seen as representative of high-level interactive engagement between the user and content
and between the users themselves, respectively.
Ksiazek et al. 505

We distinguish interactivity with online news videos in two broad ways, building on
theoretical work with interactivity in a variety of online contexts (Chung, 2008; Chung
and Yoo, 2008; Kiousis, 2002; McMillan, 2002, 2005; Massey and Levy, 1999; Nagar,
2011; Schultz, 2000; Stromer-Galley, 2000; Yoo, 2011). User–content interactivity
involves a user interacting with content and producers, such as posting an initial com-
ment to a video thread. This represents a basic form of feedback for the content creator.
Alternatively, user–user interactivity consists of interactions among two or more users,
such as a user replying to another comment already posted on the video thread by a dif-
ferent user. This back and forth among viewers of the news video is representative of a
dialogue, or conversation, between the commenters.
Both forms of interactivity signal a highly engaged user, where the individual is not
just exposed to content, but demonstrates evidence of cognitive processing. The practice
of commenting highlights an active user that is challenging, supporting, or at the very
least reflecting on the news. While it is one thing to simply read or watch a news story,
making the decision to publicly contribute your reaction or opinion in response to the
story indicates an individual that is more invested, aware, and attentive—in other words,
more engaged—with the content.
To sum up, we view the ability to act, interact, and co-create online, as a key charac-
teristic of online media, which distinguishes it from other media platforms. We theorize
a continuum of engagement, from exposure to interactivity, where more (quantity) and
better (quality) ways to interact with content and with other users indicate deeper engage-
ment. We now turn to a discussion of how these concepts are measured.

Measuring media use: From exposure to engagement

Despite the growing body of work on engagement and interactivity with the news, the vast
majority of research on news consumption has traditionally relied on rather simplistic
measures of raw exposure, with user behavior captured by gross metrics like ratings, circu-
lation, and, more recently, page views and the like. Extending this line of inquiry, we argue
that interactivity is distinct from popularity (i.e. exposure), though it is related. We agree
with Beckett (2010) that engagement is a composite of various metrics of exposure and
interactive behaviors, but make the case for thinking about engagement as a continuum that
ranges from less (popularity or exposure) to more (interactivity) user engagement.
Research on audience behavior has long been interested in examining general expo-
sure to media, both from a practical standpoint for its producers and also for the theoreti-
cal implications it brings forth. The delivery of news online contributes to a more refined
measurement of usage for two reasons: (a) it allows measurement at the level of an
individual story rather than the entire news product (e.g. newspaper, television news
broadcast), and (b) it provides more precise ways to measure whether a specific piece of
content was consumed. These new measures of usage, which aim to capture popularity
in some form, typically consist of raw measures of exposure (e.g. page views, traffic,
hits, etc.). For instance, Chatzopoulou et al. (2010) treat the number of views of a
YouTube video as a basic index of popularity.
While these measures of exposure are certainly important, they are limited in what
they tell us beyond whether or not a user encountered the content. Without some measure
of engagement, we cannot assess how users respond to what is in front of them. In the
506 new media & society 18(3)

past, researchers relied on surveys, self-reports, or letters to the editor, but the Internet
has dramatically broadened the range of behaviors vis-a-vis content: emailing, sharing,
“liking,” recommending, social bookmarking, and so on. Indeed, we are increasingly
seeing the integration of traditional exposure metrics and new indicators of engagement
to provide a more complete picture of user behavior (Napoli, 2011).
We distinguish here between “viewing” and other common popularity indicators,
such as “rating,” “ranking,” “favoriting,” and “liking.” These behaviors suggest a differ-
ent level of user experience, one that involves some action on the part of the user (Deuze,
2003). While viewing is a classic measure of exposure, the other metrics essentially
operationalize the concept of recommendation. Moreover, while all forms of online use
require some activity, we argue that viewing requires minimal interactivity and does not
necessarily signal engagement. Instead, exposure is but one aspect of engagement.
Likewise, recommendation metrics capture very basic user activity (i.e. clicking), but it
is questionable whether they represent deeper engagement.
Interactive features available online, such as direct feedback, comment sections, and
interactions with other users, allow users to take a much more active (and interactive)
role in the user experience, and thus they clearly suggest deeper engagement with the
content. Like page views, these features are publicly available and can be tracked, mak-
ing it possible to more accurately measure these types of social behaviors.
Recent scholarly work is beginning to take advantage of publicly available user com-
ments and highlights a growing interest in this phenomenon. Nagar (2011) analyzed user
comments on mainstream news sites as an expression of public opinion and forum for the
surveillance and exchange of ideas. Larsson (2011) studied commenting as one of sev-
eral interactive features available on Swedish newspaper websites, finding the option
varies across types of users in terms of appreciation and use. For instance, “Prosumers”
were active commenters, while “Lurkers” were more likely to read others’ comments
without partaking. Hujanen and Pietikäinen (2004) studied commenting as a new mode
of participation, although they found that Finnish youth rarely engaged in this practice
when visiting news websites. This work exemplifies a new trajectory in research on news
consumption that focuses on new ways of understanding user behavior.
As digital platforms afford us greater ability to track behavior in more subtle ways, we
can disentangle popularity and interactivity, examine the relationship between them, and
better understand these alternative measures of engagement. We proceed to analyze sev-
eral different indicators of popularity (from viewing to recommendation metrics), along
with the two modes of interactivity, user–content and user–user. The analysis will shed
light on the relationships among interactivity and popularity metrics, as well as the role
that certain content features may play in helping to explain these measures of user behav-
ior. In the next section, we outline three sets of hypotheses. The first deals with the rela-
tionship between popularity and interactivity, while the last two make inferences about
the role of content in explaining interactivity.

Research hypotheses
The first goal of our study is to examine the two types of interactive engagement (user–
content and user–user) in relation to broad patterns of popularity. As both types are
increasingly integrated in the study of user behavior, we propose two hypotheses to
Ksiazek et al. 507

better understand the relationship between popularity and interactivity. The predicted
relationships are based on a common notion that more popular content is also more
engaging. Siersdorfer et al. (2010) focused on the comment sections of YouTube videos,
but with an emphasis on popularity (i.e. “likes”) of comments, ultimately finding that
positive comments are associated with greater popularity. Studying the behavioral char-
acteristics of users on the social networking site Digg, Jamali and Rangwala (2009) also
link interactivity with popularity ratings; they found that the time of the comment (i.e.
time passed since original post) as well as the actual word length of a post or comment
was related to high levels of popularity. Lee et al. (2010) proposed a model to predict the
popularity of online content derived from the lifetime of threads as well as the number of
comments on such threads, where longer lifetime/more comments leads to greater popu-
larity. Taken together, these studies offer empirical justification consistent with the theo-
retical link between popularity and interactive engagement. Therefore, we propose the
following hypotheses:

H1a: There is a positive relationship between popularity and comments (user–content

interaction).
H1b: There is a positive relationship between popularity and conversation (user–user
interaction).

A secondary goal of this study is to explore the relationship between interactivity and
a variety of content characteristics. In the following sections, we review two primary
distinctions made in the research on journalism content, hard/soft news and objective/
biased news, and propose additional hypotheses to explore the relationship between
interactive engagement and these content features.

News: hard versus soft

In recent studies of news content, the provision and selection of hard versus soft news is
an area that is garnering a great deal of research attention (Baum, 2002, 2003, 2004;
Boczkowski, 2009; Boczkowski and Peer, 2011; Gans, 1980; Hamilton, 2004; Patterson,
2003; Prior, 2003; Schaudt and Carpenter, 2009; Tewksbury, 2003; Zaller, 2003). For
instance, Boczkowski and Peer (2011) found a “choice gap,” where journalists provide
more hard news than users seem to desire. Despite this gap, other research suggests that
news organizations are still trending toward offering more soft news, overall. Hamilton
(2004) argues that profit motivations lead to more soft news across all types of news
organizations, where the quest for marginal viewers that prefer soft news provides an
economic explanation for the rise in soft news over recent years. As we witness the “sof-
tening of news” (Boczkowski and Peer, 2011), many wonder about the social and politi-
cal implications of less public affairs content. For instance, will this lead to a less
politically involved citizenry? What about the impact on informed voters? How does this
contribute to or detract from the health of a deliberative democracy?
To date, most research on hard/soft news has explored content (subject and format),
production, antecedents, and outcomes. Comparing the frequency of comments and con-
versation posted to hard and soft news videos would offer an additional contribution to
508 new media & society 18(3)

this body of research, specifically as it relates to the generative deliberation prospects of

each content type. Based on past evidence of a preference for soft news among audience
members, and assuming a positive relationship between popularity and interactivity, we
should see a positive relationship between more popular content (i.e. soft news) and both
types of interactive engagement. We propose the following hypotheses:

H2a: Soft news videos will exhibit more comments (user–content interaction) than
hard news videos.
H2b: Soft news videos will exhibit more conversation (user–user interaction) than
hard news videos.

Views: Bias versus objectivity

In addition to exploring hard and soft news content, we also compare interactivity across
several indicators of bias and objectivity in news videos. The “objectivity norm” has
been a feature of modern journalism since the late 19th and early 20th century (Schudson,
2001). Recently, however, many scholars wonder whether market forces and economic
imperatives resulting from an environment of increased competition, niche content, and
declining profits will lead to a shift away from objective news. A recent study of online
news videos showed that biased content was more popular than objective content (Peer
and Ksiazek, 2011). Similar to the previous hypotheses, this suggests that biased news
videos, because they are more popular, should also exhibit a positive relationship with
interactive engagement. Thus, we offer the following hypotheses:

H3a: Videos that break from the objectivity norm will exhibit more comments (user–
content interaction).
H3b: Videos that break from the objectivity norm will exhibit more conversation
(user–user interaction).

Method
In order to test these hypotheses, we collected a sample of YouTube news videos over a
3-month period. The resulting dataset includes measures of popularity and commenting
behaviors, as well as hand-coded content characteristics for a subsample of the videos.
The following sections describe the sampling and data collection, details of the specific
measures used in this study, and the analytical procedures used to test the hypotheses.

Sampling and data collection

The dynamic nature of the Web makes it extremely difficult to select a purely random sam-
ple of news videos. To cope with this, we first chose a single site to study, as even a sample
of sites would not be random. YouTube seemed a natural choice, as it has long offered vari-
ous measures of user behavior (Burgess and Green, 2009; Cheng et al., 2008; Jarboe, 2011).
It is the most popular online video property in the United States, with over 136 million
unique viewers—a figure that is about three times that of the next closest competitor—and
Ksiazek et al. 509

about 16.5 billion total streams during May 2012 (NielsenWire, 2012, 22 June). It is also an
aggregator of video content, thus offering a site that includes videos from a variety of news
outlets throughout the world, both professional and amateur (Pew Research Center, 2012).
Next, we sampled a composite week, a form of cluster sampling that is common in
content analysis of television programs. We set our parameters for March–June 2008 and
selected a random Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, and Saturday
during that time frame. We crawled the pages of the top 100 videos in YouTube’s “News”
category, ranked by number of views, for each of the 7 days. These videos include both
repurposed news clips (e.g. a 60-Minutes segment uploaded to YouTube) along with
videos made exclusively for the web, which are often produced by amateur or citizen
journalists. We collected publicly available popularity and interactivity metrics for each
video. This provided our initial dataset of 700 news videos to test H1a and H1b.
We also collected a smaller subsample of videos that consisted of the top 40 news
videos for 5 days in the March–June 2008 time frame.1 These 191 videos were hand-
coded for a variety of content and production characteristics, and crawled for popularity
and interactivity metrics.2 These data were used to test H2a, H2b, H3a, and H3b.

Measures
In order to test the first two hypotheses about the relationships between popularity and
interactivity, we used five variables measured for all 700 videos. To measure the popular-
ity of each video, we used three indicators that are publicly available on YouTube:
Number of Views is a measure of raw exposure to a video (i.e. the number of times a
video has been viewed), Number of Favorited captures the number of times a video is
marked as a favorite, and Number of Ratings indicates the number of times a video
received a rating. To gauge the level of interactivity for each video, we used two varia-
bles. Number of Comments represents the total number of comments posted directly to a
video (i.e. by entering text in the “Respond to this video” text box) and indicates the
degree of user–content interactivity for a given video. To measure user–user interactiv-
ity, we captured the number of replies within the set of comments for each video. A reply
to an existing comment is registered when a user chooses to click the “Reply” button
associated with a particular comment rather than post a comment as described above.
Instead of using the absolute number of replies, we created a Ratio of Replies variable,
which represents the proportion of replies to total comments for a given video, and thus
offers a more conservative measure than the raw number of replies.3 Table 1 provides
descriptive statistics for all five variables. While the values presented in the table are real
values, all variables were log transformed to address skewed distributions before con-
ducting the analyses.
To test the remaining hypotheses, we used the same interactive engagement metrics
above, first comparing them across hard and soft news videos, then between biased and
objective videos. Four journalism graduate students coded the content of the subsample
of videos. The students received extensive training and were closely supervised. They
followed a strict protocol and achieved an overall percent agreement score of 82% and a
Kn score of .72 (see Peer and Ksiazek, 2011).4
Videos were classified as hard or soft news by recoding an existing “content topic” vari-
able in our dataset. Hard news consisted of the following topics: politics, US elections,
510 new media & society 18(3)

Table 1.  Descriptive statistics.

Min Max Mean SD

Popularity ←→ Comments
  N = 700
Number of Comments .00 3375.00 78.013 213.61
(User–content interactivity)
Ratio of Replies .00 .84 .19 .18
(User–user interactivity)
Number of Views 201.00 772,637.00 12,083.47 37,832.76
Number of Favorited .00 2209.00 31.32 142.70
Number of Ratings .00 3035.00 72.44 174.94
  Content ←→ Comments
  N = 191
Number of Comments .00 3375.00 145.8115 337.17332
(User–content interactivity)
Ratio of Replies .00 .64 .1879 .15327
(User–user interactivity)

SD: standard deviation.

Note: fairness, sourcing, and agenda are nominal variables and thus are not included.

economics and business, foreign affairs, War on Terror, crime, and natural disasters. Soft
news included the following topics: sports, arts, entertainment, culture, science, technol-
ogy, health, medicine, fitness, weather, sex, animals, and weird/humorous. Videos were
characterized as biased or objective based on three hand-coded variables: Fairness,
Sourcing, and Agenda. Fairness indicates whether the video presents more than one side of
the relevant issue, with multiple sides indicating objectivity. Sourcing asks whether the
video uses an outside/secondary source for information, balance, or an alternative point of
view, where the use of sources is an indicator of objectivity. Agenda asks coders to deter-
mine whether the video seems overly opinionated or seems to be promoting an agenda,
where the lack of an agenda characterizes more objective videos.5

Analysis
The unit of analysis in our study is the video, such that each popularity and interactivity
metric as well as all content variables are measured for each individual video. To test
H1a and H1b, we analyzed the correlations among the popularity and interactivity meas-
ures. To test the other hypotheses, we used t-tests to compare the mean scores for both
interactivity variables across different types of content. For H2a/H2b, we compared the
mean number of comments and ratio of replies between hard and soft news videos. For
H3a/H3b, we did the same between objective and biased videos, using fairness, sourcing,
and agenda as our indicators of the relative objectivity of a video.

Results
The first two hypotheses predict a positive relationship between popularity and com-
menting (H1a), as well as conversation (H1b). Hypothesis 1a is supported (see Table 2).
Ksiazek et al. 511

Table 2.  Pearson correlation matrix for popularity, user–content interactivity, and user–user
interactivity variables (n = 700 videos).

Number of Ratio of Number of Number of Number of

Comments Replies Views Favorited Ratings
Number of Comments – −.500* .470* .511* .594*
Ratio of Replies – −.181* −.215* −.207*
Number of Views – .522* .527*
Number of Favorited – .663*
Number of Ratings –

*p < .001.

There are moderately strong correlations between the number of comments posted to a
video and all three popularity variables (Number of Views, Number of Favorited, and
Number of Ratings). Commenting is positively associated with more exposure to a video,
greater likelihood of marking a video as a favorite, and choosing to rate a video. Overall,
we can say there is a positive relationship between popularity—both in terms of exposure
and recommendation—and user–content interaction.
Hypothesis 1b predicts a positive relationship between popularity and user–user interac-
tion, or conversation within a comment thread. This hypothesis is not supported. In fact,
Table 2 shows a negative relationship between ratio of replies and all three popularity vari-
ables. This suggests that less popular videos—in terms of raw exposure (Number of Views)
and being marked favorite or rated—are more likely to generate conversation among users.
The remaining hypotheses predict that soft (H2a/H2b) and biased (H3a/H3b) news
videos will exhibit more commenting and conversation. These hypotheses were not sup-
ported. Regarding both types of interactivity, no significant differences were found
between hard and soft news or biased and objective news (see Table 3). For ease of
interpretation, Table 3 lists the objective categories in the left column (multiple sides,
sources, no agenda) and the biased ones in the right column. While soft news and biased
news have been shown to be more popular than hard and objective news, respectively,
there are no significant differences in the likelihood of commenting or conversing across
these types of news content.
A post hoc analysis of interactivity between repurposed and non-repurposed videos
suggests that non-repurposed videos, or those made exclusively for the web, garner
higher levels of user–content interactivity, t (164) = 3.297, p < .01, but reveals no signifi-
cant differences in terms of user–user interactivity. In other words, the distinction
between professionally produced content and news videos created by citizen journalists
does seem to predict interactive engagement, at least in terms of posting a comment to a
video. Users appear more likely to post comments to videos produced by non-profession-
als than to those videos that originally appeared as segments on a TV news program.

Discussion
Producers and consumers of news exist in an increasingly interconnected world, with
interactions among journalists, users, and content occurring at many levels. This study
512 new media & society 18(3)

Table 3.  T-test results comparing user–content and user–user interactivity across hard/soft,
objective/biased, and repurposed/non-repurposed videos.a.

  Hard and soft news (n = 162)

 
Hard Soft

M SD M SD t df
Number of Comments 160.737 370.419 56.286 39.885 −.002 24.538b
Ratio of Replies .207 .155 .141 .127 1.850 160

  Fairness (n = 94)
 
  Multiple sides One side

M SD M SD t df
Number of Comments 169.821 236.577 184.564 318.563 .924 92
Ratio of Replies .211 .139 .208 .133 −1.399 92

  Sourcing (n = 156)

Sources No sources

M SD M SD t df
Number of Comments 140.590 299.347 171.808 423.037 1.408 154
Ratio of Replies .194 .151 .219 .161 −.037 154

    Agenda (n = 125)

No agenda Has agenda

M SD M SD t df
Number of Comments 153.098 230.296 218.156 507.363 1.261 113.026b
Ratio of Replies 0.208 0.139 0.196 0.149 −0.657 123

  Repurposed vs non-repurposed (n = 166)

 
  Repurposed Non-repurposed

M SD M SD t df
Number of Comments 103.487 214.069 239.327 521.528 3.297* 164
Ratio of Replies 0.193 0.156 0.209 0.155 −0.084 164

SD: standard deviation; df: degrees of freedom (both italic).

aMeans are based on real values for ease of interpretations, while t statistics are calculated with log-trans-

formed values to normalize skewed distributions.

bEqual variances not assumed.

*p < .01.
Ksiazek et al. 513

set out to better understand user engagement and interactivity with online news videos.
We began by theorizing that the primary ways in which users interact with the news are
manifestations of high-level engagement. We then offered a broad picture of the relation-
ship between traditional popularity metrics, long used to gauge exposure and usage, and
more recent engagement metrics. We integrated those popularity metrics under the
umbrella of engagement and conceptualized engagement on a continuum from exposure
to interactivity. In doing so, we proposed a distinction between user–content interactions
(e.g. commenting on a news video) and user–user interactions (e.g. replying to another
user’s comment). While we found a positive relationship between popularity and posting
a comment to a video, we found a negative relationship between popularity and conver-
sation among users. In other words, viewers of online news videos are more likely to
engage in user–content interaction for popular videos, but are more likely to engage in
user–user interaction with less popular ones.
These findings lend empirical support to our conceptual framework for interactive
engagement with online news. Building on recent work that theoretically and empirically
conceptualized interactivity in a variety of new media contexts (Chung, 2008; Chung and
Yoo, 2008; Kiousis, 2002; McMillan, 2002, 2005; Massey and Levy, 1999; Nagar, 2011;
Schultz, 2000; Stromer-Galley, 2000; Yoo, 2011), this study establishes that we can dis-
tinguish between user–content and user–user interactivity with the news. The study pre-
sents evidence that these behaviors are distinct in that they manifest differently with
more or less popular content and depending on the source of the video.
We would encourage researchers to continue to explore the complexities of engage-
ment and interactivity as new contexts emerge. For instance, news websites now regu-
larly display a variety of more granular metrics alongside stories (e.g. Facebook likes,
Tweets, and the number of shares and recommendations through a variety of specific
social media platforms like Facebook, Google+, LinkedIn, Delicious, Pinterest, and
StumbleUpon). These offer opportunities to explore engagement beyond general meas-
ures like views, favorites, and ratings that stem from manifestations of preference that
are internal to a given news site. It is possible that users interact in different ways with
the news when using social sharing and recommendation options that demonstrate their
preferences to their social network, external to the site where they first encounter con-
tent. For instance, there may be differences in how and why users demonstrate anony-
mous preference on a given site (e.g. by simply viewing or rating content), as compared
to identifiable manifestations of preference that are accessible to one’s social networks
beyond that site. Similarly, higher level engagement with news may take place outside
the original news site. Many of these social platforms allow users to engage in user–user
interactivity vis-a-vis content that is shared (e.g. engaging in a discussion on Facebook
in response to a news story that was shared by a friend). It would be interesting to com-
pare user–content and user–user interactivity as they play out on news sites and on other
sites, and to further explore news sites that do it well.
We also analyzed user–content and user–user interactivity as they relate to different
types of news content. Looking specifically at two themes that have garnered a great
deal of research attention, we found no differences in the likelihood of engaging with
hard versus soft news or objective versus biased news. Significantly, while there has
been a strong interest in studying these content distinctions, and past empirical research
514 new media & society 18(3)

has demonstrated significant differences in terms of popularity, this study found that
these content distinctions have little relationship with interactive engagement at multi-
ple levels. While the news industry comes under increasing scrutiny for offering softer
and more partisan news coverage, these trends may hold less weight when it comes to
engaging users.
From both applied and normative perspectives, interactivity and engagement are posi-
tive user behaviors that should be encouraged. For news organizations seeking to “pull”
users in (Hayes, 2008; Webster, 2011), evidence of more engaged users can signal greater
potential for paid subscribers, as well as more lucrative advertising rates. In other words,
interactive engagement translates into increases in both revenue streams for news organi-
zations (selling content to users and selling eyeballs to advertisers). Engagement is also
positive in a normative sense, where higher levels of interactive engagement with the
news may signal more active citizens, as well as the application of “cognitive surplus” to
the realm of public affairs (Shirky, 2010), and perhaps the existence of a kind of public
discussion that is essential to democracy (Barber, 2004; Herbst, 1998; Kim et al., 1999;
Manosevitch and Walker, 2009).
Yet if the content of the news does not predict these behaviors (at least in terms of the
hard/soft and objective/biased distinctions), news organizations need to look elsewhere.
In our post hoc analysis, we found that non-repurposed videos exhibit more user–content
engagement. This suggests that news organizations may need to look to citizen journal-
ism for ways of creating more engaging content. For instance, a recent study found sub-
stantial differences in production and content elements when comparing professional and
citizen-produced news videos (Peer and Ksiazek, 2011).
Importantly, these findings suggest that the influence of “public measures”—indica-
tors of popularity or preference made publicly available (Espeland and Sauder, 2007;
Webster, 2010, 2011)—may be greater than that of the actual content. That is, public
measures serve as a heuristic, a signal to the user that a video is worth viewing. It is now
common to find lists of “Most viewed” or “Most recommended” stories on a news web-
site, and other equivalent lists are now commonplace across the Internet. Similarly, the
popularity indicators used in this study are publicly available to YouTube users. In other
words, a user is able to see the popularity of a video prior to viewing, commenting, or
engaging in any other behavior relevant to that video. The strong positive relationships
between popularity and commenting could indicate the influence of these public meas-
ures on the likelihood of engaging in user–content interactivity. At the same time, the
negative relationship between popularity and user–user interaction may indicate that
negative public measures inspire more conversation among users, or vice versa, which
could be due to the perception of polarized tastes. Moreover, the lack of evidence that the
content of the videos makes a difference for either type of interactivity would seem to
lend further plausibility to these interpretations about the influence of public measures.
Future research can address this question by tracking changes in interactivity and popu-
larity over time, making use of the dynamic nature of the Web to explore causality.
While public measures of popularity may influence user interactivity, it is also pos-
sible that underlying social and psychological drivers of interactivity may explain the
divergent relationships between popularity and the two forms of interactivity. Earlier, we
noted that user–content interactivity is motivated by the need for information, while
Ksiazek et al. 515

user–user interactivity is driven by the need for social interaction (Chung and Yoo, 2008;
Ko et al., 2005; Yoo, 2011). It is quite possible that users post comments to stories (user–
content) when they are already primed to be interested in the story (i.e. through the
information-seeking function of media use). At the same time, users may reply to other
users (user–user) on more specialized, niche topics (i.e. less popular videos) where they
may perceive a greater likelihood of connecting with other users on a social level.

Conclusion
This study took a quantitative approach to understanding the relationships between pop-
ularity and interactive engagement metrics, as well as between these metrics and differ-
ent types of news content. While it is common to quantify engagement, future research
would also benefit from a qualitative understanding of these interactive behaviors.
Analyzing what users are saying in their comments and conversations would contribute
to a richer understanding of interactive engagement practices, how these practices relate
to news content, and begin to address questions about the nature and quality of com-
ments and conversations. For instance, Houston et al. (2011) found that qualitative
dimensions of comments (e.g. partisan slant) can lead to perceived bias in the news con-
tent to which those comments are posted.
Future research could also benefit from addressing some limitations of this study.
First, we only recorded replies when the user clicked the “reply” button associated with
an existing comment. While this was the most efficient way to collect data on 700 videos
with thousands of comments and replies in many cases, we do not account for comments
that may have signaled their reply status by addressing another user or comment in the
text of the comment (e.g. “NewsJunkie, …” or “@NewsJunkie: …”) or in a video reply.
Second, while we were able to capture a snapshot of user behavior over a 3-month period,
longitudinal research would contribute to a more complex understanding of the relation-
ships among interactivity, popularity, and content features. For example, Lee et al. (2010)
found a positive relationship between the lifetime of comments and popularity. It would
be useful to expand that research to include interactivity and content. Finally, longitudi-
nal data would also allow future research to disentangle causality—something we could
not do with our data.
Another fruitful line of inquiry could investigate the antecedents of interactivity.
Building on recent work (Chan and Leung, 2005; Chung, 2008; Chung and Yoo, 2008;
Ko et al., 2005; Yoo, 2011), it would be useful to know more about the motivations and
predictors of user comments and conversation around news videos and other types of
content. Related, understanding the nature of the conversation surrounding news videos
would be of both theoretical and practical interest. In a normative sense, we might be
interested in the degree to which conversations around news videos are civil or hostile.
In a political era that many characterize as polarized, it would be interesting to see
whether and how this plays out in online deliberation. In a practical sense, news execu-
tives have spent considerable time and effort to manage the interactive capabilities of
their digital presence. Many organizations have policies to discourage inter-user hostility
among commenters (e.g. required registration, protocol for reporting hostility, encourag-
ing digital media personnel and journalists to moderate). The industry would benefit
516 new media & society 18(3)

from research that aims to better understand the conversations users are having in
response to the news they present. Finally, future research might explore potential out-
comes of interactivity. These could include loyalty to a news brand, learning (Tremayne,
2008), and influence on perceptions about content (Houston et al., 2011).
In the context of an increasingly interconnected world, this study focused on interac-
tive engagement with online news videos. The results show that users engage with con-
tent in various ways and at differing levels, ranging from exposure to recommendation to
interactivity. News and other online organizations may wish to apply lessons from this
research, as they continue to pursue greater user engagement. But even as we advance
our understanding of how people engage with the news, many questions still remain
about why they choose to do so.

Acknowledgements
We thank four coders—Josephine Lee, Jon Sonnheim, Jessica Bobula, and Daniel Ellman—for
their careful work and dedication, Susan Anderson and Beth Bennett for their guidance and experi-
ence, and Mike Smith and the Media Management Center at Northwestern University for financial
and logistic support.

Funding
This research received no specific grant from any funding agency in the public, commercial, or
not-for-profit sectors.

Notes
1. The top 40 videos include every news video on the first two pages of news videos. The drop
in viewership after this point was substantial, so we considered it a natural cutoff point.
2. As a result of minor and temporary problems with our Web crawler, nine videos in this dataset
were missing the comments. Thus, the final count for this dataset is 191, not 200.
3. The preference for using a proportional measure, rather than the absolute number, is based on
the observation that with more comments, there are bound to be more replies, just by chance
alone. In fact, we see a very strong relationship between the number of comments and num-
ber of replies (r = .872; p < .001). To control for this, the Ratio of Replies variable expresses
user–user interaction as the proportion of replies to total comments for each video, giving an
unbiased indicator of conversation.
4. Codebook is available upon request. Kn is a version of the Kappa inter-rater reliability
coefficient.
5. To clarify, coders were instructed to identify a video as having an agenda if the video editori-
alizes, is very obviously promoting an agenda or point of view, offers an opinion, intends to
change the viewer’s disposition, intends to persuade the viewer, or drive the viewer to action.

References
Barber BR (2004) Strong Democracy: Participatory Politics for a New Age. Berkeley, CA:
University of California Press.
Baum MA (2002) Sex, lies, and war: how soft news brings foreign policy to the inattentive public.
American Political Science Review 96(1): 91–110.
Baum MA (2003) Soft news and political knowledge: evidence of absence or absence of evidence?
Political Communication 20(2): 173–190.
Ksiazek et al. 517

Baum MA (2004) Circling the wagons: soft news and isolationism in American public opinion.
International Studies Quarterly 48(2): 313–338.
Beckett L (2010) Getting beyond just pageviews: Philly.com’s seven-part equation for measur-
ing online engagement. Nieman Reports, 26 October. Available at: http://www.niemanlab.
org/2010/10/getting-beyond-just-pageviews-philly-coms-seven-part-equation-for-measur-
ing-online-engagement/ (accessed 27 July 2012).
Boczkowski PJ (2009) Rethinking hard and soft news production: from common ground to diver-
gent paths. Journal of Communication 59(1): 98–116.
Boczkowski PJ and Peer L (2011) The choice gap: the divergent online news preferences of jour-
nalists and consumers. Journal of Communication 61(5): 857–876.
Brodie RJ, Hollebeek LD, Juric B, et al. (2011) Customer engagement: conceptual domain, fun-
damental propositions, and implications for research. Journal of Service Research 14(3):
252–271.
Burgess J and Green J (2009) YouTube: Online Video and Participatory Culture. Cambridge:
Polity Press.
Calder BJ, Malthouse EC and Schaedel U (2009) An experimental study of the relationship
between online engagement and advertising effectiveness. Journal of Interactive Marketing
23: 321–331.
Chan JK and Leung L (2005) Lifestyles, reliance on traditional news media and online news adop-
tion. New Media & Society 7(3): 357–382.
Chatzopoulou G, Sheng C and Faloutsos M (2010) A first step towards understanding popularity
in YouTube. In: Proceedings of INFOCOM IEEE conference on computer communications
workshops, San Diego, CA, 15–19 March, pp. 1–6. New York: Institute of Electrical and
Electronics Engineers (IEEE).
Cheng X, Dale C and Liu J (2008) Characteristics and potentials of YouTube: a measurement
study. In: Noam EM and Pupillo LM (eds) Peer-to-Peer Video: The Economics, Policy, and
Culture of Today’s New Mass Medium. New York: Springer, pp. 205–217.
Chung DS (2008) Interactive features of online newspapers: identifying patterns and predicting
use of engaged readers. Journal of Computer-Mediated Communication 13(3): 658–679.
Chung DS and Yoo CY (2008) Audience motivations for using interactive features: distinguish-
ing use of different types of interactivity on an online newspaper. Mass Communication &
Society 11(4): 375–397.
Deuze M (2003) The web and its journalisms: considering the consequences of different types of
newsmedia online. New Media & Society 5(2): 203–230.
Espeland WN and Sauder M (2007) Rankings and reactivity: how public measures recreate social
worlds. American Journal of Sociology 113(1): 1–40.
Gans H (1980) Deciding What’s News: A Study of CBS Evening News, NBC Nightly News,
Newsweek, and Time. New York: Random House.
Hamilton JT (2004) All the News That’s Fit to Sell. Princeton, NJ: Princeton University Press.
Hargittai E and Walejko G (2008) The participation divide: content creation and sharing in the
digital age. Information, Communication & Society 11(2): 239–256.
Hayes T (2008) Jump Point: How Network Culture Is Revolutionizing Business. New York:
McGraw Hill.
Herbst S (1998) Reading Public Opinion: Political Actors View the Democratic Process. Chicago,
IL: The University of Chicago Press.
Houston JB, Hansen GJ and Nisbett GS (2011) Influence of user comments on perceptions of
media bias and third-person effect in online news. Electronic News 5(2): 79–92.
Hujanen J and Pietikäinen S (2004) Interactive uses of journalism: crossing between technologi-
cal potential and young people’s news-using practices. New Media & Society 6(3): 383–401.
518 new media & society 18(3)

Jamali S and Rangwala H (2009) Digging digg: comment mining, popularity prediction, and social
network analysis. In: Proceedings of the international conference on web information systems
and mining, Shanghai, China, 7–8 November, pp. 32–38. New York: Institute of Electrical
and Electronics Engineers (IEEE).
Jarboe G (2011) YouTube and Video Marketing: An Hour a Day. 2nd ed. Hoboken, NJ: Wiley.
Jenkins H (2006) Convergence Culture: Where Old and New Media Collide. New York: New
York University Press.
Kim J, Wyatt RO and Katz E (1999) News, talk, opinion, participation: the part played by conver-
sation in deliberative democracy. Political Communication 16(4): 361–385.
Kiousis S (2002) Interactivity: a concept explication. New Media & Society 4(3): 355–383.
Ko H, Cho CH and Roberts MS (2005) Internet uses and gratifications: a structural equation model
of interactive advertising. Journal of Advertising 34(2): 57–70.
Lange P (2007) Searching for the “you” in “Youtube”: an analysis of online response ability. In:
Proceedings of the ethnographic praxis in industry conference, Keystone, CO, 3–6 October,
pp. 31–35. Hoboken, NJ: Wiley.
Larsson AO (2011) Interactive to me—interactive to you? A study of use and appreciation of inter-
activity on Swedish newspaper websites. New Media & Society 13(7): 1180–1197.
Lee JG, Moon S and Salamatian K (2010) An approach to model and predict the popularity of
online contents with explanatory factors. In: Proceedings of the international conference
on web intelligence and intelligent agent technology, Toronto, ON, Canada, 31 August–3
September, pp. 623–630. New York: Institute of Electrical and Electronics Engineers (IEEE).
MacGregor P (2007) Tracking the online audience: metric data start a subtle revolution. Journalism
Studies 8(2): 280–298.
McMillan SJ (2002) Exploring models of interactivity from multiple research traditions: users,
documents and systems. In: Lievrouw L and Livingstone S (eds) The Handbook of New
Media. London: SAGE, pp. 163–182.
McMillan SJ (2005) The researchers and the concept: moving beyond a blind examination of
interactivity. Journal of Interactive Advertising 5(2): 1–4.
Malthouse EC and Calder BJ (2011) Engagement and experiences: comment on Brodie,
Hollenbeek, Juric, and Ilic (2011). Journal of Service Research 14(3): 277–279.
Manosevitch E and Walker DM (2009) Reader comments to online opinion journalism: a space of
public deliberation. Paper prepared for presentation at the 10th international symposium on
online journalism, Austin, TX, 17–18 April. Available at: https://online.journalism.utexas.
edu/2009/papers/ManosevitchWalker09.pdf (accessed 2 November 2013).
Massey BL and Levy MR (1999) Interactivity, online journalism, and English-language web
newspapers in Asia. Journalism & Mass Communication Quarterly 76(1): 138–151.
Nagar N (2011) The loud public: the case of user comments in online news media. PhD Dissertation,
University at Albany, State University of New York, Albany, NY.
Napoli P (2011) Audience Evolution: New Technologies and the Transformation of Media
Audiences. New York: Columbia University Press.
NielsenWire (2012) May 2012: top US online video sites. Available at: http://blog.nielsen.com/
nielsenwire/online_mobile/may-2012-top-u-s-online-video-sites/ (accessed 24 July 2012).
O’Brien HL and Toms EG (2008) What is user engagement? A conceptual framework for defining
user engagement with technology. Journal of the American Society for Information Science
and Technology 59(6): 938–955.
Oh J, Bellur S and Sundar SS (2010) A conceptual model of user engagement with media. In: 60th
annual conference of the International Communication Association, Singapore, 22–26 June.
Patterson T (2003) The search for a standard: markets and media. Political Communication 20(2):
139–143.
Ksiazek et al. 519

Peer L and Ksiazek TB (2011) YouTube and the challenge to journalism: new standards for news
videos online. Journalism Studies 12(1): 45–63.
Peterson ET and Carrabis J (2008) Measuring the immeasurable: visitor engagement. Report,
Web Analytics Demystified, 7 September. Available at: http://www.webanalyticsdemysti-
fied.com/downloads/Web_Analytics_Demystified_and_NextStage_Global_-_Measuring_
the_Immeasurable_-_Visitor_Engagement.pdf (accessed 5 January 2012).
Pew Research Center (2012) YouTube and news: a new kind of visual news. Report, Pew Research
Center, Project for Excellence in Journalism, 16 July. Available at: http://www.journalism.
org/2012/07/16/youtube-news/ (accessed 18 July 2012).
Prior M (2003) Any good news in soft news? The impact of soft news preference on political
knowledge. Political Communication 20(2): 149–171.
Ruggiero TE (2000) Uses and gratifications theory in the 21st century. Mass Communication &
Society 3(1): 3–37.
Schaudt S and Carpenter S (2009) The news that’s fit to click: an analysis of online news val-
ues and preferences present in the most-viewed stories on azcentral.com. Southwestern Mass
Communication Journal 24(2): 17–26.
Schmitz Weiss A and De Macedo Higgins Joyce V (2009) Compressed dimensions in digital
media occupations: journalists in transformation. Journalism: Theory, Practice, Criticism
10(5): 587–603.
Schudson M (2001) The objectivity norm in American journalism. Journalism 2(2): 149–170.
Schultz T (1999) Interactive options in online journalism: a content analysis of 100 US news-
papers. Journal of Computer-Mediated Communication 5(1): 0. DOI: 10.1111/j.1083-
6101.1999.tb00331.x.
Schultz T (2000) Mass media and the concept of interactivity: an exploratory study of online
forums and reader e-mail. Media, Culture & Society 22(2): 205–221.
Shirky C (2010) Cognitive Surplus: Creativity and Generosity in a Connected Age. New York:
Penguin Press.
Siersdorfer S, Chelaru S and Nejdl W (2010) How useful are your comments? Analyzing and
predicting YouTube comments and comment ratings. In: Proceedings of 19th international
World Wide Web conference, Raleigh, NC, 26–30 April. Available at: http://www.l3s.
de/~siersdorfer/sources/2010/wfp0542-siersdorfer.pdf (accessed 27 July 2012).
Stromer-Galley J (2000) Online interaction and why candidates avoid it. Journal of Communication
50(4): 111–132.
Tewksbury D (2003) What do Americans really want to know? Tracking the behavior of news
readers on the Internet. Journal of Communication 53(4): 694–710.
Thurman N (2008) Forums for citizen journalists? Adoption of user generated content initiatives
by online news media. New Media & Society 10(1): 139–157.
Tremayne M (2008) Manipulating interactivity with thematically hyperlinked news texts: a media
learning experiment. New Media & Society 10(5): 703–727.
Webster JG (2010) User information regimes: how social media shape patterns of consumption.
Northwestern University Law Review 104(2): 593–612.
Webster JG (2011) The duality of media: a structurational theory of public attention. Communication
Theory 21(1): 43–66.
Yoo CY (2011) Modeling audience interactivity as the gratification-seeking process in online
newspapers. Communication Theory 21(1): 67–89.
Zaller J (2003) A new standard of news quality: burglar alarms for the monitorial citizen. Political
Communication 20(2): 109–130.
520 new media & society 18(3)

Author biographies
Thomas B Ksiazek (PhD, Northwestern University) is an assistant professor in the Department of
Communication at Villanova University. His research interests include digital media use, audience
behavior, journalism studies, and network analysis.
Limor Peer (PhD, Northwestern University) is Associate Director for Research at the Institution
for Social and Policy Studies (ISPS) at Yale University. Her prior position was Research Director
at Northwestern University’s Media Management Center and Readership Institute, where she
supervised research on media audience, content, and management strategy and taught classes on
communication theory, public opinion, media and society, and statistics for journalists.
Kevin Lessard completed his Master of Arts in Communication at Villanova University in 2012,
with a certificate in Journalism and Public Relations. His research interests primarily revolve
around the intersections of political communication, journalism and mass media studies. He is cur-
rently a Media Analyst with Visit Philadelphia™, the official tourism agency for the Philadelphia
region.