SocArXiv SocArXiv Preprint : November 2, 2018 https://doi.org/10.31235/osf.io/me8zd A Study of Citation Motivations in HCI Research Jordan Beck, Bikalpa Neupane, & John M. Carroll The Pennsylvania State University, University Park, PA Abstract What paper should be cited where, and why? Gaining insight into citation practices in HCI can enrich our understanding of the structure of research, as well as social and political factors that influence knowledge production. Recent research into HCI citation practices have identified problematic trends: inaccurate citing and a lack of critical citations. Directly investigating authors’ motivations for citing can provide a fuller picture of citation practices. It can help us understand HCI research as a social system and develop a theory of citing. We report an interview study designed to identify authors’ motivations to cite. We found three primary motivations: affirming personal relationships, building intellectual narratives, and serving strategic objectives. We discuss our findings from a social constructivist perspective of citation practices. In addition, we explore how these findings can strengthen efforts to write CHI papers. Finally, we outline steps to develop a theory of citing in HCI research. Keywords: citations, citation motivations, scholarly communication. 1. Introduction The HCI research community has a strong tradition of examining its research practices (10; 12). Previous work has explored the interrelationship between the soft and hard sciences (11) that constitute large portions of the field. Others have reflected on the structure and development of HCI theory (41) as well as how theory supports practice (4). And there have also been general discussions of knowledge production (29; 40) and particular consideration given to new ways of producing and contributing knowledge, e.g. Research through Design (24; 51; 2). Scholarly communication–publishing posters, conference papers, and journal articles, for example–is the primary way researchers share and contribute knowledge within an intellectual community. Different communities have different approaches to scholarly communication, including: different publication templates, peer review processes, archival practices, and, re- 2 Citation Motivations in HCI lated to our purpose, citation practices. Citations can have different functions (17; 22), and researchers have identified different authorial motivations for citing (9; 14). Citation analysis has a long academic history (17; 16; 44), and many researchers over the years have studied citations in different research communities. Citation studies have been said to provide insight into the structure of scientific research (37), including the social factors that influence knowledge production (36; 28) Some HCI researchers have studied citations over the years. For example, MacKenzie (32) studied citations to argue for the impact of HCI research. Touscsik (43) found that academic mathematicians cite an open science platform, MathOverflow, as a legitimate source of scientific knowledge. And at CHI2017, Marshall & Linehan (33) and Marshall et al. (34; 35) examined citations to an influential study pertinent to exertion games research and general citation patterns in CHI literature. Key findings from their work revealed a pattern of inaccurate citing and very little critical engagement with cited sources. In this paper, our goal is to contribute a novel perspective on citations in HCI research. To accomplish this goal, we conducted an interview study with HCI researchers (n=9) in order to understand their motivations for citing. We partially transcribed each interview and performed thematic analysis on the transcripts. From this analysis, we synthesized three primary motivations for citing. They are: affirming personal relationships, building intellectual narratives, and serving strategic objectives. These motivations draw attention to the social factors involved in citations and suggest that HCI manifests both normative and social constructivist citation practice. Furthermore, they have interesting implications for how HCI researchers write research papers. We end by outlining steps to develop a theory of citing in HCI and the unique contribution HCI research can make by studying the interactive systems and artifacts that shape citation practices. 2. Background Citations have been studied by information scientists, bibliometricians, and discourse analysts. Eugene Garfield was one of the first to study citations. In one of his earliest papers (17), Garfield proposed building citation indexes to enable scientists to check cited or criticized source material more efficiently. Garfield was also the first to propose a citation classification scheme listing different reasons why authors cite (18, p. 85). Many other classification schemes have since been proposed. As of 2004, White counted about 20 different ones (48, p. 100), each of which attempts to describe what citations do (function) or why they were cited (author motivation). In what follows, we summarize two categories of existing research on citations: (1) citation classification studies and (2) citation motivation studies. We end by identifying synergies and opportunities for studying citations in the context of HCI research. 2.1. Citation Classification Since the early days of citation analysis, researchers have used content analysis to distinguish different kinds of citations. Moravcsik & Murugesan (37) published one of the earliest studies examining 30 theoretical high-energy physics articles using a classification scheme distinguishing 8 kinds of citations (37, p. 88). A key finding in this study drew critical attention to citations as measures of the intellectual quality of scholarship. Moravcsik & Murugesan identified a high number of ’perfunctory’ citations in their sample. Perfunctory citations get cited SocArXiv 3 because they exist (16) not because they are needed for understanding the paper (37, p. 88). The question of citations as measures of quality remains relevant in citation studies today. However, researchers have used content analytic techniques to answer other kinds of questions about citations, such as: who are the most highly cited authors (15), which authors are cited together, and how do doctoral students’ citing behaviors compare with those of faculty (27). Answers to these questions potentially contribute to an understanding of the structure of scientific research (37). Researchers in different fields have taken up citation classification schemes and applied them to corpuses of texts to try and understand “their” field’s citation practices. For example, Beck & Chiapello (3) analyze how authors publishing at the Design Research Society (DRS) conference cite the work of Donald Schön. They use Nigel Harwood’s citation function framework (22, p. 501-511) as a code and find that authors cite Schön primarily to credit him for ideas or to support their research topics, methods, or arguments. They find little evidence of critical engagement with Schön’s work. That there are few critical citations to Schön mirrors findings from a recent study in HCI research by Marshall et al. (34), and both of these studies are consistent with earlier analyses reporting low numbers of critical citations (46; 13). White (47) proposes a provocative yet plausible explanation for this: it is harder to criticize than it is to agree with sources. Content analyses can tell us about which texts get cited (15), where citations appear in a text (e.g. introduction, background), how citations change over time (30) and how readers interpret citations (e.g. as critical or perfunctory). However, content analyses are limited in what they can tell us about authors’ motivations for citing. Authors motivations have been characterized as personal and private (9), and researchers have questioned whether readers can reliably reconstruct authorial intent without consulting the writers themselves, e.g. (22, p. 497) and (48, p. 101). Getting at authors’ citation motivations is pertinent both for generating deeper understanding of research as a social system (36; 25) and, specifically, for developing a theory of citing (16, p. 25-34). 2.2. Citation Motivations White (47) identifies Hodges (23) study of reasons for citing, published in her dissertation, as the earliest example of a live interview study with citers. Hodges’ work remained obscure until the mid-1980s when Peritz (39) attempted to simplify Hodges’ citation classification scheme since the categories proved too difficult to apply during content analysis. Brooks (9) interviewed scholars from different academic departments, including: anthropology, art, education, hydraulic research, nursing, philosophy, and others. To conduct his interviews, Brooks synthesized a citation classification scheme on the basis of several (9, p. 225-226). This means participants chose from an existing set of possible motivations, which is typical of survey studies, e.g. (13), and which does reduce the likelihood of discovering ’novel’ motivations. However, Brooks reported that two participants did provide original answers after finding the existing choices inadequate (9, p. 228). It is still common to prompt interviewees (or survey respondents) with existing citation classification schemes. However, there have been interview studies designed to prompt original answers, too. White and Wang (50) conducted interviews about citer motivations as part of a broader, longitudinal study of document use during research. They interviewed scholars in the early stages of their projects as they searched for and selected relevant documents to cite. Then, 4 Citation Motivations in HCI they conducted a second round of interviews upon completion of the project. Project completion meant producing a manuscript for publication. Their longitudinal approach enabled White and Wang to draw conclusions about decisions to cite as well as decisions not to cite. Moreover, they curated a list of “meta-level” questions that authors posed when considering whether to cite. Secondary citing, which involves citing documents that you have not read but encounter in other texts, which is an issue that Marshall & Linehan address (33), is one such meta-level concern. Like content analyses, interview studies have been conducted to understand different disciplines. Harwood (22) interviewed computer scientists and sociologists and developed a citation function framework describing 11 possible functions. Harwood also models the discourse-based interview approach (22, p. 500) during which authors re-read and reconsider one of their own papers in terms of its citations rather than speak more generally about citation functions or their motivations for citing. One of his key insights, that is evident in other work, is that individual citations can have multiple functions. For example, one citation can credit a source text for ideas, demonstrate an author’s mastery of the literature, and evidence knowledge claims. It seems reasonable, then, that authors might also have multiple motivations for citing individual sources. 2.3. Citation Studies in HCI Research HCI research has had some interest in citations over the years. Citations have informed studies of the impact of HCI research. For example, MacKenzie examined a corpus of publications (CHI, TOCHI, and HCI) to assess the “impact” of the scholarship produced in these venues and found little support for the “phenomenon of uncitedness” (32, p.2545). Researchers have also explored how novel platforms for scholarly communication affect citations. For example, Tausczik (43) studied how content published on MathOverflow, an open science platform, has been cited within mathematics literature. They found that some scholars cited MathOverflow as a legitimate source of scholarly knowledge. Finally, and most recently, Marshall & Linehan published a full CHI paper (33) and two alt.CHI papers (35; 34) examining citation practices in exertion games literature in general and CHI literature in particular. They synthesize two key insights based on content analyses of research publications: (1) scholars routinely mischaracterize or misattribute findings to prior research, and (2) there appears to be very little critical engagement with prior work. We agree with the underlying premise driving these studies: that citations are worthy of closer examination in HCI research and that they are useful for answering questions about scholarly impact and the structure of HCI research. 3. Method To generate a deeper understanding of citation practices in HCI research, we conducted an interview study with researchers in the field about their reasons for citing. Getting at authors’ reasons contributes to a deeper understanding of HCI research as a social system (36) and to developing a theory of citing (16, p. 25-34) within the HCI community. We conducted interviews modelled on Harwood’s discourse based interview approach (22, p. 500) with nine HCI researchers who were at comparable, early stages in their careers. For each interview, we asked the interviewee to provide us with one of their recent (< 3 years old) publications, and together we read through and talked about each in-text citations. For SocArXiv 5 each citation, we prompted interviewees to explain their decision. “Why did you cite this text? ” We also asked probing questions to unpack motivations that may not have been clear to us. Talking through each in-text citation allowed for the possibility that there might be different motivations for citing the same source in different locations in the paper. We partially transcribed each interview and conducted a thematic analysis (8) on the transcriptions. Our analysis was iterative, and we relied on peer debriefing and member checking to validate our findings. 4. Results Interviewees gave many different reasons for citing prior work. Through our thematic analysis, we iteratively developed three themes to capture these different reasons. The themes include: (1) affirming personal relationships, (2) building intellectual narratives, and (3) serving strategic objectives. 4.1. Affirming Personal Relationships Interviewees frequently pointed out personal relationships with cited authors. These relationships tended to be one of three kinds. Some cited authors were mentors or advisors. “[This] is my thesis committee member ” (P1). “One of the authors was on my PhD committee” (P7). “[They were] one of my teachers” (P2). Others were colleagues. ”This citation is one of my former lab mates” (P3). “The author was in our lab at the university” (P5). “They were in my cohort at university” (P7). And some seemed to be casual acquaintances. “This one probably wasn’t incredibly necessary, but I’d actually run into the author the year before [at a conference] ” (P6). Having a personal relationship with the cite author would, in this last case, seem to be sufficient grounds for citing. This does not mean that personal relationships are the only or even the most dominant reasons for citing. Some relationships were framed as casual. “I know this guy” (P1). “He was one in my cohort” (P7). However, they do appear to play a role, and, in a few cases, personal relationships seem to play the most significant role. ”I knew one of the authors and knew they were going to be miffed if I didn’t cite them” (P4). This was one of several reasons given for a group of citations (e.g. [2, 3, 7, 11, 13, 17]). P4 gave different reasons for each citation in the group. For example, they cited the need to include more papers from a particular venue (e.g. [2, 3, and 7] came from CHI) and the need to balance citations from different fields with HCI citations (e.g. [11, 13] came from psychology literature). Citing to appease an author was the only reason given for [17], so, it is possible that maintaining a personal relationship was the only reason for citing in that case. Most interviewees acknowledged personal relationships alongside other reasons and did not reflect on the significance of the relationship when compared to these other reasons. They tended to acknowledge the relationship and then continue listing reasons. “The author was in my lab, so I know the paper really well.. It has a really compelling argument” (P4). The interviewee first mentions the relationship but then pivots to the paper’s compelling argument, which they know well. The relationship seems secondary. However, one interviewee called attention to personal relationships with cited authors as a primary motivation to cite. ”It would be really interesting to see how many [citations] are because we know these people” (P7). This came at a point in the interview after we had discussed several citations in a row 6 Citation Motivations in HCI featuring personal relationships with citees. We do not present these results as an indication that personal relationships with a citees are the only reasons motivating a decision to cite. However, they are potentially significant contributing factors. All of our interviewees acknowledged personal relationships as they explained their reasons for citing, and some even reflected on just how significant they could be with regard to citation practices. 4.2. Building Intellectual Narratives In addition to affirming personal relationships, interviewees described citations in terms of how they contribute to broader intellectual narratives. These narratives describe both how a research project grows over time (i.e. that the paper we discussed built on the author’s previous work) and how a research project fits within particular HCI research discourses. When discussing existing discourses, we were especially interested in the way authors distinguished important papers from others. The first category situates citations as part of a broader lineage, and the motivation is to extend the line. “We needed to cite our original work ” (P1). “This paper builds on my previous two” (P4). “I wanted readers to follow my path” (P2). “My co-author wrote this last year; we build in it quite a bit in this work ” (P7). “We want to move forward from the type of work done in [citations a-d] ” (P3). This lineage also accounts for particularly influential work. “This is one of my ’go-to’ citations whenever I need to warrant a claim for more humanities/art and media-theory-backed work, [which is what I do] ” (P6). “I read this one for the first time a few years ago, and it made me think I had to change everything I was doing” (P4). “You can’t write a paper like this without Paul Dourish showing up” (P5). The second category constructs one’s role or position in a scholarly community, and the motivation is to demonstrate awareness and competence with regard to playing that role. “I want to show that I understand the foundations of the topic” (P1). “I wanted people to know that I read about this stuff ” (P2). It can involve situating one’s ideas in relation to relevant threads of a discourse. “I wanted to show that my ideas connect with others’ ideas” (P2). “It’s a group of people that wrote it that are pretty connected to my work ” (P5). It can distinguish the approach one takes or the theory one draws upon does through a kind of negative comparison to existing work. “We have to cite to justify not doing it this way” (P1). It can demonstrate knowledge of the expectations of a particular venue or genre characteristics of writing. “There is a long gap in terms of citations because we are in the discussion [section now] ” (P3). “The paper straddles psychology and sociology, but it gets cited a lot in HCI research” (P5). Interviews frequently distinguished important papers from others. “This [citation] is actually important” (P1). “The ones that are repeated... They’re pretty fundamental to the paper ” (P2). One interviewee even suggested that there might be a need to develop prioritized reference lists “with the most important ones on top of the list” (P7). The interviewee reported that the most important texts shaped the way they thought about key concepts in the paper. While citations can be important to a particular paper, importance can transcend a single paper and cut across a fuller body of scholarship. “This is one of my basic references regarding rhetoric” (P2). “I use this paper a lot [in my other work, too] ” (P4). A key insight is that “importance” is not a monolithic concept. On the one hand, a citation might be important for its intellectual content. “This first citation is like a glossary of basic definitions for the field ” (P2). “They are good at framing speculative design as philosophical SocArXiv 7 inquiry, which is a key reference for a lot of our work ” (P7). It might also be important for the personal relationship it acknowledges or reinforces. “[This citation] is important. First, I know the authors...” (P1). However, as we explain in the next section, citations might also be important as part of a political project or strategy. Interestingly, several interviewees suggested that, when a citation serves as an example of a design or concept, it is less important. For instance, “These five are examples of designs that have done what we are describing” (P3). “These are just examples of systems people have built” (P4). “They’re cited just as examples” (P5). Here, the interviewees’ use of the word “just” could mean that these citations are less important than others. We found that interviewees characterized citations as unimportant in other ways, too. In one case, a citation was a one off. “I have no idea who these authors are, but it’s sort of a one-off citation” (P5). Interviewees alluded to their lack of engagement with a citation.“I don’t have the book on my bookshelf [so to speak] even though I reference it here” (P7). This person explained that they did not possess much knowledge of the content of the citation, but that it was not necessary for their purpose. In a few cases, citations were clearly articulated as unimportant. “Individually, [these papers] aren’t so important” (P4). Some citations that were deemed intellectually unimportant, however, could have been important for other reasons, such as serving strategic objectives. 4.3. Serving Strategic Objectives Interviewees described different strategic reasons for citing. For example, they explained how citations anticipated reviewer and reader critiques or responses. “We needed to justify our results to reviewers” (P1). This comment was not based on previous critical reviews of their paper. We were discussing a paper that had been accepted at its first submission venue. Instead, it seemed to be a forward-looking decision that was based on the collective experiences of the author team. Knowing what kinds of criticisms a paper might get guides citation decisions. For example, as one interviewee explained, “Citing prevents a few ’gotchas’ from hardcore theorists who might criticize me for being out of step with contemporary work ” (P5). Whereas P1 describes anticipating reviewer critique, P5 orients more toward a broader audience. Assuming the paper gets published, who will be in the audience and what might they critique? P2 made frequent references to their “insecurity” as a writer and framed several citations as a way of “reminding readers that I was drawing on authoritative sources” (P2). Furthermore, P2 described citing authoritative sources as a tactic to persuade readers to accept the legitimacy of their argument. At one point during our interview, they imagined a conversation with a reader and said: “So, ok, if you don’t believe me, go and read the book ” (P2). Interestingly, strategic objectives can also inform decisions not to cite. P1 was the only interviewee to point out to us an instance where they could have cited but did not. They offered a succinct explanation as to why: “We didn’t solve the problems we raised, so we didn’t cite anything here” (P1). They elaborated that citing something here would have emphasized that they offered no solutions in their paper, and that this could have been problematic for reviewers. In general, reviewer and reader criticisms motivated strategic citing decisions. But interviewees also saw citations as a way to “game” the reviewing process. “There’s a little bit of a strategy there–trying to cite things so that you get assigned to a specific [editor] or 8 Citation Motivations in HCI trying to guide [the editor] on who you want your reviewers to be” (P4). P9 gave a similar account of citing:“It’s important to cite in a way that will help an AC find the right reviewer for your paper.” (P9) Other interviewees discussed citing in order to get a paper in front of a target audiences. “Those authors are people that I’d like to read the paper if it gets published ” (P2). The key assumption here is that citing someone will get their attention, and several interviewees expressed some version of this. “I like those authors; I want them to see my work ” (P4). In some cases, however, interviewees had bigger aspirations, such as fostering new research collaborations with (or among) citees. P3 explained that many of the citations in their paper “focus...on fostering collaboration across different communities” (P3). Their paper contained citations lists, e.g. [1,3,5,7,11,13]. Consulting the reference list was intended to reveal connections between what the interviewee described as largely disconnected scholarly communities. Citing them together is one way to establish ties between them, and the interviewees’ hope was that these ties would result in readers seeing new potential directions to take their research. Visibility (“seeing”) resonated with other strategic objectives, too. P4 described a lack of research on a topic that was central to their work, and so they made an effort to “amplify the citations that do exist” whether they are necessary for understanding the paper and its contribution or not. P5 reaffirmed the importance of citing “in spite of its broader lack of relevance because it’s someone who is keeping this idea alive and pushing it forward ” (P5). P8 took this one step further when they described citing a source, whose content was not important to their paper, as “part of a political project” to give the citee more visibility and notoriety within the HCI research community. 4.4. Summary We interviewed nine HCI researchers about their citation motivations. We recorded and partially transcribed each interview, and, using thematic analysis, we developed three high-level motivations to cite. These are: affirming personal relationships, building intellectual narratives, and serving strategic objectives. We relied on peer debriefing and member checking to validate our findings. In the next section, we discuss what these findings mean for understanding citation practices in HCI research. We also discuss the implications of these findings for teaching citation practices, writing papers, and the design of reference management systems. 5. Discussion Our findings draw attention to important reasons for citing that merit attention in the literature. These reasons may be “known” in the sense that researchers talk informally about citing colleagues’ work or citing strategically in anticipation of reviewers and readers’ possible criticisms. But our interviewees have provided practical and personal insights into citation practices that go beyond what we have encountered. These insights can be of practical importance to writers preparing manuscripts and to reviewers evaluating them. Furthermore, these insights motivate questions about whether citation practices in HCI are problematic or if they simply reflect that research–in HCI as in other disciplines–is a social and political project and, as such, we can expect citations to do things like affirm personal relationships and serve strategic objectives in addition to representing intellectual content and demonstrating SocArXiv 9 knowledge contributions. 5.1. Normative vs Social Constructivist Citing Practices in HCI Research Citation analysts have debated whether to characterize citation practices as normative or social constructivist (49). The normative perspective argues that authors cite primarily to acknowledge intellectual influence, which includes things like giving credit for an idea, giving a nod to a classic study or “pioneer” researcher, or justifying a research design or method (14). The social constructivist perspective argues that authors cite primarily for reasons other than intellectual influence. Typically, these other reasons have to do with persuading reviewers and readers to accept the legitimacy or significance of an argument. In 2004, White made the case that, among citation analysts, the normative view was making “a comeback” after a period of social constructivist dominance (49). Case & Higgins made a similar argument in 2011 (14, p.430). Our findings are intriguing to think about in light of these arguments. Interviewees described motivations that align with both the normative and social constructivist view. All of our interviewees spoke about intellectual influences, classic (important) studies, and “pioneer” researchers (e.g. “You can’t write a paper like this without Paul Dourish showing up” (P5). However, they also acknowledged personal relationships and strategic or political motivations to cite, and this makes sense. To our knowledge, there are no researchers who argue that a discipline’s citation practices would be either normative or social constructivist. Terttu Luukkonen suggests that Merton (36) and Latour (28), canonical figures representing the normative and social constructivist perspectives, respectively, “might” both be right. Citations can acknowledge intellectual influence and serve some rhetorical function (31, p. 33). Harwood (22) found that computer scientists and sociologists tended to give reasons for individual citations that aligned with normative and social constructivist views, and we found similar accounts in our study. Finding evidence of normative and social constructivist motivations strikes us as a good first step toward the development of a theory of citing in HCI. However, we ought to consider the interplay between the two. Are citation motivations equal parts normative and social constructivist? Do they skew more towards one than the other? The question of significance is reflective of a broader issue in the normative vs. social constructivist debate. The issue is not whether authors cite either to acknowledge intellectual influence or for other reasons. The issue is: which reason(s) are more significant? For example, during their interview, P1 acknowledged that a citee was both a personal relation and a well-established domain expert. Which reason matters more? P7 commented on how many citations “are because we know the author[s],” but personal relationships were rarely given as the only reason for citing. When we asked interviewees to explain why they cited a source, if a personal relationship was part of their answer, then it was almost always the first thing they mentioned. This gave the impression that personal relationships are important factors in the decision to cite. However, it might also be the case that personal relationships were easier to “call up” from memory during interviews, which means we cannot say personal relationships are more significant than intellectual influence–only that they are contributing factors. On the other hand, we cannot rule out that personal relationships could be the most significant factors, but more work is needed to examine this possibility, and it would be worthwhile to undertake in the 10 Citation Motivations in HCI HCI community. Harriet Zuckerman (52) explains why the issue of significance is so compelling. The normative perspective holds that citations are a reasonable way of assessing a scientists’ research in terms of its scientific merit. The constructivist perspective calls this into question on the grounds that social factors, such as personal relationships, contribute to citing decisions, which means that citations are not just expressions of scientific quality or merit. They also express “personal relations, social origins, and social statuses,” (52, p. 518). This means that other decisions where citations play a significant role, such as faculty appointments, funding proposals, and awards for achievement and influence, are thus based on a complex composite of scientific merit and social factors. This might be troubling to some, since it means that HCI researchers could “game the system” by networking and building personal relationships and, thus, gain appointments, tenure, funding, and other benefits not on the basis of scientific merit. These possibilities run counter to the “storybook” (16) version of science, which elevates scientific merit. According to this account, personal origins and social status do not determine your trajectory as a scientist. What matters is that you do good science. Doing good science will yield citations. Citations will thus primarily reflect scientific merit even if there is some rhetoric involved. Our interviewees expressed motivations that align with the storybook version of science. However, they also expressed motivations that align with what we will refer to as a “literary realist” version of science that takes into account the social and political aspects of doing research. This is not a criticism of the community. Rather, we see it as a strength. Research in HCI–and in other intellectual communities–is a social process. Our interviewees seem keenly aware of this to the extent that they can be more strategic and intentional in their use of citations when they write research papers, e.g. by deploying citations to get assigned a particular reviewer, to recruit a potential future collaborator, or to reaffirm or build a personal relationships with an advisor, mentor, or colleague. 5.2. Writing Papers With Citation Motivations In Mind Our interviewees responses are particularly interesting to consider in light of findings from a recent study of CHI publications. In 2017, Marshall, Linehan, Spence, & Egglestone reported findings from a content analysis of CHI publications (34). Using a coding scheme derived from (7), the authors analyzed 3,080 citations from 69 papers published at CHI2016 (34, pp. 831-832). They found that CHI has an unusually high number of cursory citations, which are similar to what Moravcsik & Murugesan (37) called “perfunctory” citations, and that it has an unusually low number of critical citations. Some of the citations we discussed with interviewees seemed cursory. Interviewees encountered citations that they could not recall. In some cases, they did not recognize the authors or title and speculated as to why it may have been included. In other cases, they recognized the citation but did not provide a rationale for including it in the paper. When P5 said, “I have no idea who these authors are, but it’s sort of a one-off citation,” it seems reasonable to characterize this citation as cursory. The interviewee does not recognize the authors or the text, and they describe the citation as though it could have been excluded from the paper without any value lost. This particular citation was largely irrelevant to the intellectual content of the paper. SocArXiv 11 However, intellectual content is not the only indicator of relevance or importance. Cursory citations can still perform important functions. For example, Small (42) has shown that cursory citations tie discourses together. This is interesting because it means that citations do not have to be fully relevant to the content of a paper in order to add value to it. They may appear cursory to some readers and, at the same time, reflect an intentional, strategic effort to accomplish a different primary objective, like appease a colleague or attempt to guide an AC to assign a paper to a reviewer that might not be so critical of a draft manuscript. We suggest that citations such as these demonstrate how HCI researchers effectively strike a balance between normative and social constructivist motivations when they write papers and that it is important to make this balance explicit. Citations can be important for different reasons. Some are important for their intellectual influence. Some are important for the personal relationships they reaffirm. And some are important for their capacity to persuade reviewers. Crucially, as we discussed in the previous section, these are not mutually exclusive. An individual citation can be important for multiple reasons, and our interviewees seemed highly aware of this. One even proposed redesigning reference lists to distinguish ones whose intellectual content is particularly influential or important, which, in their view, would give reviewers (and readers) a sense for authors’ “intellectual influences” (CHI) as well as a sense for which citations merit more amplification or critical engagement than others. We can capitalize off of ideas like this to innovate new ways of managing citations. It would not make sense to expect authors to describe in detail or critique citations whose primary purpose is to give a nod to an advisor, teacher, or lab mate. Nor would it make sense for authors to discuss citations whose purpose is to get the attention of “people [they’d] like to read the paper ” (P4) unless those people produced work that was relevant or influential to the paper. On the other hand, it would be interesting and useful to gain insight into how authors take these different purposes into consideration during the writing process–and how the software they use could somehow leverage these purposes. We presume that striking the right balance when it comes to citation motivations takes some skill. Although normative citations probably do not call attention to themselves, one would expect citations with social or political purposes to stand out. Then again, since social and political citations also serve other purposes, they may not stand out quite so much. When our interviewees described intellectual narrative building, they talked about intellectual influences, demonstrating their knowledge of different academic discourses, and of demonstrating the novelty of their contributions. All of these reasons align with the normative perspective. At the same time, they spoke of citing for social and political reasons (e.g. appeasing authors who might be “miffed ” if they weren’t cited, trying to get the attention of certain readers, and amplifying marginalized voices). Our findings thus lend crucial support to past arguments that authors “reward citees and seek to mobilize them for [other] ends” (49, p. 97). Neither the normative or social constructivist perspectives provide a complete account of why authors cite. Our findings reveal motivations–often for the same, individual citation–that fit with both perspectives. It would benefit HCI researchers to incorporate these perspectives into existing writing resources, such as the guide to successful CHI submissions (CHI) or the recurring CHI course on how to write and review CHI papers (38). Incorporating these perspectives would provide 12 Citation Motivations in HCI a space for authors to think about why they cite and, perhaps, to encourage them to become more strategic and intentional in their citation practices. Recognizing that a citation serves primarily to reaffirm a personal relationship, to amplify a marginalized voice, or to acknowledge intellectual influence both for an author’s time and attention during the writing process. Similar to Garfield’s famous call for tutorials on “when” to cite (19) we call for teaching and learning about “why” to cite in HCI research publications. 5.3. Steps to Develop A Theory of Citing in HCI There are no hard and fast rules for citing (19), and the debate between the normative and social constructivist perspectives is still alive and well. There are three research trajectories that would be useful for HCI researchers to pursue. First, our interview study yielded interesting results, but more work is needed to strengthen our understanding of citation motivations in HCI research and, thus, to contribute to a theory of citing. Second, additional citation classification studies, e.g., (35), are needed. Such studies would provide critical insight into historical patterns of citing in HCI research and give insight into how readers and reviewers interpret citation function. Third, and finally, HCI research is unique among disciplines with an interest in citations since its experts can study the interactive systems and artifacts that authors use to cite. Citation Motivation. Our findings show that HCI researchers cite to affirm personal relationships, build intellectual narratives, and serve strategic objectives. Since our interviewees were at comparable, early stages in their careers, we became curious to know whether our findings would hold for a larger group of interviewees, including ones at different career stages. For example, we might find different citation motivations if we interviewed senior researchers (27) or HCI researchers who live and work in different cultural contexts (26). In addition, it would be interesting to study citation motivations in the context of a particular venue, such as CHI, and even within different subcommittees. What motivates authors who submit to the design subcommittee when compared to the health or understanding users subcommittees? Studies that attempt to parse the interplay between normative and social constructivist motivations for citing might be especially interesting as well as practically useful. For example, writers could become more intentional and strategic in their citation practices–developing expert judgment about when to cite primarily for strategic purposes or to acknowledge intellectual debt. Developing richer accounts of citation motivation may also be useful to reviewers and readers. For example, recognizing that cursory citations add value to a paper–that they may not be there just because they exist–could improve assessments of a paper’s overall quality. This could have significant implications for decisions to accept manuscripts at venues like CHI, which, in turn, have implications for things like promotion & tenure and research funding. However, it is also important to continue to build on existing work in the field when it comes to citation classification (34). Citation Classification. To develop a theory of citing in HCI research, which is important for our understanding of HCI research as a social system (25, p. 179), it is necessary to complement citation motivation studies with citation classification and to explore the interplay between the two. SocArXiv 13 Citation classification studies can be especially valuable from a historical perspective. For example, is there a tradition of uncritical citation practices in HCI research? Or is this a recent development in the field? The development of HCI research has been characterized in terms of three waves (5). Do citation patterns “shift” with these waves? One way to reflect on whether HCI researchers are perpetuating low quality citation practices is to examine publications that serve as models for good research, such as CHI best papers. What kinds of citations will we find in best papers? Do they reflect the citation practices we wish to encourage and develop? What about design papers? What should we expect from citations in good design papers (21; 20)? A key question to examine moving forward is how citation classification studies and citation motivation studies coalesce to inform a more general theory of citing in HCI research. Equally important, we need to identify opportunities for HCI research to ask research questions that it is in unique position to answer. Reference Management Systems. There is a noticeable lack of research about how interactive systems and artifacts that support citation practices actually support and influence authors, not only by recommending sources (45; 6), but also by taking citation motivations into account. This is a great opportunity for HCI research. How do these systems influence or constrain practices? Do reference management systems reflect the normative or the social constructivist perspective? If so, how? If not, why not? Should they? One of our interviewees suggested redesigning reference lists to prioritize intellectually significant texts from others. Could this be accomplished through a redesign of reference management systems to support categorizing references according to different motivations or enabling authors to “rank” the importance of citations as they write? These possibilities are exciting, and they could yield novel contributions both to HCI and to other scholarly communities studying citations. 6. Conclusion We set out to study citation motivations in HCI research. Why do authors cite the papers they do? Based on an interview study with nine HCI researchers at comparable, early stages in their careers, we synthesized three primary motivations to cite. These are: affirming personal relationships, building intellectual narratives, and serving strategic objectives. Our findings suggest that HCI researchers are adept at balancing normative and social constructivist motivations to cite when they write. Moreover, they reveal how the intellectual content of a citation is not the only measure of importance or relevance when it comes to citation practices. Perfunctory or cursory citations can still add value to a paper; though the value may not be apparent to readers. We suggest that becoming more aware of the different possible motivations for citing is beneficial to HCI researchers, and that these different motivations could be examined in more depth in writing resources, such as (38). We identify opportunities for more inquiry into citation practices in HCI research, including examining the interplay between normative and social constructivist perspectives. Finally, we suggest that HCI research is in a unique position to study citation practices by studying the tools authors use to collect, manage, and cite sources. Further examination of citations can contribute both to a deeper understanding of HCI research as a social system and the development of a theory of citing. 14 Citation Motivations in HCI References [CHI] Guide to a Successful Submission–CHI 2019. [2] Bardzell, J., Bardzell, S., and Koefoed Hansen, L. (2015). Immodest Proposals. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, pages 2093–2102, New York, New York, USA. ACM Press. [3] Beck, J. and Chiapello, L. (2018). Schön’s intellectual legacy: A citation analysis of DRS publications (2010–2016). Design Studies, 56:205–224. [4] Beck, J. and Ekbia, H. R. (2018). The Theory-Practice Gap as Generative Metaphor. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, pages 1–11, New York, New York, USA. ACM Press. [5] Bødker, S. (2006). When second wave HCI meets third wave challenges. In Proceedings of the 4th Nordic conference on Human-computer interaction changing roles - NordiCHI ’06, pages 1–8, New York, New York, USA. ACM Press. [6] Bogers, T. and van den Bosch, A. (2008). Recommending scientific articles using citeulike. In Proceedings of the 2008 ACM conference on Recommender systems - RecSys ’08, page 287, New York, New York, USA. ACM Press. [7] Bornmann, L. and Daniel, H.-D. (2008). What Do Citation Measures Count? A Review of Studies on Citing Behavior. Journal of Documentation, 64(1):45–80. [8] Braun, V. and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2):77–101. [9] Brooks, T. A. (1985). Private acts and public objects: An investigation of citer motivations. Journal of the American Society for Information Science, 36(4):223–229. [10] Carroll, J. M. (2003). Introduction: Towards a Multidisciplinary Science of HumanComputer Interaction. In Carroll, J. M., editor, HCI Models, Theories, and Frameworks: Toward a Multidisciplinary Science, chapter 1, pages 1–10. Morgan Kaufmann, San Francisco, CA, USA. [11] Carroll, J. M. (2006). Soft Versus Hard: The Essential Tension. In Galetta, D. and Zhang, P., editors, Human-Computer Interaction in Management Information Systems, pages 424–432. M.E. Sharpe, Inc., Armonk, NY, USA. [12] Carroll, J. M. (2010). Conceptualizing a possible discipline of human-computer interaction. Interacting with Computers, 22(1):3–12. [13] Case, D. O. and Higgins, G. M. (2000). How can we investigate citation behavior? A study of reasons for citing literature in communication. Journal of the American Society for Information Science, 51(7). [14] Case, D. O. and Miller, J. B. (2011). Do bibliometricians cite differently from other scholars? Journal of the American Society for Information Science and Technology, 62(3):n/a– n/a. SocArXiv 15 [15] Chai, K.-H. and Xiao, X. (2012). Understanding design research: A bibliometric analysis of Design Studies (1996-2010). Design Studies, 33(1):24–43. [16] Cronin, B. (1984). The Citation Process: The Role and Significance of Citations in Scientific Communication. Taylor Graham, London. [17] Garfield, E. (1955). Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas. Science, 122(3159):108–111. [18] Garfield, E. (1965). Can Citation Indexing Be Automated. In Stevens, M. E., Giuliano, V. E., and Heilprin, L. B., editors, Statistical Association Methods for Mechanized Documentation: Symposium Proceedings, pages 189–192, Washington D.C. National Bureau of Standards. [19] Garfield, E. (1996). When to Cite. Library Quarterly, 66(4):449–458. [20] Gaver, W. and Höök, K. (2017a). In search of the elusive CHI design paper. interactions, 24(2):22–23. [21] Gaver, W. and Höök, K. (2017b). What makes a good CHI design paper? interactions, 24(3):20–21. [22] Harwood, N. (2009). An interview-based study of the functions of citations in academic writing across two disciplines. Journal of Pragmatics, 41(3):497–518. [23] Hodges, T. L. (1972). Citation Indexing: Its Potential for Bibliographical Control. PhD thesis, University of California Berkeley. [24] Höök, K., Bardzell, J., Bowen, S., Dalsgaard, P., Reeves, S., and Waern, A. (2015). Framing IxD knowledge. interactions, 22(6):32–36. [25] Kaplan, N. (1965). The norms of citation behavior: Prolegomena to the footnote. American Documentation, 16(3):179–184. [26] Kapseon, K. (2004). The motivation for citing specific references by social scientists in Korea: The phenomenon of co-existing references. Scientometrics, 59(1):79–93. [27] Larivière, V., Sugimoto, C. R., and Bergeron, P. (2013). In their own image? a comparison of doctoral students’ and faculty members’ referencing behavior. Journal of the American Society for Information Science and Technology, 64(5):1045–1054. [28] Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Harvard University Press. [29] Liu, Y., Goncalves, J., Ferreira, D., Xiao, B., Hosio, S., Kostakos, V., Liu, Y., Goncalves, J., Ferreira, D., Xiao, B., Hosio, S., and Kostakos, V. (2014). CHI 1994-2013. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, pages 3553–3562, New York, New York, USA. ACM Press. [30] Lu, C., Zhang, C., and Ma, S. (2015). How Does Citing Behavior for a Scientific Article Change over Time?: A Preliminary Study. In Proceedings of the 78th ASIS&T Annual Meeting: Information Science with Impact: Research in and for the Community, ASIST ’15, pages 97:1–4, Silver Springs, MD, USA. American Society for Information Science. 16 Citation Motivations in HCI [31] Luukkonen, T. (1997). Why has Latour’s theory of citations been ignored by the bibliometric community? discussion of sociological interpretations of citation analysis. Scientometrics, 38(1):27–37. [32] MacKenzie, I. S. (2009). Citedness, uncitedness, and the murky world between. In Proceedings of the 27th international conference extended abstracts on Human factors in computing systems - CHI EA ’09, page 2545, New York, New York, USA. ACM Press. [33] Marshall, J. and Linehan, C. (2017). Misrepresentation of Health Research in Exertion Games Literature. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI ’17, pages 4899–4910, New York, New York, USA. ACM Press. [34] Marshall, J., Linehan, C., Spence, J., and Rennick Egglestone, S. (2017a). Throwaway Citation of Prior Work Creates Risk of Bad HCI Research. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17, pages 827–836, New York, New York, USA. ACM Press. [35] Marshall, J., Linehan, C., Spence, J. C., and Rennick Egglestone, S. (2017b). A Little Respect. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17, pages 848–857, New York, New York, USA. ACM Press. [36] Merton, R. K. (1957). Priorities in Scientific Discovery: A Chapter in the Sociology of Science. American Sociological Review, 22(6):635. [37] Moravcsik, M. J. and Murugesan, P. (1975). Some Results on the Function and Quality of Citations. Social Studies of Science, 5:86–92. [38] Nacke, L. E. (2018). How to Write CHI Papers. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18, pages 1–4, New York, New York, USA. ACM Press. [39] Peritz, B. C. (1983). A classification of citation roles for the social sciences and related fields. Scientometrics, 5(5):303–312. [40] Reeves, S. (2015). Locating the ’big hole’ in HCI research. interactions, 22(4):53–56. [41] Rogers, Y. (2012). HCI Theory: Classical, Modern, and Contemporary. Synthesis Lectures on Human-Centered Informatics, 5(2):1–129. [42] Small, H. (1982). Citation Context Analysis. In Dervin, B. and Voigt, M., editors, Progress in Communication Sciences, pages 287–310. Ablex, Norwood, NJ, USA. [43] Tausczik, Y. R. and R., Y. (2016). Citation and Attribution in Open Science: A Case Study. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing - CSCW ’16, pages 1522–1532, New York, New York, USA. ACM Press. [44] Teufel, S., Siddharthan, A., and Tidhar, D. (2006). An Annotation Scheme for Citation Function. In Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue, SigDIAL ’06, pages 80–87, Stroudsburg, PA, USA. Association for Computational Linguistics. 17 SocArXiv [45] Vargas, S., Hristakeva, M., and Jack, K. (2016). Mendeley. In Proceedings of the 10th ACM Conference on Recommender Systems - RecSys ’16, pages 365–365, New York, New York, USA. ACM Press. [46] Vinkler, P. (1998). Comparative investigation of frequency and strength of motives toward referencing. The reference threshold model. Scientometrics, 43(1):107–127. [47] White, H. D. (2001). Authors as citers over time. Journal of the American Society for Information Science and Technology, 52(2):87–108. [48] White, H. D. (2004a). Citation Analysis and Discourse Analysis Revisited. Applied Linguistics, 25(1):89–116. [49] White, H. D. (2004b). Reward, persuasion, and the Sokal Hoax: A study in citation identities. Scientometrics, 60(1):93–120. [50] White, M. D. and Wang, P. (1997). A Qualitative Study of Citing Behavior: Contributions, Criteria, and Metalevel Documentation Concerns. 67:122–154. [51] Zimmerman, J., Stolterman, E., and Forlizzi, J. (2010). An analysis and critique of Research through Design: Toward a Formalization of a Research Approach. In Proceedings of the 8th ACM Conference on Designing Interactive Systems - DIS ’10, page 310, New York, New York, USA. ACM Press. [52] Zuckerman, H. (1988). The Sociology of Science. In Smelser, N., editor, Handbook of Sociology, pages 511–574. Sage, Newbury Park. Affiliation: Jordan Beck The Pennsylvania State University University Park, PA E-mail: jeb560@ist.psu.edu URL: https://designinquiry.me SocArXiv Website SocArXiv Preprints Preprint https://doi.org/10.31235/osf.io/me8zd https://socopen.org/ https://osf.io/preprints/socarxiv Submitted: November 2, 2018 Accepted: November 2, 2018