research-article

Open access

Towards Knowledge Maintenance in Scientific Digital Libraries with the Keystone Framework

Authors:

Jodi SchneiderAuthors Info & Claims

JCDL '20: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020

Pages 217 - 226

https://doi.org/10.1145/3383583.3398514

Published: 01 August 2020 Publication History

Abstract

Scientific digital libraries speed dissemination of scientific publications, but also the propagation of invalid or unreliable knowledge. Although many papers with known validity problems are highly cited, no auditing process is currently available to determine whether a citing paper's findings fundamentally depend on invalid or unreliable knowledge. To address this, we introduce a new framework, the keystone framework, designed to identify when and how citing unreliable findings impacts a paper, using argumentation theory and citation context analysis. Through two pilot case studies, we demonstrate how the keystone framework can be applied to knowledge maintenance tasks for digital libraries, including addressing citations of a non-reproducible paper and identifying statements most needing validation in a high-impact paper. We identify roles for librarians, database maintainers, knowledgebase curators, and research software engineers in applying the framework to scientific digital libraries.

References

[1]

Alvarenga, E.S., Santos, J.O., Moraes, F.C. and Carneiro, V.M.T. 2019. Quantum mechanical approach for structure elucidation of novel halogenated sesquiterpene lactones. Journal of Molecular Structure. 1180, (2019), 41--47.

[2]

Avenell, A., Stewart, F., Grey, A., Gamble, G. and Bolland, M. 2019. An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature. BMJ Open. 9, 10 (Oct. 2019), e031909.

[3]

Bar-Ilan, J. and Halevi, G. 2018. Temporal characteristics of retracted articles. Scientometrics. 116, 3 (Jun. 2018), 1771--1783.

Digital Library

[4]

Bender, M. 2019. A code glitch may have caused errors in more than 100 published studies. Vice. https://www.vice.com/en_us/article/zmjwda/a-code-glitch-may-have-caused-errors-in-more-than-100-published-studies

[5]

Bhandari Neupane, J., Neupane, R.P., Luo, Y., Yoshida, W.Y., Sun, R. and Williams, P.G. 2019. Characterization of Leptazolines A--D, polar oxazolines from the cyanobacterium Leptolyngbya sp., reveals a glitch with the "Willoughby--Hoye" scripts for calculating NMR chemical shifts. Organic Letters. 21, 20 (Oct. 2019), 8449--8453.

[6]

Borgman, C.L., Edwards, P.N., Jackson, S.J., Chalmers, M.K. and Bowker, G.C. 2013. Knowledge Infrastructures: Intellectual Frameworks and Research Challenges. https://escholarship.org/uc/item/2mt6j2mh

[7]

Bornmann, L. and Daniel, H. 2008. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation. 64, 1 (Jan. 2008), 45--80.

[8]

Bracegirdle, J., Robertson, L.P., Hume, P.A., Page, M.J., Sharrock, A.V., Ackerley, D.F., Carroll, A.R. and Keyzers, R.A. 2019. Lamellarin sulfates from the Pacific tunicate Didemnum ternerratum. Journal of Natural Products. 82, 7 (Jul. 2019), 2000--2008.

[9]

Burnett, S., Singiser, R. and Clower, C. 2014. Teaching about ethics and the process of science using retracted publications. Journal of College Science Teaching. 043, 3 (2014), 24--29.

[10]

Chan, J., Chang, J.C., Hope, T., Shahaf, D. and Kittur, A. 2018. SOLVENT: A Mixed Initiative System for Finding Analogies between Research Papers. Proceedings of the ACM on Human-Computer Interaction. 2, CSCW (Nov. 2018), 1--21.

[11]

Chandrasekaran, M.K., Yasunaga, M., Radev, D., Freitag, D. and Kan, M.-Y. 2019. Overview and results: CL-SciSumm Shared Task 2019. Proceedings of the 4th Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries (BIRNDL 2019) co-located with the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019) (2019), 153--166.

[12]

Clark, T. 2015. Argument graphs: Literature-data integration for robust and reproducible science. First International Workshop on Capturing Scientific Knowledge Collocated with the Eighth International Conference on Knowledge Capture (K-CAP) (Palisades, NY, Jul. 2015), 1--8.

[13]

Clark, T., Ciccarese, P.N. and Goble, C.A. 2014. Micropublications: a semantic model for claims, evidence, arguments and annotations in biomedical communications. Journal of Biomedical Semantics. 5, 1 (2014), 28.

[14]

De Calignon, A., Polydoro, M., Suárez-Calvet, M., William, C., Adamowicz, D.H., Kopeikina, K.J., Pitstick, R., Sahara, N., Ashe, K.H. and Carlson, G.A. 2012. Propagation of tau pathology in a model of early Alzheimer's disease. Neuron. 73, 4 (2012), 685--697.

[15]

Dinh, L., Sarol, J., Cheng, Y.-Y., Hsiao, T.-K., Parulian, N. and Schneider, J. 2019. Systematic examination of pre- and post-retraction citations. Proceedings of the Association for Information Science and Technology (2019), 390--394.

[16]

Eemeren, F.H. van, Garssen, B., Krabbe, E.C.W., Snoeck Henkemans, A.F., Verheij, B. and Wagemans, J.H.M. 2014. Handbook of argumentation theory. Springer Reference.

[17]

Elkin, M., Scruse, A.C., Turlik, A. and Newhouse, T.R. 2019. Computational and synthetic investigation of cationic rearrangement in the putative biosynthesis of justicane triterpenoids. Angewandte Chemie International Edition. 58, 4 (2019), 1025--1029.

[18]

Erikson, M.G. and Erlandson, P. 2014. A taxonomy of motives to cite. Social Studies of Science. 44, 4 (Aug. 2014), 625--637.

[19]

Freeman, J.B. 2011. Argument Structure: Representation and Theory. Springer.

[20]

Garfield, E. 1965. Can citation indexing be automated. Statistical Association Methods for Mechanized Documentation, Symposium Proceedings (1965), 189--192.

[21]

Grabmair, M., Ashley, K.D., Chen, R., Sureshkumar, P., Wang, C., Nyberg, E. and Walker, V.R. 2015. Introducing LUIMA: An experiment in legal conceptual retrieval of vaccine injury decisions using a UIMA type system and tools. Proceedings of the 15th International Conference on Artificial Intelligence and Law (New York, NY, USA, 2015), 69--78.

[22]

Gray, R., Al-Ghareeb, A., Davis, J., McKenna, L. and Amichai Hillel, S. 2018. Inclusion of nursing trials in systematic reviews after they have been retracted: Does it happen and what should we do? International Journal of Nursing Studies. 79, (Mar. 2018), 154.

[23]

Green, N.L. 2018. Towards mining scientific discourse using argumentation schemes. Argument & Computation. 9, 2 (Jul. 2018), 121--135.

[24]

Gross, C.G. 2009. Three before their time: neuroscientists whose ideas were ignored by their contemporaries. Experimental Brain Research. 192, 3 (Jan. 2009), 321--334.

[25]

Guillen, P.O., Jaramillo, K.B., Jennings, L., Genta-Jouve, G., de la Cruz, M., Cautain, B., Reyes, F., Rodríguez, J. and Thomas, O.P. 2019. Halogenated tyrosine derivatives from the Tropical Eastern Pacific zoantharians Antipathozoanthus hickmani and Parazoanthus darwini. Journal of Natural Products. 82, 5 (May 2019), 1354--1360.

[26]

Hoang, L., Boyce, R.D., Brochhausen, M., Utecht, J. and Schneider, J. 2019. A proposal for determining the evidence types of biomedical documents using a drug-drug interaction ontology and machine learning. Proceedings of the AAAI 2019 Spring Symposium on Combining Machine Learning with Knowledge Engineering (AAAI-MAKE 2019) (2019), 1--2. http://ceur-ws.org/Vol-2350/xposter3.pdf

[27]

Ioannidis, J.P.A. 2005. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 294, 2 (Jul. 2005), 218--228.

[28]

Kasza, P., Trybula, M.E., Baradziej, K., Kepczynski, M., Szafra'ski, P.W. and Ceg?a, M.T. 2019. Fluorescent triazolyl spirooxazolidines: Synthesis and NMR stereochemical studies. Journal of Molecular Structure. 1183, (May 2019), 157--167.

[29]

Keseler, I.M., Skrzypek, M., Weerasinghe, D., Chen, A.Y., Fulcher, C., Li, G.-W., Lemmer, K.C., Mladinich, K.M., Chow, E.D. and Sherlock, G. 2014. Curation accuracy of model organism databases. Database. 2014, (2014), bau058.

[30]

Kircz, J.G. 1991. Rhetorical structure of scientific articles: the case for argumentational analysis in information retrieval. Journal of Documentation. 47, 4 (Apr. 1991), 354--372.

[31]

Moravcsik, M.J. and Murugesan, P. 1975. Some results on the function and quality of citations. Social Studies of Science. 5, 1 (Feb. 1975), 86--92.

[32]

Munafò, M.R. and Davey Smith, G. 2018. Robust research needs many lines of evidence. Nature. 553, 7689 (Jan. 2018), 399--401.

[33]

Neupane, R., Parrish, S.M., Bhandari Neupane, J., Yoshida, Wesley Y., Yip, M.L.R., Turkson, J., Harper, M.K., Head, J.D. and Williams, P.G. Cytotoxic sesquiterpenoid quinones and quinols, and an 11-membered heterocycle, kauamide, from the Hawaiian marine sponge Dactylospongia elegans. Marine Drugs. 17, 7, 423.

[34]

Sandhu, N. and Schneider, J. 2018. Argument analysis of Alzheimer's Disease. Poster at University of Illinois Undergraduate Research Symposium. https://www.ideals.illinois.edu/handle/2142/106017

[35]

Schneider, J. and Sandhu, N. 2018. Modeling Alzheimer's Disease research claims, evidence, and arguments from a biology research paper. (Jul. 2018). Presentation at the 9th International Conference on Argumentation, International Society for the Society of Argumentation, Amsterdam, Netherlands, https://www.ideals.illinois.edu/handle/2142/100340

[36]

Schneider, J., Yi, D., Hill, A.M. and Whitehorn, A.S. 2020. Continued post-retraction citation of a fraudulent clinical trial report, eleven years after it was retracted for falsifying data. Under submission to Scientometrics Special Issue on "Bibliometrics and Information Retrieval." (2020).

[37]

Schrag, M., Mueller, C., Oyoyo, U., Smith, M.A. and Kirsch, W.M. 2011. Iron, zinc and copper in the Alzheimer's disease brain: A quantitative meta-analysis. Some insight on the influence of citation bias on scientific opinion. Progress in Neurobiology. 94, 3 (Aug. 2011), 296--306.

[38]

Small, H. 2018. Characterizing highly cited method and non-method papers using citation contexts: The role of uncertainty. Journal of Informetrics. 12, 2 (May 2018), 461--480.

[39]

Small, H. 1982. Citation context analysis. Progress in Communication Sciences. 3, (1982), 287--310.

[40]

Spaltenstein, P., Cummins, E.J., Yokuda, K.-M., Kowalczyk, T., Clark, T.B. and O'Neil, G.W. 2019. Chemoselective carbonyl allylations with alkoxyallylsiletanes. The Journal of Organic Chemistry. 84, 7 (Apr. 2019), 4421--4428.

[41]

Stede, M., Schneider, J. and Stede, G. 2019. Modeling Arguments [Chapter 3]. Argumentation mining. Morgan & Claypool. 27--43.

[42]

Suelzer, E.M., Deal, J., Hanus, K.L., Ruggeri, B., Sieracki, R. and Witkowski, E. 2019. Assessment of citations of the retracted article by Wakefield et al with fraudulent claims of an association between vaccination and autism. JAMA Network Open. 2, 11 (Nov. 2019), e1915552.

[43]

Teufel, S. 1999. Argumentative Zoning: Information Extraction from Scientific Text. University of Edinburgh. https://www.cl.cam.ac.uk/~sht25/thesis/t.pdf

[44]

Teufel, S. and Kan, M.-Y. 2011. Robust argumentative zoning for sensemaking in scholarly documents. Advanced Language Technologies for Digital Libraries (2011), 154--170.

[45]

Valenzuela, M., Ha, V. and Etzioni, O. 2015. Identifying meaningful citations. Scholarly Big Data: AI Perspectives, Challenges, and Ideas: Papers from the 2015 AAAI Workshop (Apr. 2015), 21--26.

[46]

Wager, E., Barbour, V., Kleinert, S. and Yentis, S. 2019. COPE Guidelines for retracting articles [2019]. Committee on Publication Ethics. https://publicationethics.org/node/19896

[47]

Walton, D., Reed, C. and Macagno, F. 2008. Argumentation Schemes. Cambridge University Press.

[48]

Weitz, D. 2017. Report of the NSF workshop on Robustness, Reliability, and Reproducibility in Science Research. http://www.mrsec.harvard.edu/2017NSFReliability/include/NSF_Workshop_Robustness.Reliability.Reproducibility.Report.pdf

[49]

White, P.F., Kehlet, H. and Liu, S. 2009. Perioperative analgesia: What do we still know?: Anesthesia & Analgesia. 108, 5 (May 2009), 1364--1367.

[50]

Xu, H.-C., Hu, K., Sun, H.-D. and Puno, P.-T. 2019. Four 14(13-12)-abeolanostane triterpenoids with 6/6/5/6-fused ring system from the roots of Kadsura coccinea. Natural Products and Bioprospecting. 9, 3 (Jun. 2019), 165--173.

[51]

Zhu, J.S., Li, C.J., Tsui, K.Y., Kraemer, N., Son, J.-H., Haddadin, M.J., Tantillo, D.J. and Kurth, M.J. 2019. Accessing multiple classes of 2H-indazoles: Mechanistic implications for the Cadogan and Davis--Beirut reactions. Journal of the American Chemical Society. 141, 15 (Apr. 2019), 6247--6253.

[52]

Zhu, X., Turney, P., Lemire, D. and Vellino, A. 2015. Measuring academic influence: Not all citations are equal. Journal of the Association for Information Science and Technology. 66, 2 (2015), 408--427.

Digital Library

[53]

Zou, Y., Wang, X., Sims, J., Wang, B., Pandey, P., Welsh, C.L., Stone, R.P., Avery, M.A., Doerksen, R.J., Ferreira, D., Anklin, C., Valeriote, F.A., Kelly, M. and Hamann, M.T. 2019. Computationally assisted discovery and assignment of a highly strained and PANC-1 selective alkaloid from Alaska's deep ocean. Journal of the American Chemical Society. 141, 10 (Mar. 2019), 4338--4344.

Cited By

Usman MBalke W(2024)Tracing the Retraction Cascade: Identifying Non-retracted but Potentially Retractable ArticlesLinking Theory and Practice of Digital Libraries10.1007/978-3-031-72437-4_7(109-126)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-72437-4_7
Hoppe TArabi SHutchins B(2023)Predicting substantive biomedical citations without full textProceedings of the National Academy of Sciences10.1073/pnas.2213697120120:30Online publication date: 18-Jul-2023
https://doi.org/10.1073/pnas.2213697120
Usman MBalke W(2023)On Retraction Cascade? Citation Intention Analysis as a Quality Control Mechanism in Digital LibrariesLinking Theory and Practice of Digital Libraries10.1007/978-3-031-43849-3_11(117-131)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-43849-3_11
Show More Cited By

Index Terms

Towards Knowledge Maintenance in Scientific Digital Libraries with the Keystone Framework

Recommendations

Preprint citation practice in PLOS
Abstract
The role of preprints in the scientific production and their part in citations have been growing over the past 10 years. In this paper we study preprint citations in several different aspects: the progression of preprint citations over time, their ...
Towards a Social Trust Based Measure of Scientific Productivity
ASONAM '17: Proceedings of the 2017 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2017

Quantifying scientific productivity has traditionally been one of the major areas of research. An important component in measuring the scientific productivity of a researcher has been the number and citation count of his publications. However, that ...
An analysis of the scientific activities of researchers from Belarus according to bibliometric indicators

This paper considers the results of scientific research conducted by the Central Scientific Library of the National Academy of Sciences of Belarus (CSL NAS of Belarus) using the bibliometric indicators of the Web of Science Core Collection Database (WoS ...

Comments

Information & Contributors

Information

Published In

JCDL '20: Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020

August 2020

611 pages

ISBN:9781450375856

DOI:10.1145/3383583

General Chairs:
Ruhua Huang
Wuhan University, China
,
Dan Wu
Wuhan University, China
,
Gary Marchionini
University of North Carolina at Chapel Hill, USA
,
Program Chairs:
Daqing He
University of Pittsburgh, USA
,
Sally Jo Cunningham
University of Waikato, New Zealand
,
Preben Hansen
Stockholm University, Sweden

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
SIGIR: ACM Special Interest Group on Information Retrieval
IEEE: Institute of Electrical and Electronics Engineers

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 August 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

JCDL '20

Sponsor:

JCDL '20: The ACM/IEEE Joint Conference on Digital Libraries in 2020

August 1 - 5, 2020

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 415 of 1,482 submissions, 28%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
759
Total Downloads

Downloads (Last 12 months)228
Downloads (Last 6 weeks)29

Reflects downloads up to 23 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Usman MBalke W(2024)Tracing the Retraction Cascade: Identifying Non-retracted but Potentially Retractable ArticlesLinking Theory and Practice of Digital Libraries10.1007/978-3-031-72437-4_7(109-126)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-72437-4_7
Hoppe TArabi SHutchins B(2023)Predicting substantive biomedical citations without full textProceedings of the National Academy of Sciences10.1073/pnas.2213697120120:30Online publication date: 18-Jul-2023
https://doi.org/10.1073/pnas.2213697120
Usman MBalke W(2023)On Retraction Cascade? Citation Intention Analysis as a Quality Control Mechanism in Digital LibrariesLinking Theory and Practice of Digital Libraries10.1007/978-3-031-43849-3_11(117-131)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-43849-3_11
Schneider JWoods NProescholdt RBurns HHowell KCampbell MHsiao TYip YFu YArianlou Y(2022)Reducing the Inadvertent Spread of Retracted Science: recommendations from the RISRS reportResearch Integrity and Peer Review10.1186/s41073-022-00125-x7:1Online publication date: 19-Sep-2022
https://doi.org/10.1186/s41073-022-00125-x
Balke WKroll HFu YSchneider Jde Waard AAizawa AMandl TCarevic ZHinze AMayr PSchaer P(2022)2nd Workshop on Digital Infrastructures for Scholarly Content Objects (DISCO'22)Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries10.1145/3529372.3530943(1-2)Online publication date: 20-Jun-2022
https://dl.acm.org/doi/10.1145/3529372.3530943
Avenell ABolland MGamble GGrey A(2022)A randomized trial alerting authors, with or without coauthors or editors, that research they cited in systematic reviews and guidelines has been retractedAccountability in Research10.1080/08989621.2022.208229031:1(14-37)Online publication date: 26-Jun-2022
https://doi.org/10.1080/08989621.2022.2082290
Kroll HMainzer NBalke W(2022)On Dimensions of Plausibility for Narrative Information Access to Digital LibrariesLinking Theory and Practice of Digital Libraries10.1007/978-3-031-16802-4_43(433-441)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-16802-4_43
Addepalli ASubin KSchneider J(2022)Testing the Keystone Framework by Analyzing Positive Citations to Wakefield’s 1998 PaperInformation for a Better World: Shaping the Global Future10.1007/978-3-030-96957-8_9(79-88)Online publication date: 23-Feb-2022
https://doi.org/10.1007/978-3-030-96957-8_9
Fu YSchneider JBlake C(2021)Finding Keystone Citations for Constructing Validity Chains among Research PapersCompanion Proceedings of the Web Conference 202110.1145/3442442.3451368(451-455)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442442.3451368
Schneider Jde Waard ABalke WWang XSong NHua BFu YDownie J(2021)Digital Infrastructures for Scholarly Content ObjectsProceedings of the 2021 ACM/IEEE Joint Conference on Digital Libraries10.1109/JCDL52503.2021.00069(346-347)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1109/JCDL52503.2021.00069
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents

pFad - Phonifier reborn

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.

Alternative Proxies:

Alternative Proxy