Poor citation practices undermine scientific credibility, so apply these principles

Research papers in academic journals refer to other information, typically other journal papers, and citations are the way of doing this. But are citations being done appropriately and correctly?

The case of global jellyfish populations

An article in The Conversation looks at citation practices in regard to the perception that there has been a global increase in jellyfish blooms. Such emerging patterns and trends in nature are usually reported in narrative reviews, and eventually the purported patterns will be either confirmed as theory or refuted through meta-analyses of the published literature. A network of citations will trace the path from primary observation to theory.

In the case of global jellyfish populations, reports of increased populations in some areas started to appear in the 1990s and 2000s, and reviews discussed the possibility that populations might be increasing globally. The conclusions of the reviews became increasingly strong over time, asserting that global oceans were awash with jellyfish. Two meta-analyses were conducted to determine if the global jellyfish boom was real or not, and they drew different conclusions.

Despite the scientific community being divided in regard to a global increase in jellyfish populations, there is a perception that such an increase has occurred. The article discusses research by Sanz-Martín et al.¹ that explored if poor citation practices had contributed to this perception.

The article states that citations can be flawed in several ways:

• Unsupported citations are when authors cite sources that contain no evidence that could possibly support the author’s claim.
• Selective citations happen when a paper is cited to support a claim but contrasting evidence provided in the same paper is ignored, or when authors choose to cite earlier papers that have since been refuted.
• Ambiguous citations happen when an author’s sentence contains multiple phrases, but the citations used to support each phrase are clustered at the end of the sentence, preventing readers from telling which is which.
• Empty citations are when authors cite a paper that cites another paper as evidence for the claim, rather than the original source (also called “lazy author syndrome”).

Sanz-Martín et al. searched for papers published before the meta-analyses, and classified the statements in these papers in regard to whether they said jellyfish are “increasing”, “may be increasing”, “decreasing”, or “not sure”, as well as their geographic extent.

They found that:

• 10% of papers said the data were equivocal
• Only 51% of papers cited were considered to provide unambiguous support for the statements made by the authors
• Almost all statements based on unsupportive citations were those claiming that jellyfish were increasing globally
• In all cases, selective citations were biased towards claims that jellyfish populations were increasing.

Occurrence of incorrect citations in ecology research

The findings of Sanz-Martín et al. align with the findings of research quoted in a blog post by Caroline Tucker. This research considers flawed citations in ecology, the same field of research as the jellyfish studies.

Research by Todd et al.² found that:

…the original assertion was “clearly supported” by the citation in 76.1% of the cases; the support was “ambiguous” in 11.1% of the cases; and the citation did “not support” the original statement in 7.2% of the cases. The remaining 5.6% of the cases were classified as “empty”.

Research by Teixeira et al.³ found that:

Reviews were significantly more often cited than regular articles. In addition, 22% of citations were inaccurate, and another 15% unfairly gave credit to the review authors for other scientists’ ideas.

Unethical journals have contributed to the situation by telling academics to pad their papers with citations.

Citation principles

Sanz-Martín et al., Caroline Tucker, Todd et al. and Teixeira et al. all recommend improved citation practices.

In a book chapter, West and Stenius⁴ propose the following citation principles. West and Stenius recommend that the principles be applied only to informal review documents because the principles for formal reviews are well-documented. However, the findings of the research conducted by Sanz-Martín et al., Todd et al. and Teixeira et al. suggests that the principles should be applied to all reviews.

The citation principles are:

• All non-obvious, substantive claims should be supported by citation or direct evidence
• If there is an authoritative review on a well-supported statement, this may be used in place of original papers
• Where original papers are cited, the discovery paper should be cited together with a small number of other papers that illustrate the generality of the phenomenon
• Authors should resist the propensity to:
  • prefer citations from their own country of origin unless the finding in question is country-specific
  • prefer citations from themselves and colleagues
  • limit citations to those that support a contention, when in fact there are others that conflict with it
  • cite output which is readily retrievable if there are more appropriate references
  • provide an unnecessarily large number of citations for a single statement
• Authors should where possible avoid citing inaccessible sources
• When using citations in support of substantive statements one should either use references that have been through some kind of peer review process or provide an appropriate caveat.

References:

Also published on Medium.