The big problems in science: what do scientists think, and what are the solutions?

Here in RealKM Magazine, we have a series of articles exploring the big problems in science, such as the reproducibility crisis, the hyping of science findings, and poor science communication.

But what do scientists think? In an article in Vox, Julia Belluz, Brad Plumer, and Brian Resnick discuss their survey of 270 scientists across the world that asked a simple question: “If you could change one thing about how science works today, what would it be and why?”

The responses show that scientists are acutely aware of the big problems in science. Most importantly, they also reveal the serious systemic issues that underlie the problems, and the potential solutions.

The big problems they identified are:

• Academia has a huge money problem
• Too many studies are poorly designed
• Replicating results is crucial — and rare
• Peer review is broken
• Too much science is locked behind paywalls
• Science is poorly communicated
• Life as a young academic is incredibly stressful

Problems and solutions

Academia has a huge money problem

Problem: Obtaining and sustaining enough funding to carry out research is a perennial obstacle. University funding is insufficient and ever-shrinking, which causes researchers to seek outside grants, but these grants are always short-term which discourages long-term research. It takes longer to produce truly novel research, and there’s a higher risk because unconventional research tends to be poorly cited, so scientists opt for short-term “safe” papers. Grant funds are also ever-shrinking, so scientists can then be drawn to industry or interest groups, for example the food industry funds much of nutrition science and drug trials are funded by drug companies. However, “Studies have found that private industry–funded research tends to yield conclusions that are more favorable to the sponsors.” Another issue with grant funding is that chasing grants takes up considerable time, reducing the time available for research.

Solutions: The current funding culture rewards fast, sensational (and potentially wrong) research findings. An obvious solution is for governments to increase research funding, but this alone is not enough – reforms are needed. Science funding programs could be given long-term stability instead of being subjected to short-term political policy whims. Long-term stability could also be facilitated by funding specific people and labs rather than individual projects. Highly competitive grants could also be awarded through random selection from the eligible pool which would help to prevent the jockeying for funding that currently occurs. The conflicts of interest that occur with industry funding could be minimised through industry contributing to a single fund from which grants are independently allocated.

Too many studies are poorly designed

Problem: The pressure to publish pushes scientists to come up with sensational results, but groundbreaking findings don’t occur often so scientists face pressure to game their studies. Scientists report that this can lead to biases in the study design or the way in which data is analysed. Scientists often hype their research results, not always consciously, and one of the ways they do this is through p-hacking. This is where researchers use data mining to test their data against many hypotheses, and only report those with statistically significant results.

Solutions: Two ways to encourage stronger study design and discourage scientists from chasing positive results involve rethinking the rewards system and making the research process more transparent. Rewards could be based on scientific rigour and the advancement of science through idea sharing, rather than research outcomes. To improve transparency, researchers can be encouraged to register their study design and methods on public websites ahead of their research, and to then report their progress on those websites.

Replicating results is crucial – and rare

Problem: Replication is a fundamental concept in science, where research findings are validated through retesting. However, it doesn’t happen as often as it should, and when it is attempted, researchers often find that they can’t reproduce the original findings. For example, “recently, a landmark study published in the journal Science demonstrated that only a fraction of recent findings in top psychology journals could be replicated.” Reasons for the lack of replication include little incentive to do so, with funding agencies preferring to fund new research, reluctance by journals to publish replication studies, and that studies can be difficult to replicate.

Solutions: Researchers should be incentivized to carry out replication studies, and journals should be incentivized to publish negative results. The replication of studies should be made easier through the more robust sharing of research methods in published papers. Data should also be shared more openly, including through the use of new software tools. Scientists could also engage in replication prior to the publication of research findings.

Peer review is broken

Problem: The peer review process is supposed to prevent poor science from being published, but numerous studies and reviews have shown that it doesn’t reliably achieve this. Because of critical time constraints, reviewers often don’t review papers carefully enough, so they fail to detect research fraud or other problems with manuscripts. Because editors and peer reviewers typically know who the authors are, but not the other way around, there can be bias against researchers and institutions, including bullying.

Solution: There are diverse views among scientists about how to overcome the problems with peer review. To reduce bias, some scientists have suggested double-blind peer review, where paper authors can’t see the names of peer reviewers and editors, and peer reviewers and editors can’t see the names of paper authors. Others have suggested the opposite: full openness where everyone can see everyone else’s name. Another approach is collaborative peer review, where editors and reviewers work together on a consolidated list of comments in regard to a paper. Peer review could also be thought of as a continuous process, where studies are repeatedly and transparently updated. Papers could be posted on the internet for comment before being peer-reviewed and published, and/or feedback could be encouraged and facilitated after publication.

Too much science is locked behind paywalls

Problem: Too much science is locked away behind journal paywalls, which limits the public dissemination of scientific research.

Solutions: Many survey respondents urged their peers to publish in open access journals, but career advancement can depend on publishing in prestigious journals that still have paywalls, for example Science or Nature. The question of how a transfer from paywalls to open access can be financed needs to be answered, because the operational costs of journals mean they can never be free. Open access journals typically charge publication fees which are an added burden to researchers who are already starved for funds. A radical solution is to replace for-profit publishing with a non-profit model. Researchers can also take the step of publishing papers or research materials online themselves.

Science is poorly communicated

Problem: “Science journalism is often full of exaggerated, conflicting, or outright misleading claims.” But it’s not only journalists who hype science findings. Sometimes it’s university media departments, with a study finding problems in one-third of university press releases, and sometimes scientists oversell their work.

Solutions: Improving scientific research itself by tackling the problems identified above should help to address the hyping problem. Science journalists could also move away from hyping single studies, and pay more attention to the scientific rigour of studies than sensational findings. Scientists themselves could learn more about communicating effectively with the public, an under-recognised skill, and the public could be better educated in science literacy.

Life as a young academic is incredibly stressful

Problem: Many young academics reported that their life as a scientists was grueling and unrewarding. Early career researchers are often the primary authors on many studies, a requirement for career advancement, and work long hours for relatively low pay. Contracts are often short-term, which discourages rigorous research and means that there is a lack of career security. Women are affected more severely than men, particularly those wanting to raise families. Many scientists choose to leave science because of the stress and pressure.

Solutions: Using early career researchers as a source of cheap labour is related to the funding problems discussed above, so addressing those problems is seen as essential to overcoming the difficulties faced by young academics.

Also published on Medium.