• Skip to main content
  • Skip to secondary menu
  • Skip to footer
  • Add Event
  • Directory
  • Submit News

MyZA

Free Events Calendar and Business Directory

  • SA News
  • Contact
  • Submit News
  • Directory
  • Quickly Add Listing

Eight common problems with science literature reviews and how to fix them

1 November 2020 By admin

Researchers regularly review the literature that’s generated by others in their field. This is an integral part of day-to-day research: finding relevant research, reading and digesting the main findings, summarising across papers, and making conclusions about the evidence base as a whole.

However, there is a fundamental difference between brief, narrative approaches to summarising a selection of studies and attempting to reliably, comprehensively summarise an evidence base to support decision-making in policy and practice.

So-called “evidence-informed decision-making” relies on rigorous systematic approaches to synthesising the evidence. Systematic review has become the highest standard of evidence synthesis. It is well established in the pipeline from research to practice in several fields including health, the environment and social policy. Rigorous systematic reviews are vital for decision-making because they help to provide the strongest evidence that a policy is likely to work (or not). They also help to avoid expensive or dangerous mistakes in the choice of policies.

But systematic review has not yet entirely replaced traditional methods of literature review. These traditional reviews may be susceptible to bias and so may end up providing incorrect conclusions. This is especially worrying when reviews address key policy and practice questions.

The good news is that the limitations of traditional literature review approaches could be improved relatively easily with a few key procedures. Some of these are not prohibitively costly in terms of skill, time or resources. That’s particularly important in African contexts, where resource constraints are a daily reality, but should not compromise the continent’s need for rigorous, systematic and transparent evidence to inform policy.

In our recent paper in Nature Ecology and Evolution, we highlighted eight common problems with traditional literature review methods. We gave examples for each problem, drawing from the field of environmental management and ecology. Finally, we outlined practical solutions.

Problems

These are the eight problems we identified in our paper.

First, traditional literature reviews can lack relevance. This is because limited stakeholder engagement can lead to a review that is of limited practical use to decision-makers.

Second, reviews that don’t publish their methods in an a priori (meaning that it is published before the review work begins) protocol may suffer from mission creep. In our paper we give the example of a 2019 review that initially stated it was looking at all population trends among insects. Instead, it ended up focusing only on studies that showed insect population declines. This could have been prevented by publishing and sticking to methods outlined in a protocol.

Third, a lack of transparency and replicability in the review methods may mean that the review cannot be replicated. Replicability is a central tenet of the scientific method.

Selection bias is another common problem. Here, the studies that are included in a literature review are not representative of the evidence base. A lack of comprehensiveness, stemming from an inappropriate search method, can also mean that reviews end up with the wrong evidence for the question at hand.

Traditional reviews may also exclude grey literature. This is defined as any document

produced on all levels of government, academics, business and industry in print and electronic formats, but which is not controlled by commercial publishers, i.e., where publishing is not the primary activity of the producing body.

It includes organisational reports and unpublished theses or other studies. Traditional reviews may also fail to test for evidence of publication bias; both these issues can result in incorrect or misleading conclusions. Another common error is to treat all evidence as equally valid. The reality is that some research studies are more valid than others. This needs to be accounted for in the synthesis.

Inappropriate synthesis is another common issue. This involves methods like vote-counting, which refers to tallying studies based on their statistical significance. Finally, a lack of consistency and error checking (as would happen when a reviewer works alone) can introduce errors and biases if a single reviewer makes decisions without consensus.

All of these common problems can be solved, though. Here’s how.

Solutions

Stakeholders can be identified, mapped and contacted for feedback and inclusion without the need for extensive budgets. Best-practice guidelines for this process already exist.

Researchers can carefully design and publish an a priori protocol that outlines planned methods for searching, screening, data extraction, critical appraisal and synthesis in detail. Organisations like the Collaboration for Environmental Evidence have existing protocols from which people can draw.

Researchers also need to be explicit and use high-quality guidance and standards for review conduct and reporting. Several such standards already exist.

Another useful approach is to carefully design a search strategy with an info specialist; to trial the search strategy against a benchmark list; and to use multiple bibliographic databases, languages and sources of grey literature. Researchers should then publish their search methods in an a priori protocol for peer review.

Researchers should consider carefully planning and trialling a critical appraisal tool before starting the process in full, learning from existing robust critical appraisal tools. Critical appraisal is the carefully planned assessment of all possible risks of bias and possible confounders in a research study. Researchers should select their synthesis method carefully, based on the data analysed. Vote-counting should never be used instead of meta-analysis. Formal methods for narrative synthesis should be used to summarise and describe the evidence base.

Finally, at least two reviewers should screen a subset of the evidence base to ensure consistency and shared understanding of the methods before proceeding. Ideally, reviewers should conduct all decisions separately and then consolidate.

Collaboration

Collaboration is crucial to address the problems with traditional review processes. Authors need to conduct more rigorous reviews. Editors and peer reviewers need to gate-keep more strictly. The community of methodologists needs to better support the broader research community.

Working together, the academic and research community can build and maintain a strong system of rigorous, evidence-informed decision-making in conservation and environmental management – and, ultimately, in other disciplines.

Source: The Conversation





Previous: Coronavirus – Malawi: COVID-19 Daily Information Update (1st November 2020)
Next: Coronavirus: African Union Member States reporting COVID-19 cases as of 1 November 2020, 9 am EAT





Filed Under: Africa Tagged With: mypr

RSS Events

  • Soweto Gets a New Summer Lifestyle Festival as Sugar People & Friends Celebrates Its First Birthday
  • Deftones Setlist: What to Expect From Their 2027 Live Show
  • Hyde Park Corner Brings Out the Stars for This Year’s Spectacular Tree Lighting Celebration!
  • Step Inside: Prudential Center – Home of the New Jersey Devils
  • Exclusive November Imax Lineup Features Iconic Music Legend and a Hollywood Blockbuster
  • Kasi Republic Brings Kwaito, Kasi Cars and Ispantsula Style to Riversands Farm Village This Reconciliation Day
  • Festive Shopping Made Simple at Solly Kramer’s Parkhurst
  • LEDBox Hires Outdoor LED Screen Trailers with On-Site Crew
  • From Busking to the Business of Music: MEX Opens New Doors
  • P!NK Setlist: What to Expect From Her 2027 Live At Sphere Residency

About admin

The one with a "My Complex" - incurable but fun. Grumpy, Cynical, Wise and Idiotic. I can also be found HERE.

Footer

Latest News from Africa

  • Prince Albert – the New King of Jazz
  • Eritrea: Independence Cup in Southern Region
  • President El-Sisi Follows-up on the Evacuation of Egyptian Citizens from Sudan
  • United Arab Emirates welcomes initiation of talks between Sudanese Armed Forces and Rapid Support Forces
  • Ghana’s regions: why creating new territories has caused problems
  • Uganda Red Cross Delivers Relief Aid to Kisoro Floods-Affected Communities
  • President Ramaphosa alarmed over conflict, violence in eastern DRC
  • Mental health: almost half of Johannesburg students in new study screened positive for probable depression
  • Dietary Supplements for Dogs and Cats: Enhancing Well-being
  • Midwives scramble to ensure safe deliveries amid violence in Sudan

RSS MyPR’s Free Press Releases

  • Mphahlele, More crowned 2026 Boxer Super Run Joburg Champions
  • Border Management Conference and Expo 2026 Brings Government, Industry and Regional Partners Together in Pretoria on 18 November
  • Damaged Cars Wanted Pays Gauteng Sellers Before the Tow Truck Leaves
  • RM Capital Explains How It Pays Suppliers Directly to Get Orders Moving
  • Acepak Highlights Tray Forming and Stretch Wrapping for Summer Drinks Runs

© 2026 MyZA · Port Elizabeth, South Africa | Privacy Policy | MyZA content usage