Skip to main content

Literature Reviewing: Critically Appraise

What is Critical Appraisal

What is critical appraisal? Why do we do it?

Retrieved all pertinent research papers from your literature search?  Critical  appraisal will identify strengths and weaknesses in what you have found. Authors may exaggerate findings or there may be methodological flaws in the research.  Critical appraisal lets you make informed decisions about the quality of the research evidence.

Critical appraisal is often carried out using checklists that help signpost areas to look for while reading a paper. There are different types of checklist depending on the type of research you are reviewing.

What to read first?

Overwhelmed by the amount of literature retrieved? Don’t know which papers to read first?  Are there any recent review (summary) papers to start with? Look at recent papers from good quality journals first – hopefully they summarise earlier research and will help in identifying key  works/authors in your field. Be ruthless – apply your  inclusion/exclusion  search criteria to your evaluation of the papers to cut down on quantity.  Take interesting looking papers– check the importance of the paper by running the title through Google Scholar – how many citations does it have? That might indicate its importance, or controversy within your field.  Use the Library’s inter library loan system to get hold of books or papers your institution has no access to.

 

Critical Appraisal Tools / Checklists

These tools are mainly used in the health field, but may provide useful background information on how to evaluate various research designs. 

CASP - eight critical appraisal tools  for use when reading different types of research. The most well known list from the Critical Appraisal Skills Programme.

SIGN six lists from the Scottish Intercollegiate Guidelines Network

CEBM four lists from Oxford's Centre for Evidence Based Medicine

Cardiff University - Critical Appraisal Checklists

 

Data Collection and Thematic Analysis

Keep a track of your reading and identify themes from the literature using a data extraction table

Study

Aims /study design

Sample & Intervention

Findings

Themes

Saeterdal (2014)

Systematic lit review – community knowledge of child vaccination. RCTs.

interventions to inform and/or educate  versus routine practices

improved knowledge of vaccines, increased rates

Attitudinal change.

Mother involvement

Kaufman (2013)

Systematic lit review –knowledge of child vaccination. RCTs

Face to face interventions to inform & educate

Little impact on immunisation status, or knowledge of vaccination

Communication and timeliness of vaccination information provision.

 

This is just one example.  Books on literature reviewing can suggest other table types to use for synthesis.

 

Reporting standards - use when writing up a research study

Equator Network - guidelines to promote transparent and accurate reporting  of health research

  • PRISMA for systematic reviews
  • CONSORT for randomised controlled trials
  • STROBE for observational studies
  • PRESS EBC Checklist for Peer Review of Electronic Search Strategies

 

Videos

Two excellent videos from Andrew Booth at SCHARR at the University of Sheffield. These take you through the actual process of appraising health research papers using the CASP tool.

Appraising a Quantitative Study             [13 mins]

Critical Appraisal of a Qualitative Study  [12 mins]

Resources - Critical Appraisal