Formulating a review question is a key stage of the review process as this impacts the development of the outcomes of the review, the eligibility criteria for selection, and the development of the search strategy. If you make changes to your review question after already moving on to other stages of the review you may need to go back and make changes to these other steps.
Ideally a review should add new knowledge to that topic or field, so you want to develop a question that has a new focus or outcomes that has not previously been explored. Sometimes it is appropriate to update a previous review using the same question and outcomes to see if the findings of the review have changed with the inclusion of new literature since the previous one was published.
If you are a Masters student it is particularly important that you choose a topic that is both viable and manageable within the word count and timescales for completion. Viable means a topic where there is published literature, you cannot do a literature review on a question where there is no available literature. Manageable means selecting a focused topic where there will not be too vast an amount of literature to include as you have a word count limit and a timescale in which to submit the assignment.
To help you develop a question try and identify an area from practice that you are interested in – ideally something the practice area can benefit from which will give value to your review findings.
The question you develop from this topic should be focused, manageable and answerable within the timescales you have.
Scoping the Literature
This is where we run initial literature searches around our topic of interest to get an initial idea of what literature is out there. This will help us to:
From these initial searches of the literature you can start to refine your review question, broadening or focusing as necessary. Please see the following video for an example of how this works in practice.
Question Formulation Frameworks
Question formulation frameworks are used particularly within healthcare to help you identify the key concepts of your topic, to then structure into a research or review question. The following document shows you examples of the most commonly used ones in healthcare, breaking down each framework into what the concepts mean, giving examples in practice of questions structured using that framework, and suggestions of review outcomes and types best suited to each framework.
What is critical appraisal?
Critical appraisal/quality assessment is a specific aspect of critical analysis where you examine and assess research in order to judge its:
You are evaluating the quality of the research and how it has been conducted, as well as the findings themselves and how it has been reported. Please see the following video by Cochrane on an Introduction to Critical Appraisal for a more in depth description.
Why do we do it?
Critical appraisal is often carried out using checklists that help signpost areas to look for while reading a paper. There are different types of checklist depending on the type of research you are reviewing.
The following document lists some of the main appraisal tools used in published reviews and would be a good place to start when deciding on which tool to use.
Further Critical Appraisal Resources
Book
Greehalgh, T (2019) How to read a paper : the basics of evidence-based medicine. 6th ed. Wiley Blackwell
Videos
Two excellent videos from Andrew Booth at SCHARR at the University of Sheffield. These take you through the actual process of appraising papers using the CASP tool.
Appraising a Quantitative Study [13 mins]
Keeping track of literature
Writing a literature review will mean that you will collect a large number of pieces of information from many sources. Before you begin searching, give some thought as to how you are going to manage this information.
Reference management software will enable you to automatically export references you collect from database searches and store them in the reference manager. Once you have read each paper you can then make personal research notes and store these within each reference inside the reference manager.
Use the software to format the citations within the text of your review. It will also produce the reference list at the end of your document formatted in a style of your choosing e.g. APA 7th.
See our Reference Management LibGuide on how to get started with Endnote or Mendeley, Edinburgh Napier’s referencing management software.
NHS Scotland users can also use the Refworks ref management software supplied on the NHS Knowledge network site instead of Endnote, if they would prefer.
When searching in databases most of the time you want to use the advanced search feature to build a search that will find a more relevant set of search results. To do this you need to be able to plan effective search strategies, using appropriate keyword search terms, and inputting these into the database in the most effective combination.
The videos below demonstrate how to input a planned systematic search strategy into a database. Different database platforms will look slightly different, but the principles for doing an advanced search are the same across them all, but differences are demonstrated. These videos are a starting point and aim to cover EBSCO in the most depth, with the remaining videos covering the key differences in adapting a search to the other platforms. For more support with using databases each database has its own help pages and you can use a video platform such as YouTube to search for videos from other university and health libraries, as well as videos created by the database companies themselves.
Searching in EBSCO databases (CINAHL, Medline, APA PsycInfo etc.) (video, 20 min)
Searching in PubMed (video, 7 min)
Searching in Ovid (video, 7 min)
Searching in Web of Science (video, 3 min)
Searching in Proquest (video, 6 min)
As well as searching in databases it is good practice when undertaking a literature review to use other search methods to find things that you will not find in the databases. Not all journals are indexed in databases so sometimes you will never find certain articles if you only search in databases. The slides and recording below cover a range of other methods for searching beyond databases including via Google Scholar, uses references for citation tracking, searching by author in Web of Science, and more.
Recording of a workshop on Searching beyond Databases (video, 37 min)
The selection process is where you will use your eligibility criteria to select the literature for inclusion in your review. Considerations needed are:
The eligibility criteria can also be referred to as the inclusion and exclusion criteria. This is a set of criteria you will develop which you will use during the selection process of the review to decide which sources of literature to include and exclude. This criteria helps to reduce selection bias, because every decision you make should be based on this pre-determined set of criteria.
When take a systematic approach to searching and selecting the literature your eligibility criteria needs to be very detailed, both for you to be able to make decisions for each of the pieces of literature you have found, but also for someone else to be able to use the criteria with the same set of literature and make the same decisions as you. If you are doing a review as part of a review team for publication then there should be a minimum of two people involved in the selection of the literature, both using the same criteria to make selection decisions. This aligns to the systematic criteria of transparency.
When developing your eligibility criteria think about the following elements:
E.g. your population group is people with dementia, so as inclusion criteria you would state that each literature source needs this population group and any source without this population group would be excluded. But what about literature where participants and both people with dementia and people with Parkinsons. Would this be included or excluded? Your criteria needs to be detailed enough to capture all of the potential decisions you would need to make.
E.g. you're question is exploring the experiences and views of a particular group of participants, therefore the data most appropriate to 'answer' this question would be qualitative.
E.g. there has been a new guideline in your topic area published in a specific year with major changes to how a specific procedure is done in practice, meaning that older literature is not relevant to the current guideline. Topics related to technology could be outdated more easily due to specific technological developments in a specific field or equipment.
A search strategy includes where and how you are searching. Can someone else use your process to find what you found? This aligns with the systematic criteria of being transparent.
You need to plan and include the following detail in your write up to allow someone else to replicate your search:
-Planning Search Strategies (video, 7 min)
Data Extraction and Charting
Your literature findings need to be presented and discussed both descriptively and analytically. It is usually to present a summary of the included sources in the form of a data extraction or study characteristics table, a process also referred to as data extraction and charting your results. The video below covers how to present your findings in this way.
Analysing and Synthesising the Findings of the Literature
Depending on the type of review you are doing and also whether the review is being done as an assignment, there may be differing expectations of how you analyse the included literature sources.
At Masters dissertation level you would be expected as a minimum to provide a narrative thematic analysis, where you compare and contrast the literature to identify patterns and themes and interpret these in relation to your review question. You can use a deductive approach where you start with a pre-existing framework of themes, or an inductive approach where themes are generated from reading the literature.
At PhD or researcher for publication level there would be an expectation of a more complex analysis of the literature, appropriate to the literature sources. A scoping review including a wide range of source types would likely best be suited to a narrative analysis, but if the review literature is all research then an appropriate quantitative, qualitative, or mixed methods form of analysis of the data would be expected.
Most Systematic Review conduction and reporting guidelines are designed around an analysis of quantitative data, so if this does not fit the data of your literature you may need to use different analysis and synthesis guidance. There are a number of different analysis methods, some examples and resources are listed below as a starting point but you may also want to look at examples of similar reviews fur further methods.