Critical Appraisal Tools

For critically appraising guidelines we use the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument, The AGREE Collaboration, Appraisal of Guidelines for Research & Evaluation (AGREE) Instrument, www.agreecollaboration.org.

For systematic reviews and clinical trials, we use the following critical appraisal tools, which are published in: Churchill, R. Critical Appraisal, in Prince, M., Stewart, R., Ford, T., Hotopf, M. (eds) Practical Psychiatric Epidemiology, 2003, Oxford University Press.

Checklist for appraising a systematic review

A. Internal validity of the review

(1) Did the review address a clearly focused question (including the population, interventions and outcome)?
(2) Were a priori inclusion and exclusion criteria explicitly stated (including the population, interventions and outcomes of interest)?
(3) Were appropriate studies included, was a thorough and comprehensive search strategy used and could important studies have been missed?
(4) Were data extracted accurately?
(5) Was the validity (or quality) of the included studies assessed properly?
(6) Was missing information sought?
(7) Was clinical and statistical heterogeneity examined and investigated?
(8) Were sensitivity analyses undertaken to explore the possible introduction of bias in the way the review was conducted?

B. What were the results

(9) What are the overall results in this review (how large was the treatment effect)?
(10) How precise were the results (confidence intervals, p-values)?

Checklist for appraising a clinical trial

A. Internal validity of the trial

(1) Did the trial address a clearly focused question (including the population, interventions and outcome)?
(2) Was the source and type of patients properly described and could the choice of subjects have influenced the outcome?
(3) Was the assignment of patients to the intervention and control group randomised and was the randomisation list concealed?
(4) Were the groups similar at the start of the trial?
(5) Were participants, staff, study personnel and observers ‘blinded’ to group assignment (was the trial single, double or triple blinded and if not, could it have been)?
(6) Aside from the intervention, were the two groups treated equally (were there systematic differences other than those under study in experience of the groups)?
(7) Did the study have adequate power to see an effect if there was one?
(8) What was the attrition rate and were all the patients who entered the trial properly accounted for?

B. What were the results

(9) Were all patients analysed in the groups to which they were randomised (were they analysed by intention to treat)?
(10) What was the effect of treatment (note which outcomes are used to demonstrate this) and how large was this effect?
(11) How precise was the estimate of the treatment effect (confidence intervals, p-values)?

The funding for this service has now ceased so we are unable to accept new questions. Please search our answers database for details of previously answered questions. These answers are not updated, so please note the date each was prepared.

This web site will remain available, but it will no longer be updated.

Disclaimer

Individual health care providers are responsible for assessing the applicability of BEST in MH answers to their clinical practice. BEST in MH is not responsible or liable for, directly or indirectly, any form of damage resulting from the use/misuse of information contained in or implied by these documents.

In this section