12 questions to help you make sense of cohort study Checklist – Critical Appraisal Skills Programme

(A) Are the results of the study valid?

Screening Questions

1. Did the study address a clearly focused issue?
HINT: A question can be ‘focused’ In terms of

• The population studied
• The risk factors studied
• The outcomes considered
• Is it clear whether the study tried to detect a beneficial
  or harmful effect?

Yes Can’t tell No

2. Was the cohort recruited in an acceptable way?
HINT: Look for selection bias which might compromise the generalisability of the findings:

• Was the cohort representative of a defined population?
• Was there something special about the cohort?
• Was everybody included who should have been included?

Yes Can’t tell No

Is it worth continuing?

Detailed questions

3. Was the exposure accurately measured to minimise bias?
HINT: Look for measurement or classification bias:

• Did they use subjective or objective measurements?
• Do the measurements truly reflect what you want them to (have they been validated)
• Were all the subjects classified into exposure groups using the same procedure

Yes Can’t tell  No

4. Was the outcome accurately measured to minimise bias?
HINT: Look for measurement or classification bias:

• Did they use subjective or objective measurements?
• Do the measures truly reflect what you want them to (have they been validated)?
• Has a reliable system been established for detecting all the cases (for measuring disease
  occurrence)?
• Were the measurement methods similar in the different groups?
• Were the subjects and/or the outcome assessor blinded to exposure (does this matter)?

Yes Can’t tell  No

5. (a) Have the authors identified all important confounding factors?

List the ones you think might be
important, that the author missed.

Yes Can’t tell  No

(b) Have they taken account of the confounding factors in the design and/or analysis?
HINT: Look for restriction in design, and techniques e.g. modelling, stratified-, regression-, or sensitivity analysis to correct, control or adjust for confounding factors

Yes Can’t tell  No

6. (a) Was the follow up of subjects complete enough?

Yes Can’t tell  No

(b) Was the follow up of subjects long enough?
HINT: Consider

• The good or bad effects should have had long enough to reveal themselves
• The persons that are lost to follow-up may have different outcomes than those available for assessment
• In an open or dynamic cohort, was there anything special about the outcome of the people leaving, or the exposure of the people entering the cohort?

Yes Can’t tell  No

(B) What are the results?

7. What are the results of this study?
HINT: Consider

• What are the bottom line results?
• Have they reported the rate or the proportion between the exposed/unexposed, the ratio/the rate difference?
• How strong is the association between exposure and outcome (RR,)?
• What is the absolute risk reduction (ARR)?

Yes Can’t tell No

8. How precise are the results?
HINT: Look for the range of the confidence intervals, if given.

Yes Can’t tell No

9. Do you believe the results?
HINT: Consider

• Big effect is hard to ignore!
• Can it be due to bias, chance or confounding?
• Are the design and methods of this study sufficiently flawed to make the results unreliable?
• Bradford Hills criteria (e.g. time sequence, dose-response gradient, biological plausibility, consistency)

Yes Can’t tell No

(C) Will the results help locally?

10. Can the results be applied to the local population?
HINT: Consider whether

• A cohort study was the appropriate method to answer this question
• The subjects covered in this study could be sufficiently different from your population to cause concern
• Your local setting is likely to differ much from that of the study
• You can quantify the local benefits and harms

Yes Can’t tell No

11. Do the results of this study fit with other available evidence?

Yes Can’t tell No

12. What are the implications of this study for practice?
HINT: Consider

• One observational study rarely provides sufficiently robust
  evidence to recommend changes to clinical practice or
  within health policy decision making
• For certain questions observational studies provide the only
  evidence
• Recommendations from observational studies are always stronger
  when supported by other evidence

Yes Can’t tell No