Está en la página 1de 30

How to appraise evidence &

research paper
Presented by
Dr khamis elessi

Evaluate your information
Everything written has at least some bias or point-of-
view. You need to evaluate how much that bias affects
the content of the article.
Who is the author?
Did the author have any authority in what they wrote? What
credentials do they have?
Why was the article written?
Many articles and websites were written to present specific
arguments or theories. Make sure you know if the
information you are using was written for a specific purpose.
Where was it published?
Was it published in a peer-reviewed, scholarly, or otherwise
authoritative journal? Or, merely on someones personal
website?
When was it published?
Obvious, yes. But, make sure that the website you use is
not outdated.
Simple steps for appraisal of research paper
At first: Scan abstract for a few seconds
Briefly assess study design
Briefly assess statistical precision of results
Are the authors conclusions of interest?
Formulate a brief summary
Critically appraise methods section for validity
Critically appraise results section (especially the
tables and figures) for validity & relevance
Draw your own conclusions about applicability

Step 3: Critically appraise the evidence (cont.)

There are 4 issues in critical appraisal of any paper:
Validity: validity is the extent to which the results are
free from bias. It Measures dimensions of occlusion
that are considered clinically important.
Relevance: refers to the extent to which the research
paper matches your needs.
Not all papers are good
Not all papers/thesis are interesting for you
Reliability (Consistency): refers to the extent to which
the results are similar across different analysis in the
study & in agreement with evidence from other studies
Importance & significance of Results ( analyzed in
light of type of Study
General points when appraising Evidence
Critique Requires some knowledge of diff. study types
ASSUMING THAT IT IS A WELL DESIGNED STUDY
Check for appropriate sample size, randomization,,
treatment allocation, stats. used, etc.

Meta-analysis of RCTs > RCT > Cohort > Case Control >
Case Series > Case Report.
Retrospective studies weaker than prospective studies

Critique requires basic knowledge of essential terms of
biostatistics
Sensitivity, specificity, prevalence, likelihood ratios
Absolute risk reduction, relative risk reduction, odds ratios,
number needed to treat, numbers needed to harm.
Critical appraisal of evidence cont.
Check Validity of the evidence
Internal Validity
External Validity
Relevance of the evidence
Did they measure something pts care about?
Is population similar (enough) to mine?
Importance of the evidence
Magnitude of effect or clinical significance?
P values, confidence intervals, relative risk or
absolute risk reduction
Internal Validity

It is the extent to which the observed
difference in outcomes between the two
comparison groups can be attributed to
the intervention rather than other factors.

study design, blinding, randomized, Bias.
sample size, appropriate statistics, etc

Appraisal of Internal Validity
Was assignment of patients to treatments randomised?
Were groups similar at start of trial?
Were groups treated similarly, apart from the
experimental treatment?
Were all participants accounted for in the conclusions?
Were all participants analysed in the groups to which
they were randomised (Intention To Treat analysis)?
Were participants and clinicians kept blind to treatment
received?
Appraise study design
A quality case-
control study is
more meaningful
than a flawed RCT
External Validity
is the validity of generalized (causal) conclusions or
inferences in scientific studies. in simple terms, It is
the degree to which the study conclusions would hold
for other persons in other places and at other times.
Appraisal of External Validity
Do the inclusion and exclusion criteria make sense?
What proportion of the screened population was recruited?
Can the results be reasonably applied to a definable group of
patients in a particular clinical setting?
Where were the participants recruited from (primary care /
referral centre)?
To whom do the results of this trial apply?
Are the results generalizable beyond the trial setting?


Reliability (consistency)
Reliability refers to the consistency of a
measure.
A measure is said to have a high reliability if
it produces consistent results under
consistent conditions.

For example, measurements of peoples
height and weight are often extremely
] 1 [
reliable.
13
How to assess reliability
Intra-rater
Have same person rate the case more than once.
Inter-rater
Have different people rate the case.
14
Valid and Reliable
Reliable but NOT
valid
NOT reliable or
valid
Reliable and valid
Cant be valid unless reliable
Bias
Allocation (Selection) Bias a systematic error in creating intervention
groups,

causing them to differ with respect to prognosis. Failure of
randomisation Systematic differences in comparison groups

Performance Bias: Systematic differences in interventions
received by the two groups.

Attrition Bias: Systematic differences in withdrawals from the trial

Measurement (Detection) Bias Failure of blinding
Systematic differences in outcome assessment

Reporting Bias Selective reporting of results
Drawing conclusions that are not supported by the results

Publication Bias Failure to publish (Systematic reviews)


Funding Bias company or sponsor interests
Performance Bias
Confounding: a situation

in which the estimated intervention effect is
biased because

of some difference between the comparison groups apart
from

the planned interventions - such as baseline characteristics,

prognostic factors, or concomitant interventions.
Contamination: Provision of the intervention to the control group.
Compliance: Poor compliance with the allocated intervention.
Co-interventions: Provision of unintended additional interventions to
either group.
Note: both contamination & compliance problems tend towards no
effect.
Co-interventions may bias in either direction
Attrition Bias: Loss to follow-up
Loss to follow-up (Drop out) rate: should not exceed
outcome event rate and should be equal in all
groups.
Rough guide: 5% - OK
If >20% - validity doubtful
Intention-to-treat analysis
Maintains the randomisation and analyse all despite
the drop outs.

Summary: Analysescompare all simvastatin-allocated
versus all placebo-allocated participants. These
intention-to-treat comparisons

Measurement (Detection) Bias

Best: Double-blind
Both patient and investigator unaware of treatment
allocation

Less important if outcome is objective (e.g. death)
Critical if outcome is subjective

Impossible for some comparisons eg medical vs
surgical intervention
What are the possible causes of an
effect in a RCT?

Bias
Placebo
Chance
Real effect

Placebo Effect
Placebo : is a false or fake intervention (Drug,
Manoeuvre, procedure)
You can only know the size of a placebo effect if
a placebo has been used!

Ex. in depression we know that antidepressant
therapy is not much more effective than placebo
However because of the medical attention and
follow-up received it is quite likely that placebo is
better than no treatment.



Could the treatment effect have arisen by
chance?
p-values
Statistical test of the (null) hypothesis (means the
intervention had no effect)
How large was the treatment effect?

If p<0.05 then the result is statistically significant
(i.e. effect would occur by chance less than 5% of the time)
The smaller the p-value the less likely is the effect to
occur by chance.

Confidence interval (CI): An interval within which the
population parameter (the true value) is expected to
lie with a given degree of certainty (eg 95%,99%).
Appraisal of significance/ Chance
Were all the outcomes studied important?
Was sub-group analysis pre-planned?
Could the treatment effect have arisen by
chance?
How large was the treatment effect? P
value

24
Measures of Quality of a
Diagnostic Test/outcome measure
Sensitivity
Specificity
Accuracy
Predictive Value (positive and negative) -- The
higher these numbers - the better the test.

25
the Gold Standard
The definitive diagnostic technique
Often expensive, elaborate, or difficult to
perform.
However, We are always looking for faster,
cheaper, better ways to diagnose disease
(and to determine treatment).
26
Sensitivity
The number of people with the disease (Gold
Standard) who have a positive test result.
sensitivity (how often we get positive results in people
with the condition)
Relates Gold Standard to New Test.
A sensitive test rarely misses people with disease.
Sensitive tests should be selected when there is an
important penalty for missing disease (i.e., cancer
diagnosis, AIDS)
27
Specificity
specificity (how often we get negative results in people
without the condition). (Rule out the false positive)
A specific test will rarely misclassify people without
disease as diseased.
Specific tests are used to rule in a diagnosis that has
been suggested by other tests.
Ideally, these measures should both be 100% but they
rarely are! More often there will be some false
positives and some false negatives.
Appraising Applicability
Is my patient similar to the study population?
Is the treatment feasible in my clinical
setting?
Will potential treatment benefits outweigh
potential harms of treatment for my
patient?

También podría gustarte