Quantitative Methods for Evaluating Health Care Practices, Coursework Example

Discuss why it is increasingly common for published reports to include both effect size estimates and statistical significance test results.

The issue at stake is the difference between statistical significance and clinical significance.

Often times, researchers will provide a p-statistic that can be (roughly) interpreted as the percent chance that the research finding was due to change rather than having an actual impact.  The p-statistic is important, with the usual threshold set at 5% (.05) representative of a “significant finding” or a finding that is due to an actual effect rather than simply chance.

While a p-statistic can intimate whether the finding was due to chance or an actual effect, it cannot say whether that effect is significant enough for clinical practice.  For example, if an intervention has a statistically significant effect, but the effect only saved one life in the cohort of 10,000s- that is not ultimately a useful finding for clinical practice.  Thus, journals are increasingly asking  researchers to report the effect size, in addition to the p-statistic, in order to assess the  clinical significance of the finding.

If you are conducting a small pilot randomized clinical trial (RCT) as part of mixed method study, why is it often recommended that the final report include effect size results only?

If one is running a small pilot randomized clinical trial, the sample size is likely small enough that the study lacks enough power to give an accurate p-value- that is, the p-value is not really worth anything. Also due to the introductory nature of the study (pilot study), researchers are primarily concerned with the question: does this intervention plausibly work?  While the effect size is not an adequate answer to that question, it gives researchers enough information to assess whether a larger, follow-up trial should be conducted.

Discuss what “number needed to treat” (NNT) represents and how it can be used to address clinical significance of a treatment or intervention.

At the core, the ‘number needed to treat’ metric estimates how many people would need to receive an intervention to prevent one undesirable event.  That is, in aggregate how many individuals would receive the new intervention (rather than the old) in having a negative or adverse event.  This is really measuring the riskiness of the new intervention, and its ability to treat patients versus the standard treatment.

The NTT statistic must be used carefully when assessing the clinical significance of a treatment.  Since the statistic is calculated on the absolute scale, the clinical significance may deemed small (as a number) but when actually used in treatment, it saves or prolongs many lives.

Discuss and interpret the NNT estimate for the following study.

A multicenter clinical trial involving 17,000 patients suspected of having acute myocardial infarction (AMI) was conducted. The vascular death rate after 5 weeks was compared in patients receiving aspirin versus those not receiving aspirin, regardless of any other concomitant treatment.  The relative risk (RR) for the effect of aspirin relative to no aspirin is 0.80 and the relative risk reduction (RRR) is 0.20.  The number needed to treat (NNT) = 42.

Answer: This essentially means that a doctor would have to treat roughly 42 patients in order to observe a benefit in at least one of them. This is a fairly high risk population, and the results  given are not better than those for aspirin, so it would not seem that this intervention is particularly effective

Explain what “number needed to harm” (NNH) represents. Discuss how NNT and NNH information contributes to the evaluation of the risk-benefit ratio for a given intervention.

NNH represents the number of patients who received the intervention that would have lead to a single additional patient being harmed.  That is, the NNH tells us how many patients need to be treated with the new intervention before an adverse effect can be expected.

NNH provides the clinicians with a risk estimate how many would need to be treated before an adverse effect is expected; the NNT provides an estimate of how many individuals essentially needed to be treated before the benefit of the intervention can be seen.  Both figures, thus, should be used together in order to understand the risk profile of the treatment.