“Using Real-World Data & Published Evidence in Pharmacy Quality Improvement Activities”
On Monday, May 20, 2013, we presented a webinar on “Using Real-World Data & Published Evidence in Pharmacy Quality Improvement Activities” for the member organizations of the Alliance of Community Health Plans (ACHP).
The 80-minute discussion addressed four topic areas, all of which have unique critical appraisal challenges. Webinar goals were to discuss issues that arise when conducting quality improvement efforts using real world data, such as data from claims, surveys and observational studies and other published healthcare evidence.
Key pitfalls were cherry picked for these four mini-seminars—
- Pitfalls to avoid when using real-world data, dealing with heterogeneity, confounding-by-indication and causality.
- Key issues in evaluating oncology studies — outcome issues and focus on how to address large attrition rates.
- Important issues when conducting comparative safety reviews — assessing patterns through use of RCTs, systematic reviews, observational studies and registries.
- Key issues in evaluating studies employing Kaplan-Meier estimates — time-to-event basics with attention to the important problem of censoring.
A recording of the webinar is available at—
Improving Results Reporting in Clinical Trials: Case Study—Time-to-Event Analysis and Hazard Ratio Reporting Advice
We frequently see clinical trial abstracts—especially those using time-to-event analyses—that are not well-understood by readers. Fictional example for illustrative purposes:
In a 3-year randomized controlled trial (RCT) of drug A versus placebo in women with advanced breast cancer, the investigators presented their abstract results in terms of relative risk reduction for death (19%) along with the hazard ratio (hazard ratio = 0.76, 95% confidence interval [CI] 0.56 to 0.94, P = 0.04). They also stated that, “This reduction represented a 5-month improvement in median survival (24 months in the drug A group vs. 19 months in the placebo group).” Following this information, the authors stated that the three-year survival probability was 29% in the drug A group versus 21.0% in the placebo group.
Many readers do not understand hazard ratios and will conclude that a 5 month improvement in median survival is not clinically meaningful. We believe it would have been more useful to present mortality information (which the authors frequently present in results section, but is not easily found by many readers).
A much more meaningful abstract statement would go something like this: After 3 years, the overall mortality was 59% in the drug A group compared with 68% in the placebo group which represents an absolute risk reduction (ARR) of 9%, P=0.04, number needed to treat (NNT) 11. This information is much more impressive and much more easily understood than a 5-month increase in median survival and uses statistics familiar to clinicians.