G-I-N Webinar: Guideline Development & Evidence-based Quality Improvement

Status

Guidelines International Network Webinar: How to Develop Guidelines Within the Context of a Clinical Quality Improvement Program

Thanks to the Guidelines International Network, a webinar we did for them is available online.  To access the recording and slide show presentation, go to—

http://www.g-i-n.net/activities/g-i-n-na/g-i-n-na-events-activities/webinar-series/delfini

For information about the case study we showcased for our presentation, go to—

http://www.delfini.org/Showcase_Project_VTE.htm

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

Critical Appraisal Tool for Clinical Guidelines & Other Secondary Sources

Status

Critical Appraisal Tool for Clinical Guidelines & Other Secondary Sources

Everything citing medical science should be appraised for validity and clinical usefulness. That includes clinical guidelines and other secondary sources. Our tool for evaluating these resources— the Delfini QI Project Appraisal Tool—has been updated and is available in the Delfini Tools & Educational Library at www.delfini.org.  For quick access to the PDF version, go to—

http://www.delfini.org/delfiniNew.htm

 

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

Reliable Clinical Guidelines

Status

Reliable Clinical Guidelines—Great Idea, Not-Such-A-Great Reality

Although clinical guideline recommendations about managing a given condition may differ, guidelines are, in general, considered to be important sources for individual clinical decision-making, protocol development, order sets, performance measures and insurance coverage. The Institute of Medicine [IOM] has created important recommendations that guideline developers should pay attention to—

  1. Transparency;
  2.  Management of conflict of interest;
  3.  Guideline development group composition;
  4. How the evidence review is used to inform clinical recommendations;
  5.  Establishing evidence foundations for making strength of recommendation ratings;
  6. Clear articulation of recommendations;
  7. External review; and,
  8. Updating.

Investigators recently evaluated 114 randomly chosen guidelines against a selection from the IOM standards and found poor adherence [Kung 12]. The group found that the overall median number of IOM standards satisfied was only 8 out of 18 (44.4%) of those standards. They also found that subspecialty societies tended to satisfy fewer IOM methodological standards. This study shows that there has been no change in guideline quality over the past decade and a half when an earlier study found similar results [Shaneyfeld 99].  This finding, of course, is likely to have the effect of leaving end-users uncertain as to how to best incorporate clinical guidelines into clinical practice and care improvements.  Further, Kung’s study found that few guidelines groups included information scientists (individuals skilled in critical appraisal of the evidence to determine the reliability of the results) and even fewer included patients or patient representatives.

An editorialist suggests that currently there are 5 things we need [Ransohoff]. We need:

1. An agreed-upon transparent, trustworthy process for developing ways to evaluate clinical guidelines and their recommendations.

2. A reliable method to express the degree of adherence to each IOM or other agreed-upon standard and a method for creating a composite measure of adherence.

From these two steps, we must create a “total trustworthiness score” which reflects adherence to all standards.

3. To accept that our current processes of developing trustworthy measures is a work in progress. Therefore, stakeholders must actively participate in accomplishing these 5 tasks.

4. To identify an institutional home that can sustain the process of developing measures of trustworthiness.

5. To develop a marketplace for trustworthy guidelines. Ratings should be displayed alongside each recommendation.

At this time, we have to agree with Shaneyfeld who wrote an accompanying commentary to Kung’s study [Shaneyfeld 12]:

What will the next decade of guideline development be like? I am not optimistic that much will improve. No one seems interested in curtailing the out-of-control guideline industry. Guideline developers seem set in their ways. I agree with the IOM that the Agency for Healthcare Research and Quality (AHRQ) should require guidelines to indicate their adherence to development standards. I think a necessary next step is for the AHRQ to certify guidelines that meet these standards and allow only certified guidelines to be published in the National Guidelines Clearinghouse. Currently, readers cannot rely on the fact that a guideline is published in the National Guidelines Clearinghouse as evidence of its trustworthiness, as demonstrated by Kung et al. I hope efforts by the Guidelines International Network are successful, but until then, in guidelines we cannot trust.

References

1. IOM: Graham R, Mancher M, Wolman DM,  et al; Committee on Standards for Developing Trustworthy Clinical Practice Guidelines; Board on Health Care Services.  Clinical Practice Guidelines We Can Trust. Washington, DC: National Academies Press; 2011 http://www.nap.edu/catalog.php?record_id=13058

2. Kung J, Miller RR, Mackowiak PA. Failure of Clinical Practice Guidelines to Meet Institute of Medicine Standards: Two More Decades of Little, If Any, Progress. Arch Intern Med. 2012 Oct 22:1-6. doi: 10.1001/2013.jamainternmed.56. [Epub ahead of print] PubMed PMID: 23089902.

3.  Ransohoff DF, Pignone M, Sox HC. How to decide whether a clinical practice guideline is trustworthy. JAMA. 2013 Jan 9;309(2):139-40. doi: 10.1001/jama.2012.156703. PubMed PMID: 23299601.

4. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA. 1999 May 26;281(20):1900-5. PubMed PMID: 10349893.

5. Shaneyfelt T. In Guidelines We Cannot Trust: Comment on “Failure of Clinical Practice Guidelines to Meet Institute of Medicine Standards”. Arch Intern Med. 2012 Oct 22:1-2. doi: 10.1001/2013.jamainternmed.335. [Epub ahead of print] PubMed PMID: 23089851.

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

Quickly Finding Reliable Evidence

Status

Quickly Finding Reliable Evidence

Good clinical recommendations for various diagnostic and therapeutic interventions incorporate evidence from reliable published research evidence. Several online evidence-based textbooks are available for clinicians to use to assist them in making healthcare decisions. Large time lags in updating are a common problem for medical textbooks.  Online textbooks offer a solution to these delays.

For readers who plan to create decision support, we strongly recommend DynaMed [full disclosure: we are on the editorial board in an unpaid capacity, though a few years ago we did receive a small gift]. DynaMed is a point-of-care evidence-based medical information database created by Brian S. Alper MD, MSPH, FAAFP. It continues to grow from its current 30,000+ clinical topics that are updated frequently. DynaMed monitors the content of more than 500 medical journals and systematic evidence review databases.  Each item is thoroughly reviewed for clinical relevance and scientific reliability. DynaMed has been compared with several products, including in a new review by McMaster University. The DynaMed website is https://dynamed.ebscohost.com/.

The McMaster University maintains a Premium Literature Service (PLUS) database which is a continuously updated, searchable database of primary studies and systematic reviews. Each article from over 120 high quality clinical journals and evidence summary services is appraised by research staff for methodological quality, and articles that pass basic criteria are assessed by practicing clinicians in the corresponding discipline.  Clinical ratings are based on 7-point scales, where clinical relevance ranges from 1 (“not relevant”) to 7 (“directly and highly relevant”), and newsworthiness ranges from 1 (“not of direct clinical interest”) to 7 (“useful information, most practitioners in my discipline definitely don’t know this).

Investigators from McMaster evaluated four evidence-based textbooks—UpToDate, PIER, DynaMed and Best Practice [Jeffery 12].  For each they determined the proportion of 200 topics which had subsequent articles in PLUS with findings different from those reported in the topics. They also evaluated the number of topics available in each evidence-based textbook compared with the topic coverage in the PLUS database, and the recency of updates for these publications.  A topic was in need of an update if there was at least one newer article in PLUS that provided information that differed from the topic’s recommendations in the textbook.

Results

The proportion of topics with potential for updates was significantly lower for DynaMed than the other three textbooks, which had statistically similar values. For DynaMed topics, updates occurred on average of 170 days prior to the study, while the other textbooks averaged from 427 to 488 days. Of all evidence-based textbooks, DynaMed missed fewer articles reporting benefit or no effect when the direction of findings (beneficial, harmful, no effect) was investigated. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks’ treatment recommendations was 23% (95% CI 17 to 29%) for DynaMed, 52% (95% CI 45 to 59%) for UpToDate, 55% (95% CI 48 to 61%) for PIER, and 60% (95% CI 53 to 66%) for Best Practice (?23=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131 to 209) for DynaMed, to 488 days (range 423 to 554) for PIER (P<.001 across all textbooks).

Summary

Healthcare topic coverage varied substantially for leading evidence-informed electronic textbooks, and generally a high proportion of the 200 common topics had potentially out-of-date conclusions and missing information from 1 or more recently published studies. PIER had the least topic coverage, while UpToDate, DynaMed, and Best Practice covered more topics in similar numbers. DynaMed’s timeline for updating was the quickest, and it had by far the least number of articles that needed to be updated, indicating that quality was not sacrificed for speed.

Note: All textbooks have access to the PLUS database to facilitate updates, and also use other sources for updates such as clinical practice guidelines.

Conclusion

The proportion of topics with potentially outdated treatment recommendations in on-line evidence-based textbooks varies substantially.

Reference

Jeffery R, Navarro T, Lokker C, Haynes RB, Wilczynski NL, Farjou G. How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks. J Med Internet Res. 2012 Dec 10;14(6):e175. doi: 10.2196/jmir.2105. PubMed PMID: 23220465.

 

 

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

Delfini Treatment Messaging Scripts™ Update

Status

 Messaging Scripts ™ Update

Delfini Messaging Scripts  are scripts for scripts. Years ago we were asked by a consultancy pharmacy to come up with a method to create concise evidence-based statements for various therapies.  That’s how we came up with our ideas for Messaging Scripts, which are targeted treatment messaging & decision support tools for specific clinical topics. Since working with that group, we created a template and some sample scripts which have been favorably received wherever we have shown them.  The template is available at the link below, along with several samples.  Samples recently updated: Ace Inhibitors, Alendronate, Sciatica (Low Back Pain), Statins (two scripts) and Venous Thromboembolism (VTE) Prevention in Total Hip and Total Knee Replacement.

 http://www.delfini.org/page_SamePage_RxMessagingScripts.htm

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

Canadian Knowledge Translation Website

Status

Canadian Knowledge Translation Website

The Knowledge Translation (KT) Clearinghouse is a useful website for EBM information and tools. It is funded by the Canadian Institute of Health Research (CIHR) and has a goal of improving the quality of care by developing, implementing and evaluating strategies that bridge the knowledge-to-practice gap and to research the most effective ways to translate knowledge into action. Now added to Delfini web links.

http://ktclearinghouse.ca/

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

Critical Appraisal Matters

Status

Critical Appraisal Matters

Mike and I make it a practice to study the evidence on the evidence.  Doing effective critical appraisal to evaluate the validity and clinical usefulness of studies makes a difference.  This page on our website may be our most important one and we have now added a 1-page fact sheet for downloading: http://www.delfini.org/delfiniFactsCriticalAppraisal.htm

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email

The Elephant is The Evidence—Epidural Steroids

Status

The Elephant is The Evidence—Epidural Steroids: Edited & Updated 1/7/2013

Epidural steroids are commonly used to treat sciatica (pinched spinal nerve) or low back pain.  As of January 7, 2013 at least 40 deaths have been linked to fungal meningitis thought to be caused by contaminated epidural steroids, and 664 cases in 19 states have been identified with a clinical picture consistent with fungal infection [CDC]. Interim data show that all infected patients received injection with preservative-free methylprednisolone acetate (80mg/ml) prepared by New England Compounding Center, located in Framingham, MA. On October 3, 2012, the compounding center ceased all production and initiated recall of all methylprednisolone acetate and other drug products prepared for intrathecal administration.

Thousands of patients receive epidural steroids without significant side effects or problems every week. In this case, patients received steroids that were mixed by a “compounding pharmacy” and contamination of the medication appears to have occurred during manufacture. But let’s consider other patients who received epidural steroids from uncontaminated vials. How much risk and benefit are there with epidural steroids? The real issue is the effectiveness of epidural steroids. Yes, there are risks with epidural steroids beyond contamination—e.g., a type of headache that occurs when the dura (the sac around the spinal cord) is punctured and fluid leaks out. This causes a pressure change in the central nervous system and a headache. Bleeding is also a risk. But people with severe pain from sciatica are frequently willing to take those risks if there are likely to be benefits. But, in fact, for many patients who receive epidural steroids the likelihood of benefit is very low. For example, patients with bone problems (spinal stenosis) rather than lumbar disc disease are less likely to benefit. Patients who have had a long history of sciatica are less likely to benefit.

We don’t know how many of these patients were not likely to benefit from the epidural steroids, but if the infected patients had been advised about the unproven benefits of epidural steroids in certain cases and the known risks, some patients may have chosen to avoid the injections and possibly be alive today.  This is an example of the importance of good information as the basis for decision-making. Basing decisions on poor quality or incomplete information and intervening with unproven—yet potentially risky treatments puts millions of people at risk every week.

Let’s look at the evidence. Recently, a fairly large, well-conducted RCT published in the British Medical Journal (BMJ) reported that there is no meaningful benefit from epidural steroid injections in patients who have had long term (26 to 57 weeks) of sciatica [Iverson].  As pointed out in an editorial, epidural steroids have been used for more than 50 years to treat low back pain and sciatica and are the most common intervention in pain clinics throughout the world [Cohen]. And yet, despite their widespread use, their efficacy for the treatment of chronic sciatica remains unproven. (We should add here that many times lacking good evidence of benefit does not mean a treatment does not work.) Iverson et al conclude that, “Caudal epidural steroid or saline injections are not recommended for chronic lumbar radiculopathy [Iverson].”

Of more than 30 controlled studies evaluating epidural steroid injections, approximately half report some benefit. Systematic reviews also report conflicting results. Reasons for these discrepancies include differences in study quality, treatments, comparisons, co-interventions, study duration and patient selection. Results appear to be better for people with short term sciatica, but improvement should not be considered to be curative with epidural steroids. In this situation, it is very important that patients understand this fuzzy benefit-to-risk ratio. For many who are completely informed, the decision will be to avoid the risk.

With this recent problem of fungal meningitis from epidural steroids, it is important for patients to be informed about the world of uncertainty that surrounds risk, especially when science tells us that the evidence for benefit is not strong.  Since health care professionals frequently act as the eyes of the patient, we must seriously consider for every intervention we offer whether benefits clearly outweigh potential harms—and we must help patients understand details regarding the risks and benefits and be supportive when patients are “on the fence” about having a procedure. Remember Vioxx, arthroscopic lavage, vertebroplasy, encainide and flecainide, Darvon and countless other promising new drugs and other interventions? They seemed promising, but harms outweighed benefits for many patients.

References

1. http://www.cdc.gov/HAI/outbreaks/meningitis.html accessed 12/10/12

2.  Cohen SP. Epidural steroid injections for low back pain. BMJ. 2011 Sep 13;343:d5310. doi: 10.1136/bmj.d5310. PubMed PMID: 21914757.

3.  Iversen T, Solberg TK, Romner B, et al.   Effect of caudal epidural steroid or saline injection  in chronic lumbar radiculopathy: multicentre, blinded, randomised controlled trial. BMJ. 2011 Sep 13;343:d5278. doi: 10.1136/bmj.d5278. PubMed PMID: 21914755.

 

Facebook Twitter Linkedin Digg Delicious Reddit Stumbleupon Tumblr Email