Annals of Oncology Advance Access published online on July 28, 2008
Annals of Oncology, doi:10.1093/annonc/mdn418
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Heterogeneity in cancer guidelines: should we eradicate or tolerate?
1 Department of Medical Oncology, Ioannina University Hospital, Ioannina, Greece
2 Clinic and Polyclinic of Oncology, University Hospital, Zürich, Switzerland
3 The Finsen Centre, Copenhagen University Hospital, Copenhagen, Denmark
* Correspondence to: Dr N. Pavlidis, Professor of Oncology, Department of Medical Oncology, Ioannina University Hospital, Niarxou Avenue, 45500 Ioannina, Greece. Tel/Fax: +30-2651099394; E-mail: npavlid{at}uoi.gr
 |
Abstract |
|---|
 Top Abstract introductionmethodsresultsdiscussionReferences |
|---|
Background: Heterogeneity in aspects of development, structure and context of oncology guidelines was not evaluated. We analysed and critically examined its implications.
Materials and methods: Nine cancer clinical practice guidelines were selected on the basis of popularity among oncologists. The relevant Web sites and publications on three tumours were examined and characteristics grouped in the data domains: producing organisation, methodology, guideline structure and content, implementation and evaluation and scientific agreement.
Results: ASCO, ESMO, NICE, SIGN, START, NHMRC, NCI, NCCN and CCO guidelines were examined. Development was initiated by stakeholders or authorised bodies, run by task forces with varying degrees of multidisciplinarity, with rare endorsement of external guidelines. Recommendation formulation was on the basis of evidence, shaped via interactive processes of expert review and public consultation-based modifications. Guidelines varied in comprehensiveness per tumour type, number, size, format, grading of evidence, update and legal issues. Orientation for clinic use or as reference document, end-users and binding or elective nature also varied. Standard dissemination strategies were used, though evaluation of adoption and of impact on health outcomes was implemented with considerable heterogeneity.
Conclusions: Heterogeneity in development, structure, user and end points of guidelines is evident, though necessary in order to meet divergent demands. Crucial for their effectiveness are adherence to methodological standards, a clear definition of what the guideline intends to do for whom and a systematic evaluation of their impact on health care.
cancer, clinical practice, guidelines, oncology
|
introduction |
|---|
|
TopAbstractintroductionmethodsresultsdiscussionReferences |
|---|
Clinical practice guidelines (CPG) are defined as "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" [1]. The purposes of CPG are to improve the quality of patient care and health care outcomes, make clinical decisions more transparent, promote efficient use of resources and prioritise research goals. Moreover, CPG may aim to provide guidance for involved stakeholders (health professionals, patients and carers, industry, health care providers and policy makers) and support quality control [2, 3]. Oncology guidelines are nowadays produced by several organisations and have been shown to improve both the care process and patient outcomes in several studies [4, 5]. Increasing concern about the methodology, reporting and quality of cancer guidelines provided the incentive for international collaboration which led to the development of tools and checklists [6]. These were intended to provide standardised methods of development and reporting of guidelines, of assessment of their quality and tools for adaptation and implementation [7–11]. These features are regarded as pivotal, since only high-quality, methodologically sound, clear guideline statements are likely to be user-friendly and adoptable by consumers [12]. In contrast, heterogeneity in several aspects of guideline development is controversially seen. It is regarded by some as an impediment to international collaboration, to standardisation of methodology/quality control and leading to duplication of efforts while other investigators consider it a necessary tool for guideline flexibility, adaptation and for meeting diverse needs [13–15]. In this study, we sought to examine heterogeneity in several aspects of the most commonly used oncology guidelines produced in English language, critically analyse it and discuss its implications.
|
methods |
|---|
|
TopAbstractintroductionmethodsresultsdiscussionReferences |
|---|
Two on-line guideline databases (Guidelines International Network, www.g-i-n.net, and National Guideline Clearinghouse, www.guideline.gov) as well as Web sites of professional health organisations and governmental agencies were visited in order to identify oncology CPG produced in English language. As our intention was not to produce a systematic review of quality/structure of existing guidelines, our sample was neither global (encompassing all guidelines) nor random. Instead, we chose to evaluate the most commonly used oncology guidelines of the highest reputation and popularity among practicing oncologists, as judged by a consensus survey carried out between authors of this manuscript.
The quality of guidelines can be assessed by several suggested tools, of which the AGREE (Appraisal of Guidelines Research and Evaluation) instrument became popular and has been endorsed by the World Health Organisation [7]. Appointing quality scores for the studied guidelines was not our intention, as this has been done in the medical literature and can be easily carried out by any interested person or party [16]. Instead, we sought to provide descriptive characteristics of the nine examined guidelines in order to highlight similarities and differences in their structure, format/reporting, context, methodology, orientation/intent, scientific content and evaluation and to critically analyse them. Accordingly, we recorded guideline characteristics and classified them in the following data domains, each containing a number of data modules: producing organisation (seven modules), methodology (five modules), guideline structure and content (11 modules), implementation and evaluation (six modules). Moreover, we sought to study the impact of heterogeneous guideline development processes in the scientific content of formulated recommendations (scientific agreement of formulated recommendations, 10 modules).
Relevant data were retrieved by visiting the Web sites of the producing organisations. Information, statements and documents/publications that contained data on the structure of the organisation and details on the methodology followed for developing, reporting and implementing the guidelines (GL) were studied. Moreover, three topic guidelines (advanced lung cancer, advanced breast cancer, advanced colon cancer or guidelines of similar topics) were scrutinised from each organisation so as to confirm or refute guideline features described in methodology documents and to study the agreement in the scientific content of formulated recommendations. The principles and definitions below were applied in extracting, recording and presenting data in each domain.
producing organisation characteristics
organisation and guideline web address.
year of initiation.
The earliest publication year of any guideline was recorded.
body responsible for GL development.
The expert panel, development group or committee undertaking the task of developing a guideline.
proposal/decision for GL development.
Data on individuals or groups entitled to submit a proposal for GL development and on those taking the final decision on the proposal.
check for duplication and potential endorsement of external GL.
Explicit statement whether the responsible body checked for presence of already developed guidelines on the same topic from other organisations and whether it analysed/considered them for potential endorsement.
funding.
Source of financial support, exclusively or predominantly, for the development of GL.
editorial independence from funding body.
Presence of data or statements affirming that views or interests of the funding body (government, industry, charity or professional organisations) did not influence the guideline recommendations.
methodology characteristics
composition of GL development group.
Data on the professional skills of individuals participating in the development group as well as on the presence of methodological experts (statisticians, health economists, information technology specialists, epidemiologists), patient/carer representatives, representatives of policy makers (government) or industry.
review of the evidence.
A literature review was defined as systematic only if information such as search terms, time interval, selection criteria and databases searched were set upfront before guideline development and were provided. In the absence of the above, the literature review was termed narrative.
process of GL development.
An outline of the administrative/organisational algorithm or process of guideline production.
consensus and authorship.
Data on methods for reaching consensus and processing disagreements between members of the responsible body for guideline development. Upon explicit statement of use of consensus techniques (Delphi, nominal group, consensus development conference or others), the consensus was termed formal; otherwise, it was presumed to be informal [17]. Authorship could involve one or several authors or committee authorship (committee name furnished with members in an appendix).
legal review.
Check of the guideline draft by legal counsellors before publication.
guideline structure and content characteristics
comprehensive for tumour type and site.
CPG globally providing diagnosis, staging and management recommendations for a solid tumour (i.e. CPG on non-small-cell lung cancer (NSCLC) or lung cancer or advanced lung cancer) in contrast to ‘narrow’ CPG focusing on specific topics within a given solid tumour (i.e. second-line therapy for irresectable NSCLC, chemoradiation for locally advanced NSCLC).
number of existing guidelines.
As appearing online on 1 January 2008 and in use.
advanced breast, colon, lung cancer.
Presence or absence of a comprehensive CPG on each of these tumour types with the year of issue. If a comprehensive CPG is not present, number of guidelines with more focused topics within the tumour type (in parentheses).
pages and references.
Size of each full CPG version and number of references cited (on the basis of the breast, colon and lung cancer CPG studied).
structure.
Format of the guideline in thematic sections.
grading of evidence.
Scales used to grade levels of evidence and strength of recommendations. If none provided, the narrative approach summarily describes characteristics and findings of clinical trials.
cost data.
Information on cost, cost-effectiveness, cost utility or formal health economic analyses.
literature search and update.
Time interval for screening literature for emergence of new relevant data and time interval for issue of an updated guideline version.
conflict of interest stated.
Explicit statement whether members of the developing group have any financial, research or other interests potentially conflicting with their property of independent guideline developers.
implementation and evaluation characteristics
use.
Orientation for use of the CPG as a highly detailed reference source of data/recommendations or as a short, flexible summary recommendation tool for the clinic.
dissemination and implementation tools.
Tools used to disseminate and enhance adoption/implementation of the CPG among stakeholders.
binding for physicians.
Statement whether physicians are legally bound to adhere to guidelines.
targeted health professionals and range.
Health workers for whom the CPG were developed and geographic/political/cultural areas where relevant.
GL evaluation.
Methods used and data produced on the implementation of recommendations and on their impact on patient outcomes (survival, quality of life, satisfaction), physician outcomes (adoption of GL, satisfaction, quality care) and health system outcomes (cost-effectiveness, optimal utilisation of resources).
scientific agreement of formulated recommendations
Similarities and differences in the scientific content of formulated recommendations for the management of patients with NSCLC were recorded by study of relevant guidelines produced by the nine organisations. The modules that were examined were: Epidemiology, Screening and presentation, Diagnosis, Staging, Surgery, Radiotherapy, Chemotherapy, Other therapies (biologics), Palliative/supportive care, Follow-up, Implementation and Research. The rate of agreement between guidelines was scored in each module by the use of the following simple scale:
- 0%–20%: Radically different
- 20%–40%: Numerous major differences
- 40%–60%: Some major differences
- 60%–80%: Only minor differences
- 80%–100%: Essentially identical
- 20%–40%: Numerous major differences
A mean score of agreement was calculated for each guideline. The SIGN guidelines for NSCLC were randomly chosen to be used as the common comparator for the remaining eight guidelines.
|
results |
|---|
|
TopAbstractintroductionmethodsresultsdiscussionReferences |
|---|
The CPG examined were those issued by the following:
- ASCO (American Society of Clinical Oncology)
- ESMO (European Society of Medical Oncology)
- NICE (National Institute for Health and Clinical Excellence, England and Wales)
- SIGN (Scottish Intercollegiate Guidelines Network)
- START (State of the Art Oncology in Europe, Italy)
- NHMRC (National Health and Medical Research Council, Australia)
- NCI (National Cancer Institute, USA)
- NCCN (National Comprehensive Cancer Network, USA)
- CCO (Cancer Care Ontario, Canada)
- ESMO (European Society of Medical Oncology)
producing organisation characteristics
Characteristics of the organisations responsible for production of cancer CPG are shown in Table 1. Four professional organisations of oncologists (ASCO, ESMO, START and NCCN) and five government-funded multidisciplinary agencies or institutes (NICE, SIGN, NHMRC, NCI and CCO) have been developing CPG since the 1990s. A task force of 10–30 individuals with both expertise and motivation in applications of evidence-based medicine actually constituted the workhorse for development of CPG in virtually all organisations. Differences were evident between organisations in the size of the task forces (10 to >45 members) and the frequency of task force meetings (ranging from one per year to monthly sessions). These CPG development groups were appointed and overseen by boards of experts and executives that lay higher in the decision-making hierarchy of each organisation. The proposals for new guideline development could stem from any individual or society member in some organisations (ASCO, ESMO, NICE, SIGN and NCCN), while only from registered groups or relevant committees in others (START, NHMRC, NCI and CCO). Detailed data on the body responsible for CPG development and the decision-making hierarchical structures of each organisation could be retrieved in all nine cases. Only ASCO and NHMRC specifically stated that a formal screen for existence of external, high-quality CPG on the relevant topic was undertaken and that endorsement was possible, after appropriate analysis and quality control. Still, in both cases, it was the independent organisation that had developed the external CPG that should apply to ASCO or NHMRC for endorsement evaluation. Other societies did not examine external CPG at all (ESMO, START, NCI and NCCN) while others only used them as relevant evidence upon which their own CPG were constructed (NICE, SIGN, NHMRC and CCO). Financial support for the guideline developmental process originated from either the budget of professional organisations/societies (ASCO, ESMO, START and NCCN) or grants issued by the government or government-affiliated agencies (NICE, SIGN, NHMRC, NCI and CCO). However, editorial independence from the funding body, as well as this could be examined on the basis of legal and administrative documents, the composition of development groups, the reputation and expertise of involved members and the statement of independence in produced guidelines, was a pivotal prerequisite respected in all nine organisations.
|
methodology characteristics
Characteristics of methods for guideline development are shown in Table 2. The composition of the CPG development group in NICE, SIGN, NHMRC and CCO was multidisciplinary encompassing several professional qualifications; they included medical, surgical and radiation oncologists, other medical specialities, nurses, pharmacists, psychologists, various methodological experts, health managers or providers and patient/carer representatives. CPG working groups from most other institutions included multidisciplinary oncology specialists and some times, but not always, methodological experts and patient representatives. No patient representative involvement took place in ESMO, START and NCI and was not consistent in ASCO and NCCN. No methodological experts were present in development task forces of ESMO, NCI and NCCN, while in START one statistical editor operated across all panels. A consistent finding in all development groups was nonparticipation of industry representatives in order to safeguard the integrity of produced recommendations. The review of relevant medical literature was carried out systematically with a priori definition of search strategy and strict selection criteria in the ASCO, NICE, SIGN, NHMRC and CCO. A thorough literature search was carried out by the guideline development group not according to the strict definition of systematic review (usually without provision of data on the selection and synthesis of evidence) in the case of ESMO, START, NCI and NCCN. The process of guideline drafting followed a standard core format in all cases: a task force or single author, on the basis of existing relevant evidence, produced a first draft which was shaped into the final guideline through an interactive process of peer review and feedback-based modifications. Committee authorship was the rule in most guidelines with the exception of ASCO and ESMO in which a number of scientists authored each GL (ranging from 2 to 10). The review process focused mainly on peer review by independent experts in ASCO, ESMO, START, NCI and NCCN guidelines, while it included a public consultation process with feedback received from stakeholders (patients, public, industry, professional organisations, community health professionals and health providers) in NICE, SIGN, NHMRC and CCO. Moreover, some organisations (NICE, START, NHMRC) implemented a second ‘validation’ phase, during which guidelines were available for scrutiny and feedback from professionals and public before finalisation. Decisions were reached by formal or informal consensus in the development groups, with appropriate care taken to provide information on divergent views or lack of evidence for unanimous strong recommendations in all cases. Following review, the revised guidelines were formatted and approved by the CPG development groups and supervising editorial boards.
|
guideline structure and content characteristics
Most organisations issued ‘comprehensive’ guidelines that encompass all diagnosis, staging, risk assessment and management recommendations for a specific tumour type (Table 3). Notable exceptions were ASCO and CCO that produced guidelines focused on ‘narrower’ topics or questions set within a specific tumour type. The scope of the latter was to study available evidence and provide recommendations for specific clinical circumstances (i.e. adjuvant taxanes in early breast cancer). NICE produced both comprehensive and more focused guidelines, the latter being more numerous. The number of active guidelines ranged from 12 to 108, the societies covering all tumours with comprehensive guidelines being ESMO, START, NCI and NCCN. The size of produced guidelines exhibited significant variation, ranging from 2 to 368 pages and citing from 15 to 1000 references. Consequently, some GLs were meant to be summary clinic guides while others served as reference source documents. The structure of each guideline in thematic sections was grossly standardised in all cases. START and NCI had their guidelines formatted as on-line hypertexts with convenient hyperlinks incorporated in them. Most guidelines included synopses of recommendations in the form of summaries or tables. A characteristic feature of NCCN CPG was the presentation of recommendations in flow diagrams or algorithms according to disease and patient characteristics. Although in the majority of cases, a system for grading the level of supporting evidence (LOE) and the strength (grade) of recommendations (GOR) was used, it was not standardised: the US Preventive Services Task Force, ASCO, SIGN, START, NCI, NCCN and GRADE tools were applied. Most used the study design and type as the main LOE criterion, methodology rigour and consistent results as GOR criteria. Still in some instances, strength of end points and consensus level were considered too. In view of the absence of unanimous agreement on an ideal, common LOE-rating scheme, ASCO and CCO adopted a narrative description of the type and quality of evidence. Cost data for implementation of recommendations were furnished in practice only by NICE, SIGN and NHMRC. Quantitative cost reports were given in all cases by NICE and occasionally by NHMRC, while SIGN only provided verbal, qualitative estimates of resource implications. Of note, CPG of importance for the definition of reimbursement policies were commonly the ones produced by governmental agencies and containing more detailed health economic data (NICE, SIGN and NHMRC). Periodic screens for emerging, relevant medical literature were carried out by the editors or the CPG development groups either with reiteration of the review/validation cycle (ASCO, ESMO, NHMRC, NICE, SIGN, NCCN and CCO) or without it (NCI and START). In practice, some guidelines took 2–3 years to develop and updates were issued at more sparse (3- to 5-year) intervals (NICE, SIGN, NHMRC and CCO). The net effect, updated guidelines, occurred at yearly intervals only for ESMO, NCI and NCCN. Finally, conflict of interest statements of guideline developers could be retrieved in all guidelines except for START, NCI and NHMRC before 2004 and ESMO before 2007.
|
implementation and evaluation characteristics
The composition of the development group, methodology followed, size, structure and target users of each guideline ultimately defined its orientation for use and context (Table 4). CPG issued by ESMO and the summary Quick Reference Guides issued by NICE, SIGN and CCO as well as the Flow Chart features of NCCN were intended for quick consultation by the physician in the office or clinic. The full guideline versions of NICE, SIGN, NCCN and CCO and the CPG of ASCO, START, NHMRC and NCI had the hallmark features of reference documents: considerable size, detailed data and numerous references. Most organisations seemed to promote dissemination of issued CPG via standard strategies, such as medical journals, publications, seminars or workshops, sessions in professional organisation meetings, patient materials and Web site availability of downloadable CPG. Flow sheets for hospital practices, implementation plans for regional practices and other forms of electronic material were used as well. Adherence to the guidelines was not legally binding for the practicing physician. However, though acknowledging that the CPG does not override the responsibility of health care professionals to make individualised decisions appropriate to the circumstances of each patient, NICE and SIGN stated that institutions providing NHS-funded care would be expected to adhere to the guidelines. Moreover, health care professionals were requested to document the reasons for not following a guideline. These facts, along with the importance of the NICE and SIGN guidelines in the definition of reimbursement health care policies, were likely to constitute them more binding for the physician. Targeted health professionals encompassed a broad group of physicians, nurses and other health care workers in NICE, SIGN, NHMRC, NCI and CCO. In contrast, it was more restricted to physicians caring for cancer patients in ASCO, ESMO, START and NCCN guidelines. The region for guideline implementation was defined by the developing body and ranged from regions (Ontario and Canada) to countries (Australia, United States and UK) and continents (Europe). The extent of the ‘catchment area’ was inversely related to the binding nature, regulatory aspect, health economic analysis and reimbursement impact of guidelines.
|
The evaluation of the implementation of guidelines and their impact on patient and health system outcomes (survival, quality of life, cost-effectiveness, resource utilisation) presented considerable heterogeneity among issuing organisations. Audits on the basis of electronic databases, health system registries, physician/patient surveys, opinion meetings or mailed questionnaires, publication titrage and number of Web site downloads were most commonly used in various settings. However, the extent, systematic nature and data capture of these audits varied significantly. Some organisations (CCO, NICE, SIGN, NCI, ASCO) regularly issued evaluation reports based mainly on electronic/registry data capture while others (NHMRC, ESMO, NCCN, START) carried out periodic assessment of GL impact by means of surveys at irregular time intervals. It should be stressed that only CCO and NCI had evaluation data indicative of their implementation/impact on health care freely available on their Web sites.
scientific agreement of formulated recommendations
All screened organisations showed significant agreement in the scientific content of formulated recommendations for the management of patients with NSCLC, the leading cause of cancer mortality in developed societies. Using the SIGN guidelines, issued in 2005, as a common comparator, we observed only minor and few, if any, major disagreements in the other CPG (Table 5). They were observed in areas such as the application of low-dose spiral CT for NSCLC screening, the role of positron emission tomography and routine brain imaging in staging, the recommendation for adjuvant chemotherapy, the role of continuous hyperfractionated accelerated radiotherapy (CHART) as definite radiotherapy for medically inoperable cases or locally advanced disease. Differences were also seen in the number of palliative chemotherapy cycles to be administered, active second-line chemotherapy drugs, recommendation of anti-EGFR-targeted agents and on the intensiveness, timing, modalities of follow-up. Most minor differences were attributable to different philosophies on resource availability and utilisation, rapidity of adoption of emerging medical breakthroughs and role of the patient in clinical decision making. Differing views on matters such as persistence to active treatment with antineoplastic therapies, evaluation of quality-of-life values, cost-effectiveness and quality-adjusted life year thresholds may also have played a role. Moreover, the fear of litigation, the source of health care funds and the psychological/social contexts probably differed between geographical areas, contributing to some minor disagreements. Most importantly, the rare major scientific disagreements were due to different rate of update of guidelines and are quite likely to disappear when the chronologically older versions of some CPG are replaced by updated ones. For example, guidelines issued before 2006 did not contain recommendations on some second- to third-line chemotherapy or anti-EGFR regimens for patients with advanced NSCLC. This highlights the importance of timely updates of cancer guidelines.
|
|
discussion |
|---|
|
TopAbstractintroductionmethodsresultsdiscussionReferences |
|---|
The rapid progress in the fields of molecular biology, oncology and supportive care faced oncologists with an agreeable challenge: to make individualised judgements on the best available treatment of the most relevant health outcome, while respecting patient preferences. CPG are recommendations systematically developed by experts with access to available evidence in order to help doctors and patients identify appropriate health care for each setting. Soon after embarking on guideline development projects, the scientific community became aware that standardisation was the key to success: the processes of laying down the clinical question to be answered by the guideline, the development of the guideline, the collection and synthesis of the evidence, the formulation of recommendations, the report, evaluation and adaptation of guidelines needed to become standardised in order to guarantee high methodological quality and dissemination [3, 6, 7, 13, 14]. Several tools and instruments were developed for this goal and were met with satisfactory acceptance. PICO, an acronym for Population, Intervention, Comparison, Outcome, is a structured approach to formatting questions that a guideline should address [11]. Formal consensus techniques have been developed to facilitate merging and synthesis of different opinions among working group members [17]. Investigators proposed several algorithms for development of guidelines, like the Practice Guidelines Development Cycle devised by the Cancer Care Ontario group [18]. Several quality-rating scales have been published so as to evaluate the quality, quantity, rigour and consistency of the evidence base (GRADE, SIGN, US Preventive Services Task Force, SORT), thus assessing the level of evidence and strength of recommendations [9, 10, 19, 20]. The AGREE instrument is a validated systematic framework for assessing key components of guideline quality, including scope/purpose, stakeholder involvement, rigour of development, clarity, applicability and editorial independence [7]. Adaptation tools, like ADAPTE, represent validated processes for adapting existing guidelines to different economic, cultural or geographical contexts [8].
In our examination of nine popular Anglophone CPG, we observed that the realisation of the need for adherence to methodological standards resulted in relative homogeneity in several key components of guidelines. They were developed by working groups of 10–30 members, a number allowing for both global representation of stakeholders and flexibility, composed of multidisciplinary oncology-oriented health professionals in the absence of industry or government [11, 21]. Editorial independence from the funding body was guaranteed and recommendations were on the basis of available evidence, searched in on-line databases. The final product was shaped through an interactive process of opinion synthesis taking place between authors, reviewers, supervising bodies and involved stakeholders. Most CPG-provided comprehensive recommendations for diagnosis, staging, management and follow-up of patients with specific malignancies, were updated periodically and disseminated via printed and electronic material in target users. Finally, the scientific content of formulated recommendations showed high rates of agreement between guidelines issued.
However, a deeper examination of data reveals divergence within similarity. CPG examined were produced by both professional organisations and ones funded by or affiliated to the government. This might bear on the end-product, the parameters to consider, the priorities for intended outcomes and the targeted users. We believe that CPG issued by professional organisations may rank patient benefit as the absolute priority, while governmental CPG be obliged to consider cost-effectiveness data and optimal allocation of finite health resources. Proposals for guideline development could stem from any interested person versus members of the organisation or expert committees only. The composition of guideline development groups varied considerably: the definition of multidisciplinarity ranged from involvement of surgical, medical and radiation oncologists to that of physicians of different specialities or involvement of physicians, nurses, pharmacists, physical therapists, social workers, epidemiologists, statisticians, health care managers, economists and patient representatives. Recommendations were on the basis of evidence. Still, on several occasions that were distilled via on-line database searches with no methodology data provided, a strategy with implicit faith on expert opinion and ‘illuminated’ bias of the developer. The latter factor is, in our opinion, often a strong point but may well become a major source of inappropriate bias. In other cases, exhaustive search strategies with strictly defined criteria were implemented. This approach emphasises systematician belief in the value of data mining, though at risk of drawbacks such as the burden of effort, the risk of mining outdated, irrelevant or redundant data and of producing extensive guideline statements. Moreover, the review process varied from peer review by experts to extensive public consultation with patients, public, industry and organisations involved. Even the structure, focus and size of guidelines showed considerable variability, a comprehensive approach encompassing all aspects of management of a specific tumour was observed in parallel to more focused guidelines providing recommendations for specific clinical situations, therapeutic interventions or risk groups. Guidelines appeared as a basic set of recommendations, as web-based hypertexts or as voluminous textbooks up to 1000 pages long, implying different philosophies of use as either clinic guides or reference manuscripts. Levels of evidence and strength of recommendations were classified by means of several different schemes or simply narratively described [9, 10, 19, 20, 22]. Cost data were considered crucial, optional or irrelevant and guidelines were regarded as supportive or at times restrictive for treating physicians [23]. Updates were produced whenever relevant data appeared or periodically every 1 year or up to 5 years. Various CPG were applicable in regions, countries or continents and targeted as users oncologists only, physicians of different specialities or globally health professionals, health managers and patients.
The observed heterogeneity could be interpreted by two distinct phenomena: lack of standardisation and lack of unanimous acceptance of developing tools or unavoidable emergence of flexible divergence in order to accommodate different socioeconomic contexts. We believe that both mechanisms are at play. No standard algorithm for producing CPG is of proven superiority, the parameters to consider and methods to measure such superiority being poorly defined. No unique rating scale for evaluating level of evidence/strength of recommendation has been widely adopted so far. We believe that the GRADE scheme is the most likely candidate to gain broad acceptance in health care disciplines, as it quantifies the strength of recommendations taking into account the estimate of effect of an intervention, the design, execution, consistency, precision, reporting bias of studies, the patient health benefits versus harms and the health system costs. Still, drawbacks are present: it does not completely eliminate subjectivity, it loses information through categorisation, there is no guarantee about its generalisability. Finally, its application will require significant additional resources. Adaptation policies for CPG are only recently beginning to be developed. The varying guideline update intervals reflect both lack of standardised, electronic update tools as well as differential health system flexibility. There are varying amount of efforts invested in guidelines with extensive data mining and consultation processes versus more flexible datasets. They also mirror different perception of the rapidity and clinical applicability of research breakthroughs [24].
We found guideline evaluation to be one of the most poorly implemented and among the most diverse aspects of the organisations that we screened. Any evaluation strategy may study the implementation/adoption of the guideline by the target user and most importantly, its impact on patient, physician and health system outcomes. We encountered a range of evaluation philosophies ranging from periodic, ‘loose’ evaluations on the basis of surveys at irregular intervals to regular ones with well-organised data capture. Even among the latter, in only two instances were evaluation data freely available online. Inherent problems with evaluation approaches are the definition of parameters (end points) to be studied, the resources and infrastructure required for reliable data collection and the techniques used to analyse the data. In oncology, most evaluation analyses study survival or quality of life as relevant patient outcome indices, GL adoption as physician outcomes and optimal resource allocation, cost-effectiveness, timely and quality care as health system outcome measures. We strongly believe that further validation and improvement of issued guidelines can only be sought after systematic, reliable evaluation of their impact on health outcomes. Such evaluation could generate invaluable information on benefits, deficits and areas for change of aspects of guideline development, reporting and end-products.
Heterogeneity stemming from flexibility in order to accommodate different socioeconomic and cultural backgrounds is more controversial. Such heterogeneity of CPG was often seen as a negative phenomenon, interfering with their quality, dissemination and adoption, thus worth eliminating. International collaboration has been deemed necessary to deal with it, avoid duplication of efforts, minimise costs and guarantee high quality [3, 14, 25]. However, a prerequisite for such cooperative efforts is consensus on the end-product to be had. Such consensus is not always established, as may be seen by examining the definition of guidelines cited in the introduction. CPG are ‘systematically developed statements’, but how do we define ‘systematic’? Should one use the strict definition of the term, which is evaluated by 21 parameters in the QUOROM instrument and by 29 in the one suggested by Vigna-Taglianti, or stick to the more liberal definition of literature search in on-line databases [26, 27]? The goal is to identify the most relevant evidence which is still evaluated and synthesised by the expert opinions of developers and reviewers. Accordingly, evidence-based recommendations should perhaps more realistically be termed bias-minimised recommendations. CPG aim to ‘assist practitioner and patient decisions’. It is crucial to define which practitioner would that be. The medical oncologist only? The medical, radiation or surgical oncologists, based on tertiary reference centres? The community physicians, other health specialists or health professionals? It is also pivotal to consider whether health provider input is expected to influence decisions reached. Moreover, patient involvement in decision making may vary according to social, financial and cultural contexts, therefore making this parameter less concrete [15, 28]. Finally, decisions are expected ‘about appropriate health care for specific clinical circumstances’, the latter definition being as heterogeneous at it gets in real life. Appropriate health care could be any relatively safe treatment providing some survival or quality-of-life benefit or only a cost-effective therapy that provides substantial clinical benefit with reasonable and affordable resource allocation. Is it feasible to homogeneously rank benefits and costs of interventions in different societies, such as poor or rich countries, privately funded or nationally funded health systems where resources and criteria for their allocation differ? Specific clinical circumstances also vary by culture, health system, country, medical technology and resources available [29–31]. Moreover, there is disagreement on how quickly appropriate health care changes and specific clinical circumstances evolve, hence a different opinion on the optimal frequency of guideline updates [24].
We argue that such CPG heterogeneity is not detrimental for guideline quality, dissemination and adoption. A key point is to define clearly what the guideline intends to do, for whom and in which circumstances. There are different needs to be met by CPG in various health systems, societies, among health professionals, patients and organisational structures. Provided methodological standards are adhered to so as to guarantee high-quality, heterogeneity in aspects of development, structure, context, target user and end point definitions may be needed in order to better meet divergent patient and physician demands in a caleidoscopic world. Evaluation of guideline impact on health outcomes in each societal background should help further refine the end-product. International collaboration is certainly the one to build on, though with room for variance.
Received for publication February 4, 2008. Revision received May 31, 2008. Accepted for publication June 9, 2008.
|
References |
|---|
|
TopAbstractintroductionmethodsresultsdiscussionReferences |
|---|
1. Committee to Advise the Public Health Service on Clinical Practice Guidelines. Institute of Medicine. In: Clinical Practice Guidelines: Directions for a New Program—Field MJ, Lohr KN, eds. (1990) Washington, DC: National Academy Press.
2. Wollersheim H, Burgens J, Grol R. Clinical guidelines to improve patient care. Neth J Med (2005) 63:188–192.[Web of Science][Medline]
3. Woolf SH, Grol R, Hutchinson A, et al. Clinical guidelines: potential benefits, limitations, and harm of clinical guidelines. BMJ (1999) 318:527–530.
4. Smith TJ, Hilnner BE. Ensuring quality cancer care by the use of clinical practice guidelines and critical pathways. J Clin Oncol (2001) 19:2886–2897.
5. Ray-Coquard I, Philip T, de Laroche G, et al. A controlled "before-after" study: impact of a clinical guidelines programme and regional cancer network organisation on medical practice. Br J Cancer (2002) 86:313–321.[CrossRef][Web of Science][Medline]
6. Shiffman RN, Shekelle P, Overhage M, et al. Standardising reporting of clinical practice guidelines: a proposal from the conference on guideline standardisation. Ann Intern Med (2003) 139:493–498.
7. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care (2003) 12:18–23.
8. ADAPTE. Manual for Guideline Adaptation (2007) ADAPTE Group. www.adapte.org (31 May 2008, date last accessed).
9. GRADE Working Group. Grading quality of evidence and strength of recommendations. BMJ (2004) 328:1490–1497.
10. SIGN 50. A Guideline Developer's Handbook (2004) Edinburgh: Scottish Intercollegiate Guidelines Network. www.sign.ac.uk/guidelines/fulltext/50/index.html (31 May 2008, date last accessed).
11. National Institute for Health and Clinical Excellence. The Guidelines Manual (2007) London: National Institute for Health and Clinical Excellence. www.nice.org.uk.
12. Woolf SH, Di Guiseppi CG, Atkins D, Kamerow DB. Developing evidence-based clinical practice guidelines: lessons learned by the US Preventive Services Task Force. Annu Rev Public Health (1996) 17:511–538.[Web of Science][Medline]
13. Pavlidis N. Towards a convenient way to practice medical oncology. Ann Oncol (2007) 18(Suppl 2):ii3–ii4.
14. Fervers B, Philip T, Haugh M, et al. Clinical practice guidelines in Europe: time for European co-operation for cancer guidelines. Lancet Oncol (2003) 4:139–140.[CrossRef][Web of Science][Medline]
15. Eisinger F, Geller G, Burke W, Holtzman NA. Cultural basis for differences between US and French clinical recommendations for women at increased risk of breast and ovarian cancer. Lancet (1999) 353:191–200.
16. Fervers B, Burgers JS, Haugh MC, et al. Predictors of high quality clinical practice guidelines: examples in oncology. Int J Qual Health Care (2005) 17:123–132.
17. Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technology Assessment (1998) 2:1–88.[Medline]
18. Browman GP, Levine MN, Mohide EA, et al. The practice guidelines development cycle: a conceptual tool for practice guidelines development and implementation. J Clin Oncol (1995) 13:502–512.
19. Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force. A review of the process. Am J Prev Med (2001) 20(3 Suppl):21–35.[Web of Science][Medline]
20. Ebell MH, Siwek J, Weiss BD, et al. Simplifying the language of evidence to improve patient care. Strength of Recommendation Taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. J Fam Pract (2004) 53:111–120.[Web of Science][Medline]
21. Choudhry NK, Stelfox HT, Desky AS. Relationships between authors of clinical practice guidelines and the pharmaceutical industry. JAMA (2002) 287:612–617.
22. American Society of Clinical Oncology Guideline Procedures Manual (December 2006). Expert Panel Version 2.0. Alexandria, VA: ASCO;. www.asco.org (31 May 2008, date last accessed).
23. Eccles M, Mason J. How to develop cost-conscious guidelines. Health Technol Assess (2001) 5:1–69.[Medline]
24. Shekelle PG, Ortiz E, Rhodes S, et al. Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated? JAMA (2001) 286:1461–1467.
25. Fervers B, Philip T, Browman GP. Critical appraisal of the MCR of the European Society for Medical Oncology: challenges for a European framework for the development of clinical practice guidelines. Ann Oncol (2002) 13:1507–1510.
26. Mother D, Cook DJ, Eastwood S, et al. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Quality of Reporting of Meta-analyses. Lancet (1999) 354:1896–1900.[CrossRef][Web of Science][Medline]
27. Vigna-Taglianti F, Vineis P, Liberati A, Faggiano F. Quality of systematic reviews used in guidelines for oncology practice. Ann Oncol (2006) 17:691–701.
28. Surbone A. Cultural aspects of communication in cancer care. Support Care Cancer (2008) 16:235–240. doi: 10.1007/s00520-007-0366-0.[CrossRef][Web of Science][Medline]
29. Ward E, Halpern M, Schrag N, et al. Association of insurance with cancer care utilisation and outcomes. CA Cancer J Clin (2008) 58:9–31.
30. Norheim OF. High-cost therapy. Ethical principles of allocation of scarce resources. Tidsskr Nor Laegeforen (1992) 112:3115–3118.[Medline]
31. World Health Organisation. Science and technology for health promotion in developing countries. WHO Chron (1979) 33:399–406.[Web of Science][Medline]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  N. Pavlidis, H. Hansen, and R. Stahel ESMO Clinical Recommendations: using the easier and faster approach to oncology guidelines Ann. Onc., May 1, 2009; 20(suppl_4): iv7 - iv9. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||