© 2007 European Society for Medical Oncology
lung cancer |
The impact of anemia on outcome of chemoradiation for limited small-cell lung cancer: a combined analysis of studies of the National Cancer Institute of Canada Clinical Trials Group
1 The Ottawa Hospital Regional Cancer Centre, Ottawa
2 National Cancer Institute of Canada Clinical Trials Group, Kingston
3 Princess Margaret Hospital, Toronto
4 British Columbia Cancer Agency, Vancouver, Canada
* Correspondence to: Dr S. A. Laurie, The Ottawa Hospital Regional Cancer Centre, 501 Smyth Road, Ottawa, Ontario, K1H 8L6 Canada. Tel: +1-613-737-7700 ext. 70175; Fax: +1-613-247-3511; E-mail: slaurie{at}ottawahospital.on.ca
 |
Abstract |
|---|
 Top Abstract introductionpatients and methodsresultsdiscussionReferences |
|---|
Background: Associations between anemia and outcomes of chemoradiation have been documented in several malignancies, but few data exist for limited small-cell lung cancer (LD-SCLC). This combined analysis of 652 patients in two randomized clinical trials in LD-SCLC carried out by the National Cancer Institute of Canada Clinical Trials Group was undertaken to explore the relationship between anemia at baseline and anemia arising during therapy, and outcomes of chemoradiation in this cancer.
Patients and methods: The relationships between overall survival and local control with hemoglobin levels at baseline and those arising during therapy (nadir hemoglobin (Hb) and maximum percentage drop from baseline values) were explored.
Results: No Hb parameter was associated with either outcome. Baseline anemia was found in one-third of patients, was more common in males, in those with a poorer performance status and those with an elevated lactate dehydrogenase; all of these latter factors were associated with shorter survival. A trend towards improved local control in patients with the greatest drop in their Hb did not remain significant in a multivariate analysis.
Conclusions: Anemia is common in patients with LD-SCLC. Anemia at diagnosis may have a different prognostic implication than that arising during therapy, and correction of anemia may have no impact on outcomes.
Key words: anemia, chemotherapy, radiation, lung cancer
|
introduction |
|---|
|
TopAbstractintroductionpatients and methodsresultsdiscussionReferences |
|---|
Anemia is common in cancer patients, and its etiology is multifactorial, including blood loss, bone marrow invasion, the anemia of chronic disease, and treatment-induced inhibition of erythropoiesis. Anemia, whether present at baseline before the initiation of therapy or arising during treatment, has been associated with a poorer outcome from radiation or chemoradiation for cancers of the head and neck [1], cervix [2], and non-small-cell lung cancer (NSCLC) [3]. It is not known whether this association is causative (anemia leading to relative tumor hypoxia and thus to radioresistance) or simply prognostic or predictive.
Small-cell lung cancer (SCLC) accounts for 
15% of all lung malignancies. For patients with limited-stage disease, standard therapy consists of cisplatin–etoposide (EP) combination chemotherapy [4] given with early, concurrent thoracic radiotherapy [5, 6]. In patients who are able to tolerate such combined modality therapy, 5-year survival rates of up to 26% have been reported [7]. There are several accepted prognostic factors for outcome of therapy with chemoradiation in limited small-cell lung cancer (LD-SCLC), including gender, performance status (PS), and serum lactate dehydrogenase (LDH) levels [8–10]. There are limited and conflicting data on the impact of baseline anemia on LD-SCLC, with some reporting a negative impact [9] while others report no association [11, 12].
To examine the potential association between anemia and outcome in LD-SCLC, we undertook this retrospective analysis of data from two previously reported multicenter randomized controlled trials conducted by the National Cancer Institute of Canada Clinical Trials Group (NCIC CTG) [5, 13]. Compared with institutional retrospective reviews, this analysis has the benefit of a large number of homogenously treated patients with a low rate of missing data.
|
patients and methods |
|---|
|
TopAbstractintroductionpatients and methodsresultsdiscussionReferences |
|---|
patients
The NCIC CTG has conducted two randomized controlled trials in LD-SCLC, BR.3, and BR.6, both of which have been previously published [5, 13]. BR.3 randomized patients to receive six cycles of chemotherapy with EP and cyclophosphamide–doxorubicin–vincristine (CAV) given in either a sequential (three cycles of CAV followed by three cycles of EP) or alternating fashion. In this trial, thoracic radiotherapy (either 37.5 Gy in 15 fractions or 25 Gy in 10 fractions, at the institution's discretion) was administered to all patients following the completion of chemotherapy, while prophylactic cranial irradiation (PCI) (20 Gy in five fractions) was administered to all patients during week 7. BR.6 was designed to evaluate the importance of early versus late concurrent thoracic radiotherapy. All patients received six cycles of alternating CAV/EP, with thoracic radiotherapy (40 Gy in 15 fractions) administered concurrently with either cycle 2 or 6 of chemotherapy. PCI (25 Gy in 10 fractions) was administered following the completion of chemoradiation.
Three hundred and twenty-two patients were enrolled to BR.3 and 330 to BR.6. Thus, a total of 652 patients are the subject of this report.
statistical methods
Exploratory analyses were carried out to characterize the relationships between various hemoglobin (Hb) parameters and patient characteristics and treatment outcomes. The Hb parameters examined were those that had been found to be associated with outcome in other malignancies [1–3, 14, 15] and included baseline Hb (before initiation of any therapy) dichotomized as normal versus low (<136 g/l for men, <120 g/l for women); nadir Hb during treatment (<100 g/l versus 
100 g/l); maximum percentage drop from baseline (<10% versus 10%–30% versus >30%); and Hb level immediately before the initiation of PCI (pre-PCI Hb) (<100 g/l versus 100 g/l).
Overall and progression-free survivals were defined as the time from randomization until death from any cause or until disease progression or death, respectively. Local disease control was defined as the time from randomization to the time of disease progression in the chest (or in the central nervous system (CNS), for analyses related to PCI). Kaplan–Meier curves were used to estimate the distributions of the time-to-event outcomes [16]. The log-rank test [17] was used to assess the association between Hb level and these time-to-event outcomes. A chi-square test or Fisher's exact test [18] was used to assess associations between categorical variables, while the Cox regression model [19], stratified by treatment arm, using a stepwise procedure with a significance level of 0.1 for factors to stay, was used to study the effect of Hb level on time-to-event outcomes while adjusting for baseline prognostic factors. These factors included gender, age, Eastern Cooperative Oncology Group (ECOG) PS, body surface area (BSA), race, and LDH. The data were analyzed using the statistical package SAS® Version 8.2 (SAS Institute Inc., Cary, NC). All reported P values are from two-sided tests unless otherwise specified.
|
results |
|---|
|
TopAbstractintroductionpatients and methodsresultsdiscussionReferences |
|---|
baseline Hb
Of the 652 patients randomized in BR.3 and BR.6, a low baseline Hb was present in 32% (210) of patients (Table 1). Males (P < 0.001), patients >65 (P = 0.03), those with ECOG PS
2 (P = 0.02), and those with increased LDH (P = 0.03) were more likely to present with low baseline Hb. Race, BSA, trial or treatment arm were not associated with baseline Hb value.
|
Tables 2, 3 and 4 summarize the impact of the various Hb parameters on outcomes, as well as those baseline prognostic factors that did have an independent association with outcome. Baseline Hb was not statistically associated with either overall (P = 0.58) or progression-free survival (P = 0.35) in univariate analysis (log-rank test stratified by treatment). In the multivariate Cox regression model stratified by treatment arm, male gender, a PS
2, and an elevated LDH were associated with poorer overall and progression-free survival, while baseline Hb remained nonsignificant. Similarly, baseline Hb had no impact on the rate of local (chest) progression/relapse (P = 0.62) (Table 2).
|
|
|
nadir Hb during therapy and maximum percentage drop
Of the 652 patients in BR.3 and BR.6, 633 had at least one Hb value recorded during treatment and were included in the analysis. Nadir Hb during treatment of
grade 2 (<100 g/l) was observed in 55% of patients. Baseline factors that were associated with grade 2 nadir Hb during treatment included female gender (P < 0.001), presence of a baseline low Hb (P < 0.001), age >65 (P = 0.002), and a BSA of <2 (P < 0.001). Univariate analysis (log-rank test stratified by treatment arm and baseline Hb level) showed that nadir Hb was not independently associated with overall (P = 0.79) or progression-free survival (P = 0.2). Those with a nadir Hb <100 g/l, however, had a longer duration of freedom from local (chest) recurrence [Hazard ratio (HR) = 0.70, 95% confidence interval (CI) 0.54–0.90, P = 0.005; Figure 1]. Multivariate Cox regression analysis showed that males had a higher risk of local progression (HR = 1.6, 95% CI 1.2–2.1; P = 0.001), while a nadir Hb <100 g/l remained marginally significant (P = 0.06) (Table 3). Similar analyses showed that the maximum percentage drop in Hb from baseline did not predict for survival (P = 0.27), nor did it predict local progression (P = 0.33) (Table 4).
|
immediate pre-PCI Hb
This analysis was restricted to the 523 patients who received PCI. A lower Hb immediately before the initiation of PCI was associated with pretreatment anemia (P < 0.001), age >65 (P = 0.004), female gender (P < 0.001), ECOG PS
2 (P = 0.04), BSA <2 (P < 0.001), and enrollment to trial BR.6 (P < 0.001). A low pre-PCI Hb, however, was not associated with an increased risk of local (CNS) relapse (P = 0.31).
effect of dose intensity of chemotherapy and dose of radiation on outcomes
Multivariate Cox regression analyses stratified by treatment arm and baseline Hb level were also carried out to evaluate the effect of minimum Hb level on outcomes, while adjusting for chemotherapy and radiotherapy dose/dose intensity, as well as baseline age, gender, ECOG PS, BSA, race, and LDH. These analyses revealed that there was no impact of either chemotherapy or radiotherapy doses or dose intensity on outcomes.
|
discussion |
|---|
|
TopAbstractintroductionpatients and methodsresultsdiscussionReferences |
|---|
The current analysis of two randomized trials carried out by the NCIC CTG enrolling >600 patients with LD-SCLC was designed to investigate the impact of anemia, both at baseline and arising during treatment, on outcomes. It confirmed, as have other studies, that male gender, poor PS, and an elevated LDH are negative prognostic factors in this disease. The impact of anemia, however, is less clear. Consistent with what others have observed [11, 20], anemia is common at diagnosis in patients with LD-SCLC, seen in one-third of patients enrolled to these two trials. The data on whether baseline pretreatment anemia is a negative prognostic factor in LD-SCLC are conflicting, with reports both supporting [9] and, as in the current study, not observing [10–12] an association. Any association between baseline anemia and poorer outcome is not necessarily causative, as it may be that anemia at presentation is simply another surrogate marker for increased burden of disease (much like LDH), and/or indicates the presence of an intrinsically more resistant tumor. Supporting this latter hypothesis is the fact that baseline anemia in the current analysis was associated with several factors that are themselves associated with poorer outcome in SCLC.
Significant anemia of at least grade 2 was observed in 55% of patients during their treatment course; this was more likely to occur in females, those >65, and those who were anemic at presentation. There are less data on the impact of anemia arising during therapy for patients treated with chemoradiation for LD-SCLC. In keeping with the hypothesis that anemia increases tumor hypoxia and thus radioresistance, others have found that preradiotherapy anemia [11] or a drop in Hb of >10 g/l [12], are associated with decreased overall survival in these patients. The current study, however, did not find any negative association between outcomes and the nadir Hb or the maximum percentage drop in Hb during therapy. Instead, a trend towards ‘improved’ local control was observed among patients who had the greatest drop in their Hb during their therapy. The failure of an improvement in local control to translate into an improvement of overall survival is not unexpected in a systemic disease such as SCLC. It is interesting to note that while males were more likely to have anemia at baseline, females were more likely to develop anemia during therapy. In another analysis of trials carried out by the NCIC CTG in both LD- and extensive stage-SCLC [21], females were more likely to suffer hematological toxicity, including anemia, but despite this, females did not have lower dose intensity than males, and female gender is consistently found to be a favorable prognostic factor in SCLC
An association between increased hematological toxicity and improved survival has been observed in other malignancies. For example, an analysis of another NCIC CTG clinical trial (JBR.10) of adjuvant chemotherapy versus observation following complete resection of NSCLC has shown that those patients who have the greatest percentage drop in their Hb while receiving adjuvant chemotherapy have a significantly longer survival [22]. Neutropenia has been shown to be associated with improved outcome in patients receiving adjuvant chemotherapy for breast cancer [23, 24] and in patients receiving chemotherapy for metastatic NSCLC [25].
If this trend reflects a true association, and not merely a chance finding of multiple statistical comparisons, it indicates that this may be a marker for some facet of increased treatment intensity in those patients who develop the greatest degree of anemia. Although the results of the current analysis did not change when dose intensity of chemotherapy and radiation were incorporated into the statistical model, dose intensity is not necessarily indicative of dose effect for an individual patient. It is recognized that dosing based on BSA has little relationship to individual patient pharmacokinetics or pharmacodynamics. For example, in a study of patients with SCLC undergoing chemotherapy with a regimen of doxorubicin–ifosfamide–etoposide, no correlation with BSA and the clearance of any of the drugs was detected [26]. Others have likewise shown that etoposide clearance is related to several factors, including serum albumin and creatinine, but not to BSA [27], while only a weak correlation has been detected between cisplatin clearance and BSA [28]. Both cisplatin and etoposide show large interpatient variability which is not explainable by BSA [26, 28]. The most common method of calculating BSA, the DuBois formula [29], has been reported to overestimate the true BSA by at least 15% in one-sixth of patients [30], and this may partly explain the increased treatment-emergent anemia in those with a BSA <2.
Anemia may have different prognostic implications at different times in the disease course. At diagnosis, it may be a negative prognostic factor, reflecting disease burden, while if it arises during therapy (particularly in those who were not anemic at baseline), it may be a positive factor. In either situation, correction of the anemia per se is unlikely to have an affect on outcome. In fact, it may be that to improve outcome, hematologic toxicity should be a target of therapy. It is possible, however, that correction of such a resulting anemia would improve quality of life and/or tolerance of therapy, and this requires further study.
Given that the data regarding the impact of anemia on outcomes of therapy in this and other malignancies are conflicting, caution is warranted regarding the routine use of erythropoietin in patients undergoing curative combined modality treatment protocols. A blinded randomized trial adding erythropoietin to standard chemoradiation for LD-SCLC was halted early due to a higher rate of thrombotic events in the experimental arm [31]; no improvement in outcome was observed despite a clear increase in Hb levels in the erythropoietin arm. Two randomized trials adding erythropoietin to definitive radiotherapy for squamous cell carcinoma of the head and neck have been reported [32, 33]. Neither showed a benefit from the addition of erythropoietin, despite higher mean Hb levels in the active treatment arms; in fact one trial [32] indicates that the addition of erythropoietin may be detrimental to outcome. Thus, despite the consistent finding in head and neck cancer that preradiotherapy anemia is negatively prognostic, correction of such an anemia with the use of erythropoietin has not been shown to improve outcomes of therapy. These findings challenge the assumption that correction of anemia with erythropoietin will improve tumor oxygenation and hence radiosensitivity. A mathematical model of blood flow indicates that maximal oxygen deliver occurs at a Hb value of 110 g/l, and that a significant drop in tumor oxygenation would not occur until <80 g/l [34]. Further, erythropoietin receptors have been detected on a variety of tumor cells, and erythropoietin may have antiapoptotic and/or proliferative activity [35]. A retrospective review of a subset of patients enrolled to one of the randomized trials of erythropoietin or placebo added to radiotherapy for advanced squamous cell carcinoma of the head and neck has been carried out [36]. It revealed that two-thirds of tumors stained positively for erythropoietin receptors, and in those receptor-positive patients, the use of erythropoietin was associated with a poorer locoregional progression-free survival; erythropoietin use was not detrimental in those patients with receptor-negative tumors.
Thus, in the absence of data that use of erythropoietin improves outcome in LD-SCLC or any other malignancy and given the theoretical risk of tumor protection and promotion and the known risk of thrombotic events, transfusions remain an acceptable, and cheaper, option for the treatment of anemia in these patients. Prospective clinical trials are needed to evaluate the relative merits of transfusions versus exogenous erythropoietin, different target Hb levels, and the differential impact of erythropoietin on receptor-positive and -negative tumors.
Received for publication November 13, 2006. Revision received February 2, 2007. Accepted for publication February 5, 2007.
|
References |
|---|
|
TopAbstractintroductionpatients and methodsresultsdiscussionReferences |
|---|
1. Glaser CM, Millesi W, Kornek GV, et al. Impact of hemoglobin level and use of recombinant erthyropoietin on efficacy of preoperative chemoradiation therapy for squamous cell carcinoma of the oral cavity and oropharynx. Int J Radiat Oncol Biol Phys (2001) 50:705–715.[CrossRef][Web of Science][Medline]
2. Obermair A, Cheuk R, Horwood K, et al. Impact of hemoglobin levels before and during concurrent chemoradiotherapy on the response of treatment in patients with cervical carcinoma. Cancer (2001) 92:903–908.[CrossRef][Web of Science][Medline]
3. MacRae R, Shyr Y, Johnson DH, et al. Declining hemoglobin during chemoradiotherapy for locally advanced non-small cell lung cancer is significant. Radiother Oncol (2002) 64:37–40.[CrossRef][Web of Science][Medline]
4. Laurie SA, Logan D, Markman BR, et al. Practice guideline for the role of combination chemotherapy in the initial management of limited-stage small-cell lung cancer. Lung Cancer (2004) 43:223–240.[CrossRef][Web of Science][Medline]
5. Murray N, Coy P, Pater JL, et al. Importance of timing for thoracic irradiation in the combined modality treatment of limited-stage small-cell lung cancer. J Clin Oncol (1993) 11:336–344.
6. Takada M, Fukuoka M, Kawahara M, et al. Phase III study of concurrent versus sequential thoracic radiotherapy in combination with cisplatin and etoposide for limited-stage small-cell lung cancer: results of the Japan Clinical Oncology Group study 9104. J Clin Oncol (2002) 20:3054–3060.
7. Turrisi AT, Kim K, Blum R, et al. Twice-daily compared with once-daily thoracic radiotherapy in limited small-cell lung cancer treated concurrently with cisplatin and etoposide. N Engl J Med (1999) 340:265–271.
8. Sagman U, Feld R, Evans WK, et al. The prognostic significance of pretreatment serum lactate dehydrogenase in patients with small-cell lung cancer. J Clin Oncol (1991) 9:954–961.[Abstract]
9. Bremnes RM, Sundstrom S, Aasebo U, et al. The value of prognostic factors in small cell lung cancer: results from a randomised multicenter study with minimum 5 year follow-up. Lung Cancer (2003) 39:303–313.[CrossRef][Web of Science][Medline]
10. Paesmans M, Sculier JP, Lecomte J, et al. Prognostic factors for patients with small cell lung carcinoma: analysis of a series of 763 patients included in 4 consecutive prospective trials with a minimum follow-up of 5 years. Cancer (2000) 89:523–533.[CrossRef][Web of Science][Medline]
11. Laurie SA, Jeyabalan N, Nicholas G, et al. Association between anemia arising during therapy and outcomes of chemoradiation for limited small cell lung cancer. J Thorac Oncol (2006) 1:146–151.[Medline]
12. Ash R, Videtic G, Rodrigues G, et al. The impact of hemoglobin levels on outcome in limited stage small cell lung cancer. Lung Cancer (2005) 49(Suppl 2):S316. (Abstr P-749).
13. Feld R, Evans WK, Coy P, et al. Canadian multicenter randomized trial comparing sequential and alternating administration of two non-cross-resistant chemotherapy combinations in patients with limited small-cell carcinoma of the lung. J Clin Oncol (1987) 5:1401–1409.
14. Lee WR, Berkey B, Marcial V, et al. Anemia is associated with decreased survival and increased locoregional failure in patients with locally advanced head and neck carcinoma: a secondary analysis of RTOG 85-27. Int J Radiat Oncol Biol Phys (1998) 42:1069–1075.[CrossRef][Web of Science][Medline]
15. Grogan M, Thomas GM, Melamed I, et al. The importance of hemoglobin levels during radiotherapy for carcinoma of the cervix. Cancer (1999) 86:1528–1536.[CrossRef][Web of Science][Medline]
16. Kaplan E, Meier P. Nonparametric estimation from incomplete observations. J Am Stat Assoc (1958) 53:457–481.[CrossRef][Web of Science]
17. Mantel N. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer Chemother Rep (1966) 50:163–170.[Medline]
18. Pearson K. On the 
2 test of goodness of fit. Biometrika (1922) 9:186–191.
19. Cox DR. Regression models and life tables. J R Stat Soc B (1972) 34:181–220.
20. Sundstrom S, Bremnes RM, Kaasa S, et al. Cisplatin and etoposide regimen is superior to cyclophosphamide, epirubicin, and vincristine regimen in small-cell lung cancer: results from a randomized phase III trial with 5 years' follow-up. J Clin Oncol (2002) 20:4665–4672.
21. Singh S, Parulekar W, Murray N, et al. Influence of sex on toxicity and treatment outcome in small-cell lung cancer. J Clin Oncol (2005) 23:850–856.
22. Gauthier I, Ding K, Winton T, et al. Do hemoglobin (Hb) levels influence outcomes of adjuvant chemotherapy (ACT) with cisplatin/vinorelbine in patients (pts) with completely resected non small cell lung cancer (NSCLC)? The JBR.10 trial experience. Lung Cancer (2005) 49(Suppl 2):S280. (Abstr P-615).
23. Cameron DA, Massie C, Kerr G, et al. Moderate neutropenia with adjuvant CMF confers improved survival in early breast cancer. Br J Cancer (2003) 89:1837–1842.[CrossRef][Web of Science][Medline]
24. Mayers C, Panzarella T, Tannock IF. Analysis of the prognostic effects of inclusion in a clinical trial and of myelosuppression on survival after adjuvant chemotherapy for breast carcinoma. Cancer (2001) 91:2246–2257.[CrossRef][Web of Science][Medline]
25. DiMaio M, Gridelli C, Gallo C, et al. Chemotherapy-induced neutropenia and treatment efficacy in advanced non-small-cell lung cancer: a pooled analysis of three randomized trials. Lancet Oncol (2005) 6:669–677.[Web of Science][Medline]
26. Freyer G, Tranchand B, Ligneau B, et al. Population pharmacokinetics of doxorubicin, etoposide and ifosfamide in small cell lung cancer patients: results of a multicenter study. Br J Clin Pharmacol (2000) 50:315–324.[CrossRef][Web of Science][Medline]
27. Nguyen L, Chatelut E, Chevreau C, et al. Population pharmacokinetics of total and unbound etoposide. Cancer Chemother Pharmacol (1998) 41:125–132.[CrossRef][Web of Science][Medline]
28. de Jongh FE, Verweij J, Loos WJ, et al. Body-surface area-based dosing does not increase accuracy of predicting cisplatin exposure. J Clin Oncol (2001) 19:3733–3739.
29. DuBois D, DuBois EF. A formula to estimate the approximate surface area is height and weight be known. Arch Intern Med (1916) 17:863–871.[Web of Science]
30. Gehan EA, George SL. Estimation of human body surface area from height and weight. Cancer Chemother Rep (1970) 54:225–235.[Web of Science][Medline]
31. Goss G, Feld R, Bezjak A, et al. Impact of maintaining hemoglobin with epoetin alfa on time to progression, overall survival, quality of life and transfusion reduction in limited disease SCLC patients. Lung Cancer (2005) 49(Suppl 2):S53. (Abstr O-154).[Medline]
32. Henke M, Laszig R, Rube C, et al. Erythropoietin to treat head and neck cancer patients with anaemia undergoing radiotherapy: randomised, double-blind, placebo-controlled trial. Lancet (2003) 362:1255–1260.[CrossRef][Web of Science][Medline]
33. Machtay M, Pajak T, Suntharalingam S, et al. Definitive radiotherapy +/- erythropoietin for squamous cell carcinoma of the head and neck: preliminary report of RTOG 99–03. Int J Radiat Oncol Biol Phys (2004) 60(Suppl 1):S132. (Abstr 5).
34. Fyles AW, Milosevic M, Pintilie M, et al. Anemia, hypoxia and transfusion in patients with cervix cancer: a review. Radiother Oncol (2000) 57:13–19.[CrossRef][Web of Science][Medline]
35. Hardee ME, Arcasoy MO, Blackwell KL, et al. Erythropoietin biology in cancer. Clin Cancer Res (2006) 12:332–339.
36. Henke M, Mattern D, Pepe M, et al. Do erythropoietin receptors of cancer cells explain unexpected clinical findings? J Clin Oncol (2006) 24:4708–4713.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||