{'fraction_non_alphanumeric': 0.025399077687123094, 'fraction_numerical': 0.0071301880099326, 'mean_word_length': 5.18766461808604, 'pattern_counts': {'":': 1, '<': 0, '<?xml version=': 0, '>': 2, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Analysis of the impact of professional involvement in evidence generation for the HTA Process, subproject "cochlear implants": methodology, results and recommendations.

SUMMARY The aim of Health Technology Assessment (HTA) is to provide decision-makers, distributors and recipients with information on the effectiveness, cost and impact of health technologies. The present study constitutes a subproject within the wider project “Analysis of the impact of professional involvement in evidence generation for the HTA process”, which is part of the strategic programme “Transfer of the results of the research in clinical practice and organisation of healthcare services”, coordinated by Laziosanità – Agency of Public Healthcare of the Lazio Region and AgeNaS (National Agency for Regional Healthcare Services). The objectives of the present subproject (cochlear implants) are as follows: a) to produce a report regarding the health impact of cochlear implants (CI) on their recipients, through a systematic review of literature and extensive selection of relative studies, combining the outcomes with metanalytical techniques. Output: report on the indications of usage in the groups of population for which benefits are controversial; b) to create a registry of patients using cochlear implants. The registry should contain a selection of anagraphic and clinical information relative to patient follow-up in order to assess factors associated with safety and impact on cochlear implant users. This source of information is essential for future observational studies. This was divided into 4 phases: 1st phase: definition of key participants in the assessment process; 2nd phase: definition of methods and timing of “Aims” (definition of the objective); 3rd phase: definition of the methods and times of the “assessment process”, 4th phase: production of the final report. From the analysis of systematic reviews and italian and international guidelines, the Working Group members approved recommendations on the following topics: results after CI in children in relation to age at implantation, bilateral CI in children, CI in deaf children with associated disabilities, CI in adults with advanced age, bilateral CI in adults and CI in adults with pre-lingual deafness. These recommendations have also been evaluated by the Consulting Committee members and approved with minimal suggestions.


Introduction
Why a report on Health Technology Assessment? over the last decades, the continuous growth of medical technology in the healthcare system has led to an unlimited expansion of services, but at the same time has made it clear that the amount of resources that the national health Service can offer to its patients is limited 1 . Therefore, health policy decisions are becoming increasingly important, and it seems necessary to consider carefully investment choices that can be can afforded and worth making. For this reason, decision-makers within the public health system should be provided with specific tools to assess the validity, effectiveness, safety and cost efficacy of the different healthcare improvements suggested in order to decide whether they are worth adopting. The aim of Health Technology Assessment (hTA) is to provide decision-makers, distributors and recipients with information on the effectiveness, cost and impact of health technologies [2][3][4][5] . An hTA review should have solid scientific foundations that support its conclusions, but should also be simple and easily understood by decision-makers who are not experts in the field. For these reasons, the report includes an "executive summary", written in simple, non-technical language, and a "full report" containing various chapters and appendices with all the scientific information conveyed in the executive summary. The discussion, conclusions and recommendations of the present article reflect those reported in the executive summary of the report on the web site https://sites.google.com/site/impianticoclearipisa/, and will be soon inserted in the Age.na.S website (www. agenas.it/). Within the italian health System, many of the decisions as to what should be considered an essential level of Assistance are made by on a regional level. unfortunately, this often leads to overlap and redundancy; at the same time, individual regions often lack sufficient resources to ensure assessment of all new technologies. For these reasons, regions have started to organize themselves in a national network coordinating the hTA activities, aimed at reducing waste and duplication of labour, while respect-ing the autonomy of each region. Age.na.S. (national Agency for regional healthcare Services) coordinates this network, which is now called the italian network of hTA, rihTA (rete italiana hTA). At the onset of this process, the Tuscany region submitted the following crucial questions to the newly-formed Coordinating group: who should be guaranteed a cochlear implant? Where is it possible to find the best cost-effective benefits? The proposal, which falls within a larger development project of hTA in italian regions, has resulted in the publication of this project. As the reader may observe, the Working group (Wg) engaged in the drafting of the report took into account all the above-mentioned issues concerning cochlear implants in both children and adults, through careful analysis not only of the most significant literature in the field, but also using less conventional information sources, with particular regard to organisational, ethical and social implications. it determined the most critical areas in which scientific evidence was still weak, and it recommended methods which could help current clinical practice to produce evidence through the creation of registries. The Wg has provided recommendations for better and more correct organizational modes for intervention. Finally, it indicates effectiveness and cost efficacy. hTA is a process of assessment according to which technicians and specialists cannot work isolated from society and citizens who will be benefit from the technology under scrutiny. This request will not only be represented by the political power elected, which in any case has full decisional independence after having been informed by the report, but should also be represented by active interlocutors in some of the crucial phases of the assessment process. For this reason, the Wg involved all stakeholders, i.e. patient associations, device manufacturers and scientific societies from the earliest phases of definition of the aims of the report, submitting a detailed list of questions to be answered and asking whether they were exhaustive, pertinent and correctly posed. This study constitutes a subproject within the broader project "Analysis of the impact of professional involvement in evidence generation for the hTA process", which is part of the strategic programme "Transfer of the results of the research in clinical practice and organisation of healthcare services". The project was coordinated by laziosanità -Agency of Public healthcare of the lazio region, while the strategic programme was coordinated by Age.na.S. The principle aim of the project, which is part of the strategic programme to assess the involvement of the specialists of national Sanitary System (nSS), is the production of scientific evidence. Within the framework of the project "Analysis of the impact of professional involvement in evidence generation for the hTA process", the Tuscany region proposed the Cochlear implants Technique, with the priority to clearly identify criteria for reimbursement by the national health Service. The methodology employed for the hTA report was tailored to the needs of the italian situation starting with the indications contained in the "guide to Technology Appraisal Process" of the uK national institute for Clinical excellence (niCe) 6 . The objectives of the present subproject (cochlear implants) are as follows: a) To produce a report concerning the health impact on cochlear implant recipients, through a systematic review of literature and extensive selection of relevant studies, combining outcomes with metanalytical techniques. output: report on the indications of usage in the patient groups for which benefits are controversial. b) To create a registry of patients using cochlear implants. The registry should contain a selection of anagraphic and clinical information that is relative to patient follow-up to assess factors associated with safety and impact on cochlear implant users. This source of information is essential for future observational studies. The Wg of the present project, after discussion and consensus, assumed that Ci is a well established clinical treatment for specific patient groups. These include adults with severe to profound sensorineural hearing loss, children under two years of age with profound sensorineural hearing loss and children between 2 and 18 years of age with severe to profound sensorineural hearing loss [7][8][9] . As a consequence, the Wg decided to focus on the following controversial and emergent Ci issues. Objective a1) With regard to the assessment of clinical effectiveness of CI procedure in children, we analyzed the following topics: post-implant outcome in relation to early implantation; bilateral cochlear implantation (simultaneous/sequential) vs. unilateral Ci and vs. bimodal stimulation; benefits from cochlear implantation in children with multiple disabilities. With regard to the assessment of clinical effectiveness of CI procedure in adults, we analyzed the following topics: unilateral Ci in the elderly; bilateral cochlear implanta-tion (simultaneous/sequential) vs. unilateral Ci and vs. bimodal stimulation; benefits from unilateral Ci in prelingual deafened adults. Objective a2) The systematic economic review evaluated the cost effectiveness of CI in children and adults. The review included primary studies of economic evaluation (analyses of cost, cost effectiveness, cost utility and cost benefit). With regard to children, the following categories have been analyzed: unilateral Ci; bilateral simultaneous Ci and bilateral sequential Ci. in adults, the following categories have been analyzed: unilateral Ci, unilateral Ci in the elderly, unilateral Ci in pre-lingual deafened adults and bilateral cochlear implantation (simultaneous/sequential). Objective b) A model of Ci recipient registry derived from the consensus of the Wg members is proposed, collecting demographic data, clinical and audiological data, surgical data, device data, rehabilitative and follow-up data. The full-length of the report is on the web site https://sites.google.com/site/impianticoclearipisa/ and will soon be inserted on the Age.na.S website (www.agenas.it/).

Methodology
The work was divided into 4 phases: 1 st phase: definition of key participants in the assessment process A Working group (Wg) was set up in the early phase of this work -June 2009. The creation of the Wg, and in particular the choice of members and recruitment, was carried out by the Scientific Coordinator of the subproject (Prof. Stefano Berrettini). All candidates in the Wg were invited to participate by the Tuscany region, specifying the aims of the project and methodology to be followed. The Wg was formed of italian researchers and clinicians from both hospital divisions and university structures with experience in the field of cochlear implants, confirmed by extensive case studies of both adult and paediatric patients, as well as scientific production. All the members of the Committee responsible for laying down the guidelines for cochlear implantation for the italian Society of otorhinolaryngology were included in the Wg 7 . experts in healthcare organization, economy and epidemiology were also included. The members of the Wg are reported in Table i. The Consulting Committee (CC) (which includes all "Stakeholders") was established. The creation of the CC, and in particular the choice of members and their recruitment, was the responsibility of the Scientific Coordinator of the subproject (Prof. Stefano Berrettini), in agreement with the members of the Wg and in collaboration with the Tuscany region. The member candidates were invited to participate in the CC. The Scientific Coordinator invited the members of the CC to participate in the project after explaining the aims and role of stakeholders. The members of the CC are listed in Table ii. in September 2009, the Wg appointed Prof. Berrettini, Prof. Turchetti and dr. Forli to proceed with the project and compile the final report. in particular, Prof. Berrettini and dr. Forli were entrusted with the elaboration of the project and final version of the report relative to clinical effectiveness of the cochlear implant procedure in adults and children, while Prof. Turchetti was assigned the report on cost-efficacy. Prof. Berrettini and dr. Forli were also appointed to proceed with the implementation of a registry to collect data on implanted patients.

nd phase: definition of methods and times of "Aims" (definition of the objective)
The creation of the Wg and CC was followed by the definition of the objectives (Aims). This document was approved by members of the Wg during the first meeting and successively sent via email and approved by members of the CC.
The aims of the subproject on Cochlear implants in the document on Aims (october 2009) is reported in the introduction section. during the meeting on 30 January 2010, the members of the Wg, given the uniform evidence of national and international literature and having studied the main international guidelines concerning the procedure in question, considered as universally accepted the usefulness/effectiveness of the unilateral Ci procedure in severe/profound deaf adults and children, and to focus the systematic reviews on clinical effectiveness and cost-efficacy of the cochlear implant procedure on more controversial issues for which international consensus is lacking. Furthermore, after a first qualitative analysis of the literature on the Ci procedure in adult and paediatric patients and a study of the main international guidelines on this procedure, the Wg reformulated the objectives of systematic reviews on the clinical effectiveness and cost efficacy of the Ci. Therefore, the definite objectives faced by the systematic reviews were the following:

rd phase: definition of the methods and times of the "assessment process"
The third phase of the project consisted in a systematic review of the literature regarding aims a1), a2) and formulation of a registry proposal of cochlear implant users b). Both the partial and definite results deriving from the systematic reviews (aims a1 and a2) were submitted to, discussed and approved by the members of the Wg during the meeting. on the basis of the data obtained from the reviews, the members of the Wg decided not to carry out a meta-analysis, owing to the heterogeneous outcomes.
According to the evidence resulting from a systematic analysis of the literature, the appropriateness of the procedure was identified. A registry proposal to be used with cochlear implant recipients was also developed. The final text of the report (containing some suggestions of the CC members) was approved by all members of the Wg (may 2011). With regard to the CI in children older than 12 months, the selected studies show better hearing and linguistic results in children implanted at earlier ages. nevertheless, the samples are heterogeneous with regard to the age at implantation and the analyzed outcome, and not all studies report statistical significance. nonetheless, a sensitive period under 24-36 months has been identified over which cochlear implantation is reported to be less effective in terms of improvements in speech and hearing. With regard to clinical effectiveness of bilateral CI, the included studies report greater benefits from bilateral implants compared to monolateral implants when assessing hearing in quiet environments, and overall in noise and sound localization abilities. Benefits are reported to be present for simultaneous or sequential bilateral implantation. however, with regard to the delay between surgeries in sequential bilateral implantation, although benefit is reported to be present even after very long delays, on average a long delay between surgeries seems to negatively affect outcome with the second implant. none of the studies report on the impact of bilateral Ci on linguistic skills and cognitive abilities.

Cost effectiveness. Systematic review
Published studies in english language were searched using the following electronic databases: Pubmed medline, Cochrane Controlled Trials register and Cochrane Systematic reviews database and Centre for reviews and dissemination (Crd) which includes database of Abstracts of review of effects (dAre), nhS economic evaluation database (nhS eed) and hTA database. The search was performed for the period 2000-September 2009 and updated to 31 may 2010. The methodology is fully described in the individual reports.
Children 68 articles were extracted by the search procedure. The reviewers eliminated 54 articles that did not meet the inclusion criteria. After reading the abstract, 11 articles were retrieved and assessed for eligibility. The reviewers manually added 5 articles identified from bibliography research. on reading the full text copies, 9 publications were considered relevant to the review. The review reports the results in terms of cost analysis, cost effectiveness and educational costs. The studies included had different characteristics in terms of nature, design and methodology of analysis: 7 studies were conducted in europe (5 in uK, 1 in germany and 1 in France), 1 in uSA and 1 in Canada. Two studies were multicentre and 4 single centre. Three studies were prospective, 3 ret-rospective and 3 cross-sectional. Five studies presented partial economic evaluation (cost analysis). Four studies presented complete economic evaluation: 2 cost utility analyses, 1 cost benefit analysis and 1 cost effectiveness analysis.
The articles related to cost analysis considered the direct and/or indirect costs of Ci. only two publications report both categories of costs. one paper discusses in detail the direct and indirect costs and the relative methodology used. The other reports the costs in an aggregate form. The direct costs ranged from € 39,172 to € 69,297 a . The main cost items were pre-operative costs, operative costs and post-operative costs.
The studies that performed a cost effectiveness analysis were different: one publication described a cost utility analysis as related to the instrument utilized (cost per QAly ranges between € 4720 and € 8200); another reported cost per QAly ranging from € 1998 to € 15,359; an uK study claimed that the benefits in terms of health utility are significant and sufficiently large to justify costs. The systematic review includes the analysis of educational costs as these are very significant from an economic point of view: the costs increase according to the level of hearing loss and the type of school attended. Ci in children opens important debates from economic, clinical and ethical points of view. A relevant limit in the economic review is the paucity of published studies. Ci is expensive, in particular because of the cost of the device and lifelong support. if healthcare costs are high, the savings in terms of indirect costs and quality of life are also important. Ci, in fact, has a positive impact in terms of quality of life and is strongly recommended for children before two years of age.
Adults 64 publications were identified, and according to inclusion and exclusion criteria 57 studies were excluded, leaving 7 eligible articles for the systematic review on the economic aspects for cochlear implants in adults. From the references of the selected articles, 1 additional study was considered eligible for a total of 8 eligible publications. Four studies were excluded, and at the end of the inclusion-exclusion process 4 articles were selected to assess costs and benefits of cochlear implantation in adults. Three studies were conducted in europe (2 in uK and 1 in France) and 1 in uSA; 3 studies were multicentre and 1 was monocentric; 2 studies were prospective and 2 were retrospective; 1 study presented a partial economic evaluation (cost analysis) and 3 presented a cost utility analysis; 3 studies used the healthcare system perspective for cost analysis and 1 study did not specify the perspective. All 4 studies analyzed unilateral cochlear implants, and 1 of the 4 studies also exam- ined bilateral Ci. All 4 studies included patients with postlingual deafness, and 1 also examined a small group of six pre-lingual deaf adults. The systematic review of the economic literature suggested that monolateral cochlear implants are cost effective in adults. The monolateral Ci has a cost/QAly ranging from € 24,000/QAly to € 32,000/QAly compared to no intervention for patients with post-lingual deafness. Few economic studies were available, and future research is recommended to provide further insights on the cost effectiveness of monolateral and bilateral cochlear implants.

Discussion
This is one of the first hTA reports on a complex medical device/surgical intervention in italy by governmental agencies and universities 10 . We can use this experience as a case study to better understand barriers and opportunities for hTA in italy. The production of the report did not face substantial barriers, and the Wg easily agreed on the criteria for systematic review and literature analysis. The Wg included most of the centres performing Ci in italy, comprising both research centres (university hospitals and irCCS) and non-research hospitals. Scientific societies had a determinant role in maintaining a balance in the different points of view (clinicians and economists) and the recruiting of experts from different centres. Finally, the CC participated actively and constructively, and the external review was performed without problems. in particular, the participation of stakeholders, both manufacturers and patients associations, was successful and allowed meaningful assessment of ethical and social issues, which are usually neglected in most hTA reports 11 . We hope that such a collaborative process will make the report acceptable by all interested parties, and consequently a useful tool for decision making.
With regard to the second aim of the project, namely the proposal of a registry for cochlear implant recipients, is one of the first efforts in italy in this field. All members of the Wg actively participated in the creation of a registry that was approved, with minimal suggestions by the members of the CC. We hope that, if adopted by Ci teams, it will be useful for future observational studies.

Conclusions and recommendations
The following recommendations derive from analysis of the results of systematic reviews and italian and international guidelines. They were approved by the Wg members after several meetings. They were also approved with minimal suggestions by CC members. Some of the obser-vations suggested by the CC were approved by all Wg members.

Children under 2 years of age
Ci is indicated when all the three below criteria are satisfied: • Age: ≥ 12 months. The actual level of evidence does not justify systematic implantation in the first year of life. This indication should be limited to cochlear ossification or to selected cases reliably evaluated by experienced teams, with a defined diagnosis with regards to hearing threshold, aetiology and site of lesion. • Hearing threshold level: CI is indicated in children with bilateral profound deafness (mean threshold between 0.5-1-2 kHz ≥ 90 dB HL), detected with both subjective and objective methods. • Results with hearing aids: no significant communicative and hearing improvement after a period of 3-6 months with hearing aids and speech therapy training (except in the case of documented incipient cochlear ossification).

Children between 2 and 18 years of age
Ci is indicated when all three of the following criteria are satisfied: • Hearing threshold level: CI is indicated in children with bilateral severe to profound hearing loss (mean threshold between 0.5-1-2 khz > 75 dB hl) detected with both subjective and objective methods. • Results with hearing aids: no significant b communicative and hearing improvement after a period of 3-6 months with hearing aids and speech therapy training (except in the case of documented incipient cochlear ossification). • Speech perception abilities: evaluation of speech perception abilities is recommended using materials appropriate to age and speech development. Ci is indicated if the open-set speech recognition score is ≤ 50% in the best aided condition without lip reading. in selected cases, Ci is indicated if the open-set speech recognition score is ≤ 50% in the best aided condition without lip reading with background noise (signal to noise ratio Snr+10).

Deaf children with associated disabilities
Ci in children with multiple disabilities is indicated. indications and prognosis should be considered on a caseby-case basis. Comprehensive counseling with family members and caregivers is mandatory. The benefits after implantation should be expected both in terms of speech and language improvement and quality of life.

Adult patients (post-verbal deafness)
Ci is indicated in adult patients with bilateral severe to profound hearing loss (mean threshold between 0.5-1-2 khz > 75 dB hl), with open-set speech recognition score ≤ 50% in the best aided condition without lip reading. in selected cases, Ci is indicated if the open-set speech recognition score is ≤ 50% in the best aided condition without lip reading with background noise (signal to noise ratio Snr + 10). Ci is admitted in selected cases with better residual hearing at low and middle frequencies and hearing threshold between 2 and 4 kHz ≥ 90 dB, with an open-set speech recognition score ≤ 50% in the best aided condition without lip reading.

CI in the elderly
Ci in the elderly is admitted, without any upper limit of age. general health problems and life expectancy should be taken into account, and Ci indications should be considered on a case-by-case basis.
6. Adults with pre-verbal deafness indications for Ci and prognostic factors should be analyzed on a case-by-case basis. Factors to be taken into account mainly include progression of deafness, hearing aid use and rehabilitation (in particular methodology of rehabilitation), results with hearing aids, patient motivations and psychological aspects.
7. Bilateral CI is indicated in the following conditions a. Adult patients: • patients with deafness and initial bilateral cochlear ossification (ex postmeningitic); • deaf-blind patients or patients with multiple disabilities (that increase their reliance on auditory stimuli as a primary sensory mechanism for spatial awareness); • unsatisfactory results with unilateral CI if better results are achievable with a contralateral Ci; • patients with CI failure if reimplantation in the same ear is contraindicated. b. Children: • children with bilateral profound sensorineural hearing loss, who cannot achieve significant benefits with bimodal hearing; • children with postmeningitic deafness and initial bilateral cochlear ossification (ex postmeningitic); • deaf-blind children; • unsatisfactory results with unilateral CI, if better results are achievable with a contralateral Ci; • children with CI failure if reimplantation in the same ear is contraindicated; • children with multiple disabilities (case-by-case evaluation). Both simultaneous and sequential procedures are admitted, although the simultaneous procedure is recommended. in the case of sequential bilateral implantation, a short delay between surgeries is recommended.

CI recipient registry
A registry to collect data on Ci recipients in Ci centres in italy is recommended to pool data, which should be made available for epidemiological studies.