{'fraction_non_alphanumeric': 0.02741264284298831, 'fraction_numerical': 0.011955875553206948, 'mean_word_length': 5.412537060567556, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Creating a Validated Simulation Training Curriculum in Otolaryngology

Simulation-based training is an integral component of surgical training. It allows practice of technical skills within a safe environment without compromising patient safety. This article seeks to review current virtual and non-virtual reality simulation models within the literature and review their validation status. Many simulation models exist within otolaryngology and are currently being used for education. New models are also continuously being developed; however, validity should be proven for the models before incorporating their use for educational purposes. Validity should be determined by experts and trainees themselves. A validated simulation curriculum should be incorporated within the otolaryngology training programme. A curriculum based on the current training programme at our institution serves as an exemplar for local adoption.


Introduction
The benefits of simulation training have been well documented in medical education, as well as other industries such as aviation and military services. Time for training has been affected in recent years due to continuing changes in the healthcare system [1][2][3]. Barriers to learning include infrequent training opportunities as well as societal pressures for increased patient safety. Learning and practising skills by novices on any patient raise patient safety concerns [4].
Operative experience is essential in acquiring surgical skills and the time in which trainees have to gain these experiences has changed due to a variety of factors including reduced training opportunities and a restriction of working hours.
Simulation allows a safe platform for education and assessment where there is deliberate practice under supervision as well as determining competency without compromising patient safety. This is becoming increasingly important as a recent study from John Hopkins University suggests that medical error is the third leading cause of death in the USA [5]. In the UK, the annual report in 2017/2018 showed that the NHS spent £1.95 billion in clinical negligence claims [6].
Many other specialties have successfully integrated simulation-based education opportunities into their curriculum [4] and simulation-based education is now considered an integral component of surgical training [7].
In a recent review, simulation resulted in reduced surgical time and higher performance rates for laparoscopic surgery when compared with traditional teaching methods [8]. Simulation-based teaching has led to improved patient outcomes in many clinical settings including technical skills and crisis situations [9], as well as allowing time for reflection with the assurance of patient safety [10].

Simulation in Otolaryngology Training
The current cohort of surgical trainees are required to gain the same surgical competencies as previous generations during a time of limited operative exposure, reduced trainee-trainer interaction time and increased workload of supervising surgeons [2,3,11,12].
In the UK, the Otolaryngology curriculum currently requires trainees to show competence in performing key indicative procedures before completion of training. The Joint Committee on Surgical Training (JCST) states that all trainees seeking certification in otolaryngology specifically must obtain, as an absolute minimum, ten of the following procedures as the principal surgeon: mastoidectomy, major neck surgery, tracheostomies, paediatric endoscopies, septorhinoplasties, endoscopic sinus surgery and removal of foreign bodies from the upper aerodigestive tract. The methods in which these procedures are acquired are by practical teaching, observing, assisting and operating under supervision. No uniformity or proven validity exists for any of these methods.
All simulated platforms aim to consolidate techniques and accelerate skill acquisition needed to complete training, without compromising patient safety [13]. Simulation within otolaryngology is not a new concept. Cadaveric temporal bone dissection for training in mastoid surgery is one of the oldest simulators [14,15].
Currently, due to the limited access to live animal and cadaveric human tissue imposed by the Human Tissue Act [16], many practical otolaryngology courses use alternative platforms as simulators. These include physical "task trainers" and computer-assisted virtual reality platforms, which have been and are in the process of development. However, many of these models have not yet been validated [17•].
Trainees have engaged with practical skills courses including simulated training platforms such as animal models, cadavers and synthetic material for several decades. There are many descriptive studies in the literature. However attempts to validate the models are less evident [18] and there is no standardisation in the method of validation of such training platforms. In addition, although a model may not appear realistic, it can still be used to achieve desired training by enhancing the skill in question [13]; this has led to the development of low-cost trainers for skill transference.

How to Validate Simulation Models
As simulators vary and teach different skill sets, this makes it difficult to validate the models in a uniform way. The European Association for Endoscopic Surgery produced guidelines to outline the keystones of simulator validation [19•].
Face validity reflects the ability of a simulator to produce a realistic environment that resembles the actual surgical procedure, and can be assessed by the trainer and trainee.
Content validity is the ability of the simulator to deliver what is expected to be achieved, which can be done by satisfying pre-determined learning objectives. This can then be subdivided into global, task-specific, construct, predictive and concurrent validity. Construct validity can be used to differentiate between different levels of expertise amongst participants and is essential prior to incorporating models into training to use them on a regular basis. Predictive validity can be used to predict future performance. Concurrent validity is used to how a model compares with another that has already been validated or considered as a gold standard. Transfer validity is used to ascertain whether the simulator has the effect it proposes to have [20].
Validity can be measured via a structure questionnaire or via physical measurements of size, durability or use of instruments [21,22].
As well as validating simulation models practically, simulation-based teaching is also important to develop nontechnical skills of trainees. A recent study showed that senior trainees valued the non-technical skills of communication, leadership and teamwork [23] within simulation-based training.

Validated Simulation in Otolaryngology
Simulators can be broadly divided into virtual reality (VR) and non-virtual reality (non-VR) simulators. Nonvirtual reality simulators include human, animal and artificial models. These simulators remain popular as they have been used for a long time and are generally affordable and reproducible.
Human cadavers provide the greatest anatomical accuracy; however, there is lack of availability due to escalating costs and changes to the Human Tissue Act [16] as well as the lack of tissue realism due to the effect of formaldehyde on tissues and the absence of bleeding.
Dead animal tissue does not require any special licence within the UK except standard health and safety protocols. Compared with synthetic material, animal tissue offers greater realism of tissue handling and is far more affordable and accessible.
Non virtual reality simulators can be specific body parts or total body simulators, such as SimMan, SimNewB and SimBaby [24]. Other simulators relating to specific body parts can include animal/human tissue, task trainers or simulation mannequins including hybrid models. Full-body mannequins have been found to be useful in setting up complex patient scenarios [10] whereas partial body simulators are useful for task-specific training and are more practical with regard to portability and storage.
Otolaryngology is a speciality with a wide range of procedures, requiring different instruments and skills. In order to look at simulation within otolaryngology, skill sets needed to achieve global operative competence, sometimes irrespective of the anatomical area, need to be assessed. For this reason, we divided the syllabus procedures into:

Otology Simulators
Otology is probably the most developed subspecialty within otolaryngology with regard to simulation (Table 1). Myringotomy and grommet insertion is one of the commonest otolaryngology procedures and there are many physical simulators for it. Most models are inexpensive, easily made and reproducible [25][26][27][28]. Other physical otology models are also available including 3D models for mastoidectomy, stapedectomy and various other middle ear surgeries [29][30][31][32]. Cadaveric temporal bone dissection has been used for mastoidectomy training for many years with high validity [14,15]. However, due to limitations of cadaveric material, synthetic temporal bones have been developed for training such as the Pettigrew temporal bone with variable success [33]. There have been other non-virtual reality models that have been developed for temporal bone drilling, with different forms of validation [34][35][36][37][38].
Otology as a subspecialty has the most virtual reality simulators available. Arora et al. [39] demonstrated that the incorporation of temporal bone virtual reality simulation into the otolaryngology curriculum is beneficial especially for junior trainees.
Few virtual reality airway simulators are reported in the literature. Some of these allow the practice of flexible bronchoscopy [93][94][95][96][97] and provide haptic feedback. However, these have not been validated.

Rhinology Simulation
Endoscopic sinus surgery (ESS) is one of the key skills in otolaryngology training and it lends itself very well to simulation. Biologic models have traditionally been used. Ovine models have been used for ESS training before due to similarity in human anatomy [98], and demonstrated that this model achieved face and content validity as well as discriminating between different experience levels of participants therefore demonstrating construct validity of this model. It has been the largest study to our knowledge which looked at alternative methods for ESS training [99,100].
Due to cost and storage, other physical models have been developed and tested as task trainers [101][102][103]. These may use biologic or non-biologic materials. The low-cost task trainer described by Malekzadeh et al. [101] allows trainees to perform removing fluid from maxillary sinus and targeted injections. This has achieved face, content and construct validity, and is an easily affordable model. This demonstrates skill transference as mentioned previously.
Sinus procedures lend themselves well to virtual reality simulation due to heavy instrumentation; therefore, these simulators allow more analysis and feedback as they are computer generated [90]. Simulators allowing procedures such as nasendoscopy [107][108][109] and FESS [110] have been developed.

Other Simulations Within Otolaryngology
Due to robotic surgery developing within otolaryngology, models have been developed so trainees can gain appropriate  [123][124][125]; the model allows trainees to practise on da Vinci controls. There has also been development of models for subtotal or intracapsular tonsillectomies [126] and for FNA of thyroid lesions [127]; however, to our knowledge, these have not been validated. Other physical models have been developed to allow practice of robotic surgery setup [128], rigid oesophagoscopy [129] and neck dissection [130].
For the purpose of this review, we have focussed on validated models; however, it should be noted there are other simulated platforms which have been developed which have not yet been validated. The teams that use these models should be encouraged to validate them as outlined above.

Selecting Models for a Particular Curriculum
A simulation curriculum should cover the competencies required for completion of training. Currently at Imperial College London, a simulation programme for North Thames ST3-ST5s runs annually. Work-based assessments (WBA) are part of the ISCP curriculum for CT1-ST8s. These can take the format of procedure-based assessments (PBA), direct observation of procedural skills (DOPS), clinical evaluation exercise (CEX) and case-based discussion (CBD) [131]. Awad et al. [132••] showed over a period of 6 years, 3264 procedure-based assessments were submitted, and these were by far the most popular form of WBA. Common procedures a s s e s s e d u s i n g P B A i n c l u d e d t o n s i l l e c t o m y, microlaryngoscopy and endoscopic sinus surgery. Therefore, the focus on a technique for a practical assessment tool is warranted, and this can be achieved by validated simulation platforms. Programme directors may choose different simulated training platforms depending on facilities, faculty and funding.

Conclusion
Simulation is a continually developing field within otolaryngology training. Due to reduced operative experience, it is crucial that trainees have adequate exposure to the core procedures they will be experiencing as a registrar and are required to perform to complete their training. In addition, simulation is also useful for practising real-life scenarios that trainees will commonly encounter during training. We believe a national simulation-based training programme should be developed and offered to all trainees with focus on different skills required at that level of training, whether it be technical or non-technical operative skills. In addition, a regular simulation-based training programme will allow development of simulators within otolaryngology to continuously improve educational opportunities for trainees.

Compliance with Ethical Standards
Conflict of Interest The authors declare that they have no conflict of interest.
Human and Animal Rights and Informed Consent This article does not contain any studies with human or animal subjects performed by any of the authors.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.