Simulator training for endobronchial ultrasound: a randomized controlled trial.
Konge L1, Clementsen PF2, Ringsted C3, Minddal V2, Larsen KR4, Annema JT5.
1Centre for Clinical Education, University of Copenhagen and The Capital Region of Denmark, Copenhagen, Denmark [email protected]dk.
2Dept of Pulmonology, Gentofte Hospital, University of Copenhagen, Hellerup, Denmark.
3The Wilson Centre and Dept of Anesthesiology, University of Toronto and University Health Network, Toronto, ON, Canada.
4Dept of Pulmonology, Bispebjerg Hospital, University of Copenhagen, Copenhagen, Denmark.
5Dept of Pulmonology, Leiden University Medical Center, Leiden, The Netherlands Dept of Pulmonology, Academic Medical Centre, University of Amsterdam, Amsterdam, The Netherlands.
Eur Respir J. 2015 Jul 9.
Endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) is very operator dependent and has a long learning curve. Simulation-based training might shorten the learning curve, and an assessment tool with solid validity evidence could ensure basic competency before unsupervised performance.A total of 16 respiratory physicians, without EBUS experience, were randomised to either virtual-reality simulator training or traditional apprenticeship training on patients, and then each physician performed EBUS-TBNA procedures on three patients. Three blinded, independent assessor assessed the video recordings of the procedures using a newly developed EBUS assessment tool (EBUSAT).The internal consistency was high (Cronbach’s α=0.95); the generalisability coefficient was good (0.86), and the tool had discriminatory ability (p<0.001). Procedures performed by simulator-trained novices were rated higher than procedures performed by apprenticeship-trained novices: mean±sd are 24.2±7.9 points and 20.2±9.4 points, respectively; p=0.006. A pass/fail standard of 28.9 points was established using the contrasting groups method, resulting in 16 (67%) and 20 (83%) procedures performed by simulator-trained novices and apprenticeship-trained novices failing the test, respectively; p<0.001.The endobronchial ultrasound assessment tool could be used to provide reliable and valid assessment of competence in EBUS-TBNA, and act as an aid in certification. Virtual-reality simulator training was shown to be more effective than traditional apprenticeship training. [/av_textblock] [av_hr class='default' height='50' shadow='no-shadow' position='center' custom_border='av-border-thin' custom_width='50px' custom_border_color='' custom_margin_top='30px' custom_margin_bottom='30px' icon_select='yes' custom_icon_color='' icon='ue808'] [av_textblock size='' font_color='' color='']
Construct validity of the BRONCH Mentor simulator for essential bronchoscopic skills.
Pastis NJ1, Vanderbilt AA, Tanner NT, Silvestri GA, Huggins JT, Svigals Z, Shepherd RW.
1*Division of Pulmonary/Critical Care, Medical University of South Carolina, Charleston, SC †Virginia Commonwealth University School of Medicine, Richmond, VA.
J Bronchology Interv Pulmonol. 2014 Oct;21(4):314-21.
BACKGROUND: Although simulation-based bronchoscopy has been shown to be an effective training modality, formal assessment should still be performed as new technology emerges. We sought to validate a simulator in essential bronchoscopic tasks, and survey perceptions of bronchoscopists on simulation.
METHODS: A cohort study at 2 medical centers used 3 groups to assess construct validity of the Simbionix Bronchoscopy Simulator: 7 first-year fellows with <10 bronchoscopies each (novice), 6 pulmonologists with 200 to 1000 bronchoscopies each (experienced), and 7 pulmonologists with >1000 bronchoscopies each (expert). Participants were tested in 4 tasks (1: scope manipulation, 2: guided anatomic navigation, 3: airway anatomy, and 4: lymph node anatomy). Participants were scored and surveyed on their impressions of simulation. The means and Kruskal-Wallis test among groups were compared by task item (P<0.05).RESULTS: There were statistically significant differences in mean ranks among groups for tasks 1 and 3. For task 1, final score, time, mid-lumen time, and wall hits were discriminative (P=0.006, 0.006, 0.012, and 0.014, respectively). For task 3, time, bronchial segments identified, bronchial segments incorrectly identified, and bronchial segments skipped were discriminative (P=0.04, 0.012, 0.013, and 0.013, respectively). There was no statistically significant difference for task 2 and task 4. All participants agreed that simulation training is helpful and should be incorporated into bronchoscopic training.
CONCLUSIONS: The simulator demonstrated validity in differentiating skill in scope manipulation and airway anatomy, but did not discriminate skill levels in anatomic orientation or identification of lymph nodes. Bronchoscopy simulation was viewed as helpful by all levels and should be considered before performance on patients.
Construct Validity of the BRONCH Mentor Simulator for Essential Bronchoscopic Skills
Nicholas Pastis, MD; Allison Vanderbilt, EdD; Nichole Tanner, MD; Gerard Silvestri, MD; John Huggins, MD; Molly Madden, EdD; Philip Svigals, MD; Ray Shepherd, MD
Chest. 2013;144(4_MeetingAbstracts):581A. doi:10.1378/chest.1702165
PURPOSE: In an era of heightened awareness for patient safety, simulation-based bronchoscopy has emerged as a zero risk training modality, however, its effectiveness remains uncertain. We sought to demonstrate that the simulator could distinguish between 3 groups of bronchoscopists (novice, experienced, and expert), and surveyed their perceptions of simulation.
METHODS: A cohort study was conducted at 2 academic medical centers. Three groups were used to assess construct validity of the Simbionix Bronchoscopy Simulator: a novice group(< 10 bronchoscopies), an experienced group(200-1000 bronchoscopies), and an expert group(>1000 bronchoscopies). Participants were tested in 4 tasks(1-manipulation through the center of airways, 2-anatomic orientation, 3-airway anatomy, and 4-identification of lymph node stations). Participants’ performances were scored by the simulator, and participants were surveyed on their impressions of simulation training for bronchoscopy. Means and Kruskal-Wallis Test among groups were compared by task item(p<0.05).
RESULTS: There were statistically significant differences among the groups for tasks 1 and 3. For task 1, final score, total time, percent time at mid lumen, and total wall hits were most favorable (p=0.006,0.006,0.012, and 0.014, respectively) for expert(n=7), then experienced(n=6), and then novice bronchoscopists(n=7). For task 3, the total time, number of bronchial segments identified on first attempt, number of bronchial segments incorrectly identified after 3 attempts, and bronchials segments skipped were most favorable(p=0.04, 0.012,0.013,0.013, respectively) for expert, then experienced, and then novice bronchoscopists. There was no difference between groups for tasks 2 and 4. All participants agreed that simulation training is helpful and should become part of bronchoscopic training.
CONCLUSIONS: The bronchoscopy simulator demonstrated construct validity in differentiating skill levels in scope manipulation and airway anatomy tasks, however was not validated in differentiating skill levels for anatomic orientation or lymph node station identification. Users of all skill level rated bronchoscopy simulation as helpful.
CLINICAL IMPLICATIONS: Bronchoscopy simulation is a tool that should be considered to enhance teaching of scope manipulation and airway anatomy prior to first real world bronchoscopy.
Simulation-Based Bronchoscopy Training
Jack A. Kastelik, MD; Faiza Chowdhury, MBChB; Anthony Arnold, MD
Chest. 2013;144(2):718-719. doi:10.1378/chest.13-0880
We read with great interest an article by Kennedy and colleagues1 in a recent issue of CHEST (July 2013) in which a systematic review and meta-analysis of studies revealed significant improvements in skills and behaviors when comparing simulation-based bronchoscopy with no intervention. However, the article also identified gaps in evidence, such as the lack of clear understanding of optimal design or choice of modalities in relation to simulation-based bronchoscopy training. Based on our experience of setting up a regional simulation bronchoscopy program, we are able to address these issues.
We established five clinical skills laboratories that deliver simulation bronchoscopy training. We also set up a group of regional experienced bronchoscopists responsible for the development of simulation bronchoscopy training. We have run 15 courses and trained >60 candidates. Although initially we used different formats for the courses, the trainees’ overall experience of simulation-based bronchoscopy was extremely positive. Based on our initial experience, we established that the optimal design for delivering simulation-based bronchoscopy courses should incorporate a blend of short lectures, e-learning, and hands-on experience using simulation. To be successful, simulation-based bronchoscopy requires a high trainer-to-trainee ratio (ideally 1:2), and, therefore, we established a faculty of experienced bronchoscopists with a special interest in procedural training. Our results confirmed significant improvement, for both novices and more experienced trainees, in the technical ability of handling bronchoscopes, their understanding of anatomy and identification of bronchial segments, and their knowledge of the procedure; the improvements were in the range of 20% and 30% when using high fidelity alone and in combination with low-fidelity bronchoscopy simulation, respectively.2 We observed that the best outcomes were achieved by combining sessions on a virtual reality bronchoscopy simulator with low-fidelity manikin and real scope modules. Our real-life experience showed that it is possible to set up a large and successful regional simulation bronchoscopy training program, which is now offered to all of our trainees (before exposure to patients).
Training in bronchoscopy is a complex process and has been traditionally based on an apprenticeship model, which raised concerns of patient safety and variable level of experience, with reports suggesting that one-fifth of trainees may not be achieving the required number of procedures.3 Although simulation-based bronchoscopy can overcome many of these issues, it has only been incorporated into 36% programs in the United States.4 The article by Kennedy and colleagues1 and our experience, therefore, provide important evidence to encourage wider use of simulation for bronchoscopy training.
1. Kennedy CC, Maldonado F, Cook DA Simulation-based bronchoscopy training; systematic review and meta-analysis. Chest. 2013;144(1):183-192.
2. Chowdhury F, Gondker A, Acharya N, et al. Standardisation of bronchoscopy training across Yorkshire and Humber Deanery [abstract]. Thorax. 2012;67(suppl 2):A171. [CrossRef]
3. Pastis NJ, Nietert PJ, Silvestri GA; American College of Chest Physicians Interventional Chest/Diagnostic Procedures Network Steering Committee. Variation in training for interventional pulmonary procedures among US pulmonary/critical care fellowships: a survey of fellowship directors. Chest. 2005;127(5):1614-1621. [CrossRef]
4. Lucarelli MR, Lucey CR, Mastronarde JG. Survey of current practices in fellowship orientation. Respiration. 2007;74(4):382-386.
Standardisation of Bronchoscopy Training Across Yorkshire and Humber Deanery
F Chowdhury1, A Gondker2, N Acharya1, R Naseer1, J Hill1, J Hogg3, J Kastelik2, S Renshaw1, S Bianchi1
+ Author Affiliations
1. Sheffield Teaching Hospitals, Sheffield, England, United Kingdom
2. Hull Royal Infirmary, Hull, England, United Kingdom
3. Pinderfields Hospital, Wakefield, England, United Kingdom
BACKGROUND: Reduced exposure to bronchoscopyis a key issue for respiratory trainees with effect on their confidence in undertaking the procedure and thus patient safety. Studies have shown that simulation improves confidence in bronchoscopy skills but do not explore the most optimal teaching methods.
AIM: To assess two different methods of delivering bronchoscopy simulation training
METHODS: Two half day simulation bronchoscopy courses were designed independently within the Yorkshire and Humber Deanery. Course 1 concentrated on providing a knowledge based training consisting of a didactic lecture followed by equal time spent on a Symbionixsimulator and on the BTS e-learning hub website. Course 2 provided pre-course material in the form of BTS guidelines and bronchoscopy procedure pocketbook. The course focused on hands-on simulation training using a bronchoscopy manikin and the Symbionix simulator. All candidates completed pre and post course Likert scale questionnaires in six areas relating to participant knowledge and confidence in using a bronchoscope.
RESULTS: Overall 30 trainees; 15 in each course were evaluated. Candidates had performed between 0 to >300 previous bronchoscopies and were from across the SpR years. Both courses delivered significant improvement in confidence scores in all of the six areas assessed. The greatest improvement was found in confidence levels in technical ability (see table 1). Course 1 candidates showed a greater confidence improvement in factual skills (such as knowledge of contra-indications of the procedure and anatomy). Course 2 demonstrated that 93% of candidates agreed that the simulator helped to improve technical ability in contrast to 100% with manikin exposure. 100% of candidates found the pocketbook was a useful adjuvant to the course with 93% agreeing that they would find this useful to complement their training.
Conclusions: A combined and standardised bronchoscopy simulation course incorporating lectures and pre-course materials but focusing on hands on experience on both a manikin and a simulator is therefore considered to provide greatest educational benefit. This course is now active in Yorkshire and the Humber and is to be mandated for all new trainees to the programme. Each SpR will also be re-assessed after a 3-month period incorporating a competency-based assessment approach.
Authors 1 and 2 are first authors
The following abstract was presented at the British Thoracic Society June 13-14, 2013, University Place Conference Centre, University of Manchester, UK
The respiratory simulation curriculum in Yorkshire and the Humber Deanery
Dr Faiza Chowdhury (Yorkshire and the Humber)
F Chowdhury1, S Pathmanathan1, I Aslam1, A Arnold1, J Hogg2, S Renshaw3, J Hill3, N Chaudhuri4, T Rogers5, J Kastelik6
(1Health Education Yorkshire and the Humber, 2Mid-Yorkshire Hospitals NHS Trust, 3Sheffield Teaching Hospitals NHS Foundation Trust, 4University Hospital of South Manchester NHS Foundation Trust, 5Doncaster and Bassetlaw Hospitals NHS Foundation Trust, 6Hull Institute of Learning and Simulation, Hull Royal Infirmary)
INTRODUCTION: With the introduction of the European Working Time Directive, trainee exposure to vital clinical skills and knowledge is diminishing. As a result, the Annual Review of Competence Progression (ARCP) requirements are becoming increasingly difficult to achieve. The Yorkshire and Humber Deanery has developed a Respiratory Simulation Curriculum to support trainees’ achievement of the Joint Royal College Postgraduate Training Board (JRCPTB) 2010 Respiratory Specialty Training Curriculum. This is in concordance with the Department of Health Technology Enhanced Learning Framework coupled with the GMC’s guidance on simulation being “good medical practice.”
METHODS: The JRCPTB curriculum was reviewed by a Respiratory Simulation Fellow and areas for simulated curriculum delivery were identified and developed. These included:
1. Bronchoscopy simulation
2. Thoracic ultrasound
3. Difficult scenarios in respiratory medicine (eg. massive haemoptysis) and human factors
4. Medical thoracoscopy and indwelling pleural catheter insertion
5. Respiratory physiology and simulated practical skills workshops e.g. administration of non-invasive ventilation.
This innovative programme was designed for respiratory trainees and reviewed by faculty members trained in simulation delivery. To date we have successfully delivered and evaluated bronchoscopy, thoracic ultrasound, medical thoracoscopy and indwelling pleural catheter simulation courses. Pilots of the remaining courses will be delivered before August 2013. Funding has been secured to ensure future sustainability of this programme.
RESULTS: Over 60 trainees have been trained in simulated bronchoscopy and thoracic ultrasound courses to date. There were significant improvements demonstrated in relation to knowledge and technical ability. After review by the Deanery Pulmonary Procedures Committee, the programme is now a mandatory part of our regional Respiratory training. The content of all planned courses have been standardised and peer reviewed by specialists across the deanery.
CONCLUSIONS: Using simulation to support the achievement of training requirements improves the confidence and knowledge of trainees. Simulation allows for learning to occur in a safe and controlled environment without risk to the patient. In addition, we feel that our educational initiative ultimately optimises care by improving patient safety. We feel that simulation curricula should be developed nationwide mapping directly to the specialty JRCPTB curriculum to support increasing training needs
Using virtual-reality simulation to assess performance in endobronchial ultrasound.
Konge L, Annema J, Clementsen P, Minddal V, Vilmann P, Ringsted C.
Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Copenhagen, Denmark.
Respiration. 2013;86(1):59-65. doi: 10.1159/000350428. Epub 2013 May 23.
BACKGROUND: For optimal treatment of patients with non-small cell lung carcinoma, it is essential to have physicians with competence in endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA). EBUS training and certification requirements are under discussion and the establishment of basic competence should be based on an objective assessment of performance.
OBJECTIVES: The aims of this study were to design an evidence-based and credible EBUS certification based on a virtual-reality (VR) EBUS simulator test.
METHODS: Twenty-two respiratory physicians were divided into 3 groups: experienced EBUS operators (group 1, n = 6), untrained novices (group 2, n = 8) and simulator-trained novices (group 3, n = 8). Each physician performed two standardized simulated EBUS-TBNA procedures. Simulator metrics with discriminatory ability were identified and reliability was explored. Finally, the contrasting-groups method was used to establish a pass/fail standard, and the consequences of this standard were explored.
RESULTS: Successfully sampled lymph nodes and procedure time were the only simulator metrics that showed statistically significant differences of p = 0.047 and p = 0.002, respectively. The resulting quality score (QS, i.e. sampled lymph nodes per minute) showed an acceptable reliability and a generalizability coefficient of 0.67. Reliability of 0.8 could be obtained by testing in 4 procedures. Median QS was 0.24 (range 0.21-0.26) and 0.098 (range 0.04-0.21) for groups 1 and 2, respectively (p = 0.001). The resulting pass/fail standard was 0.19. Group 3 had a median posttraining QS of 0.11 (range 0-0.17). None of them met the pass/fail standard.
CONCLUSIONS: With careful design of standardized tests, a credible standard setting and appropriate transfer studies, VR simulators could be an important first line in credentialing before proceeding to supervised performance on patients.