Abstract
Purpose
Virtual reality simulator technology together with novel metrics could advance our understanding of expert neurosurgical performance and modify and improve resident training and assessment. This pilot study introduces innovative metrics that can be measured by the state-of-the-art simulator to assess performance. Such metrics cannot be measured in an operating room and have not been used previously to assess performance.
Methods
Three sets of performance metrics were assessed utilizing the NeuroTouch platform in six scenarios with simulated brain tumors having different visual and tactile characteristics. Tier 1 metrics included percentage of brain tumor resected and volume of simulated “normal” brain tissue removed. Tier 2 metrics included instrument tip path length, time taken to resect the brain tumor, pedal activation frequency, and sum of applied forces. Tier 3 metrics included sum of forces applied to different tumor regions and the force bandwidth derived from the force histogram.
Results
The results outlined are from a novice resident in the second year of training and an expert neurosurgeon. The three tiers of metrics obtained from the NeuroTouch simulator do encompass the wide variability of technical performance observed during novice/expert resections of simulated brain tumors and can be employed to quantify the safety, quality, and efficiency of technical performance during simulated brain tumor resection. Tier 3 metrics derived from force pyramids and force histograms may be particularly useful in assessing simulated brain tumor resections.
Conclusion
Our pilot study demonstrates that the safety, quality, and efficiency of novice and expert operators can be measured using metrics derived from the NeuroTouch platform, helping to understand how specific operator performance is dependent on both psychomotor ability and cognitive input during multiple virtual reality brain tumor resections.
Similar content being viewed by others
Abbreviations
- NRC:
-
National Research Council
- VR:
-
Virtual reality
- PGY:
-
Postgraduate year
- cc:
-
Cubic centimeters
- 3D:
-
Three-dimensional
References
Brydges R, Kurahashi A, Brümmer V, Satterthwaite L, Classen R, Dubrowski A (2008) Developing criteria for proficiency-based training of surgical technical skills using simulation: changes in performances as a function of training year. J Am Coll Surg 206:205–211. doi:10.1016/j.jamcollsurg.2007.07.045
Aggarwal R, Grantcharov TR, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A (2006) An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 244:310–314. doi:10.1097/01.sla.0000218094.92650.44
Ericsson KA, Charness N (1994) Expert performance. Am Psychol 49:725–747
Tsuda S, Scott D, Doyle J, Jones DB (2009) Surgical skills training and simulation. Curr Probl Surg 46:271–370. doi:10.1067/j.cpsurg.2008.12.003
Jaffer A, Bednarz B, Challacombe B, Sriprasad S (2009) The assessment of surgical competency in the UK. Int J Surg 7:12–15. doi:10.1016/j.ijsu.2008.10.006
Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Eng J Med 355:2664–2669. doi:10.1056/NEJMra054785
Brightwell A, Grant J (2013) Competency-based training: who benefits? Postgrad Med J 89:107–110. doi:10.1136/postgradmedj-2012-130881
Bhatti NI, Cummings CW (2007) Competency in surgical residency training: defining and raising the bar. Acad Med 82:569–573
Gélinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery integrating theory into practice. Neurosurgery 73(Suppl. 1):30–38. doi:10.1227/NEU.0000000000000115
Gaba DM, DeAnda A (1998) A comprehensive anesthesia simulation environment: re-creating the operating room for research and training. Anesthesiology 69:387–394
Chung J, Sackier J (1998) A method of objectively evaluating improvements in laparoscopic skills. Surg Endosc 12:1111–1116
Derossis AM, Bothwell J, Sigman HH, Fried GM (1998) The effect of practice on performance in a laparoscopic simulator. Surg Endosc 12:1117–1120
Delp SL, Loan JP, Hoy MG, Zajac FE, Topp EL, Rosen JM (1990) An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Trans Biomed Eng 37:757–767
Satava RM (1993) Virtual reality surgical simulator. Surg Endosc 7:203–205
Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA 3rd, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. doi:10.1016/j.amjsurg.2006.06.050
Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM, Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. doi:10.1097/01.SLA.0000028969.51489
Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802. doi:10.1136/bmj.b1802
Phillips NI, John NW (2000) Web-based surgical simulation for ventricular catheterization. Neurosurgery 46:933–937
Ferroli P, Tringali G, Acerbi F, Schiariti M, Broggi M, Aquino D, Broggi G (2013) Advanced 3-dimensional planning in neurosurgery. Neurosurgery 72(Suppl 1):54–62. doi:10.1227/NEU.0b013e3182748ee8
Neubauer A, Wolfsberger S (2013) Virtual endoscopy in neurosurgery: a review. Neurosurgery 72(Suppl 1):97–106. doi:10.1227/NEU.0b013e31827393c9
Mitha AP, Almekhlafi MA, Janjua MJ, Albuquerque FC, McDougall CG (2013) Simulation and augmented reality in endovascular neurosurgery: lessons from aviation. Neurosurgery 72(Suppl 1):107–114. doi:10.1227/NEU.0b013e31827981fd
Rousseau G, Bailes J, del Maestro RF, Cabral A, Choudhury N, Comas O, Debergue P, De Luca G, Hovdebo J, Jiang D, Laroche D, Neubauer A, Pazos V, Thibault T, DiRaddo R (2013) The development of a virtual simulator for training of neurosurgeons to perform and to perfect endoscopic endonasal transphenoidal surgery. Neurosurgery 73(Suppl. 1):85–93. doi:10.1227/NEU.0000000000000112
Kockro RA, Serra L, Tseng-Tsai Y, Yih-Yian Chan CS, Gim-Guan C, Lee E, Hoe LY, Hern N, Nowinski WL (2000) Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery 46:118–137
Luciano C, Banerjee P, Lemole G, Charbel F (2006) Second generation haptic ventriculostomy simulator using the immersive touch system. Stud Health Technol Inform 119:343–348
Banerjee PP, Luciano CJ, Lemole GM Jr, Charbel FT, Oh MY (2007) Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg 107:515–521. doi:10.3171/JNS-07/09/0515
Luciano CJ, Banerjee PP, Bellotte B, Oh GM, Lemole M, Jr., Charbel FT, Roitberg B (2011) Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery 69(1 Suppl Operative):ons14–19. doi:10.1227/NEU.0b013e31821954ed
Luciano CJ, Banerjee PP, Florea L, Dawe G (2005) Design of the ImmersiveTouch\(^{TM}\): a high-performance haptic augmented virtual reality system. In: 11th International conference on human-computer interaction, Las Vegas, NV
Delorme S, Laroche D, Diraddo R, F. Del Maestro R (2012) NeuroTouch: a Physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71(1 Suppl Operative):ons 32–42. doi:10.1227/NEU.0b013e318249c744
Choudhury N, Gelinas-Phaneuf N, Delorme S, Del Maestro R (2013) Fundamentals of Neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 80:E9–E19. doi:10.1016/j.wneu.2012.08.022
Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Debergue Pazos VP, Diraddo R, Del Maestro RF (2013) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J CARS. doi:10.1007/s11548-013-0905-8
Wheatstone C (1838) Contributions to the physiology of vision.-Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Philos Trans R Soc Lond 128:371–394
Kohn LT, Corrigan J, Donaldson MS (2000) To err is human: building a safer health care system. National Academy Press, Washington
Gray JD (1996) Global rating scale in residency education. Acad Med 71:555–563
Turnbull J, Gray J, MacFadyen J (1998) Improving in training evaluation programs. J Gen Intern Med 13:317–323. doi:10.1046/j.1525-1497.1998.00097.x
Katz R, Hoznek A, Salomon L, Antiphon P, de la Taille A, Abbou CC (2005) Skill assessment of urological laparoscopic surgeons: can criterion levels of surgical performance be determined using the pelvic box trainer? Eur Urol 47:482–487. doi:10.1016/j.eururo.2004.12.007
de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA (2008) The incidence and nature of in-hospital adverse events: a systemic review. Qual Saf Health Care 17:216–23. doi:10.1136/qshc.2007.023622
Gawande AA, Thomas EJ, Zinner MJ, Brennan TA (1999) The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 126:66–75
Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ (2013) Surgical skills and complication rates after bariatric surgery. N Engl J Med 369:1434–1442. doi:10.1056/NEJMsa1300625
Kockro RA (2013) Neurosurgery simulators-beyond the experiment. World Neurosurg. 80:E101–E102. doi:10.1016/j.wneu.2013.02.017
Acknowledgments
This work was supported by The Di Giovanni Foundation, the Montreal English School Board, the B-Strong Foundation, the Colannini Foundation, and the Montreal Neurological Institute and Hospital. Dr. R. F. Del Maestro is the William Feindel Emeritus Professor in Neuro-Oncology at McGill University. Dr. Azarnoush held the Postdoctoral Neuro-Oncology Fellowship from the Montreal Neurological Institute and Hospital. Dr. Gélinas-Phaneuf was funded by a generous contribution from the Harold and Audrey Fisher Brain Tumour Research Award.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (wmv 4794 KB)
Supplementary material 2 (avi 7285 KB)
Rights and permissions
About this article
Cite this article
Azarnoush, H., Alzhrani, G., Winkler-Schwartz, A. et al. Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection. Int J CARS 10, 603–618 (2015). https://doi.org/10.1007/s11548-014-1091-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-014-1091-z