{'fraction_non_alphanumeric': 0.03171247357293869, 'fraction_numerical': 0.01569607277852521, 'mean_word_length': 4.730543318649046, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Evaluation of the precision of operative augmented reality compared to standard neuronavigation using a 3D-printed skull.

OBJECTIVE
Augmented reality (AR) in cranial surgery allows direct projection of preregistered overlaid images in real time on the microscope surgical field. In this study, the authors aimed to compare the precision of AR-assisted navigation and standard pointer-based neuronavigation (NV) by using a 3D-printed skull in surgical conditions.


METHODS
A commercial standardized 3D-printed skull was scanned, fused, and referenced with an MR image and a CT scan of a patient with a 2 × 2-mm right frontal sinus defect. The defect was identified, registered, and integrated into NV. The target was physically marked on the 3D-printed skull replicating the right frontal sinus defect. Twenty-six subjects participated, 25 of whom had no prior NV or AR experience and 1 with little AR experience. The subjects were briefly trained in how to use NV, AR, and AR recalibration tools. Participants were asked to do the following: 1) "target the center of the defect in the 3D-printed skull with a navigation pointer, assisted only by NV orientation," and 2) "use the surgical microscope and AR to focus on the center of the projected object" under conventional surgical conditions. For the AR task, the number of recalibrations was recorded. Confidence regarding NV and AR precision were assessed prior to and after the experiment by using a 9-level Likert scale.


RESULTS
The median distance to target was statistically lower for AR than for NV (1 mm [Q1: 1 mm, Q3: 2 mm] vs 3 mm [Q1: 2 mm, Q3: 4 mm] [p < 0.001]). In the AR task, the median number of recalibrations was 4 (Q1: 4, Q3: 4.75). The number of recalibrations was significantly correlated with the precision (Spearman rho: -0.71, p < 0.05). The trust assessment after performing the experiment scored a median of 8 for AR and 5.5 for NV (p < 0.01).


CONCLUSIONS
This study shows for the first time the superiority of AR over NV in terms of precision. AR is easy to use. The number of recalibrations performed using reference structures increases the precision of the navigation. The confidence regarding precision increases with experience.

Augmented reality (AR) refers to the projection of virtual objects on real-world images. 11 This technology is gaining interest not only for use during tumor surgeries 10,12-14 but also in the preparation for spine surgeries [15][16][17] or cerebrovascular interventions. [18][19] Together with virtual reality (or the simulation of a virtual environment with which the user can interact), AR offers great opportunities in the fields of education and neurosurgical training. [20][21][22][23] In particular during cranial surgeries, AR has gained popularity because it offers 3 main advantages that are not offered by standard NV: 1) it allows direct projection of preregistered overlaid images on the real-world view while using the surgical microscope; 2) the surgeon keeps his or her focus on the surgical field; 24 and 3) modern recalibration tools palliate inherent imprecision, such as brain shifts during resection. The surgeon is able to reregister the AR according to intracranial reference structures such as vessels or bones. These recalibration possibilities represent an outstanding evolution of NV systems because they possibly reduce the registration error and other errors inherent in the surgery.
However, to the best of our knowledge, no protocol has studied the precision of AR compared to standard NV. The aim of this study was to analyze the precision of AR compared to NV. We hypothesized 1) that AR enhances the precision by at least a factor of 2; 2) that the trust in AR-assisted navigation precision is higher than in NV once the operator has had experience with both; and 3) that 3D video playing experience improves the accuracy of the operator.

The 3D-Printed Skull
A commercially available, standardized 3D-printed skull was used for the purpose of this study (Kezlex). The CT scan from the 3D-printed skull was fused with a CT scan and an MRI sequence originating from a patient suffering from a 2 × 2-mm right frontal sinus defect with CSF leak (Elements; Brainlab). A 7 × 5-cm frontal medial opening in the skull let participants look inside. The right frontal sinus defect was not apparent on the 3D-printed model, but its location was precisely marked by the first and the last authors. The position of the defect was hidden to participants during the task. The skull was held in a Mayfield clamp with a referencing array fixed to it, in order to simulate surgical conditions (Fig. 1).

NV and AR Registration
The software Elements (Brainlab) was used to create virtual objects such as the skull and the right frontal sinus defect based on the preoperative CT and MRI studies. For the purpose of standard NV, referencing of the 3D-printed skull was performed using 6 predefined landmarks: the 2 supraorbital foramina, the glabella, the medial posterior point of the frontal opening, and 2 points on the structures held by the Mayfield clamp.
For the purpose of the AR, the surgical microscope (Leica) was navigated and referenced after focusing on the center of the referencing array. The virtual 3D volume rendering of the referencing array and skull was then adjusted to the real-world image using the microscope AR software (Elements, Application Microscope; Brainlab).

Participants
Twenty-six participants took part in the study: 7 young neurosurgical residents from the Geneva University Hospitals without NV or AR experience, 18 medical students from the University of Geneva Faculty of Medicine without experience or training in neurosurgery, and 1 Brainlab executive with no experience in neurosurgery and little experience with AR. Each participant was asked if she or he had played at least 100 hours of video games with a 3D perspective.

Setup and Task
The skull was held in a Mayfield clamp with a referencing array fixed to it, in order to simulate surgical conditions. Participants then received instruction in NV and AR, and were invited to test the NV and AR system freely for 10 minutes.
The first task focused on using standard NV and a pointer, as described in Fig. 2. Participants were asked to register the 3D-printed skull and to evaluate the global precision of the system. If they were not satisfied, another trial could be performed. The first task consisted of pointing to the center of the defect in the skull by using a navigated pointer (Softouch pointer; Brainlab) without looking into the skull: participants navigated the pointer to reach the target while watching 2D MR-based images on the navigation screen. The distance between the tip of the navigated pointer (point) and the real center of the defect (target) was recorded (point-to-target distance).
The second task focused on using the navigated microscope and AR, as described in Fig. 3. Preregistered virtual objects such as the skull and the right frontal sinus defect were projected into the microscope overlaid on the real-world image. Using the overlaid objects, participants were instructed to target the center of the defect by using the laser of the microscope. Participants were asked to use the recalibration function of the AR system to achieve, according to them, a precision good enough to reach the center of the target. The number of readjustments was recorded. Finally, the distance between the pointed localization (point) and the target was measured (point-to-target distance).
Before and after realizing each task, all participants were asked: "On a scale from 0 to 9, how much do you trust the technology to reach a small target (AR/NV)?" with 0 being no trust and 9 being complete trust (9-level Likert scale). Assessments made before and after the experiment were compared (pretest and posttest trust). Participants were made aware of the point-to-target distance before giving the posttest score for each task.

Statistical Analysis
All statistical analyses were performed using RStudio and R version 3.6.1 software. All continuous nonparametric variables were compared using a Wilcoxon test. Results were expressed in terms of the median, first quartile (Q1), and third quartile (Q3). A p value < 0.05 was considered significant. According to pilot assessments, we assumed that AR would increase the precision (or reduce the point-to-target distance) by 2-fold. The power calculation determined that a sample size of 24 participants was needed to reach an 80% chance to detect a 2-fold increase in precision, considering an α error of 5%.

Participants
Of the 26 participants, 9 were women (34.6%). None of the participants had prior experience with surgical AR. One man had already used AR, but had no neurosurgical experience. Fifteen participants had a > 100-hour video gaming experience with 3D perspective (57.7%).

Precision
All measures for the NV and AR tasks were successful. The median distance-to-target value for the NV task was 3 mm (Q1: 2 mm, Q3: 4 mm) and for the AR task it was 1 mm (Q1: 1 mm, Q3: 2 mm) (p < 0.001) (Fig. 4). There was no statistically significant difference in precision by sex The median number of recalibrations in the AR task was 4 (Q1: 4, Q3: 4.75). The number of recalibrations was correlated with precision using AR (Spearman rho: −0.71, p < 0.05) (Fig. 5). It has to be highlighted that the extreme distance for AR (4 mm), or the lowest precision, was performed with the lowest number of recalibrations (2 readjustments).

Subjective Trust in the Systems
No difference was found in the pretest trust between  (Fig. 6).

Discussion
This study is, as far as we know, the first analysis reporting superiority in terms of accuracy of AR over NV when using a 3D-printed skull replicating a real clinical scenario. Furthermore, we showed that even if the pretest trust in precision was comparable between NV and AR, the posttest trust in AR was significantly higher than in NV. Video gaming experience did not influence the precision of the operator.
Stieglitz et al. 6 retrospectively analyzed 55 cranial procedures during which standard NV was used and recorded the target registration error immediately after registration by pointing at the center of the nasion and comparing it to preoperative image-based NV. This distance represents the inherent error of the system. Stieglitz et al. reported a mean error of 2.9 mm, consistent with our findings using a 3D-printed skull (median 3.0 mm). The authors also reported an increase in NV inaccuracy after each step of the surgery (after placement of surgical drapes, skin incision, craniotomy, etc.). 6 The 3D-printed skull was registered through 6 referencing points defined on the NV system. These points were chosen because of their invariability during the procedure, such as the supraorbital foramina in this case. The referencing points were already set in the NV system and participants were asked to target them one by one on the 3D-printed skull for the purpose of homogeneity.
Unvarying "signature" structures used for the recalibration of AR should be in the direct proximity of the target. In our case, readjustment according to the frontal opening and then according to the frontobasal bony structures offered a great configuration to achieve the 1-mm precision.
In our study, the mean distance to target was significantly shorter when using AR compared to NV (mean 1 mm for AR vs 3 mm for NV, p < 0.001). AR offers the opportunity to permanently recalibrate preregistered objects to fit to the actual surgical images. The skull and intracranial bony configuration, as well as cranial sutures and intracranial vessels, offer unvarying structures that can be used as references to readjust using AR. During our task, participants were asked to recalibrate after registration according to the skull. Bony structures of the frontobasal area offered great possibilities to readjust the AR near the target. The analysis showed that 3-4 recalibrations were necessary to achieve the 1-mm point-to-target distance (Fig. 5).
The choice to test the setup on medical students and young neurosurgical residents without experience with NV systems was based on the hypothesis that AR is an intuitive and easy-to-use method to localize a relatively small target. All participants received a short introduction and were invited to test the system before performing the task. None of the 26 participants failed. No distinction was made based on sex. This is important because AR has been used for educational and training purposes. [20][21][22][23] Apprenticeship, surgical experience, and hands-on experience have traditionally been a challenge for young neurosurgical residents. 22,23 ,25-27 Si et al. 23 have proposed in their works results from a test of a new AR-guided neurosurgical simulator with haptic answers. After producing a 3D-printed skull from a patient and registering it, 10 participants without neurosurgical experience "in real life" were enrolled. They were asked to remove an intracerebral tumor and their feedback on the realism of the procedure was analyzed. Responses from participants indicated that AR provides a novel and realistic way to obtain experience during neurosurgical training. The participants strongly approved the gain of AR in the procedure of tumor removal. Our data showed that the posttest minus pretest trust strongly favors AR (Fig. 6). After knowing their results FIG. 6. Difference of trust posttest minus pretest. Participants had significantly more trust in AR to find small targets after using it compared to standard NV. ***p < 0.001. for point-to-target distances for NV and AR, the subjects understood the potential of AR to localize relatively small targets. Furthermore, by using nonneurosurgically trained participants, our results reflect the actual inherent error of the NV system without the influence of a participant's neurosurgical skills.
To our surprise, experience with video games did not give a significant advantage regarding AR-assisted navigation accuracy. This may indicate that the 3D volume rendering in the microscope is well perceived by participants even without experience with virtual 3D images, or that our sample was too small to reveal a small effect (15 participants with video gaming experience vs 11 without).
The main limitation in our protocol is based on the following facts. 1) Participants did not engage in the "preoperative" delineation of the target object. The planning phase seems to be as important as the ability to understand intraoperative structures with the use of AR. 10,17,21 2) Errors potentially associated with multimodality image fusion to generate digital skull models and targets have not been assessed. 3) Errors potentially associated with soft-tissue deformation or brain shift have not been assessed and will be the subject of a dedicated future study.
Despite mimicking surgical conditions as much as possible, it remains a laboratory experiment; assessment of precision and trust in AR versus NV needs now to be formally assessed during cranial surgeries.

Conclusions
The comparison of AR and NV in a simulated surgical condition by using a 3D-printed skull showed a superior precision of AR-assisted navigation with respect to the point-to-target distance compared to standard pointerbased NV. AR systems offer the possibility to readjust the navigation at any time during the procedure by using realtime projections of virtual objects on real-world structures through the surgical microscope. AR systems are easy to use in daily practice.