{'fraction_non_alphanumeric': 0.02524692602297924, 'fraction_numerical': 0.010834509171537996, 'mean_word_length': 5.378977820636451, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Moving beyond two-dimensional screens to interactive three-dimensional visualization in congenital heart disease

Beginning with the discovery of X-rays to the development of three-dimensional (3D) imaging, improvements in acquisition, post-processing, and visualization have provided clinicians with detailed information for increasingly accurate medical diagnosis and clinical management. This paper highlights advances in imaging technologies for congenital heart disease (CHD), medical adoption, and future developments required to improve pre-procedural and intra-procedural guidance.


Introduction
Diagnostic imaging has played a pivotal role in the medical field since 1895 when Wilhelm Röntgen discovered X-rays. Technological advances led to fluoroscopy becoming available in 1920 for radiologists and interventional cardiologists to make improved diagnoses and perform interventions of increasing complexity [1]. The introduction of ultrasound in 1956, computed tomography (CT) in 1972, and magnetic resonance (MR) imaging in 1977 transformed the medical field. Unlike X-ray and CT, MR does not use ionizing radiation for image acquisition, and soft tissue sensitivity is found to be greater than that of CT [2]. Subsequent improvements in imaging technologies now allow higher resolution, and significant increases in computing power enable visualization of both soft tissue and bone as well as faster detection of abnormalities [3][4][5]. Fast-forward several decades; we now have CT, MR and ultrasound-based 3D imaging, 3D rendering, and full-volume imaging. These technologies are providing clearer image studies of congenital heart disease (CHD). Moreover, advanced processing of individual modalities can be achieved for virtual visualization and 3D printing of complex CHD. Initially segmentation was limited to CT and MR and is now feasible with ultrasound [6]. The integration of multiple imaging modalities has been demonstrated to improve the accuracy of hybrid 3D models over single modality models [7,8]. The advent of augmented and virtual reality in medical imaging is transforming workflows for clinical decision-making as well as enhancing education of patients/family members and medical professionals. These new advances and innovations in medical imaging are pushing the limits on how physicians and other professionals can utilize the various imaging datasets [8], which are particularly important for the care and management of children and adults with CHD ( Fig. 1).

Current state of imaging
The standard-of-care imaging modalities for congenital heart disease include CT, MR, and ultrasound. Post-processing of the imaging datasets can be performed on a picture archiving and communication system (PACS). Historically, PACS vendors have provided inconsistent tools to functionally allow 3D assessment or simultaneous viewing of multiple imaging modalities, thereby limiting full anatomic evaluation [9]. Companies currently supporting Further advances in medical imaging, namely, in multienergy CT, cardiac magnetic resonance (CMR) and 3D echocardiogram (3DE) have made it possible for high quality 3D volumetric rendering of dynamic cardiac structures [8,10]. Each imaging modality has different strengths: cardiac CT is the modality of choice for visualization of extracardiac and small vascular anatomy; CMR is the gold standard for quantification of ventricular volumes and myocardial architecture; and 3DE provides the best visualization of valve morphology and intracardiac structural anomalies [3][4][5]8]. As such, these imaging datasets have proven to be advantageous for pre-procedural planning of surgical and interventional procedures [8]. However, cumulative exposure to radiation is a major concern in patients who have image acquisition using CT or diagnostic/interventional catheterization using fluoroscopy [11,12]. In addition, there are significant limitations as viewing 3D anatomy on a 2D screen may cause each observer to interpret it from different perspectives, leading to misinterpretation and high interobserver variability [13]. In other words, the conceptualization of three-dimensionality on a 2D visualization platform requires the physician or end user to have advanced knowledge in 3D spatial orientation. The advent of 3D printing and 3D visualization  The sagittal view of the segmented virtual model shows the compressed pulmonary artery conduit (red arrows) between the sternum and the myocardium decreases this risk as it leaves no aspect of the spatial relationship of the cardiac structures to the imagination (Fig. 2) [14].

3D printing
3D printing in surgical and interventional planning in complex CHD is well recognized. Prior to 2014, CT and CMR were the only modalities used to derive 3D printing cardiac models. Printing of 3D anatomic cardiac models sourced from 3DE was shown to be feasible in 2014 [6]. Combining the strengths of each imaging modality has provided the ability to print morphologically accurate hybrid 3D models of cardiac structures [8]. The Radiological Society of North America 3D Printing Special Interest Group recently established appropriateness guidelines for 3D printing cardiac models including various other specific medical conditions. These guidelines include the best approach for acquisition of imaging, segmentation tools, 3D printing, and post-processing the models for CHD diagnosis and patient care [15]. 3D printing of CHD has been shown to be effective in double inlet left ventricle (DILV), double outlet left ventricle (DOLV), double outlet right ventricle (DORV), double inlet right ventricle (DIRV), congenitally corrected transposition of the great arteries (levo-TGA), transposition of the great arteries (dextro-TGA), unbalanced atrioventricular canal, Tetralogy of Fallot (ToF), truncus arteriosus, total anomalous pulmonary venous return (TAPVR) and partial anomalous pulmonary venous return (PAPVR) [15]. Cardiac models have proven valuable for pre-procedural planning, patient education, and training of medical professionals [16][17][18][19][20][21][22]. Research has provided insight into how valuable 3D printed models are with regards to post-operative care of CHD patients especially with patient hand-off versus a traditional verbal hand off [23]. Considering 3D printed heart models from CT or MRI have been accurate representations, yet there remains the unknown of whether the STL file and the 3D printed model are true representations in terms of measurements [24]. However, the heart is a dynamic organ and using static 3D models for cardiac visualization provides a limited representation of cardiac anatomy and physiology [8,10,25]. Further innovations in medical 3D printing include development of materials, which more accurately mimic cardiac tissues [26]. 3D bio-printing is another technology which may provide significant innovations in treatment of congenital and structural heart disease. The bioengineering of an electromechanically functional miniature ventricular heart chamber from human induced pluripotent stem cells has now been demonstrated [27,28]. Development of functional artificial chambers that can be safely implanted into patients will have a major impact on the treatment of complex CHD.

Interactive 3D visualization
Major limitations of 3D imaging are the current display methods, i.e. displaying 3D imaging datasets on a 2D screen [8,29]. The introduction of augmented reality (AR) and virtual reality (VR) in medicine is another significant advancement with potential to significantly improve the care and management of patients. However, there is some skepticism with AR/VR systems, specifically the need to wear bulky or uncomfortable glasses/headsets. Some users may also experience motion sickness with the potential of compromising patient safety during procedures [11,12]. There are several companies working on solutions. One such solution is the True 3D Viewer (EchoPixel, Inc., Santa Clara, CA, USA) where imaging datasets sourced from CT, MR, and 3DE can be viewed in 3D space with the use of 3D glasses. Moreover, the system offers the user an interactive platform with the ability to rotate the images in multiple planes, perform dissections, segment the dataset, and perform measurements [30]. While the EchoPixel screen displays the image, lightweight 3D glasses with sensors are followed by tracking built into the display allowing the image to move with the viewer. This allows the now 3D image to be in perspective with the viewer and a stylus allows the user to manipulate the image and understand the complexity of the CHD. There are six viewing modes with each providing a different function allowing for images to be differentially analyzed (intuitive 2D, True 3D, C-arm Slab, Haptic annotation, radiology, and surgery). For CHD cases, having multiple viewing planes is particularly useful allowing for the heart to be evaluated from different perspectives. This technology is able to interpret and display CT, MR and XA standard DICOM/DICOMDIR files along with certain types of DICOM ultrasounds (GE vivid E90, E95 and Philips E33, GI, 3DDCM) [31]. EchoPixel has a toolset with several functions which include making measurements (including surface and volume), region growing segmentation, region of interest (ROI) selection, volume editing, and transfer function presets. These allow for CHD defects to be measured and analyzed as well as be prepared as a model for 3D printing within EchoPixel. These optimized image datasets can then be saved and referenced for later display for surgical or catheterization planning [31]. The segmentation tool allows for parts of the image to be transformed, visualized and potentially 3D printed. The C-arm view allows physicians to better view a fluoroscopy image by showing the orientation within the patient [32]. The bookmark tool helps with planning and actuating of surgeries wherein the surgeon bookmarks steps which allows for scrolling through during surgery creating a set of instructions with images for the procedure [33]. When paired with a 3D projector, the True 3D Viewer can be used with groups for pre-procedural planning, patient/family education, and training in multiple Clinical utilization of EchoPixel has been found to improve surgical procedures in patients with CHD. In one study, approximately eighty percent of surgeons who looked at both EchoPixel and standard methods of imaging found it to be useful [34]. EchoPixel has been found to alter surgical techniques and repair strategies and offer additional insight in patients with borderline intracardiac anatomy for biventricular repair [35]. In a case at Stanford Children's Health, EchoPixel was used in the operating room to decrease the size of the surgical incision, which reduced recovery time and risk of infection for post-operative patients [35]. In another study, EchoPixel was used to measure the internal carotid artery length [36]. This measurement was taken several times on two patients and determined to be reproducible, showing the accuracy of the technology.
Real View Medical Holography (RealView Imaging Ltd., Israel) has designed a system, which uses holograph technology to display dynamic images, live holography, without requiring the user to wear a headset [37]. This has now been proven to be useful in the catheterization lab to show real time medical holograms-https ://youtu .be/KLQCb Dblji k [10]. In September 2018, the U.S. Food and Drug Administration granted NOVARAD (Salt Lake City, UT), 510(k) Clearance for use of their OpenSight AR system. This system allows the user to view AR in 2D, 3D and 4D simultaneously reducing the risk of disorientation [38]. The system supports preoperative planning providing the surgeon or interventionalist the advantage of comprehensively understanding the anatomy prior to performing a procedure. This technology has the possibility of decreasing procedure time and potential errors while improving outcomes. A recent study in eight patients undergoing cardiac catheterization procedures where RealView hologram was utilized along with standard imaging demonstrated the value of this product. The observers rated the hologram usefulness on a scale of 1-5 and all of them found it to be the most useful. Additionally none of them became nauseous, which is a common issue when utilizing augmented reality [10].

Artificial intelligence
Recently, algorithms have been developed wherein computers may aid in heart segmentation in preparation for visualization or 3D printing [39]. Efficient segmentation methods may make it possible for scaling volumes of patients or urgent cases. Patch based interactive segmentation in which an operator manually segments a few anatomic slices while an algorithm completes the rest of the dataset. Current algorithms can create accurate models using only 14 segmented anatomic slices. This is significantly less than the normal process to create segmented models. Improving these algorithms may speed up this process, eventually allow for 3D printing to be utilized commonly in clinics [39]. Another version of this patch-based segmentation is being used for mitral valve visualization. This algorithm uses scans and coordinates as well as a moldable model to create a fitted model. This works well for a small region of interest, however has not been used for a whole heart or larger anatomic areas [40]. Another use of artificial intelligence is to compile patient data to determine what type of imaging to order and produce probable diagnoses. AI can also aid in deciding whether patients are eligible for a study. These processes are fully realized when large amounts of data are compiled and analyzed in one program. AI systems are also able to search the web or cloud-based systems and find cases from there. The difficulties of medical adoption of AI are programs which are fast and user friendly enough to be used in clinics [41]. Artificial neural networks (ANNs) are the most common type of AI that are used in imaging. These systems use pathways which make connections and compare data. Along with this they weigh these connections to decide which path to go down or outcome to show. This process is developed to operate similarly to a neuron [42]. It can be "taught" but also learns from each diagnosis attempt. Through this learning process it can become increasingly accurate at choosing appropriate imaging or diagnosing patients. Considering AI and its application with CMR, challenges with AI and CMR with regards to CHD have been recognized and therefore need to be addressed moving forward [42].

Future advancements
Future systems need to become more intuitive, allowing users the capability to perform tasks quickly, decreasing risks associated with inadequate preparation for procedures, and increase patient satisfaction and outcomes. Manufactures have yet to put a system on the market that allows for integrated multimodality imaging that can support dynamic 3D datasets. Medical imaging companies have been working in isolation with technology that supports AR and VR for various uses such as movies, flight simulation and other training systems. Some healthcare systems have been evaluating advanced imaging techniques on patients with cancer or other non-cardiac medical diagnoses. The dynamic nature of the cardiac structures and the complexity of anomalies being limited to a vulnerable population of children and adults with CHD, results in significant limitations for the field of congenital cardiology. However, as precision medicine and corresponding treatment options grow, advanced imaging specialists are going to need this enhanced technology to help detect, diagnose, and guide procedures.
Clinical management of complex CHD patients often involves multiple imaging modalities to fully capture the complexity of patient anatomy. Individual imaging modalities are interpreted in isolation without integration on a common platform. Researchers and developers need to design a system with the ability to integrate the strengths of 3D imaging from CMR, CT, angiogram, 3D mapping, and 3DE with the ability to 3D print from them or display them in a dynamic format in a single platform. Moreover, developing a PACS capable of handling a large amount of data from multiple modalities would also be necessary. Current PACS may be capable; however, healthcare systems with PACS may have different software packages for various modalities using appropriate codecs. The next generation of imaging viewers must overcome these limitations. Future development of AI in the area of tissue characterization including incorporating echocardiogram dropouts with auto filling will improve the visualization of ultrasound-based 3D models.
Live imaging during the procedure could be sent directly to the AR system, and the interventionalist could perform the procedure without any fluoroscopy. This level of AR assisted procedures could reduce or eliminate radiation exposure and improve procedural outcomes. Research and development of an AR system that does not require the user to wear a cumbersome headset, while allowing multiple viewers would be ideal for procedures in the operating room or catheterization laboratory. If AR imaging advances to this level, the need for pre-procedural tests may no longer be routinely required. Moreover, adding hemodynamic data to the integrated imaging datasets will improve assessment and interventions. The advantage of AR is its ability to integrate aspects of the virtual world with the real world, enhancing the objects that a person can feel, see, and hear. This would be an excellent tool for surgeons and interventionalists. Virtual interventional or surgical procedure using image fusion technology is within reach. Ultimately, this technology has the potential to benefit patients, improve surgical and interventional outcomes, reduce procedural errors and risks, and decrease costs associated with medical procedures and hospital admissions.
In addition to this, artificial intelligence (AI) is a field which has significant potential to aid imaging. Currently there are AI systems in development which may provide significant performance and be advantageous to CHD imaging. However, developing a system which can be used across a hospital and is very user friendly is not a reality yet. Creating a system which can perform the functions of storing and organizing patient data helping with computer aided diagnoses and other AI functions would be ideal. This would allow a smooth process for adoption of these new technologies. However, it is anticipated that implementing these systems in hospitals would take a significant amount of time.

Conclusion
Assessing current imaging systems and how innovative they have become leaves us to wonder what the future will hold. Since the beginning of medical imaging, technology has been pushing the medical visualization boundaries and enhanced the capabilities and medical professionals. Current systems are providing medical professionals with advanced imaging datasets to perform what is necessary to improve patient care, reduce surgical errors, and to preplan surgical or interventional procedures. These technological advances have made it possible for physicians to better appreciate the complexity of CHD and cardiac structures. Advances in integrated multimodality imaging is expected to democratize data interpretation from highly trained professionals of individual modalities to those directly interacting with patients. Advanced 3D imaging, and visualization of dynamic structures, 3D printing, and image manipulation are paving the way for research and development of integrated imaging systems utilizing artificial intelligence that will be able to perform far beyond what is currently available.