{'fraction_non_alphanumeric': 0.022881099100919285, 'fraction_numerical': 0.007728053338721083, 'mean_word_length': 4.9761545427105345, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Impact of SciDAC on accelerator projects across the office of science through electromagnetic modeling

Electromagnetic Modelling led by SLAC is a principal component of the "Advanced Computing for 21st Century Accelerator Science and Technology" SciDAC project funded through the Office of High Energy Physics. This large team effort comprises three other national laboratories (LBNL, LLNL, SNL) and six universities (CMU, Columbia, RPI, Stanford, UC Davis and U of Wisconsin) with the goal to develop a set of parallel electromagnetic codes based on unstructured grids to target challenging problems in accelerators, and solve them to unprecedented realism and accuracy. Essential to the code development are the collaborations with the ISICs/SAPP in eigensolvers, meshing, adaptive refinement, shape optimization and visualization (see "Achievements in ISICs/SAPP Collaborations for Electromagnetic Modelling of Accelerators"). Supported by these advances in computational science, we have successfully performed the large-scale simulations that have impacted important accelerator projects across the Office of Science (SC) including the Positron Electron Project (PEP) -II, Next Linear Collider (NLC) and the International Linear Collider (ILC) in High Energy Physics (HEP), the Rare Isotope Accelerator (RIA) in Nuclear Physics (NP) and the Linac Coherent Light Source (LCLS) in Basic Energy Science (BES).


Introduction
Particle accelerators are essential tools for scientific discovery in DOE's Office of Science (SC). Close to 50% of the proposed "Facilities for the Future of Science" involve accelerators as they are being used in High Energy Physics, Nuclear Physics and Basic Energy Science plus many other fields of scientific research. Accelerators are among the largest and most complex facilities in the world with system requirements continually pushing the envelope of accelerator science and technology. In light of the significant costs, it deems necessary that the most advanced numerical tools and resources be brought to bear on these machines to optimize their design and improve their performance.
Almost all accelerators make use of electromagnetic waves to accelerate particles so electromagnetic structures comprise the core of the accelerating system. Of these, the accelerating cavity is the most important as it determines the energy reach of the machine. The proposed International Linear Collider (ILC) [1] for example, would require 20,000 cavities to accelerate the beams to 500 GeV and they constitute a large fraction of the machine cost. It is therefore not surprising that the design of the accelerating cavity and the associated electromagnetic structures is of primary importance in any accelerator R&D.
Next generation accelerators are required to meet increasingly challenging specifications in beam energy and machine current. As a result, the accelerating cavities have become more complex in geometry and the accuracy requirement on their design more stringent. While numerical modelling is already used extensively in the accelerator community, more advanced tools are needed to simulate the complicated cavity designs under consideration for future facilities, such as the International Linear Collider (ILC) and the Rare Isotope Accelerator (RIA) [2], to accuracies, in speed and problem size previously not possible.

Parallel Electromagnetic Codes based on Unstructured Grids
Under SciDAC, SLAC is developing new 3D parallel electromagnetic codes [3] that are based on unstructured grids which conform to the geometry surface for higher accuracy than standard rectangular grid representation. The list of codes developed to date is as follows: Omega3P -Parallel eigensolver for finding normal modes in accelerating cavities; S3P -Parallel S-matrix solver for computing transmission properties of traveling wave structures; T3P/Tau3P -Parallel time-domain solver with beam excitation for wakefield calculations; Track3P -Parallel particle tracking module with surface physics to simulate dark currents; V3D -Graphics tool for visualizing meshes, fields and particles.
Except for Tau3P which is formulated on the generalized Yee grid, Omega3P, S3P and T3P are based on the finite element method, employing basis functions up to the 6 th order. The most mature of these field solvers is the eigensolver Omega3P, followed by the S-matrix solver S3P while work is continuing on advancing Tau3P and T3P. Track3P is being extended to include multipacting simulation and V3D is developing capabilities to facilitate code debugging and result analysis. These codes have been ported to linux clusters, the IBM SP at NERC as well as the Cray X1 at NCCS to perform the large-scale simulations necessary for high-resolution cavity modeling, or the end-to-end simulation of electromagnetic systems.

ISICs/SAPP Collaborations in Computational Science
Parallel electromagnetic modelling is a multi-step process that involves meshing, partitioning, solvers, refinement, shape optimization, performance optimization and visualization, all of which play an important role in the overall success and productivity of the simulation. Under SciDAC, SLAC is engaged in the following collaborations with the ISICs and SAPP -Parallel meshing with TSTT (SNL and U of Wisconsin), Partitioning with SAPP (SNL, LBNL), Eigensolvers with TOPS and SAPP [4,5] (LBNL, UC Davis, Stanford), Adaptive mesh refinement with TSTT (RPI), Shape optimization with TOPS and TSTT [6,7] (CMU, Columbia, LBNL, SNL, LLNL), Performance optimization with PERC (LBNL, LLNL) and Visualization with SAPP (UC Davis). Details on these efforts are described in the poster paper "Achievements in ISICs/SAPP Collaborations for Electromagnetic Modelling of Accelerators" [8]. Advances in these computational science areas have been essential to realizing some of the challenging simulations required by the accelerators.

PEP-II IR Heating
The PEP-II B Factory is a high current positron (e+) electron (e-) collider at SLAC for studying the properties of matter and anti-matter. In its early operation, the storage ring was limited from operating at higher currents due to excessive heating at the Interaction Region (IR) chamber because of beam driven higher-order-modes (HOMs). Figure 1 shows the IR beamline complex which is a long (> 5m) structure with complicated cross sections of several cm in diameters making it challenging to model due to the large aspect ratio and detailed features. Using Omega3P the power distribution was calculated by summing the trapped modes to help in the IR redesign to reduce beam heating. Since the IR upgrade, the accelerator was able to run at 15% higher current resulting in higher luminosity which led to new particle discovery. We have also calculated the heating directly using Tau3P with beams and the simulation was made possible because of help from TSTT in making good quality hexahedral meshes.

PEP-II Vertex Bellows Damping
The PEP-II accelerator is operating at increasingly higher currents and shorter bunch lengths to increase the luminosity and thus, the event rate for physics studies. As a consequence, the beam generates more higher-order-mode (HOM) power that can cause overheating at some beamline components such as the vertex bellows near the IR. The vertex bellows [9] has a complicated 3D geometry with disparate length scales, ranging from 60 microns in the gap between the contact fingers to 2.5 cm for the radius of the vacuum chamber. Plans are to mount ceramic tiles on the bellows convolution to damp the localized modes excited by the beam. A new complex solver in Omega3P developed with TOPS/SAPP was used to evaluate the damping and the results showed that the Qo of the bellows modes can be reduced to below 30. This indicates that large-scale simulation can provide a faster and cost-effective way to validate the performance of complex engineering designs.

NLC Cell Design
The NLC was a technology option using room-temperature RF for a TeV scale e+e-linear collider targeted for HEP research. The Damped, Detuned Structure (DDS) [10] for the NLC main linac is a novel 3D design that is optimized for higher gradient (14% increase over standard design) and suppresses harmful long-range wakefields that disrupt long bunch train operation. The computational challenge was to model the complex geometry close to machining tolerance in order to obtain accuracy for the cavity frequency of better than 0.01%. This is necessary for maintaining the cavity efficiency and to avoid post-tuning of the cells. Using Omega3P on the NERSC's Cray T3E, a table of dimensions for all 206 cells along the DDS was generated for computerized machining. QC measurements on the fabricated prototypes showed that the required 0.01% frequency accuracy was met. This is a significant result which demonstrates that high resolution modeling can replace timeconsuming and costly trial-and-error prototyping and Omega3P has since been considered a core design capability for the NLC.

NLC DDS Transverse Wakefields
The NLC planned to operate with long bunch trains so the suppression of transverse wakefields to preserve the beam emittance is of primary importance. The DDS employs cell-to-cell variation to detune the dipole modes for 100 fold wakefield reduction at the trailing bunch location, and extracts the wakefields out of the cavity through the HOM couplers via the external manifolds to provide damping over the entire bunch train. Figure 5 shows the 55-cell model of the NLC baseline design. Previously, wakefield analysis of this structure had been limited to equivalent circuit models. Using Tau3P, an end-to-end simulation of the DDS in exact dimensions has been carried out for the first time with a transit beam ( Figure 5). The wakefields is shown in Figure 6 in which the wakefields obtained by summing over the eigenmodes (one such eigenmode is plotted in Figure 7) computed with Omega3P is also shown for comparison. We see there is good agreement thus validating the damped, detuned scheme for wakefield suppression while providing a good benchmark case for the two codes.

NLC Dark Current Pulse
Dark current can limit the NLC linac from operating at its designed field gradient. At high fields, the structure walls emit electrons which further generate secondary electrons upon impact with the wall surface. Electrons that are captured by the accelerating field constitute the dark current which can perturb the main beam and/or affect the diagnostics downstream at the interaction point. To understand dark current generation and capture is a fundamental goal in high gradient structure R&D. Applying Track3P with fields from Tau3P, we have simulated the dark current pulse in a 30-cell constant gradient structure for various rise-times of the RF drive pulse to compare with measurements [11]. Figure 8 shows an instance in time of the dark current generated in the 30-cell structure where the primary electrons are denoted in red and the secondaries denoted in green. In Figure 9, we compare the dark current pulses for rise-times of 10, 15, and 20 nsec in the drive pulse, with Track3P results on the left and the measured data (last row) on the right and reasonably good agreement is found.  Field analysis with Tau3P revealed that the fields in the structure are enhanced during the rising pulse due to dispersive effects. The faster the rise-time the broader range of frequencies is excited which results in a more pronounced field enhancement. Since dark current generation depends on the field gradient, one can understand the increase in the dark current pulse as the rise-time gets shorter. This work provides evidence that we have a powerful tool for reliably predicting the dark current effect in a given structure, and represents a major advance in accelerator science enabled by the large-scale simulations performed under SciDAC.

ILC Low-Loss Cavity Design
The ILC is the highest priority future accelerator project in high energy physics. This is a 0.5 to 1.0 TeV electron-positron collider to provide a tool for the study of dark matter, dark energy, and the fundamental nature of energy and matter as well as space and time. The choice of technology for the accelerator has been determined to be superconducting RF and the accelerating cavity is based on the TESLA design which came into existence in the early nineties. A new low-loss cavity design has recently been proposed that offers the possibility of improved performance (higher gradient) and reduced cost (lower cryogenic losses) relative to the standard TESLA design. Presently an international team comprising KEK, DESY, FNAL, JLab and SLAC is collaborating on the development of the low-loss cavity as a viable option to the standard TESLA cavity.
An important issue in the low-loss cavity design for the ILC is that of higher-order mode (HOM) damping. If not controlled, long-range wakefields due to beam excited HOMs can lead to emittance growth of the beam. Figure 10 shows the 9-cell low-loss cavity design under consideration by the international collaboration. It consists of 7 inner cells and two outer cells that are connected to enlarged beampipes to which one HOM coupler is attached in the front end of the cavity while a HOM and a fundamental mode (FM) coupler are attached in the back end. One essential goal in the cavity design is the optimization of the HOM couplers to ensure all HOMs are adequately damped so as to preserve beam emittance.
Although the 9 cells have relatively simple shapes the HOM couplers have complicated geometries which have to be modeled precisely if the HOM damping is to be calculated accurately. As a result, there had been no end-to-end simulation to date even of the standard TESLA cavity since its existence more than a decade ago. None of the existing codes were able to tackle such a challenging problem without the combined capabilities of parallel computing and conformal meshing. SLAC is using Omega3P with quadratic elements to model the ILC low-loss cavity [12]. Because of the coupling to external waveguides as is the case for the HOM and FM couplers, the eigenvalue problem now becomes nonlinear as the boundary conditions are also functions of the eigenvalue if the waveguide terminations are assumed to be matched. Matched termination allows only outgoing wave and therefore provides damping to the modes. In collaboration with TOPS and SAPP, two algorithms have been implemented to solve the nonlinear eigenvalue problem -a second order Arnoldi (SOAR) method and a self consistent iteration (SCI) method [13]. These methods find the complex eigenfrequencies and eigenvectors of the higher-order modes in the cavity.
The figure of merit for damping is the external quality factor or Qe which is given by ½ of the ratio between the real and imaginary parts of the complex eigenfrequency. Figure 11 shows the partitioned model of the low-loss cavity while figure 12 shows the HOM damping calculated by Omega3P for the lowest five dipole bands. On the IBM SP at NERSC, these computations use 738 CPUs, require 300 GB of memory and generate 18 modes per hour.   Work is ongoing to further improve the HOM damping of this low-loss cavity design and a prototype is expected to be built at JLab next year. At the same time, KEK is planning also to construct a lowloss cavity, named the ICHIRO cavity, for the ILC based on a modified version of the present design first proposed by DESY. SLAC is providing the simulation support to the KEK effort as well and Figure 13 is the preliminary result of the HOM damping in the ICHIRO cavity. Since this cavity has a larger beampipe, only two dipole bands are below cutoff and therefore need to be considered in the Qext in ICHIRO-2 cavity

Impact of SciDAC on RIA, a Proposed Accelerator Project in Nuclear Physics
RIA is a proposed Nuclear Physics project that would be the world's most powerful facility dedicated to producing and exploring new rare isotopes not found naturally on earth. A variety of RFQ cavities are being designed for its low-frequency linacs and some of these cavities have complex geometries that are difficult to model. Figure 14 shows the geometry of a hybrid RFQ with widely disparate length-scales. Due to lack of accurate predictions using standard codes, tuners are needed to cover frequency deviations of about 1%. We have used Omega3P to model the RFQ cavities for RIA. Working with TSTT, an adaptive mesh refinement (AMR) capability has been developed that can significantly improve the accuracy in frequency and wall loss calculations. In the case of the RFQ cavity (see poster paper "Achievements in ISICs/SAPP Collaborations for Electromagnetic Modeling), the improvement in frequency accuracy is a factor of 10 while that for wall loss accuracy is a factor 2. AMR [14] also accelerates convergence therefore substantially reduces the computing resources required. It is expected that Omega3P will be used to provide the final design for RIA's RFQ cavities. The improved accuracy from calculations will significantly reduce the number of tuners required thereby lowering the machine cost. The reduced tuning range will ease operation procedures and the improved wall loss will lead to better cooling design also. .

Impact of SciDAC on LCLS, an Accelerator Project in Basic Energy Science
The LCLS will be the world's first x-ray free electron laser for use in photon science research. It is nearing the end of a detailed project engineering and design phase and is about to start construction. The SciDAC Electromagnetic Modeling team was engaged in the design of the RF gun cavity which is an important part of the photo-injector system. A standard design of the RF gun cavity existed but the LCLS specifications required that design changes be made to improve performance and reliability.
The two important requirements for the LCLS RF gun cavity [15] are that (1) the higher-order field components (dipole and quadruple) be minimized to preserve the gun's low emittance, and (2) the RF pulsed heating at the coupling iris be reduced to a manageable level. Two modifications are needed to meet the 1 st requirement -a dual feed input coupler to eliminate the dipole component by symmetry, and a racetrack shaped cavity to reduce the resulting quadruple field component. To reduce pulsed heating, a z-coupling scheme and rounding of the coupling iris are used. The final geometry of the RF gun cavity is shown in Figure 15. .
A comparison of the quadruple field component in the new design versus that in the standard design is also shown in Figure 15. In order to be able to obtain the reduced field accurately, 4 th order finite element basis functions have to be employed. The design of a gun cavity to this level of detail and accuracy would not have been possible without the advanced simulation capabilities developed under SciDAC. Using Omega3P, we have generated the dimensions for the cavity which are ready for fabrication. Figure 15. (Left) Model of the LCLS RF gun cavity with dual-feed racetrack shaped coupler cavity and rounded coupler iris (insert shows heating at iris); (Right) quadruple field component minimized by racetrack shape cavity versus that of cylindrical cavity.

Summary
The Electromagnetic Modeling component of the HEP's Accelerator Simulation SciDAC project has developed a comprehensive set of parallel electromagnetic codes that have been significantly enhanced by ISICs/SAPP collaborations on important aspects of computational science. Using computing resources provided at NERSC and NSSC, the new simulation capability has made an impact on accelerator projects across the Office of Science by optimizing their performance and improving their reliability while lowering cost.