{'fraction_non_alphanumeric': 0.02300221502811382, 'fraction_numerical': 0.0084341455103084, 'mean_word_length': 4.498360655737705, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Data Quality and Performance of the NOvA Prototype Detector

The NOvA project is a long-baseline neutrino experiment. It utilizes the NuMI neutrino beam at Fermilab and consists of two functionally-identical liquid scintillator filled detectors. The detectors are placed 14 milliradians off-axis from the beam and 810 km apart. A 209 ton prototype detector, the Near Detector On the Surface (NDOS), was built and began taking initial neutrino data in December 2010. NDOS is 110 milliradians off-axis from the NuMI beam and also records neutrinos from the Booster Neutrino Beam. As NDOS is in the commissioning phase, metrics are being developed to improve understanding of the detector as well as monitor the quality of data. Performance of the prototype detector will be presented.


The NOνA Experiment
The NuMI 1 Off-axis ν e Appearance (NOνA) experiment will be able to address many questions about the parameters of neutrino oscillation. NOνA will be an order of magnitude more sensitive to values of sin 2 (2θ 13 ) than current limits near 0.1, as shown in Figure 1. It will be able to probe the mass hierarchy as well as examine if oscillations violate Charge-Parity. NOνA will also make precision measurements of θ 23 and |∆m 2 32 |. NOνA consists of two detectors, one located at Fermilab in Batavia, IL, (called the near detector) and one located at Ash River, MN near the Canadian border and Voyageurs National Park (called the far detector). These detectors are functionally-identical except for overall size. The near detector will be about 200 tons, while the far detector will be about 14 kton. These detectors are placed to intercept the neutrinos from the NuMI beam created at Fermilab [2].
The NOνA detectors have a cellular structure. Each cell is a reflective PVC tube that is filled with liquid scintillator (mineral oil doped with about 5% pseudocumene). Within the cell is a loop of wave-length shifting fiber whose ends are connected to an Avalanche Photodiode (APD). For the far detector, the dimensions of the cell are 4 cm x 6 cm x 15 m. The near detector cells have the same cross-sectional area, but are 4 m long. These dimensions result in 0.15 radiation lengths per plane of cells. The planes of cells are orientated perpendicular to the direction the neutrinos are traveling and each plane alternates from being horizontally or vertically aligned in the perpendicular plane. This allows for full three-dimensional reconstruction.
When a neutrino interacts within the detector volume, charged particles are created. As charged particles travel through the liquid scintillator, they create photons. These photons are absorbed by the wave-length shifting fiber and are transported to the fiber end, where they are read-out by the APD. The APD amplifies the signal by a factor of one hundred, a level capable of being read by sensitive, low-noise electronics.
Each APD has 32 pixels which allows it to read out one module. Each APD is then connected to a Front End Board (FEB). The FEB digitizes the signal from the APD and sends it to a Data Concentrator Module (DCM). Each DCM is capable of reading out up to 64 FEBs. This signal is then passed to the rest of the data acquisition system [2].

The Near Detector On the Surface (NDOS)
The NDOS is the prototype detector for the NOνA detectors. At 209 tons, it is approximately the size of the future near detector and is functionally-identical. It is located on the surface at Fermilab instead of underground, as the near detector will be. The NDOS is currently taking neutrino data. Having a prototype of this scale has proved invaluable in preparation for construction of the real detectors. It has allowed us to test and improve everything from assembly procedures to the timing systems.
NDOS detects neutrinos from the NuMI beam, currently running for the MINOS experiment [3], as well as from the Booster Neutrino beam. The Booster Neutrino beam is primarily for the MiniBooNE experiment [4]. The NDOS detector is 110 mrad off-axis from the NuMI beam and the detector axis is in the same plane as the beam axis. The NDOS is nearly on-axis relative to the Booster beam and the detector axis is rotated relative to the beam axis by 23 • .

Monitoring Systems
The NDOS has five levels of data monitoring: the Data Acquisition (DAQ) Monitor, the Memory Viewer, the Event Display, the Online Monitor, and DataCheck. The first four are real-time monitors, while DataCheck has a delay of about one hour. Both Online Monitor and DataCheck report run metrics. Under stable running conditions, runs are about a day in length, with subruns lasting approximately one hour. Subruns consist of trigger windows. Everytime the system triggers on either a neutrino spill or a regular, 10 Hz cosmic trigger, a 500 µsec time window of data is written out. The near and far detectors will have much shorter trigger windows; the longer windows for NDOS allowed us to increase the load on the DAQ system.

DAQ Monitor
The first level of monitoring is done by the DAQ Monitor. It monitors the health and performance of the entire DAQ System, from the DCMs to the network connections. It uses Ganglia as its base. Ganglia is a third-party, easily customizable, open-source software that specializes in tracking statistics like memory and network usage. NOνA has created metrics such as data and trigger rates, data sizes, error states, and the level of data corruption. All of these metrics are written to a database and accessed through a web interface which allows them to be plotted interactively. The example display shown in Figure 2 is used to monitor the status and performance of a DCM. In general, custom monitored metrics can be used to troubleshoot problems, optimize performance, and diagnose trends over both short and long time periods.

Memory Viewer
Memory Viewer is the next level of monitoring. Memory Viewer displays colored bytes of the raw data from the detector, giving an intuitive sense of the data flow. Using pattern matching, one can tell if the run is operating normally. Memory Viewer, as well as many of the other monitoring systems, relies on Event Dispatcher. The Event Dispatcher is a server program running on a DAQ host which streams event data to clients, which connect to it via a TCP socket. Memory Viewer connects to this stream and displays the bytes.  The raw data is displayed as colored bytes, giving an intuitive sense of the data. Using pattern matching, one can quickly assess the run status.

Event Display
Event Display is a detector view of the trigger windows from Event Dispatcher. It shows both of the twodimensional views: the view from the top of the detector and the view from the side. Each channel is represented in one of the views as a grey rectangle. If the channel had a hit during the trigger window, it is colored. The color can indicate the relative time of the hit, as in Figure 4, or the relative charge of the hit. The beam enters the front of the detector, the left-hand side of the Event Display. Towards the back of the detector, we have regions that are either uninstrumented or only partially instrumented. Finally, the muon catcher is represented at the right side of the Event Display; its planes are interspersed with steel so the instrumentation is less dense. Figure 4 shows cosmics rays entering the detector.

Online Monitor
Online Monitor displays run metrics such as the number of active channels and channel occupancies in real time. Figure 5 shows Online Monitor displaying the occupancies of the FEBs which gives an indication of which  FEBs are noisier than others. Online Monitor is broken into two processes to increase system stability. Online Monitor Producer reads in information from the Event Dispatcher and processes it to create run metrics. Online Monitor Viewer is a graphical user interface that communicates with the Producer through a ROOT TMapFile. The Viewer is responsible for actually displaying the information.

DataCheck
DataCheck is the final level of monitoring. It is an offline software tool and has a delay of about one hour from the end of a subrun. DataCheck has the ability to look at metrics subrun by subrun, or cumulatively across multiple subruns or runs to find trends in the data. It also has a web interface that uses PHP to make plots from a database. DataCheck monitors metrics, such as the number of active channels, the average number of hits per channel, the number of active FEBs, and the relative times of neutrino candidates. Section 4 reports some of the findings from DataCheck.

Performance of the NDOS
One fundamental metric of the performance of the NDOS is the number of active channels. Active channels are defined as those that are instrumented and operating within acceptable parameters. From Figure 6, one can see that channels were steadily installed throughout the winter of 2010 and spring of 2011. However, in May 2011, we began actively removing all channels that displayed questionable behavior. This has allowed us to fully understand all the problems with our channels before we begin procurement for the far detector. One may also note the drops in channel level throughout the run period. This is due to frequent running with only parts of the detector while work is being done on other sections. For reference, a fully-instrumented NDOS would have 15,900 channels.
Another basic metric is the number of hits per trigger window per active channel. One can see from Figure  7 that it is generally 5%. When initally commissioning NDOS, we set high thresholds and the average number of hits was lower. Once we understood our system, we lowered our thresholds to more reasonable levels, which accounts for the rise in average number of hits. One can see that sometimes the NDOS gets very noisy. This is due to humidity effects, electronics noise, etc. These effects are under study and we are working to mitigate them in the near and far detectors.
This noisy behavior is also reflected in the rate of FEB shutoffs. Our system is designed to shutoff a FEB during a run if the data buffer overflows. The FEB then remains off for the rest of the run and is re-initialized when the next run begins. Data buffers usually overflow due to noisy APDs. Thus, high rates of FEB shutoffs are a concern. Initially, we noticed noisy run conditions seemed to be correlated with high humidity in the detector hall. Then we began to cool our APDs to −15 • C to reduce the noise levels. However, when we did this on the NDOS, some channels became unusable. It was found that many had water on the surface of the APD due to condensation from air infiltration. We are investigating and redesigning our electronics installation procedure and circuit boards to ensure that the APDs are in a sealed environment.
Using rough activity, fiducial, and direction cuts, DataCheck can quickly find neutrino candidates. The time distribution of these candidates relative to the trigger window is plotted in Figure 8. A peak in this distribution within the 10 µsec NuMI beam spill indicates the timing and triggering systems are operating as expected. Figure 8 contains one week's worth of running and a peak is clearly visible. This allows us to verify if our systems are working on a reasonably short time scale. We also track the Booster neutrino peak; however, with the lower statistics, it takes more than a week to see a peak. : Times of NuMI neutrino candidates within the trigger window (500 µsec) for one week's running. Neutrino candidates are found using rough activity, fiducial, and direction cuts. If a peak is seen in the distribution within the 10 µsec NuMI beam spill, we have verified that the trigger and timing systems are working as expected.