# Search for Rare and Forbidden Dilepton Decays of the 𝐷⁺, 𝐷_𝑠⁺, and 𝐷⁰ Charmed Mesons

## Abstract

We report the results of a search for flavor-changing neutral current, lepton-flavor violating, and lepton-number violating decays of $`D^+`$, $`D_s^+`$, and $`D^0`$ mesons (and their antiparticles) into modes containing muons and electrons. Using data from Fermilab charm hadroproduction experiment E791, we examine the $`\pi \mathrm{}\mathrm{}`$ and $`K\mathrm{}\mathrm{}`$ decay modes of $`D^+`$ and $`D_s^+`$ and the $`\mathrm{}^+\mathrm{}^{}`$ decay modes of $`D^0`$. No evidence for any of these decays is found. Therefore, we present branching-fraction upper limits at 90$`\%`$ confidence level for the 24 decay modes examined. Eight of these modes have no previously reported limits, and fourteen are reported with significant improvements over previously published results.

FERMILAB-Pub-99/183-E   
UMS/HEP/99–020

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , and

The SU(2)$`\times `$U(1) Standard Model of electroweak interactions qualitatively accounts for the known decays of heavy quarks and can often quantitatively predict the decay rates. However, this model is incomplete in that it does not account for the number of quark and lepton families observed, nor their hierarchy of mass scales. Also unknown is the mechanism responsible for breaking the underlying gauge symmetry. One way to search for physics beyond the Standard Model is to search for decays that are forbidden or else are predicted to occur at a negligible level. Observing such decays would constitute evidence for new physics, and measuring their branching fractions would provide insight into how to modify our theoretical understanding, e.g., by introducing new particles or new gauge couplings.

In this letter we present the results of a search for 24 decay modes of the neutral and charged $`D`$ mesons (which contain the heavy charm quark). These decay modes<sup>1</sup><sup>1</sup>1Charge-conjugate modes are included implicitly throughout this paper. fall into three categories:

1. FCNC – flavor-changing neutral current decays ($`D^0\mathrm{}^+\mathrm{}^{}`$ and $`D_{(d,s)}^+h^+\mathrm{}^+\mathrm{}^{}`$, in which $`h`$ is $`\pi `$ or $`K`$);
2. LFV – lepton-flavor violating decays ($`D^0\mu ^\pm e^{}`$, $`D_{(d,s)}^+h^+\mu ^\pm e^{}`$, and $`D_{(d,s)}^+h^{}\mu ^+e^+`$, in which the leptons belong to different generations);
3. LNV – lepton-number violating decays ($`D_{(d,s)}^+h^{}\mathrm{}^+\mathrm{}^+`$, in which the leptons belong to the same generation but have the same sign charge).

Decay modes belonging to (1) occur within the Standard Model via higher-order diagrams, but the estimated branching fractions are $`10^8`$ to $`10^6`$ . Such small rates are below the sensitivity of current experiments. However, if additional particles such as supersymmetric squarks or charginos exist, they could contribute additional amplitudes that would make these modes observable. Decay modes belonging to (2) and (3) do not conserve lepton number and thus are forbidden within the Standard Model. However, lepton number conservation is not required by Lorentz invariance or gauge invariance, and a number of theoretical extensions to the Standard Model predict lepton-number violation . Many experiments have searched for lepton-number violation in $`K`$ decays, and for lepton-number violation and flavor-changing neutral currents in $`D`$ and $`B`$ decays. The limits we present here for rare and forbidden dilepton decays of the $`D`$ mesons are typically more stringent than those obtained from previous searches , or else are the first reported.

The data are from Fermilab experiment E791 , which recorded $`2\times 10^{10}`$ events with a loose transverse energy trigger. These events were produced by a 500 GeV/$`c`$ $`\pi ^{}`$ beam interacting in a target consisting of five thin foils that had 15 mm center-to-center separation along the beamline. The most upstream foil was 0.5 mm thick platinum. It was followed by four foils consisting of 1.6 mm thick diamond. Momentum analysis was provided by two dipole magnets that bent particles in the horizontal ($`x`$-$`z`$) plane. Position information for track and vertex reconstruction was provided by 23 silicon microstrip detectors (6 upstream and 17 downstream of the target) along with 10 planes of proportional wire chambers (8 upstream and 2 downstream of the target), and 35 drift chamber planes. The experiment also included electromagnetic and hadronic calorimeters, a muon detector, and two multi-cell Čerenkov counters that provided $`\pi /K`$ separation in the momentum range $`660`$ GeV/$`c`$ . The kaon identification criteria varied by search decay mode. We typically required that the momentum-dependent light yield in the Čerenkov counters be consistent with that of a kaon track measured in the spectrometer.

Electrons were identified by an electromagnetic calorimeter  that consisted of lead sheets and liquid scintillator located 19 m downstream of the target. Electron identification was based on energy deposition and transverse shower shape in the calorimeter. The electron identification efficiency varied from 62$`\%`$ for momenta below 9 GeV/$`c`$ to 45$`\%`$ for momenta above 20 GeV/$`c`$. The decrease in efficiency with increasing momentum reflects the fact that higher momentum electrons populate a more congested region of the spectrometer. The pion misidentification rate was approximately 0.8$`\%`$, independent of pion momentum.

Muon identification was obtained from two planes of scintillation counters. The plane that measured vertical coordinates ($`y`$) consisted of 16 scintillation counters, each 3 meters long and 14 cm wide. The plane that measured horizontal coordinates ($`x`$) consisted of 14 counters, each 3 meters long and covering a full width of 5.5 meters in the $`x`$-direction. The counters were located behind shielding with a thickness equivalent to 2.5 meters (15 interaction lengths) of iron. Candidate muon tracks projected into the muon system were required to pass a series of muon quality criteria that were optimized with $`D^+\overline{K}^0\mu ^+\nu __\mu `$ decays from our data . Timing information from the $`y`$-coordinate counters was used to improve the position resolution in the $`x`$-direction. The efficiencies of the muon counters were measured in special runs using muons originating from the primary beam dump, and were found to be $`(99\pm 1)\%`$ for the $`y`$-coordinate counters and $`(69\pm 3)\%`$ for the $`x`$-coordinate counters. The probability for misidentifying a pion as a muon decreased as momentum increased, from about 6$`\%`$ at 8 GeV/$`c`$ to $`(1.3\pm 0.1)\%`$ for momenta greater than 20 GeV/$`c`$.

After reconstruction, events with evidence of well-separated production (primary) and decay (secondary) vertices were retained for further analysis. To separate charm candidates from background, we required the following: that secondary vertices be well-separated from the primary vertex and located well outside the target foils and other solid material; that the momentum vector of the candidate charm meson point back to the primary vertex; and that the decay track candidates pass approximately 10 times closer to the secondary vertex than to the primary vertex. A secondary vertex had to be separated from the primary vertex by greater than $`20\sigma __L`$ for $`D^+`$ decays and greater than $`12\sigma __L`$ for $`D^0`$ and $`D_s^+`$ decays, where $`\sigma __L`$ is the calculated resolution of the measured longitudinal separation. In addition, the secondary vertex had to be separated from the closest material in the target foils by greater than $`5\sigma __L^{}`$, where $`\sigma __L^{}`$ is the uncertainty in this separation. The sum of the vector momenta of the tracks from the secondary vertex was required to pass within $`40\mu `$m of the primary vertex in the plane perpendicular to the beam. Finally, the net momentum of the charm candidate transverse to the line connecting the production and decay vertices had to be less than 300 MeV/$`c`$ for $`D^0`$ candidates, less than 250 MeV/$`c`$ for $`D_s^+`$ candidates, and less than 200 MeV/$`c`$ for $`D^+`$ candidates. These selection criteria and, where possible, the kaon identification requirements, were the same for the search mode and for its normalization signal.

For this study we used a “blind” analysis technique. Before our selection criteria were finalized, all events having masses within a mass window $`\mathrm{\Delta }M_S`$ around the mass of $`D^+`$, $`D_s^+`$, or $`D^0`$ were “masked” so that the presence or absence of any potential signal candidates would not bias our choice of selection criteria. All criteria were then chosen by studying signal events generated by a Monte Carlo simulation program (see below) and background events from real data. Events within the signal windows were unmasked only after this optimization. Background events were chosen from a mass window $`\mathrm{\Delta }M_B`$ above and below the signal window $`\mathrm{\Delta }M_S`$. The criteria were chosen to maximize the ratio $`N_S/\sqrt{N_B}`$, where $`N_S`$ and $`N_B`$ are the numbers of signal and background events, respectively. We used asymmetric windows for the decay modes containing electrons to allow for the bremsstrahlung low-energy tail. The signal windows are:

$$\begin{array}{c}1.84<M(D^+)<1.90\mathrm{GeV}/c^{\mathrm{\hspace{0.17em}2}}\mathrm{for}D^+h\mu \mu ,\hfill \\ 1.78<M(D^+)<1.90\mathrm{GeV}/c^{\mathrm{\hspace{0.17em}2}}\mathrm{for}D^+hee\mathrm{and}h\mu e,\hfill \\ 1.95<M(D_s^+)<1.99\mathrm{GeV}/c^{\mathrm{\hspace{0.17em}2}}\mathrm{for}D_s^+h\mu \mu ,\hfill \\ 1.91<M(D_s^+)<1.99\mathrm{GeV}/c^{\mathrm{\hspace{0.17em}2}}\mathrm{for}D_s^+hee\mathrm{and}h\mu e,\hfill \\ 1.83<M(D^0)<1.90\mathrm{GeV}/c^{\mathrm{\hspace{0.17em}2}}\mathrm{for}D^0\mu \mu ,\hfill \\ 1.76<M(D^0)<1.90\mathrm{GeV}/c^{\mathrm{\hspace{0.17em}2}}\mathrm{for}D^0ee\mathrm{and}\mu e.\hfill \end{array}$$
(1)   
We normalize the sensitivity of our search to topologically similar Cabibbo-favored decays. For the $`D^+`$ decays we use $`D^+K^{}\pi ^+\pi ^+`$; for $`D_s^+`$ decays we use $`D_s^+\varphi \pi ^+`$; and for $`D^0`$ decays we use $`D^0K^{}\pi ^+`$. The widths of our normalization modes were 10.5 MeV/$`c^{\mathrm{\hspace{0.17em}2}}`$ for $`D^+`$, 9.5 MeV/$`c^{\mathrm{\hspace{0.17em}2}}`$ for $`D_s^+`$, and 12 MeV/$`c^{\mathrm{\hspace{0.17em}2}}`$ for $`D^0`$. The events within the $`5\sigma `$ window are shown in Figs. 1a–c. The upper limit for each branching fraction $`B_X`$ is calculated using the following formula:

$$B_X=\frac{N_X}{N_{\mathrm{Norm}}}\frac{\epsilon _{\mathrm{Norm}}}{\epsilon _X}B_{\mathrm{Norm}}$$
(2)   
where $`N_X`$ is the 90$`\%`$ CL upper limit on the number of decays for the rare or forbidden decay mode $`X`$, and $`\epsilon _X`$ is that mode’s detection efficiency. $`N_{\mathrm{Norm}}`$ is the fitted number of normalization mode decays; $`\epsilon _{\mathrm{Norm}}`$ is the normalization mode detection efficiency; and $`B_{\mathrm{Norm}}`$ is the normalization mode branching fraction obtained from the Particle Data Group .

The ratio of detection efficiencies is

$$\frac{\epsilon _{\mathrm{Norm}}}{\epsilon _X}=\frac{N_{\mathrm{Norm}}^{\mathrm{MC}}}{N_X^{\mathrm{MC}}}$$
(3)   
where $`N_{\mathrm{Norm}}^{\mathrm{MC}}`$ and $`N_X^{\mathrm{MC}}`$ are the fractions of Monte Carlo events that are reconstructed and pass the final selection criteria, for the normalization and decay modes respectively. The simulations use Pythia/Jetset  as the physics generator and model the effects of resolution, geometry, magnetic fields, multiple scattering, interactions in the detector material, detector efficiencies, and the analysis selection criteria. The efficiencies for the normalization modes varied from approximately $`0.5\%`$ to $`2\%`$ depending on the mode, and the efficiencies for the search modes varied from approximately $`0.1\%`$ to $`2\%`$.

Monte Carlo studies show that the experiment’s acceptances are nearly uniform across the Dalitz plots, except that the dilepton identification efficiencies typically drop to near zero at the dilepton mass threshold. While the loss in efficiency varies channel by channel, the efficiency typically reaches its full value at masses only a few hundred MeV/$`c^{\mathrm{\hspace{0.17em}2}}`$ above the dilepton mass threshold. We use a constant weak-decay matrix element when calculating the overall detection efficiencies. Two exceptions to the use of the Monte Carlo simulations in determining relative efficiencies are made: those for Čerenkov identification when the number of kaons in the signal and normalization modes are different, and those for the muon identification. These efficiencies are determined from data.

The 90$`\%`$ CL upper limits $`N_X`$ are calculated using the method of Feldman and Cousins  to account for background, and then corrected for systematic errors by the method of Cousins and Highland . In these methods, the numbers of signal events are determined by simple counting, not by a fit. All results are listed in Table 1 and shown in Figs. 2 and 3. The kinematic criteria and removal of reflections (see below) are different for the $`D^+`$, $`D_s^+`$, and $`D^0`$. Thus, the $`D^+`$ and $`D_s^+`$ rows in Fig. 2 with the same decay particles are different, and the seventh row of Fig. 2 is different from the bottom row of Fig. 1.

The upper limits are determined by both the number of candidate events and the expected number of background events within the signal region. Background sources that are not removed by the selection criteria discussed earlier include decays in which hadrons (from real, fully-hadronic decay vertices) are misidentified as leptons. In the case where kaons are misidentified as leptons, candidates have effective masses which lie outside the signal windows. Most of these originate from Cabibbo-favored modes $`D^+K^{}\pi ^+\pi ^+`$, $`D_s^+K^{}K^+\pi ^+`$, and $`D^0K^{}\pi ^+`$ (and charge conjugates). These Cabibbo-favored reflections are explicitly removed prior to the selection-criteria optimization. There remain two sources of background in our data: hadronic decays with pions misidentified as leptons ($`N_{\mathrm{MisID}}`$) and “combinatoric” background ($`N_{\mathrm{Cmb}}`$) arising primarily from false vertices and partially reconstructed charm decays. After selection criteria were applied and the signal windows opened, the number of events within the window is $`N_{\mathrm{Obs}}=N_{\mathrm{Sig}}+N_{\mathrm{MisID}}+N_{\mathrm{Cmb}}`$.

The background $`N_{\mathrm{MisID}}`$ arises mainly from singly-Cabibbo-suppressed (SCS) modes. These misidentified leptons can come from hadronic showers reaching the muon counter, decays-in-flight, and random overlaps of tracks from otherwise separate decays (“accidental” sources). We do not attempt to establish a limit for $`D^+K^{}\mathrm{}^+\mathrm{}^+`$ modes, as they have relatively large feedthrough signals from copious Cabibbo-favored $`K^{}\pi ^+\pi ^+`$ decays. Instead, we use the observed signals in $`K^{}\mathrm{}^+\mathrm{}^+`$ channels to measure three dilepton misidentification rates under the assumption that the observed signals (shown in Figs. 1d–f) arise entirely from lepton misidentification. The curve shapes were determined from Monte Carlo simulations. The following misidentification rates were obtained: $`r_{\mu \mu }=(7.3\pm 2.0)\times 10^4`$, $`r_{\mu e}=(2.9\pm 1.3)\times 10^4`$, and $`r_{ee}=(3.4\pm 1.4)\times 10^4`$. Using these rates we estimate the numbers of misidentified candidates, $`N_{\mathrm{MisID}}^h\mathrm{}\mathrm{}`$ (for $`D^+`$ and $`D_s^+`$) and $`N_{\mathrm{MisID}}^{\mathrm{}\mathrm{}}`$ (for $`D^0`$), in the signal windows as follows:

$$N_{\mathrm{MisID}}^h\mathrm{}\mathrm{}=r_{\mathrm{}\mathrm{}}N_{\mathrm{SCS}}^{h\pi \pi }\mathrm{and}N_{\mathrm{MisID}}^{\mathrm{}\mathrm{}}=r_{\mathrm{}\mathrm{}}N_{\mathrm{SCS}}^{\pi \pi },$$
(4)   
where $`N_{\mathrm{SCS}}^{h\pi \pi }`$ and $`N_{\mathrm{SCS}}^{\pi \pi }`$ are the numbers of SCS hadronic decay candidates within the signal windows. For modes in which two possible pion combinations can contribute, e.g., $`D^+h^+\mu ^\pm \mu ^{}`$, we use twice the above rate. These misidentification backgrounds were typically small or negligible.

To estimate the combinatoric background $`N_{\mathrm{Cmb}}`$ within a signal window $`\mathrm{\Delta }M_S`$, we count events having masses within an adjacent background mass window $`\mathrm{\Delta }M_B`$, and scale this number ($`N_{\mathrm{\Delta }M_B}`$) by the relative sizes of these windows:

$$N_{\mathrm{Cmb}}=\frac{\mathrm{\Delta }M_S}{\mathrm{\Delta }M_B}N_{\mathrm{\Delta }M_B}.$$
(5)   
To be conservative in calculating our 90$`\%`$ confidence level upper limits, we take combinatoric backgrounds to be zero when no events are located above the mass windows. In Table 1 we present the numbers of combinatoric background, misidentification background, and observed events for all 24 modes.

The sources of systematic errors in this analysis include: statistical errors from the fit to the normalization sample $`N_{\mathrm{Norm}}`$; statistical errors on the numbers of Monte Carlo generated events for both $`N_{\mathrm{Norm}}^{\mathrm{MC}}`$ and $`N_X^{\mathrm{MC}}`$; uncertainties in the calculation of misidentification background; and uncertainties in the relative efficiency for each mode, including lepton and kaon tagging efficiencies. These tagging efficiency uncertainties include: 1) the muon counter efficiencies from both Monte Carlo simulation and hardware performance; 2) kaon Čerenkov identification efficiency due to differences in kinematics and modeling between data and Monte Carlo simulated events; and 3) the fraction of signal events (based on simulations) that would remain outside the signal window due to bremsstrahlung tails. The larger systematic errors for the $`D_s^+`$ modes, compared to the $`D^+`$ and $`D^0`$ modes, are due to the uncertainty in the branching fraction for the $`D_s^+`$ normalization mode. The sums, taken in quadrature, of these systematic errors are listed in Table 1.

In summary, we use a “blind” analysis of data from Fermilab experiment E791 to obtain upper limits on the dilepton branching fractions for flavor-changing neutral current, lepton-number violating, and lepton-family violating decays of $`D^+`$, $`D_s^+`$, and $`D^0`$ mesons. No evidence for any of these decays is found. Therefore, we present upper limits on the branching fractions at the 90$`\%`$ confidence level. These limits represent significant improvements over previously published results. Eight new $`D_s^+`$ search modes are reported. A comparison of our 90$`\%`$ C.L. upper limits with previously published results  is shown in Fig. 3.

We gratefully acknowledge the assistance of the staffs of Fermilab and of all the participating institutions. This research was supported by the Brazilian Conselho Nacional de Desenvolvimento Científico e Tecnológico, CONACyT (Mexico), the Israeli Academy of Sciences and Humanities, the U.S. Department of Energy, the U.S.-Israel Binational Science Foundation, and the U.S. National Science Foundation. Fermilab is operated by the Universities Research Association, Inc., under contract with the U.S. Department of Energy.