{'fraction_non_alphanumeric': 0.026083170041367298, 'fraction_numerical': 0.007750925321140866, 'mean_word_length': 5.072448439978847, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Computational ghost imaging for transmission electron microscopy

While transmission electron microscopes (TEM) can achieve a much higher resolution than optical microscopes, they face challenges of damage to samples during the high energy processes involved. Here, we explore using computational ghost imaging techniques in electron microscopy to reduce the total required intensity. The technological lack of the equivalent high-resolution, optical spatial light modulator for electrons means that a different approach needs to be pursued. To this end, we show a beam shaping technique based on the use of a distribution of electrically charged metal needles to structure the beam, alongside a novel reconstruction method to handle the resulting highly non-orthogonal patterns. Second, we illustrate the application of this ghost imaging approach in electron microscopy. To test the full extent of the capabilities of this technique, we realised an analogous optical setup method. In both regimes, the ability to reduce the amount of total illumination intensity is evident in comparison to raster scanning.


Introduction
Scanning transmission electron microscopes (STEM) achieve high resolution imaging by raster scanning a focused beam of electrons over the sample and measuring the transmission to form an image. Raster scanning is a simple approach to creating an image in the absence of complicated detector arrays. An alternative approach that has been pioneered in the optical regime has been measuring the transmission of the sample when illuminated with more complicated spatial patterns. This approach, often referred to as classical computational ghost imaging, inverts knowledge of the known patterns and their measured transmission to reveal the image. Furthermore, the inversion can be constrained using known image properties (e.g. sparsity of spatial frequencies) such that the image can be estimated from fewer measurements than would have been required by a raster scan; a form of compressed sensing [1,2].
The spatial structuring of optical beams has been extended to matter waves, primarily electrons. This structuring of electron beams paved the way for the production of vortex electron beams [3][4][5], self-accelerating beams [6], non-diffracting beams [7][8][9] and orbital angular momentum analysers [10,11]. In the case of electron beams, an arbitrary way of flexibly spatially programming a beam is problematic and existing methods for the dynamic shaping of electron beams are limited. However, recent progress could inspire to build a tuneable phase plate based on a set of needle-shaped electrodes producing a continuous phase landscape. In this work, we use electron beam shaping technologies to demonstrate that the technique can be applied in electron microscopy and show it is possible to create complicated far-field patterns that when combined with a novel inversion algorithm produce images comparable to those produced by raster scanning but using much lower peak intensities. The technique is firstly developed with a light-optics analogue toy experiment and then deployed in the electron microscope real measurement.
The potential advantages of the computational ghost imaging approach are two-fold. First, that single element detectors usually have better characteristics, e.g. for light an extended range of operating wavelengths, for electrons a larger quantum efficiency. Second, the flexibility of the illuminating patterns make them potentially more suitable for imaging optimisation by compressed sensing algorithms, reducing the number of required measurements [12][13][14]. This ability to image with fewer measurements potentially increases the frame rate and/or reduces the exposure of potentially delicate samples to the illumination. Using these computational ghost imaging techniques, novel imaging systems have been developed that include imaging at terahertz wavelengths [15], time of flight imaging [16,17], and imaging sensitive to fluorescent lifetime [18].
In addition to optical wavelengths, electron beams can also be used for precision imaging. Since the time-independent wave equation for the light and electrons are perfectly analogous [19], components like lenses for electron beams are defined with a similar nomenclature as for light. However, optical elements for light are inherently easier to build and spatial light modulators (SLM) that shape the optical beams in a programmable, and near arbitrary way, do not explicitly exist for electrons. Although single-element detector-based imaging has previously been introduced for specialised electron imaging systems [20], the barrier to more general use is absence of SLM-like technologies.
In the context of electron beams, holographic plates [19,21] laid the foundation for applications related to the shaping of electron beams. However, these are passive elements and cannot be dynamically addressed like optical SLM. A most welcomed recent development has been the introduction of active optical elements or programmable phase plates [22]. These are based on simple electrostatic elements to be positioned along the beam path. Two technological solutions are currently available: the research group led by Verbeeck has been developing devices based on Einzel lenses. These lenses work like the individual pixels of an optical spatial light modulator but the number of addressable points are few in comparison the the optical devices [22]. An additional challenge for these lenses arises from the difficulty to bias and drive many connections from outside the microscope column, which necessarily limits the useful fill fraction of such a pixel-based approach. As an alternative, other groups are using separated electrodes to define specific phase landscapes across a wider area. This electrode-based approach has the benefit that the beam itself only interacts with the electric field created rather than the modulator components themselves, avoiding the main problems of the other approach. However, whereas the solution is defined everywhere the phases are limited by the harmonic condition ∇ 2 = 0 everywhere except on the electrodes [23]. In other words, the variations in possible phase modulation is far more restricted.
The most trivial electrode used thus far is a simple metallic needle [24]. The definition of the electrostatic field around a charged microtip has been studied in detail [25,26], wherein charge applied to these needles induces a phase shift around their tips to the electron beam. Interestingly, the phase shift generated by a metallic needle was used as key to implement an orbital angular momentum (OAM) sorter [27]. More importantly, multiple metallic needles have been demonstrate to be capable of producing complex tunable caustic phenomena in the far field [28]. This current work uses an arrangement of six such needles.

Computational ghost imaging in TEM
We propose a design of the experimental setup for TEM implementation of Computational Ghost Imaging. Figure 1 shows a schematic representing the key steps of the experiment.
In complete analogy with the optical realisation of ghost imaging, the technique requires a way Fig. 1. A TEM sample holder carrying our needle-based beam-shaping device is inserted in the column at the condenser system level. Dedicated electronics are used to randomly select six random biases and apply them to the needles. The electron beam crosses the potential distribution and is imaged on the sample. Then, the total transmitted intensity is acquired by a single-pixel detector, whose output value is acquired by the electronics and recorded together with the six biases. After that, the patterns used for the measurement are calculated starting from the six biases. Finally, the combination of the single-pixel measurements and the calculated patterns through the CGI algorithm yields the ghost image of the sample.
to reliably produce a form of structured illumination. As mentioned before, in optical systems this is possible thanks to SLM technology, but there is not any equivalent form of beam-shaping technology available for electrons. Thus, we developed a device that uses six electrically biased needles to generate a potential distribution, as shown in figure 2. When the device is inserted into the TEM column the potential distribution interacts with the electron beam and applies a position-dependent phase shift to it (see figure 2(c)). The beam shaping device produces complex patterns in far field observation, with a shape dependent on the six biases applied to the needles. Thus, our device allows us to control the illumination, albeit with far fewer degrees of freedom than those available with a SLM. Some examples are shown in figure 2(d). Notice how there can be a single intensity peak or more than one with variable separation, from superimposed to very far apart.
A critical limitation of the patterns produced by the beam-shaping device is that they are highly non-orthogonal. In stark contrast, SLMs can produce completely orthogonal pattern sets, like the Hadamard basis, which are inherently better for CGI. To this end, we also developed a modified CGI algorithm that adds orthogonalisation to the image reconstruction process.

Novel algorithm for ghost imaging with non-orthogonal pattern basis
The choice of mask patterns is key to the performance of the system as it defines the resolving capability of the system and the applicability of the reconstruction algorithm. The field of ghost imaging has seen an evolution in both experimental setups and algorithms [29,30]. Although the vast majority of computational ghost imaging and the related work on single pixel cameras use a spatial modulator positioned in the image plane of the object, the original work by Shapiro [31] recognised that this need not be the case. Rather, given any specific modulation, the resulting form of the light field in the plane of the object could be calculated using numerical beam propagation techniques enabling both lens free and far field implementations [32].
Another important aspect of ghost imaging are the algorithms used to recover an image from the measurements and knowledge of the patterns used. This problem can be cast into a simple matrix form: where is a matrix containing the patterns ì as rows, ì represents the (discretised) scene, and ì is the set of measurements . The methods used to retrieve ì and thereby an image fall roughly into two categories, offline or online [33]. Offline approaches infer the image from the final complete set of patterns and corresponding measurements. The majority of well known methods for solving linear systems fall in this category, e.g. the matrix (pseudo)inverse, Newton's method, etc. Conversely, online approaches update a best guess of the image with every new measurement and light pattern. The most common online algorithm involves a straightforward weighted sum of the patterns, where the weights are adjusted based on a statistical interpretation of the measurements: Here ì is the best estimate after patterns and indicates the mean detector value up to the current pattern, and ì 0 = ì 0. This algorithm, which we will refer to as traditional ghost imaging (TGI), has been extensively explored with different setups [34,35].
The primary advantage of online reconstruction algorithms lies in the low computational overhead and robustness to under-sampling. In the absence of additional constraints, to achieve a reasonable spatial resolution, the number of illumination patterns required is typically equal to the number of pixels in the image. This quickly makes the matrix from equation 1 very large, Fig. 3. Light from a HeNe at 633 nm is polarised by a polarising beam splitter (PBS), then spatially filtered and expanded to fill the aperture of an SLM using an aperture (P), a 50 mm focal length lens (L 1 ), precision pinhole (P 50 ) and 400 mm focal length lens (L 2 ). The SLM displays a superposition of the six-needles phase masks and a carrier diffraction grating. The SLM is optimised to project the majority of the modulated light into the first diffraction order, which is selected using a 250 mm focal length lens (L 3 ) and aperture (P) placed in the Fourier plane of the SLM. This light is propagated into the far-field through a combination of lenses (250 mm focal length lens (L 4 ) and 400 mm focal length lens (L 5 )). The beam is propagated into two separate arms using beamsplitter (BS). One arm uses a power meter to measure the laser power and the second arm consists of the target (T). The photomultiplier tube (Thorlabs, PMM02) measures the transmitted light after interaction with the target. Author Figure adopted from Kallepalli et al. [39] leading to high memory requirements and long computation times for offline methods. Online methods only need to store the current best guess of ì and the current pattern, can run in parallel with the measurement, and only ever operate on vectors of limited size. Moreover, TGI is by its very nature impervious to problems with under-sampling and can show approximations to the correct image with comparatively only a handful of measurements.
Both classes of reconstruction method work significantly better when used with orthogonal pattern sets, for example Hadamard matrices [36][37][38], where each mask probes a subset of spatial frequencies. For offline methods this is equivalent to ensuring full row rank, which increases the stability of the matrix inversion results. For TGI, it ensures that each pattern's spatial frequencies are counted democratically in the mean correction term. Either way, in the absence of noise, the result approaches the true image when the number of orthogonal patterns equals the number of pixels.
If the mask patterns are implemented using a pixelated spatial light modulator, specifying their design to be orthogonal is straightforward (e.g. Hadamard, or similar). If, however, the patterns are produced in other ways, e.g. using natural scattering or with the highly limited modulators available in a TEM, then orthogonality between the patterns can no longer be enforced and the standard reconstruction algorithms no longer converge to an accurate image. The TGI algorithm in particular will overemphasise the overlapping parts of the patterns. To resolve this issue, we report an alternative approach that is computationally efficient and makes optimal use of patterns regardless of their orthogonality. This alternative method, which we will call orthogonalised ghost imaging (OGI), takes inspiration from the Kaczmarz algorithm for solving linear systems [40][41][42][43][44].
OGI essentially takes the projection of each new pattern onto the current best pattern estimate as an extra correction to the mean detector value used in TGI: where = ì · ì −1 is the predicted signal under the assumption that the previous best estimate is the correct reconstruction. This has the effect of mitigating double counting stemming from non-orthogonality and emphasising actual new information present in the measurement. Note that care needs to be taken such that both the measured and predicted values are correctly scaled to each other such that the reconstructed image approaches the true image. This is equivalent to ensuring that the quantity [( − ) − ( − )] tends to zero. The approach we adopt here to ensure this scaling is to iteratively normalise the signals such that = and = .

Optical ghost imaging
The modulation device investigated in this article is inspired from the charged microtips approach, designed by Matteucci et al. [25]. The needles, to which random voltages are applied, create a spatially structured phase change in the transmitted electron beam, leading to a complicated, spatially structured, point spread function in the far-field that illuminates the sample. An example is shown in figure 2. In our experiment, the equivalent phase change to that induced by the charged needles is, in the optical regime, created using a liquid crystal spatial light modulator (LC-SLM). Our analogous optical experimental set up [39] for demonstrating the six-needles approach to computational ghost imaging is shown in Figure 3. The expanded output beam from a HeNe laser is shaped using randomly switched six-needles that generate phase masks. These phase masks are displayed on the spatial light modulator (SLM), structuring the laser beam and propagating complex intensity patterns into the far-field.
Reflective SLMs have the advantage of faster response rates and a higher fill factor than those devices based upon transmission. However, these reflective devices are susceptible to manufacturing limitations that result in deviations from flatness of the addressed surface, resulting in aberrations such as astigmatism and defocus. Correction of these distortions is key to accurately manipulating the phase of the incoming light beam. [45,46]. Our approach to aberration correction, adapted from Turtaev et al. [47], uses a diagonal blazed phase grating with a predetermined periodicity. The aberration-corrected SLM operates in a diffractive mode, and therefore, the masks are combined with a phase-grating to diffract the desired pattern to the first-order which is selected by a pinhole in the Fourier-plane. The chosen order is then propagated through the lens system and into two optical paths (Figure 3): (a) laser power meter to monitor the laser power for any variations and (b) ghost imaging system (photomultiplier tube, PMT). The PMT is positioned to collect the light after transmission by the object. Based on the known voltage applied to each of the needles and the phase change this creates, the calculation of the far-field intensity distribution and the measured signal are used to reconstruct the image, using the various algorithms described above.
We benchmark the performance of our six-needle approach against a traditional raster scan. Raster scanning is achieved by displaying the appropriate series of linear phase gratings on the SLM to produce a scanning spot in the far-field. The experiment results are presented in figure  4, which gives a visual representation of the TGI and OGI algorithms performance compared to that of a raster scan. Clearly, the non-orthogonal nature of the six-needle pattern sets has significantly degraded the reconstructed image when using the TGI algorithm. By comparison we see that the OGI algorithm allows reconstruction an image with the same resolution as a Fig. 4. Our approach illustrates the viability of using TEM-inspired phase masks for image reconstruction using a new approach to ghost imaging known as orthogonalised ghost imaging (OGI). This approach is ideal for cases such as six-needles phase mask sets, where the pattern set is not orthogonal. The comparison shows (a) randomised raster scan using traditional ghost imaging (TGI), (b) six-needle phase masks using TGI and (c) six-needle phase masks using the proposed OGI technique. In each case, the construction was done using a set of 50000 patterns and the reconstructed images are of 512 × 512 pixel resolution. The binary amplitude target used is a laser cut 8 spoke shape. traditional raster scan. As a more quantitative comparison, the spatial frequency and resolution is assessed using a radial contrast function ( Figure 5). With this number of measurements TGI has essentially converged to its final resolution, with any further improvements primarily consisting of suppressing the noise. OGI on the other hand has not yet converged, but already significantly outperforms TGI for the non-orthogonal pattern set created by the six-needle modulator.  figure 4, quantifying the resulting reconstructions using OGI-six needles, TGI-six needles and TGI-raster scans, each using 50000 patterns. The six-needles patterns are nonorthogonal by nature.
Although the overall optical energy used in the six-needle OGI and the traditional raster scan is the same we note that the peak intensity in six-needle approach is orders of magnitude lower. This reduction would be extremely important in situations where the sample can be damaged by the illumination, especially if that damage mechanism was non-linear with respect to intensity.

TEM computational ghost imaging
Having proven the applicability of computational ghost imaging with the optical analogy, we attempted an experimental demonstration with electrons. The experiments were carried out by mounting the modulation device in a biased I-V TEM sample holder. The sample to be imaged is a Siemens star target with 5 spokes, located in the Selected Area plane of the microscope. It was fabricated on a commercially available 200 thick Si 3 N 4 membrane, covered with ∼150 of gold, so that it would act as an amplitude mask. The total transmitted intensity is collected by the annular dark-field (ADF) detector, set off-axis to acquire the whole transmission. The output value of the detector is read by the control software and synchronised with the biases from the modulation device. Fig. 6. Comparison of (a) a TEM image of the target and the images reconstructed with TGI (b) and OGI (c) algorithms. The target is a 3 m 10-spoke Siemens star, acquired on a Gatan MSC794 CCD camera.
The acquisition comprised 11500 different illumination patterns. The patterns produced by the beam shaping device were not directly recorded, but instead they were recovered using a simulation algorithm after the measurement. The simulation algorithm predicts the illumination patterns used during the experiment starting from the six biases applied to the needles of the beam-shaping device. Figure 6(a) displays the target and the ghost images obtained with both the traditional (b) and orthogonalised (c) algorithms, starting from the same data. TGI is clearly outperformed by our OGI formulation (Eq. 3), as pointed out by the differences between the two reconstructions. The OGI image exhibits higher resolution and better contrast, resulting in an overall sharper image.

Conclusions
We have shown a successful proof-of-principle ghost imaging scheme tested in both optical and electron microscopy regimes. In the optical regime, we combine non-orthogonal intensity distribution patterns with a novel ghost imaging algorithm for image reconstruction. Due to the non-orthogonal nature of the far-field intensity pattern produced by the six-needle mask, the reconstruction requires a modified algorithm if image resolution is to be maintained. The results obtained using six-needle mask in the far-field show a comparable reconstruction resolution between the proposed method and a traditional raster scanning with a focused spot. This approach was transitioned to transmission electron microscopy, wherein the modulation method approach was used to reconstruct the object, with preliminary results showing promise for image reconstruction with reduced overall illumination intensity.