{'fraction_non_alphanumeric': 0.025802997858672377, 'fraction_numerical': 0.011705924339757316, 'mean_word_length': 5.207576428887904, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 3, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Unsupervised learning of ferroic variants from atomically resolved STEM images

An approach for the analysis of atomically resolved scanning transmission electron microscopy data with multiple ferroic variants in the presence of imaging non-idealities and chemical variabilities based on a rotationally invariant variational autoencoder (rVAE) is presented. We show that an optimal local descriptor for the analysis is a sub-image centered at specific atomic units, since materials and microscope distortions preclude the use of an ideal lattice as a reference point. The applicability of unsupervised clustering and dimensionality reduction methods is explored and are shown to produce clusters dominated by chemical and microscope effects, with a large number of classes required to establish the presence of rotational variants. Comparatively, the rVAE allows extraction of the angle corresponding to the orientation of ferroic variants explicitly, enabling straightforward identification of the ferroic variants as regions with constant or smoothly changing latent variables and sharp orientational changes. This approach allows further exploration of the chemical variability by separating the rotational degrees of freedom via rVAE and searching for remaining variability in the system. The code used in the manuscript is available at https://github.com/saimani5/ferroelectric_domains_rVAE.


Introduction
Ferroic phenomena in complex materials have been a primary focus of condensed matter physics and materials science since the dawn of these disciplines. Ferroelectric materials have found a broad spectrum of applications from sensors and actuators to non-volatile memories. [1][2][3][4] Ferromagnets underpin multiple energy generation and information technologies 4 while ferroelastic materials such as shape memory alloys are broadly used as structural materials. 5,6 In ferroelectrics and ferroelastics, emergence of the corresponding order parameters is intimately tied to lattice deformation, resulting in a broad spectrum of mechanical coupling effects and enabling a new generation of strain coupled multiferroic and sensing devices. 7 Over the last two decades, the attention of the scientific community has shifted to the exploration of ferroic behavior on the nanometer and atomic scales. These advances were enabled by the synergy of improved fabrication techniques such as pulsed laser deposition (PLD) and atomic layer deposition (ALD), 8 and advances in high-resolution imaging and characterization. Techniques such as piezoresponse force microscopy and spectroscopy have enabled visualizing ferroelectric domain structures and probing the polarization dynamics on the nanometer level. [9][10][11][12] Scanning transmission electron microscopy (STEM) has allowed visualization of atomic structures of surfaces and interfaces with a high information limit, allowing determination of atomic column positions with picometer level precision. 13 Owing to the strong coupling between ferroic order parameters and mechanical and structural phenomena in ferroelectrics and ferroelastics, the direct visualization of corresponding order parameter fields and their evolution at relevant microstructural elements has been enabled. [14][15][16] While the first visualization of ferroelectric domains from atomic-scale transmission electron microscopy (TEM) imaging can be traced to early 2000, 17 this direction gained prominence with further resolution gains from the advent of spherical aberration correctors and the demonstration by Jia et al. of the direct mapping of polarization fields at interfaces and domain walls by TEM. 15,18,19 Almost immediately, a similar approach was introduced using STEM by Chisholm, 16 Borisevich,[20][21][22] Nelson and Pan, [23][24][25][26] and others, leading to rapid growth in this field over the last decade. The behavior of polarization, octahedral tilts in the image plane and in the beam direction, and chemical and physical strain fields can now be visualized. [27][28][29] Correspondingly, multiple studies of the polarization behavior at surfaces, interfaces, domain walls, and more complex topological defects such as closure domains and ferroelectric vortices, have been reported. 26 In several cases, the numerical values of polarization fields have been fitted to mesoscopic Ginzburg-Landau type models to extract the gradient and flexoelectric terms in order parameter expansions. 30,31 However, these analyses have been limited to nearly ideal materials systems imaged under optimal conditions, e.g., single interfaces or domain walls. While multiple observations of polarization fields in the vicinity of complex domain wall structures, impurities, and dislocations have been reported and are routinely available, analyses have been limited due to the large number of possible structural variants affected both by the polarization field distribution, chemical composition, and STEM imaging conditions including sample tilt and beam shape. This problem is common in STEM image analysis and has stimulated the development of machine learning (ML) methods for structural analysis (local crystallography) based on the analysis of atomic neighborhoods, [32][33][34] sliding window transforms, 35,36 and the analysis of centered sub-images. 37,38 These methods differ in the choice of the underpinning physical models used. For instance, sliding window transforms implicitly rely on continuous translational symmetry and use fast fourier transform (FFT) amplitude (rejecting the FFT phase) to extract local periodicities. Local crystallographic analysis necessitates unambiguous atom finding and position refinement, reducing the (S)TEM image to a set of atomic positions. Alternatively, the analysis can be based on sub-images centered on selected atomic features, thereby enabling physics-based descriptors as explored for ferroelectrics in Ref. [ 37 ] and for the chemically heterogeneous Si-graphene system in Ref. [ 38 ] However, the characteristic aspect of ferroic materials is the presence of multiple variants of the same deformed unit cell related though discrete rotational symmetries, which can potentially be affected by the imaging conditions used. This severely limits the applicability of image analysis methods based on convolutional neural networks (CNNs), traditional clustering or dimensionality reduction methods, since each rotational variant will be associated with an individual class in the clustering methods. In the presence of other symmetry lowering distortions or composition variations, these classes are impossible to separate since the presence of additional sources of variability results in interclass mixing and leads to physically undefined structural variants. Here, we demonstrate the applicability of rotationally invariant variational autoencoders (rVAE) for the analysis of ferroic materials in the presence of structural and chemical variants and under nonideal imaging conditions. We show that the ferroic variant corresponding to different orientations can be readily identified and that rVAEs can disentangle the chemical and physical variability. We further show that special care should be taken to identify the presence of instrumental biases, such as those induced by sample tilt variations, necessitating careful tracing of these effects during image acquisition and analysis.

Results and Discussions
As a model system we demonstrate rVAE applied to a high-angle annualr dark field (HAADF)-STEM image dataset for a rhombohedral ferroelectric BiFeO3 (BFO) thin film ( Figure  1a) grown via PLD. The BFO film adopts a polydomain configuration of the ferroelectric order parameter corresponding to nominal rotationally symmetric twins of the rhombohedral structure. In BFO this order parameter manifests in local cation sublattice as non-centrosymmetry between the 4 corner A-site (Bi) cations and the central B-site (Fe) cation of the perovskite unit cell. This non-centrosymmetry vector at each cation is the offset between the cation position and the average of the four nearest neighbors. The cation positions are plotted as scatter points in real space and are colored with the vector properties of non-centrosymmetry vector using an amplitude-phase color scheme in Fig.1b. The magnitudes of the non-centrosymmetry vectors are normalized to fall in [0,1] and then a color is picked for each vector corresponding to its position in polar space. This color variation in the polar space is shown in the inset of Fig.1b, where the radius of the circle is 1. In addition, the film interface with the SrTiO3 substrate is host to a <101> edge dislocation, which dissociates into separate <100> and <001> components as shown in Figure 1c, enlarged from the highlighted region in Figure 1a.
To explore the order parameter behavior in the BFO system, we extend a previously reported approach based on ML analysis of the structural building blocks. 37,38 In this method, atoms in the selected sub-lattice are chosen as local reference points and the sub-images centered on these atoms are chosen as descriptors. The sub-images are centered at the experimentally determined atomic coordinates rather than at atoms in an ideally periodic lattice. In this manner, the intrinsic strains and distortions present in the material are relevant only on the length scale of the individual sub-images. Comparatively, using the ideal lattice sites as reference points leads to systematic deviation of the atomic positions from the center of sub-images since in this case the relevant length scale is that of the full image.
Previously, we demonstrated that simple linear unmixing techniques such as principal component analysis (PCA) and non-negative matrix factorization (NMF) applied to thus derived sub-images can be used to explore ferroic distortions in the system, with the PCA eigenvectors providing the dominant distortion modes and the loading maps visualized at respective lattice sites yielding the domain structures and visualizing anomalies at domain walls and interfaces. 37 This approach, however, was shown to create an independent PCA component for each domain variant. However, ferroelectric domains correspond physically to identical distortion patterns differing only in the orientation in the image plane. Hence, while linear unmixing methods allow for separating ferroic variants, the smaller variability corresponding to chemical changes in the system or observational conditions in STEM will be overlooked.
Autoencoders (AE) are a class of neural networks that project the original dataset into a low dimensional latent space through a set of dense or convolutional layers. These latent representations of the dataset are subsequently deconvolved into the original dataset. The reconstruction error i.e., difference between the original dataset and the reconstructed dataset is used to train the AE. Variational autoencoder uses a similar approach but learns a continuous latent space representation of the dataset as opposed to the discrete encoding of the dataset by AEs. In their true sense, the VAEs model the generating distribution of the dataset with the latent layer representing a Bayesian prior distribution. 39 They use a stochastic encoder to approximate the true posterior of the generative model which is the decoder. 40 They achieve this by minimizing a Kullback-Leibler (KL) divergence between the true posterior and the approximated distributions in addition to minimizing a reconstruction loss between the original and the encoded-decoded datasets. rVAEs are an adaptation of VAEs to the dataset with rotational symmetry i.e., same datapoint can be present in the dataset at different orientations. rVAEs explicitly enforce one of the latent variables to represent the angle, two optional latent variables allow for the small offsets in the x-and y-directions, while the remaining latent variables encode data similar to traditional VAEs. In this manner the rVAE latent space has 1+N (pure rotation and no translations) or 3+N (rotation plus offsets) elements encoding the orientation of the building blocks, optional offsets allowing for sub-pixel displacements of the true atomic positions from the maximal pixel, and latent variables encoding the characteristic distortions. We have previously applied rVAEs to analyze the dynamics of protein nanorod's self-assembly, 41 explore the dynamic evolution of electron bean induced processes in atomic scale structures, 42 investigate the domain switching pathways in ferroelectric materials visualized by piezoresponse force microscopy, 43 create a bottom-up symmetry analysis Bayesian workflow for atomically-resolved data, 44 and study the dynamic domain evolutions in bias induced phase transitions of BaTiO3. 45 To analyze the subimages of HAADF-STEM image in Fig.1, an rVAE with symmetric encoder-decoder architecture is used. The encoder has 2 convolutional layers, each comprise 128 kernels/filters of size 3*3 with a leaky-ReLU (slope = 0.1) activation function. At the end of these two convolutional layers, a dense layer is used to reduce the dimensionality of the data to the twice the number of latent dimensions. In a symmetric autoencoder, the decoder's architecture is the mirror image of the encoder. The encoder-decoder pair is trained together to minimize a loss function that is the sum of reconstruction error and KL divergence. As can be observed from the variation of non-centrosymmetry vector across the image, the expected factors of variation in the unit cells present in the image include the presence of ferroelectric domains of dissimilar orientation, presence of non-polar phase in the substrate, emergence behavior at interfaces and domain walls, and instrumental factors. The rVAE analysis has been performed for several sets of possible degrees of freedom viz., i) no translation; two latent variables, ii) translations; one latent variable and iii) translations; 2 latent variables. The full set of analysis is provided in the jupyter notebook attached in github repository. However, since the variability in the input dataset is impossible to independently capture using only a few latent dimensions, in this manuscript, we have only discussed the rVAE with 5 latent dimensions viz., an encoded angle, translations/offsets in X (Ox) and Y (OY), two latent variables L1 and L2.
A stack of sub-images centered on the B site cations was formed as the input the rVAE. For generality, we used the data from the image shown in Figure 1a with a sub-image size of 48 pixels corresponding to 7.5Å or ~1.8-unit cells. Each of these subimages is then represented as a 5D vector in the latent space by rVAE. The resulting distribution of the encoded angle is plotted as a histogram Figure 1d depicting three sharp peaks separated by /2, indicative of the successful separation of the ferroic variants based on the in-plane rotations which are expected to be /2. The latent space of L1-L2 is uniformly sampled in the range of [-2, 2] while the other latent variables are held at constant. This set of uniformly sampled points in the latent space are then decoded back into the space of subimages and are shown in figure. 1e providing insight into traits within the data encoded in the latent variables (L1 and L2). Overall, the uniformity of the latent space suggests that the network discovered the universal representation of the sub-images, i.e., the individual building blocks. Insight into the microstructure and evaluation of the latent space segregation can be gained from simple unsupervised classification algorithms, e.g., k-means clustering and Gaussian mixture modeling (GMM). K-means clustering separates the data points based on the selected distance metrics into maximally localized clusters where the distance is the Euclidean distance in the full vector space corresponding to the sub-image size, i.e., the full 2304-dimensional space. GMM represents the probabilistic model for sub-populations (clusters) within an overall population (entire sub-image stack). In GMM, clusters are characterized as mean vectors and covariance matrices in the sub-image space. Unlike distance-based clustering methods, GMM can separate distributions characterized by similar means but with different dispersions. Note that GMM, kmeans, and other clustering techniques allow for a significant number of options in terms of distance metrics and form of the covariate matrices. The attached Jupyter notebook allows the reader to experiment with these techniques and apply them to their own data. The 5D latent encodings at each cation site act as the feature vectors (input) to the GMM algorithm. The result of a GGM k=5 cluster analysis is shown in the latent (L1 vs encoded angle) in Figure 2h and real space in (Figure 2i). The scatter points are then colored using the class labels of the clusters formed by GMM. Note, the three ferroelectric domain orientations are assigned to three clusters (#1,#2,#5), one for the poor-fit regions (#4), and one for the domain walls (#3). The class associated with domain walls also encompasses points from the right side of the image that are not part of the domain wall.
To explore the applicability of the rVAE approach for more complex materials systems, we applied the same approach to a BiFeO3−La0.7Sr0.3MnO3 (BFO-LSMO) superlattice epitaxially grown on a SrTiO3 substrate by pulsed laser deposition (PLD). An annular dark field (ADF) STEM image of the BFO-LSMO superlattice acquired along the [100] zone axis is shown in Figure 3a. Here, the bright/dark contrast corresponds to the different BFO/LSMO layers, respectively. Coherent interfaces, i.e., without dislocations, were characteristic in this system; however, the contrast variation is relatively diffuse especially at the BFO-into-LSMO interface, indicative of chemical intermixing at this growth interface. Closer examination of this image (and similar ones) reveals the presence of a defective lamellar structure in some areas within the BFO layers. The lamellae have a periodicity of 2-unit cells and lie 45° with respect to the BFO-LSMO interfaces along (110) direction. Previous image analyses concluded that a highly strained BFO orthorhombic phase coexists with the more common pseudorhombohedral phase. 46 This orthorhombic phase presents as the lamella structure with anti-parallel polarization, resulting in an antiferrolectric-like structure.
To illustrate the initial crystallographic analysis of the heterostructures that form within the BFO layers, the A and B cation sites are identified using semantic segmentation via the deep convolutional neural network (DCNN) with the U-Net architecture. The details of this DCNN architecture and its workings are discussed extensively in Ref. 47 The resulting lattice is illustrated in Figure 3b. Note that identification of the atomic sites relies purely on local contrast and no assumptions regarding long-range translational symmetry are made.
As discussed for the previous case, subimages i.e., images cropped around the centers of cations are used as the feature vectors for the cations. Initially, we explored classification of these experimental sub-images into different numbers of classes using two clustering algorithms viz., Kmeans and GMM. The results of these clustering techniques are shown in figure. 3c for k-means and figure. 3d for the GMM. In all cases, at least eight classes are necessary to visualize domain variants as separate discrete classes, whereas only a few classes separate the variability of chemical composition but not ferroic variants. This is unsurprising since sub-images encode both chemical and polarization effects and the former tend to dominate STEM contrast. This behavior is further illustrated in Figure 4 where the sub-images are visualized in the latent spaces corresponding to different dimensionality reduction methods. Here, we apply classical principal component analysis (PCA), t-distributed stochastic neighbor embedding (tSNE), multidimensional scaling (MDS), and isometric mapping (ISO). This allows us to explore the variability of the first two components, proximity in the data space, distant scaling, and isometric projections for optimal representation of the data manifold in 2D space. These transforms encode the sub-images as component vectors. Unlike clustering methods, this component vector corresponds to the continuous variables. The maps in all cases visualize the distributions of the components in 2D space corresponding to the first and second endmember. Additionally, to build the relationship between the materials structure and dimensionally reduced variables, we color code the GMM classes in the images.
Note that in all cases the distribution of points in the corresponding low dimensional spaces, while structured, are not easily separable. The PCA decomposition forms a dense cloud of the points, and no obvious clusters emerge, the GMM labels suggest considerable intermixing in the PCA space. The density based tSNE decomposition separates clearly visible subclusters, as expected for this method. However, no clear clusters corresponding to the individual components or ferroic variants in the BFO-LSMO system are observed. Similar behaviors are observed for MDS, and ISO based dimensionality reduction. Overall, these methods allow to visualize the clusters within the data but will treat the rotational variants of the same structure as a dissimilar group. Finally, an rVAE with similar architecture discussed for the previous dataset is applied on this dataset of subimage stack with two different window sizes. The results of the rVAE analysis for two different window sizes are compared in Figure 5. Latent space representation where the space of L1-L2 is uniformly sampled while all other latent variables are held at constants and these sample points are then decoded back to the subimage space is shown in Figure. 5a for window size = 54 pixels and in Figure. 5b for window size = 36 pixels. Similar to the BFO case, the dominant variability in the latent space for window size = 36 pixels is due to the atomic column intensity. This is clearly observed in the joint distribution of data in the L1-L2 space ( Figure. 5f) where the L2 varies from -1.5 to 1.5, and L1 varies from -0.08 to 0.05, i.e., 30 times smaller. Furthermore, the latent variable distribution seems to suggest a relationship between the two. The latent angle distribution shows two clear peaks separated by /2 as an indication of two ferroelectric domains.
In comparison, for a larger sub-image size the latent space becomes less degenerate, with the range of L2 variability being less than an order of magnitude smaller than that of L1 ( Figure. 5c). The structure of the distribution becomes more complex, reflecting more detail in the subimages. However, examination of the latent space representation suggests that the sub-images contain unphysical features. The decoded images are shown in Figure 6. Note that for n = 54 and n = 36 window sizes, the rotation of polarization in the ferroelectric layer, i.e., the antipolar orthorhombic phase, is clearly observed. The L2 images clearly visualize the chemical homogeneity for both window sizes, whereas the corresponding L1 image visualizes similar weak contrast. For the larger window size, n = 54, the L1 image now contains information from the ferroelectric structures. In this manner, we have discovered the factors of variability in local atomic structures that visualize ferroic variants and composition variations. Note that the analyses in Figures 5 and 6 represent the structural descriptors of the materials in terms of continuous low dimensional variables. While similar to other ML methods, these do not necessarily have well-defined physical meaning; the use of rVAEs allow separation of the rotational variants and remaining degrees of freedom. In the present case, the latter are dominated by chemical composition. This approach further allows exploration of purely chemical behavior of the system. For example, Figure 7 shows GMM clustering on the latent variables. Here, GMM is applied in the latent space of the system where each subimage is represented by its encoding in the 2D latent space with L1, L2 as the latent dimensions. This representation not only helped in the reduction of dimensions from 1296 to 2 but also helps in separating the rotational components of the ferroelectric variants. The resultant cluster centers or means for k = 4 are shown in Figure. 7a. Each subimage is then plotted as a scatter point in L1-L2 latent space and is color coded with its corresponding class label in Figure. 7b. It can be observed from this plot that we can observe four distinct clusters in the latent space, and which is why the number of classes is set to 4 in the first place. Finally, the subimage centers in real space are color coded with their respective class labels in Figure. 7c. Here, the clustering clearly separates the LSMO and BFO regions as well as the intermediate layers between the two, whereas the rotational variants of BFO are identified as a single class.
To summarize, here we explore the applicability of rVAEs for the exploratory analysis of atomically resolved scanning transmission electron microscopy data with multiple ferroic variants in the presence of imaging non-idealities and chemical variability. We show that the optimal local descriptor for the analysis is the sub-image stack centered at specific atomic units, since materials and microscope distortions preclude the use of an ideal lattice being used as the reference point. Classical unsupervised clustering methods such as k-means and GMM tend to produce clusters dominated by chemical and microscope effects and a large number of classes is required to establish the presence of rotational variants. Similarly, dimensionality reduction methods projecting the data on low dimensional spaces via PCA or density-based methods tend to produce non-easily separable distributions.
Comparatively, the rVAE allows extraction of the angle corresponding to the orientation of ferroic variants explicitly, allowing for straightforward identification of ferroic variants as regions with the constant or smoothly changing latent variables and sharp orientational changes. This approach allows for further exploration of chemical variability by separating the rotational degrees of freedom via rVAE and exploring the remaining variability in the system. This approach can be of interest for systems with more complex polarization structures, including morphotropic ferroelectrics, materials at the morphotropic phase boundaries, and can provide insight into the nature of symmetry breaking phenomena in these materials.

Materials and methods:
Sample growth: The oxide layers were grown by pulsed laser deposition (PLD) using a 48 nm wavelength KrF excimer laser. La0.7Sr0.3MnO3 layers were deposited at 600 o C using 0.8 mJ/cm 2 with an oxygen pressure of 0.15 mbar and frequency of 5 Hz. BiFeO3 layers were deposited at 600 o C using 0.97 mJ/cm 2 with an oxygen pressure of 0.15 mbar and 10 Hz pulse frequency Atomic-resolution STEM images of the monolithic BiFeO3 film were collected using a probe corrected NION ultraSTEM at 200kV with a high-angle annular dark field (HAADF) detector. The BFO sample preparation, imaging, scan correction, fitting, and image analysis were performed identically to Refs. [ 48,49 ].
Atomic-resolution STEM images of the BFO-LSMO superlattices were acquired using a double aberration-corrected Schottky emission JEOL ARM-200F operating at 200 kV. Annular dark field (ADF) images were formed using a collection angle of 45−180 mrad. To reduce scan distortion and sample drift effect, each image is the result of 20 short exposure images combined using the SmartAlign algorithm [10.1186/s40679-015-0008-4]. TEM specimens were prepared using a FEI Scios focused ion beam (FIB) using standard lift-out procedures.