{'fraction_non_alphanumeric': 0.029588267385513945, 'fraction_numerical': 0.013350020859407593, 'mean_word_length': 4.708371496611101, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Laplacian Pyramid-like Autoencoder

In this paper, we develop the Laplacian pyramid-like autoencoder (LPAE) by adding the Laplacian pyramid (LP) concept widely used to analyze images in Signal Processing. LPAE decomposes an image into the approximation image and the detail image in the encoder part and then tries to reconstruct the original image in the decoder part using the two components. We use LPAE for experiments on classifications and super-resolution areas. Using the detail image and the smaller-sized approximation image as inputs of a classification network, our LPAE makes the model lighter. Moreover, we show that the performance of the connected classification networks has remained substantially high. In a super-resolution area, we show that the decoder part gets a high-quality reconstruction image by setting to resemble the structure of LP. Consequently, LPAE improves the original results by combining the decoder part of the autoencoder and the super-resolution network.


Introduction
Deep neural networks are standard machine learning methods for diverse image processing such as object classification, image transform, image recognition. The networks have great varieties in architectures, algorithms, and processes. The autoencoder is a part of these varieties.
The autoencoder encodes a given data to some representation in a latent space, usually compressed from the input data, by a few layers. Then it decodes this representation to the reconstruction converted to have desired properties by different layers. The encoder has the advantage of analyzing the data in a low dimensional space, in a way similar to the Principal Component Analysis. Also, the simplicity of the model structure makes it easy to modify the structure according to various purposes like unsupervised pre-training, denoising, restoration of image.
Our paper is motivated by the article [6]. The authors develop the wavelet-like autoencoder (WAE) and use it for acceleration in classification networks. WAE decomposes the input image into two down-scaled images, low-frequency information I L and high-frequency information I H , through the encoder and reconstructs the original image arXiv:2208.12484v1 [cs.CV] 26 Aug 2022 at the decoder. Their use of the prefix "wavelet-like" is due to the imposed condition on I H to be sparse and the reconstruction process obtained by adding the convolution filtered versions of I L , I H . In WAE, to accelerate the classification model, they input I L as the mainstream and I H as a helper to the classification networks (e.g., VGG16, ResNet50) instead of the original image. The change made the network have smaller computational complexity; hence it takes less time for the entire process. Besides, the complementary analysis using the helper I H makes the network stay competitive in terms of accuracy.
Although WAE is good at accelerating the basic classification networks, it is not satisfactory in some crucial aspects. First, contrary to the wavelet, WAE does not impose any condition on low-frequency information. This missing condition makes the approximation I L hard to reflect the original image, which can drop classification performance. Second, such a low-frequency image can lower the quality of the reconstruction image. The WAE paper does not pay attention to the reconstructed result because its primary concern is the acceleration problem, requiring only two decomposed images in the classification networks. This limited architecture makes it hard for use in other areas requiring a reconstruction. Third, the name of the autoencoder is "wavelet-like," but the model is missing a critical feature of the wavelet, namely multi-scale property. Consequently, WAE has difficulty decomposing multiple times, resulting in a restriction on WAE so that the model is stuck in the acceleration task.
Considering the preceding, we propose a new model named the Laplacian pyramidlike autoencoder (LPAE). We impose an extra condition on the low-frequency part of WAE, and get an autoencoder with a hierarchy, similar to the shape of the Laplacian pyramid (LP) introduced in [5]. As a result, LPAE makes the approximation image with better quality. Using this approximation image, we obtain higher performance of classification but also extend to super-resolution problems with 2 k magnification for various k since LPAE decomposes and reconstructs an image multiple times.
The datasets used for the classification are ImageNet2012 (ImageNet) from [33] and Intel Image Classification (Natural Scene) from [18]. We combine two base networks, VGG16 (VGG) and ResNet50 (ResNet), with WAE and LPAE. LPAE shows better classification performance than WAE. LPAE accelerates test times sufficiently, although there is a slight drop in the acceleration time. In some cases, LPAE even has a faster total training time than WAE. For a super-resolution problem, we use three datasets, CelebA [26], DIV2K [35], and Set5 [4]. After training on CelebA, the test is done on CelebA. After training on DIV2K, the test is carried out on Set5, following the convention in the field. For the network, base network is WaveletSRNet (WaveSR) introduced in [14]. Since WaveSR uses the wavelet packet transform for image reconstruction, we can directly see the effectiveness of LPAE for super-resolution by replacing the wavelet part with LPAE. This result in super-resolution shows the potential usefulness of the proposed LPAE for solving other problems in deep learning because the classification and the super-resolution are different types of problems.

Related Works Autoencoder
Recently, most research about the autoencoder concentrates on its connection to other applications rather than developing it independently. Generative autoencoder [10,31] occupies a significant portion as a base model for application. In [17], variational autoencoder (VAE) solves a problem of insufficient data by producing synthetic data from a given sample data. [27] develops a reference-based SR approach using VAE, which can take any image as a reference. [36] suggests a novel generative model with hierarchical structure based on VAE. By exploiting unusual convolutions such as depthwise separable convolution and regular convolution, authors get high performance without modifying the loss function of VAE. Other existing autoencoders, such as sparse autoencoder and denoising autoencoder, are used to solve specific problems. For example, [15] composes stacked sparse denoising autoencoder for detecting electricity theft and [13] exploits a sparse autoencoder for landslide susceptibility prediction. Because the autoencoder extracts the feature efficiently and reconstructs data with a simple structure, it is a popular tool for developing a framework combined with machine learning. Our model, LPAE, is a model possessing the properties of LP. So LPAE can provide new approaches to diverse problems where LP properties are helpful.

Network Acceleration
After the substantial progress of the convolutional neural network (CNN), a lot of research focuses on the acceleration of the networks with keeping high performance. There are several approaches to accelerate networks. [3,6] propose methods that modify an architecture or overall structure. [3] suggests an accelerator using the point that can compute the convolution layer similar to the fully connected layer. Moreover, to accelerate model training, some researches present a new training framework. [41] makes an assistant model along with the main model to remove trivial instances during training, then gets the results with the new training algorithm. [25] suggests a new training plan for fine-tuning. It accelerates the fine-tuning by adaptively freezing the nearly converged portion of the network. Since many natural language processing and classification models exploit fine-tuning the pre-trained network, this method is tempting. Furthermore, the study to achieve harmony between the hardware and the software is actively performed [16,30]. Considering rapidly advanced hardware, both propose a hardware-based approach to speed up a CNN and reduce the energy required for computation.

Single Image Super-Resolution
In the super-resolution area, many of the studies try to build a deep convolution-based model using up-sampling. For example, [19] introduces the way of the deep network for super-resolution by a residual learning and high initial learning rate, while [22] suggests enhancing the power of the deep network by removing the batch normalization and setting a training pipeline. [12] makes a model to repeat the up-scaling and down-scaling iteratively and to reflect the feedback of error. In addition to the construction of deep models, some researchers are focusing on other points. By learning the feature correlations of hidden layers by second-order attention, [8] improves the expressional power of CNN. [34] trains the content-adaptive resampling (CAR) model with the main superresolution (SR) network to create a low-resolution image. Since unsupervised training is conducted on CAR, CAR makes a down-scaled image to keep important information for super-resolution impartially. The primary concern of [7,20] is the speed of the SR. While both of them construct a new network structure and framework, neural architecture search is a noticeable characteristic of [7]. The LPAE that we propose in this paper is an assistant model to reconstruct images trained with the main SR model, similar to the approach in [34]. Also, LPAE learns the correlation between the components for super-resolution and can improve the reconstructing power of the main SR model similar to [8]. However, LPAE is an autoencoder with a straightforward structure and is easy to connect with diverse architectures without too much modification.

Idea and Structure of LP
The Laplacian pyramid (LP) is introduced in [5] as a technique for compact image encoding. This technique has its root in subtracting a low-pass filtered image from the original image. Such subtraction reduces redundant information by decreasing the correlation between neighboring pixels in an image, in other words, data compression. Moreover, this compression process can be repeated on low-pass filtered images having different scales. Then the repetitions form a pyramid-like structure and accelerate the reduction. Since this process is similar to the Laplacian operator sampling on diverse scales, the pyramid is named the Laplacian pyramid.
The following is the overall process of establishing LP. For an input image I 0 , a filtered image I 1 is obtained with the decreased resolution by some low-pass filter. A filtered image I 2 is similarly obtained from I 1 , and proceeding successively results in a sequence {I k } K k=0 . For each fixed I k with k ≥ 1, the imageĨ k with the same size as I k−1 is defined by expanding I k by interpolation. SubtractingĨ k from I k−1 produces d k with the same size as I k−1 but with compressed image information. The sequence {d k } K k=1 of such differences corresponds to the pyramid of LP.

Strengths of LP and Applications on Neural Net
According to the above procedure, LP decomposes an image I into a low-pass filtered image I c and a difference I d after one stage. The decomposition result of LP is redundant compared to the wavelet transform often used as the analysis tool because the I d part of LP has the same size as the original image. But some advantages are originated from the structure of LP. For example, the implementation that requires only low-pass filtering and subtraction is straightforward. Another strength of LP is that there exists no scrambled frequency [9] derived from high-pass filtering. And the organizational style of LP enables the perfect reconstruction of the original image. Such advantages encourage many usages of LP as a tool in traditional image processing.
In addition, there have been lots of efforts in machine learning to treat LP, both independently and as a concept in a model. For the super-resolution problem, [2] and [43] get competitive results by introducing the style of LP in the attention module and by considering a generator of the conditional generative adversarial network, respectively. On the other hand, [21,23] use LP itself to make input images. After generating the image pyramid from the original image, [23] put components into two modules for highquality transfer, a drafting module that transforms artistic style from low-resolution image and a revision module that refines the transformed image with a high-resolution image. On top of these, there are applications for diverse problems such as object detection [42], image compression [37]. These researches show a possibility of a connection between LP and deep learning, presenting satisfactory results. As will be seen later in this paper, our LPAE presents convincing results, which add to the above possibility.

Proposed Model
LPAE has two parts, encoding (analysis) and decoding (reconstruction) parts. Fig. 1 shows the overall structure of LPAE. In the analysis part, LPAE decomposes the original image I into images I c and I d . I c is the down-sampled approximation image that is lowpass filtered, and I d is the detail image representing the difference between the original image and the prediction from approximation. Both outputs pass through 4 convolution layers. When we downsample images, we use the convolution layer with stride 2 instead of a pooling layer. In the reconstruction part, the output image I is reconstructed by an element-wise sum between the detail image I d and the prediction image φ(I c ), where φ is a deconvolution process. The process φ consists of up-sampling with 4×4 transposed convolution layer and filtering with 3 of convolution layers. In our LPAE, except for the output layers, we use convolution layers with filter-size 3 × 3 and 16 output channels for simplicity and efficiency, but the network can be made deeper or more redundant.

Loss for LPAE
Our loss function consists of three components that make the autoencoder similar to LP. Since we want LPAE to reconstruct the original image as much as possible, we define our first loss function to be the reconstruction loss, For the approximation image representing the low-frequency channel of the original image well, we prepare an approximation image I ↓ by using the bicubic interpolation and apply the mean square error (MSE). we then define our second loss function as the energy (or approximation) loss, For the approximation image I ↓ , many other methods, including the wavelet transform by CDF 9/7 filters, can also be used, but we find no significant difference in the loss for using different approximation methods. Hence in this paper, we fix the bicubic interpolation to get I ↓ . The bicubic interpolation makes a natural connection with the super-resolution problem.
To constrain the detail image to be sparse, we set our last loss function to be the sparsity loss, This sparsity loss makes the detail image carry a high-frequency channel of the original image and provide textures to the connected network. The overall loss of LPAE is defined as the weighted sum of the three losses: where α = γ = 1, β = 0.8 to give less weight on the approximation loss than other losses. Fig. 2 shows the result of LPAE and WAE applied to two different images from ImageNet. For the approximation image in Fig. 2(b), there is no doubt that the result of LPAE is more vivid, original-like image. Besides, Fig. 2(c), the result of detail images, demonstrates that this precise approximation influences the detail image to be sparse. Although sparse, the detail contains many information for the original texture since it has the same spatial size as the original image, unlike WAE. Eventually, LPAE gets a more sharp reconstruction, as seen from the images in Fig. 2(d). The differences prove the validity of our loss function.

Image Classification Problem
In [6], the authors show that WAE can accelerate classification networks while keeping the accuracy almost the same by using two outputs of the encoding part. We show that LPAE can accelerate classification networks at about the similar level, even with a slight improvement in accuracy. We think that using both the approximation and the detail, similar to [6], has contributed to getting comparable accuracy, even if the input in VGG is not the original image. In this sense, we speculate that there is an improvement in accuracy for LPAE because LPAE gets a better approximation and a high-resolution detail than WAE.
To be comparable with the experiments in [6], we set the same structure except for the autoencoder part, replacing WAE with LPAE. Below we describe the use of the autoencoder only for VGG, as the process for ResNet is similar. Recall that the encoder part of LPAE decomposes an original image I into an approximation image I c and a detail image I d . The features f c for I c are extracted from the feature extraction part of VGG. The features f d for I d are extracted from another lighter feature extraction part consisting of convolution layers with only a quarter of the output channels for f c . Using fully connected layers, we obtain the classification score s c from f c , and s d from the concatenation of f c and f d . The final score s is the average of the two scores s c and s d .
From the acceleration perspective, according to [6], the total complexity of all the convolutional layers can be represented by Here, d is the number of the convolution layers. For the l-th layer, n l is the number of the output channels, s l is the size of the kernels, and m l × m l is the spatial size of the output features. Since LPAE has 1 2 size of feature maps for the approximation, the complexity N in (1) is 1 4 of the original network. However, unlike WAE, there is no change in the size of feature maps for the detail. Still following the setup of WAE for the detail, the number of the output channels for the LPAE's detail becomes 1 4 compared to the approximation case, so the complexity becomes 1 16 of the original network. As a result, LPAE has 5 16 total complexity compared to the original. Thus the acceleration rate is about 1/ 5 16 = 3.2. This number is comparable to 3.76, the acceleration rate of WAE.

Applications to Super-Resolution Problem
As observed earlier, LPAE's approximation image in Fig. 2 tends to be similar to some other low-pass filtered images, such as the approximation image of the wavelet transform or the bicubic interpolation. This approximation image helps the detail to be more sparse. Besides, LPAE can make the hierarchical structure because the approximation carries low-frequency information sufficiently. Based on these observations, we try to expand the application domain of LPAE to super-resolution.
There are lots of algorithms in the super-resolution problem. Some of them are hard to be connected to LPAE directly in a natural way. So we try to choose the models having room for substitution, and WaveletSRNet (WaveSR) in [14] is one such model. The basic concept of the network is the use of the wavelet packet transform (WPT) as a reconstructor. For example, to train WaveSR for magnification of 4, an input image is decomposed into two levels using WPT. Then the network gets the approximation image of 1 4 size and extracts features from the approximation image through the embedding part. The features form new details of the number needed in the reconstruction process (15 for magnification of 4) and one approximation image. At last, through WPT, the network creates the high-resolution image of magnification of 4. In this procedure, the authors set the loss function of WaveSR to make the new details and approximation similar to the decomposition results of WPT.
In this algorithm, we insert LPAE as a substitute for WPT. The divided encoding and decoding parts can take the role of the existing decomposition and reconstruction, respectively. Moreover, the hierarchical structure of LPAE does not restrict its use only for a magnification of 2 but allows it for an arbitrary magnification of 2 k . Although LPAE produces larger-sized details than WPT, the redundancy of information is helpful to reconstruct. And the fact that it has a smaller number of detail images makes the network efficient. In addition, we expect the substitute to improve the high-resolution result because it has the flexibility that the reconstruction of LPAE gets better by a fine-tune on various datasets in contrast with WPT.
We modify the loss function given in [14] to fit LPAE's situation when we train WaveSR substituted with LPAE (named LPSR). The reconstruction loss is defined by using the L1 distance between the reconstruction I and the original high-resolution image I, The use of L1 loss for the whole procedure can reduce the smoothing effect and enhance the quality of the image [44]. Based on this observation, we define the pyramid loss by where {λ i } are weights for the detail part. For instance, we take λ 1 = 0.8, λ 2 = 1.2 in case of magnification of 4. Finally, the loss function of LPSR is the weighted sum of two losses: l total = γl rec + δl p .
If the new details I di and approximation I c produced by LPSR are close to the outputs {I di , I c } of LPAE sufficiently, the reconstruction using the new ones will have high quality on high-resolution stage. From this point of view, we focus on the closeness between the outputs of LPAE and LPSR. Hence in all of our experiments, weights γ = 1 and δ = 10 are used.

Details on Experiments
In classification, as mentioned in our paper, we use two backbones, VGG16, ResNet50, and use two datasets, ImageNet2012 (ImageNet), Intel Image Classification datasets (Natural Scene). ImageNet is a large dataset having train images of 1.2 million approximately and validation images of 50k. The images are classified as labels of 1000 and are with varying image sizes. In comparison to ImageNet, Natural Scene is a tiny dataset of 6 categories. There are 14k in train, 10k in validation and test. Moreover, the spatial size of each image is fixed as 150 × 150.
We mostly follow the training strategies in [6] to train the autoencoders for comparison with WAE. In particular, for both WAE and LPAE, the Xavier algorithm is used for initialization of parameters, and the SGD algorithm with momentum 0.9 and weight decay of 0.0005 is used. However, some options are chosen differently because of the dataset size. Although we keep a batch of 4 as in [6] for Natural Scene, we choose 256 for ImageNet to shorten training time. Also, our training epochs are fixed as 100 for Natural Scene and 20 for ImageNet. Considering the difference between the two autoencoders, we choose the initial learning rate differently. For WAE, it is set as 0.000001 with the decay factor of 0.1 after every 10 epochs as in [6], but for LPAE as 0.01 with the same decaying strategy.
To train the connected classification network, for Natural Scene images, we randomly crop to 128 × 128, and for ImageNet images, we resize to 256 × 256 and randomly crop to 224×224. The only data augmentation we select is the random horizontal flip. We choose the batch size to be 256 regardless of datasets, and choose the SGD algorithm with the same options. But the decay strategy and the number of epochs are different along with the dataset. For ImageNet, training is performed with 20 epochs, and then the learning rate is multiplied by 0.1 after 10 epochs. For Natural Scene, the learning rate is multiplied by 0.5 after every 10 epochs during 100 training epochs.
In the super-resolution, we train on the DIV2K dataset and test on the Set5 dataset. Additionally, we use the CelebA dataset for comparison with the original task of WaveSR. DIV2K has 800 diverse images of the large size of 2k resolution, but CelebA has 162,770 images of center-arranged (to some degree) face with the size of 178 × 218. Because of this substantial dissimilarity, we perform different data augmentations on two datasets.
The autoencoders' training options are chosen similarly to the classification. For DIV2K, we randomly crop the high-resolution image to 192 × 192 with random horizontal/vertical flips. We train LPAE using the Adam algorithm with a batch of 4 and adjust the initial learning rate of 0.001 to be divided by 2 after every 50 epochs during 400 epochs. To train LPSR, we set a batch of 8 and select the Adam algorithm with the initial learning rate of 0.001 decaying to half after every 50 epochs for 300 training epochs. For the other dataset, CelebA, the training image is resized to 144 × 144 then randomly cropped to 128 × 128 with a random horizontal flip only. LPSR is trained during 40 epochs using the Adam algorithm with a batch of 256, the initial learning rate of 0.01 multiplied by 0.1 after every 10 epochs. Options for other networks, WaveSR and WSR, are chosen to be identical to LPSR. All of our codes are available at https://github.com/sangjun7/LPAE.

Experimental Results
In this section, we examine our LPAE performances. All of the detailed options on experiments are reported in Supplementary Material. We first make a comparison between LPAE and WAE using the PSNR value in Table 1. Then we join two autoencoders to the classification networks, VGG and ResNet. These networks represent the efficiency and the power of LPAE by inference time and accuracy. The following super-resolution results show the versatility of LPAE. By checking PSNR and SSIM values, we show that replacing original reconstruction parts with LPAE enhances super-resolution abilities. All of the experiments are conducted on 6 GPUs of GeForce RTX 3090 except for measuring test time. We measure the test time of classification networks by a GPU of Tesla T4 on Google Colab.

Autoencoder
LPAE is motivated by WAE. However, there is a considerable difference between the two autoencoders in their network structure and loss formulation. As we can see in Fig. 2, LPAE gets a high-resolution detail image compared to the detail image of WAE. The large-sized detail image has more abundant information for textures and highfrequency data of the original image. Furthermore, LPAE makes a more specific approximation image than WAE, as seen in Fig. 2. This fact comes from the constraint on the approximation image to resemble the bicubic interpolated image. Also, the change of the MSE loss between the input image and the reconstructed image to the L1 loss with a large learning rate returns high-quality reconstruction [44]. Table 1 shows a difference between the two autoencoders. The PSNR values in the table are calculated between the original image and the reconstruction image. Although PSNR alone cannot determine the quality of reconstruction, many researchers indeed consider a large PSNR value as an essential indication for better reconstruction. For Bicubic, we put back to the original image using bicubic interpolation after reducing the spatial size to half. For ImageNet, we get the large PSNR value of 47.89 dB, far superior to the others, i.e., 28.57 dB for WAE and 28.44 dB for Bicubic. In the DIV2K dataset, LPAE makes the closer reconstruction image to the original than the case of ImageNet. The PSNR value of LPAE is 54.73 dB, and that of WAE is 19.90 dB, which shows a significant difference between PSNR values. LPAE gets one compressed image and another sparse image, and based on the above results, we see that LPAE restores the original image much better. Thus LPAE can accomplish the role of accelerator comparable to WAE and be extended to the super-resolution problem.  Table 1: Comparison of power for restoring the original image after encoding and decoding between WAE, LPAE and bicubic interpolation (Bicubic).

Classification
As mentioned in Section 4, we expect that the link between LPAE and a classification network reduces the computational cost and accelerates algorithms with a slight drop in accuracy. To show this, we train the basic classification networks (VGG, ResNet) and those connected with WAE and LPAE, using two datasets (ImageNet, Natural Scene). We use our codes for WAE. Although not able to reproduce exactly, we identify a tendency of accelerating presented in [6]. Table 2 shows a comparison of performances between networks. For all cases, classification networks connected with LPAE (LPVGG, LPResNet) get better precision than those connected with WAE (WVGG, WResNet). For Natural Scene, resulting in 89.97%, LPVGG even has the same accuracy as the original VGG. We speculate that more information of the original image is kept after the LPAE's encoding helps classify. Table 2 also reports the number of trainable parameters for each case for reference. The training times of LPVGG/LPResNet are reduced to a similar level as WVGG/WResNet. For ImageNet, WVGG saves about 2.63 hr than the basic, and LPVGG saves about 2.99 hr that is about 10% of the whole training time of the basic. For Natural Scene, LPVGG reduces about 31% of training time than the basic. Table 3 shows each network's FLOPs, complexity (Compl.), and test times for different batch sizes. Our complexity here is calculated similarly to N in (1) for convolution layers and fully connected layers. For instance, with VGG on ImageNet, we get an acceleration rate of 15 Table 4: Results for the super-resolution network based on WaveletSRNet.
We evaluate super-resolution networks (WaveSR, WSR, LPSR) on two datasets (CelebA, Set5) using metrics PSNR and SSIM. For the result of WaveSR, we used our codes and could not reproduce results reported in [14] despite the same options. The values in Table 4   Our PSNR values rank about the middle on average, which is good, considering the fact that LPSR is obtained by a simple change using LPAE from WaveSR. Fig. 3 shows the reconstruction of WaveSR, WSR, LPSR for two images of Set5. WaveSR works well for×2 scale, but as the scale is getting bigger, the reconstruction images of WaveSR are blurred and have a checkers pattern. However, LPSR gets high-quality reconstruction images for all scales.

Conclusion and Future Works
We organize LPAE, which assists in various problems from network acceleration to super-resolution. It reflects the structure of LP and consists of the encoder and the decoder. Using the encoder, we decompose an image into the approximation image with a low-resolution/frequency channel and the detail image with a high-frequency channel. The decoder recreates the original image correctly using the approximation and the detail. Three types of loss (approximation loss, sparsity loss, and reconstruction loss) enable us to obtain clear decomposed images and to achieve better reconstruction. Experiments in this paper show that LPAE makes the existing classification network light and preserves the original accuracy in classification. For super-resolution, it accomplishes better performance than the established wavelet-based model. In the future, we plan to explore a range of applications to different problems such as generative models, image compression, and character recognition.