{'fraction_non_alphanumeric': 0.031575637800934246, 'fraction_numerical': 0.013070427596119295, 'mean_word_length': 4.816353187042842, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Associated Learning: Decomposing End-to-end Backpropagation based on Auto-encoders and Target Propagation

Backpropagation has been widely used in deep learning approaches, but it is inefficient and sometimes unstable because of backward locking and vanishing/exploding gradient problems, especially when the gradient flow is long. Additionally, updating all edge weights based on a single objective seems biologically implausible. In this paper, we introduce a novel biologically motivated learning structure called Associated Learning, which modularizes the network into smaller components, each of which has a local objective. Because the objectives are mutually independent, Associated Learning can learn the parameters independently and simultaneously when these parameters belong to different components. Surprisingly, training deep models by Associated Learning yields comparable accuracies to models trained using typical backpropagation methods, which aims at fitting the target variable directly. Moreover, probably because the gradient flow of each component is short, deep networks can still be trained with Associated Learning even when some of the activation functions are sigmoid-a situation that usually results in the vanishing gradient problem when using typical backpropagation. We also found that the Associated Learning generates better metafeatures, which we demonstrated both quantitatively (via inter-class and intra-class distance comparisons in the hidden layers) and qualitatively (by visualizing the hidden layers using t-SNE).


Introduction
Deep neural networks are usually trained using backpropagation [1], which, although common, increases the training difficulty of deep neural networks for several reasons. First, the end-to-end training method propagates error signals layer by layer; consequently, it prevents us from updating network parameters in parallel. This problem is called backward locking and is discussed in [2]. Backward locking limits the speed of neural network training, especially when the network has many layers. Second, the weights in a deep neural network may be updated slowly because of the vanishing gradient problem [3]. Several methods have been proposed to address this issue. One approach is to replace the traditional activation functions (e.g., sigmoid function or tanh function) with functions that are less likely to become saturated, such as ReLU [4]. Another approach is to modify the network structure so that the gradients are more likely to extend across many layers. The most representative works along these lines include LSTM [5], which adds additional gates in the cells, ResNet [6], which creates explicit shortcuts to help with gradient flows, and Batch Normalization [7], which adjusts and scales activations to mitigate the internal covariate shift. Third, the weights of a deep neural network may become unstable due to the exploding gradient problem. This problem is usually addressed by switching to an LSTM or by gradient clipping [8].
Beyond these computational weaknesses, backpropagation-based learning seems biologically implausible. For example, it is unlikely that all the weights should be adjusted sequentially and in Figure 1: The structure of Associated Learning, where x 0 indicates input data, y 0 indicates labels, s i is a subset of the network, and y i is the corresponding target. The solid black arrows indicate data paths and the solid red arrows indicate gradient flows. The parameters within the region of each blue dashed line can be updated independently.
small increments based on a single objective [9]. Additionally, some components that are essential for backpropagation to work correctly have not been observed in the cortex [10]. Therefore, many works have proposed methods that more closely resemble the operations of biological neurons [11,12,13,14,15,16].
In this paper, we propose Associated Learning, a method that can either be integrated with or used to replace end-to-end backpropagation when training a deep neural network. The Associated Learning scheme consists of two parts: (1) a sequence of non-linear layers (as shown in the left side of Figure 1); and (2) a sequence of autoencoders (as shown in the right side of Figure 1). The number of non-linear layers in the left side equals the number of autoencoders in the right side. We want s i , the output of every non-linear layer in the left, to be close to y i , the bottleneck of the corresponding autoencoder. Thus, we decompose the objective function into many small objectives. Consequently, each gradient flow becomes short, and gradients do not flow to the previous layer. As a result, the parameters in each blue dashed line form a group and do not affect the parameters in other groups. Therefore, this learning method decomposes the training process into small subnetworks that are independent to other subnetworks and therefore effectively alleviates the backward lock issue. Now the training process can be parallel to improve the training throughput.
The remainder of this paper is organized as follows. In Section 2, we introduce the Associated Learning . In Section 3, we show the comparisons between Associated Learning and backpropagationbased learning using different types of neural networks and different datasets. We review the related works in Section 4. Finally, we discuss the discoveries and suggest future work in Section 5. Figure 2: The red rounded rectangle is "bridge", which includes multiple additional nonlinear layers to improve the fit of the S i s. The black arrows indicate the data flow, and the red arrows indicate the gradient flow.
independent of the objective functions of the other subnetworks. A local forward function f i can be a simple single-layer perceptron, a multilayer perceptron, a convolution layer, or other functions. We compute s i and y i using Equations 1 and 2, respectively.
(1) Note that here, s 0 equals x 0 : where y i and g i are the generated local target variable and the nonlinear mapping, respectively, of the i-th layer. Biases are included in Q i . Also note that we plotted Figure 1 based on the TensorBoard convention, where the black arrows indicate the data path, but the weights to be learned (e.g., Q i s and weights in f i s) are not shown explicitly. For example, the arrow connecting x 0 and s 1 denotes that the value of x 0 is passed to the neighboring subnetwork, which needs to compute f 1 (x 0 ) to obtain s 1 . This approach differs from some neural network figures where the arrows are associated with the weights to be learned.
We define the forward loss function for each pair of (s i , y i ) (i = 1, . . . , N ) by Equation 3. This concept is the same as in target propagation [17,18,19,15]: our goal is to minimize the distance between s i and y i for every local subnetwork i.
Because the optimizer in the i-th subnetwork updates the parameters in f i only to reduce the local loss function (Equation 3), we create N independent objectives that can be trained independently.

Inverse Loss of Associated Learning
In addition to updating all the parameters in the f i s to minimize the forward loss, L i (s i , y i ), an inverse mapping must be trained at every subnetwork-i so that at the prediction phase, we can perform a transformation from y i to y i−1 (i = 1, . . . , N ). This transformation process is shown in Equation 4: where the biases are included in V i .
We call the function h i the "inverse mapping function" because it transforms y i to y i−1 . To obtain the inverse mapping function h i , we use the mapping function g i in Equation 2, which is called the forward mapping function, such that h i (g i (y i−1 )) ≈ y i−1 . Thus, we can view g i (.) as the encoder and h i (.) as the decoder for the subnetwork i.
The inverse loss L i for the layer i is defined by Equation 5.
where MSE(.) returns the mean square error of the two arguments.
For a well-trained network, s N and y N should eventually be very similar. We then input s N into h N (.) to infer our prediction.

Bridge of Associated Learning
In early experiments, we observed that s i s are difficult to fit to their corresponding targets y i s, especially for convolutional neural networks and their extensions. Thus, we insert nonlinear layers to improve the fit between the s i s and y i s. As shown in Figure 2, each orange rounded rectangle represents the s i in Figure 1, and the solid circles represent the nonlinear layers. We call the set of all these nonlinear layers in one subnetwork a bridge, which is denoted by the red rounded rectangle. As a result, the forward loss is reformulated to the following equation: Although this approach increases many parameters and the nonlinear layers to decrease the forward loss; in the inferencing phase, except for the last bridge, these parameters do not count because they do not affect the predictions, as we will explain in the next section.

Effective Parameters and the Hypothesis Space
We can categorize the above-mentioned parameters into two parts: the effective parameters and the affiliated parameters. The affiliated parameters help the model determine the values of the effective parameters, which in turn determine the hypothesis space of the final prediction function. Therefore, while increasing the number of affiliated parameters may help in obtaining better values for the effective parameters, this will not increase the hypothesis space of the prediction model.
Specifically, in the training phase, we search for the parameters in f i s and b i s that minimize the forward loss, g i s and h i s to minimize the inverse loss. However, in the prediction phase, we make predictions based only on Equation 1, Equation 4, and b N (s N ). Therefore, the effective parameters include only the parameters in f i s, b N (i.e., the last bridge), and h i s. The parameters in the other functions (i.e., all the g i s and b j s (j = 1, . . . , N − 1)) are affiliated parameters; they do not increase the expressiveness of the model but only help determine the values of the effective parameters. Equation 7 shows the prediction function: where • denotes the function composition operation.

Experiments
In this section, we show the results of performance comparisons between backpropagation (BP) and Associated Learning (AL) from different aspects. First, we quantify the performance of AL and BP based on their test accuracies. Surprisingly, although the AL aims at minimizing the local losses, its prediction accuracy is comparable to or better than that of backpropagation-based learning, whose goal is to directly minimize the prediction error. Second, we visualize the activations of hidden layers using t-SNE [20] to measure the qualities of the metafeatures learned by AL and BP. Finally, we list the intra-class and inter-class distances of BP and AL to assess the metadata quality. We conducted these experiments by applying AL and BP to different deep neural network structures (including MLP, CNN, VGG [21], ResNet-20, and ResNet-32) and testing them on different datasets (including MNIST [22], CIFAR-10, and CIFAR-100 [23]).  In each experiment, the networks are trained using 200 epochs with a batch size of 32. We initialize all the weights based on the He normal initializer, and use Adam as the optimizer. We experimented with different activation functions and adopted ELU for all the local forward functions (i.e., f i ) and sigmoid for the functions related to the autoencoders and bridges (i.e., g i , h i and b i ). Likewise, for backpropagation, we experimented with different activation functions and decided to adopt ReLU for all the layers. In addition, because Associated Learning includes extra parameters in the function b N (the last bridge), as explained in Section 2.4, we increased the number of layers in the corresponding baseline models when training by backpropagation so that the models trained by Associated Learning and trained by backpropagation have identical parameters.
On the MNIST dataset, we conducted experiments with only two network structures, MLP and CNN, beacause using even these simple structures yielded decent test accuracy (from 98.6% to 99.5%).
The MLP contains 6 layers (5 hidden layers and 1 output layer), and the number of neurons in these layers are 1024, 1024, 5120, 1024, 1024, and 10, respectively. The CNN contains 14 layers (13 hidden layers and 1 output layer). The first 4 are convolutional layers with a size of 3 × 3 × 32 (i.e., width: 3; height: 3; and 32 kernels) in each layer, followed by 4 convolutional layers with a size of 3 × 3 × 64 in each layer, followed by a fully connected layer with 1280 neurons, followed by 4 fully connected layers with 256 neurons in each layer, and ending with a fully connected layer with 10 neurons. The initial learning rate is 10 −4 , which is reduced after 80, 120, 160, and 180 epochs. We neither augment the input images nor perform any regularization in this experiment.
The results are shown in Table 1. For both the MLP and the CNN structure, Associated Learning performs slightly better than does backpropagation. Note that half the layers (h 1 , . . . , h N , b N ) in Associated Learning use sigmoid activation functions, which suffer from the vanishing gradient problem and usually performs unsatisfactorily as the number of layers increases. However-likely because each gradient flow is short in Associated Learning-the model still manages to obtain reasonable parameters, as demonstrated by the high test accuracy.
The CIFAR-10 dataset is more challenging than the MNIST dataset. The input image size is 32 × 32 × 3 [23], i.e., the images have a higher resolution and each pixel includes RGB information.
To make good use of these abundant features, we not only applied MLP and CNN in this experiment but also VGG and ResNet. The input images are augmented by 2-pixel jittering [24]. We applied the L2 norm using 5 × 10 −4 on VGG and 1 × 10 −4 on ResNet as the regularization weight.
Because ResNet uses batch normalization and the shortcut trick, we set its learning rate to 10 −3 , slightly larger than the other models. Besides, to ensure a fair comparison, we added extra layers to ResNet-20, ResNet-32 and VGG when using backpropagation for learning.  The CIFAR-100 dataset includes 100 classes. We used model settings that were nearly identical to the settings used on CIFAR-10 but increased the number of neurons in bridge. Table 3 shows the results. As in CIFAR-10, Associated Learning performs better on the MLP and the CNN structures, but slightly worse than backpropagation on the VGG and ResNet structures-likely because AL uses only local errors to update the parameters. To determine whether the hidden layers truly learn useful metafeatures when using Associated Learning, we used t-SNE [20] to visualize the 2 nd , 4 th hidden layers and output layer in the 6-layer MLP model and the 4 th , 8 th , and 12 th hidden layers in the 14-layer CNN model on the CIFAR-10 dataset. For comparison purposes, we also visualize the corresponding hidden layers trained using backpropagation. As shown in Figure 3 and Figure 4, the initial layers seem to extract less useful metafeatures, because the labels are difficult to distinguish in the corresponding figures. However, a comparison of the last few layers shows that Associated Learning groups the data points of the same label more accurately, which suggests that Associated Learning likely learns better metafeatures.

Metafeature Visualization and Quantification
To assess the quality of the learned metafeatures, we calculated the intra-and inter-class distances of the data points based on the metafeatures. To compute the intra-class distance, we first calculated d intra k , the average distance between any two data points in class k for each class.The inter-class distance is the average distance between the centroids of the classes. We also computed the ratio between intra-and inter-class distance to determine the quality of the metafeatures generated by Associated Learning and backpropagation [25,26]. As shown in Table 4, Associated Learning outperforms backpropagation on both the CIFAR-10 and CIFAR-100 datasets.

Related Work
Backpropagation [1] is an essential algorithm for training deep neural networks and is the foundation of the success of many models in recent decades [5,22,6]. However, because of "backward lock" (i.e., the weights must be updated layer by layer), training a deep neural network can be extremely inefficient [2]. In addition, empirical evidence shows that backpropagation is biologically implausible [9,10,27]. Thus, many studies have suggested replacing backpropagation by more biologically plausible methods or by gradient-free methods [28] in the hope of decreasing the computational time and memory consumption [27,29,30].
To address the backward lock problem, the authors of [2] proposed using a synthetic gradient, which is an estimation of the real gradient generated by a separate neural network for each layer. By adopting the synthetic gradient as the true gradient, the parameters of every layer can be updated simultaneously and independently. This approach eliminates the backward lock problem. However, the experimental results have shown that this approach tends to underfit-probably because the gradients are difficult to predict.
It is also possible to eliminate backward lock by computing the local errors for the different components of a network. In [31], every layer in a deep neural network is trained by a local classifier. However, experimental results show that this type of model is not comparable with backpropagation. The authors of [16] mixed the idea of local errors and similarity measurement [32,33,34]. Their experimental results show that this technique improves testing accuracy; however, this design requires each local component to receive signals directly from the target variable to allow the calculation of both the similarity matching loss and the prediction loss. Biologically, it is unlikely that neurons distant from the target would be able to access the target signal directly. Therefore, even though these methods do not require global backpropagation, they may still be biologically implausible.
Feedback alignment [11] suggests propagating error signals similar to backpropagation, but the error signals are propagated with fixed random weights in every layer. Later, the authors in [12] suggested delivering error signals directly from the output layer using fixed weights. The result is that the gradients are propagated by weights, while the signals remain local to each layer. The problem with this approach is similar to the problem discussed in the preceding paragraph: biologically, distant neurons are unlikely to be able to obtain signals directly from the target variable.
Yet another biologically motivated algorithm is target propagation [17,18,19,13,14,15]. Rather than computing the gradient for every layer, the target propagation computes the target that each layer should learn. This approach relies on an autoencoder [35] to compute the inverse mapping of the forward pass and then pass the ground truth information to every layer. Each training step includes two losses that must be minimized for each layer: the loss of inverse mapping and the loss between activations and targets. However, this learning method alleviates the need for symmetric weights and is both biologically plausible and more robust than backpropagation when applied to stochastic networks. Nonetheless, the targets are still generated layer by layer.
Overviews of the biologically plausible (or at least partially plausible) methods are presented in [27,15]. Although most of these methods perform worse than does conventional backpropagation, optimization beyond backpropagation is still an active research area, largely for computational efficiency and biological compatibility reasons.
Our work is highly motivated by target propagation. However, we create intermediate mappings instead of directly transforming features into targets. As a result, the local signals in each layer are independent of the signals in other layers, and most of these signals are not obtained directly from the output label.

Discussion and Future Work
This paper discusses Associated Learning, a novel process for training deep neural networks. Rather than calculating gradients in a layerwise fashion based on backpropagation, Associated Learning removes the dependencies between the parameters of different subnetworks; thus allowing each subnetwork to be trained simultaneously and independently. Moreover, our method is also biologically plausible because the targets are local, and the gradients are not obtained from the output layer. Additionally, we observed that the metafeatures generated by Associated Learning seem to be better than those generated by backpropagation. Although Associated Learning does not directly minimize the prediction error, its test accuracy is comparable to that backpropagation, which does directly attempt to minimize the prediction error. Although recent studies have started using local losses instead of backpropagating the global loss [16], these local losses are largely computed based on (or at least partially based on) the difference between the target variable and the prediction. Our method is unique because in Associated Learning, most of the layers do not interact with the target variable.
One aspect of our current work involves validating Associated Learning on other datasets (e.g., ImageNet, MS COCO, and Google's Open Images). We are also interested in validating Associated Learning on datasets beyond computer vision, such as those used in signal processing, natural language processing, recommender systems, and so on. Meanwhile, we are investigating strategies to better fit s i to y i with fewer parameters. In the longer term, we are highly interested in investigating optimization algorithms beyond backpropagation and gradients. Additionally, even though AL suppose to increase the throughput of training phase through the pipeline strategy, we still need empirical validation, so integrating AL with pipeline is also one of the future work.