{'fraction_non_alphanumeric': 0.033724219372202234, 'fraction_numerical': 0.004503885974789223, 'mean_word_length': 5.043817427385892, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Spectrum-Guided Adversarial Disparity Learning

It has been a significant challenge to portray intraclass disparity precisely in the area of activity recognition, as it requires a robust representation of the correlation between subject-specific variation for each activity class. In this work, we propose a novel end-to-end knowledge directed adversarial learning framework, which portrays the class-conditioned intraclass disparity using two competitive encoding distributions and learns the purified latent codes by denoising learned disparity. Furthermore, the domain knowledge is incorporated in an unsupervised manner to guide the optimization and further boosts the performance. The experiments on four HAR benchmark datasets demonstrate the robustness and generalization of our proposed methods over a set of state-of-the-art. We further prove the effectiveness of automatic domain knowledge incorporation in performance enhancement.


INTRODUCTION
Sensor technologies have inspired a wide range of applications that help people's daily lives in many realms, such as visual recognition [4], brain-computer interface [7], and activity recognition [5,24]. One of the major components of sensing applications is the establishment of commonly-used and robust systems over diverse scenarios. Some disparities can be caused by the variety of subjects' temporal conditions and unique physical characters: people have varying habits and body shapes; they may make different movements when performing the same activities, and sensors may perceive activities differently, given the same person performing the same activities. We call such variability within a class intraclass disparity. The intraclass disparity will significantly impair the systems' performance in dealing with new subjects or new environments.
The typical work to address this challenge [4] constructs a dictionary or a projection based on the existing subjects to conjecture the possible variations that may occur in less-seen subjects; the models can thus gain robustness through embracing the variation during training. However, such studies primarily rely on the number of known subjects, which we call subject-dependent studies. Therefore, the model may suffer from significant performance degradation when handling new subjects, due to the proprietary characteristics in existing subjects.
In light of generative models' excellent performance on sparse data, recent studies are increasingly applying generative networks to improve models' robustness on new subjects. Given that generative models usually perform better on sparse data, However, most subject-independent studies [1,20,25,30] are still limited in considering the intraclass disparity as meaningless noise and neglect the point that intraclass disparity is related to the subject and the class type. They are still inaccurate in exhibiting the relationship between the subject variation and the class, e.g., subject variation within a class should be conditionally constrained.
Besides, signal data may be segmented imprecisely, and the segments may include gaps and noises. Further, the segments carry nonequivalent amounts of information, which is difficult to measure in the time domain. Since any signal wave in the time domain also presents as a variable in the frequency domain, Spectrum represents the corresponding frequency composition of signals. Thus, it is promising to use frequency-domain features (e.g., amplitude and entropy) [29] as the domain knowledge to analyze the segments. Take amplitude spectrum [27] for example. Valid information frequencies offer peak amplitudes than gaps and noises. We can thereby measure the amounts of information and mitigate the impact of imprecise segmentation. The only issue is the heuristic, hand-engineered, case-specific nature of signal domain analysis, which limits its applicability.
To address the limitations in existing intraclass disparity research, we propose a novel Spectrum-guided Adversarial Autoencoder (SAAE) for Human Activity Recognition (HAR). HAR is challenging due to its low signal-to-noise ratio. SAAE utilizes two competitive encoding distributions to make intraclass disparity classconditioned and further eliminates the subject variation through a denoising structure. This way, the purified latent codes will be robust to handle new subjects. The advantages of SAAE are two-fold. First, SAAE applies a learnable prior distribution, which enables posterior distribution to learn class-related distribution. The posterior distribution learns to generate the intraclass disparity with the class information in prior distribution, while the discriminator and decoder further supervise the prior distribution to learn the pure information by fixing posterior distribution. Second, SAAE automates spectrum analysis as domain knowledge, leading to enhanced robustness of the data segmentation. The proposed automatic spectrum guided function is capable of dynamically weighing signals and then adjusting the optimization of adversarial training.
We make several contributions in this paper: ... Figure 1: Illustration of SAAE. First, we take two samples of the same class to explain the three phases in optimization: signal weight measurement in spectrum analysis, decomposition in the regularization phase, and denoising in the reconstruction phase. Spectrum analysis (in the middle) transforms signals and projects the corresponding amplitude spectra into signal weights W A for reconstruction by S ζ . The class weight W c is summarized from all signals under a class to assist regularization. Decomposition feeds the disparity δ (labeled 'invalid') and pure information γ (labeled 'valid') to the discriminator to decompose the original data. Denoising takes δ as noises and uses an autoencoder to denoise the disparity components. Then, after optimization, the encoding distribution q φ (γ |x) could extract information γ purified from intraclass disparity δ , which enables discriminator D to precisely predict the activities.

D d Prediction
• We introduce the signal theory and design a principled unsupervised score function to weigh signals dynamically. The score function analyzes signals in the frequency domain to measure the signal information amount and to provide domain knowledge in optimization. • We propose a novel Spectrum-guided Adversarial Autoencoder (SAAE), which fuses automating spectrum analysis and adversarial training in a unified network. SAAE utilizes two competitive encoding distributions supervised by prediction validity to extract and denoise the learned intraclass disparity. We also analyze and prove the effectiveness of intraclass disparity learning through competitive encoding distributions training. • We compare SAAE with both state-of-the-art adversarial autoencoders and human activity-related algorithms in the subject-independent experiments. The superior performance of SAAE on four benchmark datasets demonstrates its effectiveness in solving the intraclass disparity problem on unknown subjects. We further exhibit the effectiveness of intraclass disparity learning through convergence analysis and spectrum analysis.

METHODOLOGY
This section introduces the methodology of AAE for the classconditioned intraclass disparity and the domain knowledge assistance. AAE consists of two phases: intraclass disparity extraction and denoising. AAE leverages the domain knowledge obtained from frequency-domain analysis and an automated spectrum guide function for optimization.

Intraclass Disparity Learning
This section first declares the class conditioned intraclass disparity definition, and then proposes a two-phase AAE which utilizes two competitive encoding distributions to handle the intraclass disparity learning. We discuss the Reconstruction phase and the Regularization phase of AAE from the generator's perspective and the discriminator's perspective, respectively. We further prove AAE's effectiveness in learning and denoising intraclass disparity.
2.1.1 Notations for Adversarial Autoencoder. Following the notations in Kingma et al.'s work [15], we denote by p dat a (x) the observed data distribution, p(x) the model distribution of reconstructing data, q(z|x) the encoding distribution, and p(x |z) the decoding distribution. The distribution q(z|x) encodes the original data into latent codes z, which denotes the latent representation for data reconstruction. Suppose that the intraclass disparity δ confuses pure information γ in a linear relationship, we define that the latent representation z is composed by γ ∼ q φ (γ |x), δ ∼ q η (δ |x) where q φ (γ |x), q η (δ |x) denote the pure information and disparity encoding distributions, respectively.

Reconstruction Phase.
In this phase, we use the denoising autoencoder to denoise disparity and to reconstruct the signal waveform. We consider the signal waveform reconstruction with the following requirements: (i) the decoder should be able to reconstruct original data by z = γ + δ ; (ii) γ should be able to learn the representative and invariant waveform information; (iii) δ should be the confusion factor in waveform only related to class information. Given the optimal disparity δ , the reconstruction can be demonstrated as a denoising autoencoder from such disparity. The autoencoder fuses the learned disparity δ with raw data as inputs and then trains autoencoder to recover the original data. Thus, the model will be less sensitive to the disparity components. In conventional denoising autoencoders [26], the disparity part in original data is vague; thus, they fuse the noise in raw data (e.g., Gaussian noise). Different from the meaningless noise, the proposed disparity encoder q η (δ |x) can extract the disparity components, which will be proved to be the disparity with class information in the regularization phase by the adversarial training. Therefore, we move the fusion process after encoding.
The requirement (i)-(iii) can be explained with the following definitions. Let γ be pure latent codes and fix the optimal disparity distribution to let δ be the constant noise. Then, the purified latent code z = γ and the confused code z ′ = γ + δ . The condition probability distribution of z ′ on x is q φ (z ′ |x). Consider an approximate posterior distribution between x and z: where c(z|z ′ ) is the observed probability distribution between confused codes and pure codes. Then, given the decoding distribution p θ (x, z) = p θ (x |z)p(z), the lower bound of the autoencoder may be formed in the following way [12] by Jensen's inequality: fi Therefore, the lower bound of autoencoder can be sorted into minimizing the negative weighted likelihood L r ec :

Regularization Phase.
In this phase, we consider the encoding distribution regularization. The conventional AAE [1] only applies binary discriminator to regularize generation distribution as a fixed prior distribution by predicting 'real' or 'fake', which fails to portray the class-conditioned intraclass disparity. Therefore, we propose a multi-label classifier and a learnable prior distribution to improve the conventional AAE by concluding the following requirements: (i) discriminator should be able to distinguish γ and δ based on the information validity in prediction; (ii) γ represents the generalized class information without subject variation; (iii) δ represents the subject variation conditioned on activity. As mentioned above, some intraclass disparity exists due to the motion and body shape variance. The similarity distribution regularization will keep this disparity in the representations. Thus, we derive the original binary classifier to a multi-label classifier to distinguish the validity of learned representations. The representations without disparity should be more robust in predicting different subjects, while the disparity representations will impair the performance. Note that, we only consider the case that the dataset contains the class information; otherwise, the prediction validity cannot be measured. Then, we can explain requirement (ii)-(iii) by denoting the low-validity distribution in regularization as the disparity distribution and high-validity distribution as the pure information in class prediction. The requirements (i) can be modified to let discriminator predict whether the latent codes are 'valid' or 'invalid' according to the likelihood of correct predictions.
First, we discuss the Categorical Cross Entropy (CCE) for multilabel classification: where y denotes a one-hot probability distribution of ground truth; y c represents the discriminator probability of true label c.
We can find that CCE loss is equivalent to the negative loglikelihood of the true prediction probability. Thus, minimizing the CCE loss equals maximizing the validity (i.e., the likelihood of correct prediction). We utilize the adversarial regularization to optimize requirements (ii)-(iii) and then conclude three requirements in an adversarial format.
Given data (X , Y ), the competitive distribution (i.e., the encoder of γ ) enables the generator distribution (i.e., the encoder of δ ) to learn the class conditioned intraclass disparity by optimizing the loss function L r eд : where d denotes the parameters of the multi-label discriminator D; D c denotes the probability of correct predictions, respectively.
Proof. We set up parameters, φ, d, η, before the optimization, and the network has enough capacity to acquire optimal solutions When φ is fixed, we can take the pure information encoding distribution as the 'real' data distribution. According to the Proposition 2 in Goodfellow et. al's work [8], at each iteration, the generator (i.e., the disparity encoding distribution q η ′ (δ |x)) will converge to the 'real' data distribution (i.e., the pure information distribution q φ (γ |x)). Hence, the network can converge when q η ′ (δ |x) = q φ (γ |x). In other words, the disparity encoder will learn the extracted class information distribution which regularizes the disparity to be conditioned on class. Then, q η (δ |x) can represent a local optimal solution from last iteration, and the optimal φ ′ will be optimized to let discriminator differentiate the validity of disparity and information. Therefore, q φ ′ (γ |x) will generate the pure information apart from the previous disparity validated by D d . Thus, when the network converges, we can get a local optimal distribution pair q η (δ |x) and q φ ′ (γ |x) to decompose latent codes into disparity and pure information, respectively. □ As mentioned in proof, the optimization D d ′ converges to precisely predicting q φ (γ |x) and q η ′ (δ |x), and q φ ′ (γ |x) is optimized to be distinct from q η ′ (δ |x). In other words, q φ ′ (γ |x) is invisible to discriminator D d ′ . Thus, only if q φ ′ (γ |x) learns the purified data will D d ′ predict the accurate classification. To further analyze the loss functions of proposed disparity learning, we specify the details of two competitive encoding training and name the two steps of the min-max optimization of L r eд as pure loss and disparity loss according to the corresponding encoding distributions: We exhibit curves of L pur , L dis to display the convergence of two competitive encoding losses and the training classification performance of using D d ′ predicting the purified representations q φ ′ (δ |x) in Section 3.4 Convergence Analysis, which proves intraclass disparity learning and denoising.

Principled Spectrum Analysis
Considering the gaps and noises inside data segmentation, invalid segments will obstruct the models from learning factual patterns.
Most deep learning algorithms only analyze data in the time domain, where it is difficult to analyze information components. Therefore, we introduce the domain knowledge of signal theory to leverage the model optimization according to the information amount from the frequency domain. This section will introduce the principles of spectrum analysis and the fused Spectrum-guided Adversarial Autoencoder, respectively.

Time Domain
Frequency Domain Activity Figure 2: An illustrative example of demonstrating spectrum analysis in the frequency domain. We present two arbitrary data segments while a subject is doing the 'weightlifting' activity. It is represented as a typical time-series sequence in the time domain. By introducing the domainspecific knowledge in the theory of signal processing. It can be transformed into a more robust representation by automatic spectrum analysis proposed in this work. The main information of signals (i.e., peak amplitudes in the frequency domain) is indicated with dotted lines.

Notations for Spectrum Analysis.
The spectrum analysis focuses on the patterns of signals in the frequency domain, which is transformed from the time domain. Since sensors are only able to record signals in a certain sampling rate, let x[n] be the observed signals in such discrete-time domain. We can obtain the corresponding frequency domain data X and amplitude spectrum A that records the signal strength of each frequency via Fourier transform [23]: where A[n] denotes the amplitude of frequency n; N denotes the discrete time point number of the sampled signal; i, e represent the imaginary number and the mathematical constant. The amplitude spectrum is perfectly symmetric, so we only analyze the positive Our goal is to design an amplitude spectrum guide function S ζ (A[i]) → R, which can measure the importance of each frequency according to the signal strength. We define the mean score of the amplitude spectrum to represent the signal information amount (i.e., the signal importance) in the numerical format. To ease our illustration, we express the mean score of an arbitrary frequency set T evaluated by spectrum guide function S ζ as where ζ denotes the function parameters; |T | represents the element number of T . Therefore, the signal importance can be measured by its amplitude spectrum S ζ (A).
To automate the spectrum analysis, we analyze amplitude spectrum in terms of the frequency set. According to [27], the signal information is mainly composed of some frequencies with the highest signal strengths. We let U be the frequency set of the highest amplitudes (i.e., information) and I be the frequency set of the lowest amplitudes (i.e., gap and noise): where U , I are two subsets of A.

Spectrum Analysis
Principles. The spectrum analysis principles are based on intra-relationship and inter-relationship in signals, which describe the relationship of frequencies within a signal and between signals, respectively. We illustrate the proposed principles by recalling the phenomenon in Fig. 2: differing from time-segments, given two arbitrary data segments clipped from a series of noisy continuous signals with gaps, we can easily tell the main composition (i.e., orange and green peak points) and the class of two segments in the frequency domain. Also, we can easily see that the green segment contains more information, because its curve has more strengths than orange segment's, especially around peak points. The conventional band-pass filter analysis [27] manually selects the frequencies with the highest strengths, i.e., the peak points.
To automate the spectrum analysis, we transform the frequency selection to weigh frequencies based on frequency set analysis and propose two principles: (1) Given an amplitude spectrum A, ∀i ∈ U , ∀j ∈ I , function S ζ should satisfy (2) Given the certain frequency's amplitudes from two arbitrary different spectra The principle (i) describes the traditional band-pass filter theory [27]: if the frequency A[i] is the most significant frequency (i.e., the information frequency), A[i] should have much higher amplitudes than others'. Since principle (i) only considers the intra-amplitude difference relationship, we further derive the principle (ii) to complement the inter-relationship.
Beginning with principle (i), refer to Eq. (6), we consider that the information set U and the noise set I (including noise and gaps) meet the significant amplitude difference requirement. To replace the traditional case-specific band selection, we utilize the information set U to represent the frequencies where information may exist. Then, we can maximize the below loss function L N to let S ζ satisfy principle (i): Proof. For any two frequencies i, j, the score function S ζ should maximize the weight difference between i, j, so we should maximize the loss function that S ζ (A[i]) −S ζ (A[j]). Then, for any two subsets U , I , where |U |, |I | denote the element number subsets. Thus, optimizing S ζ (U ) − S ζ (I ) is equivalent to the raw formula. □ Further, we utilize principle (ii) to distinguish different signal weights. Similar to principle (i), we can easily conjecture the amplitude difference relationship between the particular frequencies of two spectra in the frequency-set format: Then, for arbitrary spectra A a > A b , we can minimize the following loss function L O to meet principle (ii): where α denotes the proportionality constant.

Spectrum Guide Function.
To put the proposed principles into practice, we further pose a value range in the function S ζ (A) → (0, 1) (0 means extremely insignificant and 1 means extremely significant), and thus the optimal maximum of S ζ (U ) −S ζ (I ) should be 1. Maximizing L N is equivalent to minimising 1−L N . The empirical proportionality constant holds α = 1.
With the above empirical setting, the expected range of A should be (0, 1). To match the different analysis perspectives, we propose to rescale the amplitudes by min-max normalization in two folds. Assuming that the spectrum set A n×m := {A 1 , A 2 , . . . , A n } contains n spectra with m frequency channels. We normalize each spectrum A i in frequency-level and sample-level to be consistent with two principles (Section 2.2). Denote A N i and A O i to represent the normalized data of intra-and inter-relationships for an arbitrary spectrum A i , respectively: where A i , A j denote a row and column vector of A, respectively; , and A max j denote the minimum and maximum of each row or column. Then, the score function evaluates the ith frequency using both intra and inter normalized features by To ensure U and I satisfy our definitions, we empirically take the 20% of frequencies with the highest amplitudes as U and the lowest 50% as I in A N . With the min-max normalization, the mean amplitude difference between U and I will be large enough to distinguish the noise and information.
The optimization of the score function S ζ can be demonstrated by two steps: (i) for any amplitude spectrum A, use A N to update ζ by minimizing 1−L N ; (ii) for any amplitude spectrum pair We can then unify the steps to optimize ζ by minimizing a pairwise loss function L S (A a , A b ): where the first line denotes 1 − L N for A N a , A N b and the second line is L O for spectrum pair. Note, we adopt a Fully Connected (FC) layer followed by Sigmoid as S ζ to match the value range in the definition, and then the optimization will be a traditional regression problem, which can be converged [14].

Spectrum-guided Adversarial
Autoencoder. Given a set of signals with labels (X , Y ), we calculate the spectrum weighted loss function for an arbitrary signal (x i , y i ) as follows: where A x i denotes the amplitude spectrum of x i ; w c denotes the weight of x i 's class; |X c | denotes the number of the samples in class c. Due to the activity type and the subject variance, the information amounts carried by signals are varying from each other and thus we apply the mean class spectrum weights of the ground truth to leverage the discriminator training.
We apply two weighing methods in SAAE. Since the reconstruction phase focuses on a single signal waveform, we leverage different signals by its own spectrum. The regularization phase focuses on the class conditioned disparity, so we leverage signals by overall class information amount distribution. The optimization algorithm is shown in Algorithm 1.

EXPERIMENTS
In this section, we report our comparative experiments and ablation studies in subject-independent settings on four real-world benchmark datasets to evaluate SAAE's robustness and the impact of the spectrum guide function. The source code is publicly available 1 .

Datasets
We evaluate our approach using four public real-

Experiment Setting
We execute Leave-One-Subject-Out experiments (with the test subject being excluded from training) to evaluate algorithms' robustness on varying subjects, where we set the time windows as 20 with 50% overlapping to pre-process the time-sequence data. The spectrum data calculated by Fourier Transform are pre-calculated, so the time complexity is O(1). We compare our algorithm, SAAE with five state-of-the-art HAR models and three AAE models, and show a further study on iAAE, a version of the proposed model without the knowledge (spectrum) guide function.
• : a convolutional adversarial autoencoder enhanced with denoising operation to extract the pure information from low signal-to-noise ratio data We take MC-CNN as the base model for the encoder, three deconvolutional blocks, named the deconvolutional layer, the rectified linear unit (ReLU), and the batch normalization layer for the decoder, an FC layer, and a softmax function for the discriminator. The network architecture details are shown in the supplementary Section A. Our model applies the same encoder-decoder structure as in DAAE but differs in using the adversarial structure and the spectrum guide function. To show the robustness of methods, we use the same hyperparameter settings over all the subjects in the same datasets. For SAAE, we set the learning rate of the spectrum guide function to 1e-4 for 4 datasets, the learning rates of the adversarial network to 5e-5 for PAMAP2 and 2e-4 for other datasets.

Overall Comparison
The results (Table 1) show our models outperforming all other algorithms on the four datasets and having the smallest standard deviations, demonstrating the effectiveness of SAAE and its variant, iAAE (without Spectrum knowledge) on four benchmark datasets. We can easily observe that the proposed models, iAAE and SAAE, consistently outperform all other baselines and hold the smallest standard deviation. In terms of F1-score, SAAE improves about 2%, 3%, 4%, and 6% in four datasets, It also indicates the robustness in handling intraclass variance of new subjects.

Further Analysis
Effectiveness of Adversarial Training. From the mean performance of four datasets, SAAE achieves a 4.3% improvement in accuracy, 4.8% in precision, and 5.2% in F1 Score comparing with a  Fig. 3 (a)). For instance, MC-CNN can achieve 97% accuracy on subject 9 while only 87% in subject 3, revealing that MC-CNN may perform badly while dealing with the new subject whose samples are deviating a bit more from the common distribution of training subjects. In contrast, SAAE steadily improves the performance on most subjects, especially subject 3 and subject 7, which proves its robustness and reliability. Effectiveness of Spectrum Analysis. Comparing SAAE and iAAE, we find that the spectrum guide function can further enhance the performance by around 1% in each subject, validating the effectiveness of frequency domain analysis in model learning. We also take subject 1 in MHEALTH as an instance to illustrate the spectrum analysis's effectiveness in optimization. Our convergence comparison of discriminator D (Fig. 3 (b)) shows spectrum domain knowledge can better exclude the disparity during optimization. Hyper-parameter Analysis. We change the learning rates of different components to explore the optimal hyper-parameters. We set learning rate as 1e-4 for spectrum guide function and 2e-4 for AAE as default, and plots the mean accuracy of one-out subjectindependent experiments when learning rates range from [5e-5,5e-4] in Fig. 3 (c). We can observe that SAAE is stable over different parameters and constantly outperforms the best state-of-the-art. The spectrum learning rate merely influences the model performance while AAE's learning rate will slightly affects the results. With larger learning rate of AAE, the results gradually become better, which means SAAE could act better than our provided result. Convergence Analysis. Fig. 3 (d) -(h) plot the averaged loss over four datasets. We can observe that L r ec , L dis , L pur could quickly converge around 1000 iterations and then remain stable, where L dis and L pur represent the min-max competition loss functions of intraclass disparity and purified information, respectively. The convergence of L pur and L dis reveals the SAAE's capability of optimizing the corresponding encoding distributions as expected. Besides, the discriminator can precisely predict the pure information components, which proves that SAAE is capable of learning to exclude the disparity components in latent codes through the adversarial training. The spectrum loss L S drops rapidly at first and then smoothly decreases until convergence over iterations. Visualization of Learned Representation. Fig. 4 visualizes the raw data and the learned features. The original dimension is reduced by t-Distributed Stochastic Neighbor Embedding (t-SNE). The learned features are more similar within classes yet more dissimilar between classes over the sample space than the raw features, indicating the effectiveness of the network's feature learning.

RELATED WORK
The previous research on intraclass disparity falls into two categories: subject-dependent and subject-independent models. Some of the subject-dependent algorithms attempt to discover patterns by case-specific analysis: Ren et al. [25] captured the community in natural activities through hand trajectory; Tapia et al. [20] designed a more comprehensive system to record the invariant features of angle movement through wireless sensors. Another thread of subject-dependent research uses deep neural networks to learn representations for automated feature design and then activity recognition. Such methods focus on capturing the discriminative representations in the signal streams. Yang et al. [28] applied convolution and pooling filters along the temporal dimensions to catch the difference in the long-term time sequence signals; Fransisco et al. [18], who further improved CNN with LSTM but still neglect the varying information amounts in signals. To validate the significance of signals, Vishvak et al. [17] integrated a temporal attention module that aligns the output vector of the last time step with other steps' to learn a relative importance score to modify the learning process. However, subject-dependent methods fail to consider the unique patterns that may occur in new subjects, as the common patterns in existing subjects may not include all the potential conditions. Subject-independent studies aim to enhance the generalization ability of models into precise recognition of new persons. Sani et al. [22] proposed to construct a support set and match the most similar instances to ease the unique patterns'. Yu et al. [9] ensembled the models from different iterations and promise a more generalized model. The above non-generative models still lack generalization due to the model limitation. Balabka et al. [1] applied AAE to approximate a generalized distribution to simulate human activity distributions. Thus, the encoder in AAE could be more generalized and robust when handling new subjects. Zhang et al. [30] fused the advantages of variational autoencoders and generative adversarial networks and designed a regularized latent representations for generation. However, these works exclusively analyzed the timedomain information and all considered the intraclass disparity as meaningless noise and failed to extract the disparity distribution by adversarial training. Besides analyzing raw data patterns, some hand-engineered domain knowledge was introduced to further extract high-level discriminative information to assist predictions. The commonly used features include time-domain features (e.g., mean, variance, skewness) and frequency-domain features (e.g., power spectral density) [13]. Some studies design new features containing temporal and structural information. For example, Hammerla et al. [11] proposed the Empirical Cumulative Density Function (ECDF) to extract the spatial information of the signal frames. Such methods are generally heuristic and lack generalization in different scenarios.
Our work differs from the studies above on two aspects: utilization of spectrum analysis and specific intraclass disparity distribution learning. We implement and embrace spectrum analysis as a tool in AAE to leverage the optimization based on signal information amount. Further, we specify and precisely portray the class-conditioned intraclass disparity in a learnable competitive encoding distribution, which enables AAE effectively to extract and to denoise such disparity.

CONCLUSION
We propose novel spectrum-guided disparity learning, or SAAE, to address intraclass variability. We design two competitive encoding distributions under a unified adversarial training framework, rather than a fixed prior distribution, to learn robust embeddings that can be generalized to new subjects. We further incorporate the domainspecific knowledge in an unsupervised manner. We experimentally validate our model on four representative benchmark dataset with state-of-the-art methods. The results demonstrate the superior performance and robustness of the proposed model in predictions on unknown subjects. Given SAAE's promising performance in handling intraclass disparity, we will extend it to handle more complex scenarios in the future.

A NETWORK ARCHITECTURE
We plot the detailed network structure in Fig. 5. The decoder and encoders consist of three independent blocks and two encoders share the same structure. The architectures of Block are shown in Fig. 6 and the parameters are shown in Table 2. Specially, Conv Block 3 & 6 and Deconv Block 3 do not have Maxpool layer. The stride of layers are 1 and padding way is 0 as default. Discriminator D is composed of one FC layer, so the input dim equals the output element number of encoders and output dim equals the target class number. Given a amplitude spectrum A ∈ R 1×m , then the Spectrum Guide Function S ζ consists of two FC layers: first layer takes 2 · m dimension input (i.e., A N and A O and keeps the same output dimension; second layer generates m-dimension outputs, which represents the corresponding weights for the frequencies in spectrum.

B SUPPLEMENTARY EXPERIMENT
Due to the space limitation, we exhibit the other three datasets' analysis, i.e., specific subject accuracy reports, spectrum guided fitting curves, and embedding comparison in Fig. 7. From (a)-(c), we can observe the subject variation in other three datasets, especially the subject 8 in PAMAP2 and subject 8 in UCID-SADS, while SAAE achieves stable performance over these unstable subjects. Also, SAAE improves all three subjects in OPPORTUNITY dataset. (d)-(f) further provide the discriminator fitting curves of subject 1 on other three datasets, which indicates the effectiveness of domain knowledge in diverse scenarios. (g)-(l) exhibit the data distributions before and after SAAE's purification. We can easily see that points of same classes become more gathered and there are fewer scattered points over all three datasets.
We also exhibit the Confusion matrices over four datasets to assist proving our algorithm's outperformance and robustness. Fig.  8 plots a subject's confusion matrix of the corresponding datasets. We can observe that the classification on MHEALTH is solid, and only a few samples are misclassified. The prediction in PAMAP2 shows that it is difficult to learn the generalized representations in some classes, where the misclassified samples spread over other multiple diverse activities. However, there exists the interclass similarity in UCIDSADS and OPPORTUNITY. Most of the classes can be precisely predicted in these two datasets, but the misclassified samples mainly gather in one class. For instance, the misclassified samples of class 9 (running) centers in class 10 (exercise in steps) in UCIDSADS, which indicates the interclass similarity between class 9 and class 10.