{'fraction_non_alphanumeric': 0.027648, 'fraction_numerical': 0.010464, 'mean_word_length': 5.061093871217999, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Highly Scalable Task Grouping for Deep Multi-Task Learning in Prediction of Epigenetic Events

Deep neural networks trained for predicting cellular events from DNA sequence have become emerging tools to help elucidate the biological mechanism underlying the associations identified in genome-wide association studies. To enhance the training, multi-task learning (MTL) has been commonly exploited in previous works where trained networks were needed for multiple profiles differing in either event modality or cell type. All existing works adopted a simple MTL framework where all tasks share a single feature extraction network. Such a strategy even though effective to certain extent leads to substantial negative transfer, meaning the existence of large portion of tasks for which models obtained through MTL perform worse than those by single task learning. There have been methods developed to address such negative transfer in other domains, such as computer vision. However, these methods are generally difficult to scale up to handle large amount of tasks. In this paper, we propose a highly scalable task grouping framework to address negative transfer by only jointly training tasks that are potentially beneficial to each other. The proposed method exploits the network weights associated with task specific classification heads that can be cheaply obtained by one-time joint training of all tasks. Our results using a dataset consisting of 367 epigenetic profiles demonstrate the effectiveness of the proposed approach and its superiority over baseline methods.


I. Introduction
According to the data from the NHGRI-EBI GWAS Catalog 1 , there have been, to date, over 400,000 genetic associations being identified for a wide range of human diseases and other traits through large amounts of genomewide association studies (GWASs). Despite such a remarkable achievement, the elucidation of genetic causes for human diseases to enhance precision in their management remains elusive, hindered by intrinsic limitations of GWASs, i.e., the noninclusion of rare variants and the lack of indication of causality due to linkage disequilibrium [1]. In addition, a vast majority of genetic variants in identified associations are located in non-coding genomic 1 https://www.ebi.ac.uk/gwas regions, hampering the understanding of biological basis underlying these associations [2].
With large amounts of genome-wide profiles of varying cellular activities accumulated through both large consortia and projects, such as ENCODE [3], Roadmap [4], and GTEx 2 , and individual research laboratories 3 , training machine learning models to predict cellular actives from DNA sequence holds the promise of elucidating links between genome and phenome. Among the characterized cellular actives, there are functional genomic events, such as histone modification and chromatin accessibility that affect chromatin states and play critical roles in gene expression. The genes that are expressed and their respective expression levels in a cell largely determine the functionality of the cell and its identify, e.g., a neuron or an immune cell [5]. DNA sequence regulates chromatin states and subsequently the gene expression, contributing to the biological complexity in multi-cellular organisms, like human [4]. Similarly, mutations in DNA sequence may alter important sequence patterns, leading to perturbed chromatin states, modifications in gene expression, and eventually changes in phenotypes, e.g., risks of developing certain diseases [2], [6]. As a result, machine learning models trained to predict functional genomic events from DNA sequence can be used to evaluate the impact of mutations on chromatin states and therefore, identify genetic variants with true functional impact.
Several attempts have been recently made to train machine learning models to predict cellular activities of varying modalities from DNA sequence, including chromatin accessibility and interaction, DNA methylation, histone modification, and transcription factor binding [7]- [11]. Due to their highly expressive power and effectiveness in feature learning from sequence data, the majority of existing works exploited deep neural networks as the predictive models. On one hand, cellular activities are cell type specific, indicating the same needs to be true in models trained for predicting these activities. On the other hand, cellular constituents closely interact with each other in a unified system; therefore, cellular activities of different kinds are likely related and may be co-regulated [12]. Such shared regulations can be leveraged during training by multi-task learning (MTL) to enhance the training, which is important due to the data hungry nature of deep neural networks. Indeed, in a few studies [7], [10], models jointly trained for predicting events of different modalities show improved performance on average. However, all existing studies that exploited MTL adopted a very simple joint training framework, having all models sharing the same feature extraction component, known as hard parameter sharing [13]. As indicated in our experimental results, such a simple strategy, despite enhancing the learning on average, leads to models with decreased performance for many tasks due to conflicting training objectives, which is known as task interference or negative transfer [14].
Many methods have recently been proposed to address negative transfer while training deep neural networks simultaneous for multiple predictive tasks with hard parameter sharing. These methods generally follow four orthogonal research directions: task specific routing through a network shared across all tasks [15]- [18], branching out from shared networks [19]- [22], task grouping [23]- [25], and training objective manipulation [26]- [29]. Task grouping based methods explicitly study task relationships and only jointly train tasks that are potentially beneficial to each other. As a result, better applicability to problems in various domains and easier interpretability can be achieved compared to methods following other directions. However, all the existing task grouping based methods were originally developed for learning problems either in computation vision [23], [24] or natural language processing [25] where the number of tasks in joint training are in relatively small amount (up to 107 tasks in [25]). These methods were not designed to handle applications, such as epigenetic events predictions considered in present study, where the amount of tasks can easily reach several hundreds [7] and even many thousands [11]. They are difficult to scale and thus not readily applicable in training deep neural networks for predicting varying epigenetic events in a large number of different cell types.
In this paper we propose a highly scalable task grouping framework for deep MTL that jointly train deep neural networks for massive amount of tasks. Our method exploits the network weights associated with task specific classification heads in simple MTL framework where all tasks share the same feature extraction component. Due to the shared feature extraction component, the representation (or embedding) of any input is identical across tasks. This, together with the assumption of classification heads of tasks having the same network architecture (which is the case in all previous works on functional genomic events prediction where MTL is utilized [8], [11], [30]) mean that weights in the classification head completely identify individual tasks and importantly indicate how each task uses the shared representation of input data. Therefore, they are good source of information to use for studying relationship among tasks to determine which group of tasks should be trained together. In summary, we propose to identify task groups by performing cluster analysis using the task specific classification weights obtained from onetime joint training involving all tasks, followed by separate in-group joint training to address negative transfer.
To evaluate the proposed framework, we composed a dataset obtained from ENCODE and NCBI GEO that consists of a total of 367 profiles of cellular events of three different modalities in a number of cell types related to the human central nervous system. Besides k-means, we also exploited the utility of four other clustering methods of different nature for the task grouping in the proposed framework. Our results suggest k-means is the most effective clustering algorithm to use and demonstrate the effectiveness of the proposed method in addressing negative transfer, with 3.20% improved F1 score comparing to simple MTL. In addition, the comparison to the task grouping by either event modalities or cell types indicate the superiority of proposed grouping over the two with improved F1 scores of 1.32% and 2.55%, respectively.

A. Highly scalable task grouping framework
The key in task grouping to promote effective joint training while minimizing negative transfer is to find appropriate numerical representations of tasks followed by the definition of the affinity or similarity among tasks. It has been shown previously that just considering the distribution of the class label often does not lead to a useful similarity measure, because similar (or different) class label distribution among tasks does not necessarily imply positive (or negative) transfer in joint training [14]. Therefore, the challenge here is to find a similarity measure that encourages tasks that can learn from each other to be grouped together and those with conflict objectives, thus competing for the representation capacity of the learning model, to be grouped separately. Another challenge is that the computation cost of the similarity needs to be reasonably low. So that, it can be applicable to learning problems where the number of tasks can be in several hundreds or even many thousands, such as the epigenetic event prediction considered in this study.
Here, we propose a task grouping framework that is highly scalable and effective in reducing negative transfer without sacrificing positive transfer among tasks in joint training. All existing works that train deep neural networks with leveraging multi-task learning to predict functional genomic events adopted a simple hard parameter sharing framework [7], [8], [11], [30]. As illustrated in left most component of Figure 1, this framework includes a feature extractor shared by all tasks followed by task specific classification heads. The feature extractor is designed to learn sequence patterns in the input DNA fragment that are predictive to the functional genomic events under prediction. Even though the detailed implementation of the feature extractor varies across studies, it is typically a convolution neural network or its variants, for example with incorporating self-attention mechanisms [11]. The task specific classification head is often a subnetwork that consists of one or two fully connected layers optionally followed by a softmax layer for producing predicted probabilities if classification task. Due to the shared feature extractor, the inputs (i.e., extracted features) that task specific heads receive are identical across all tasks. Then, intuitively, the way of such inputs being used in each classification head can be used to determine the task relationship. More specifically, tasks that exploit the extracted features in ways that are more similar have higher potential to benefit each other in joint training. Given that all classification heads have the identical network architecture, what determining the way of inputs being used is the set of network parameters, represented by W i for task i. Therefore, we propose to exploit W i 's to define numerical representations of tasks for task grouping, denoted by column vectors e i 's, e i ∈ R d . Depending on the specific network architecture of the classification head, W i may be a vector (i.e., weights associated with the last fully connected layer) with/without additional weight matrices. A set of vectors can be made for each task i to include the vector and vectorized additional weight matrices if any. Then, the e i can be constructed by the concatenation of either all vectors in the set or one of its subsets. In our experiments, we used the weight vector associated with the last fully connected layer, which has been shown being effective by the results.
Let E ∈ R T ×d represent the matrix composed by stacking all obtained e i 's in rows, where T is the total number of tasks. Cluster analysis is performed to put tasks into groups by taking E as input. A variety of clustering algorithms can be potentially used in here, such as hierarchical clustering and DBSCAN. By following a previous study [31], we propose to use k-means. It identifies clusters that minimize the sum of squared distance between individual data points to the centroid (i.e., mean) of their assigned clusters, performing well when the desired clusters are in globular shape. K-means is an iterative algorithm, repeatedly assigning data points to clusters according to the centroids from previous iteration followed by updating the centroids. It is fast since each iteration in the algorithm has a linear time complexity, O(T ), adding to the scalability of the proposed framework.
In summary, as illustrated in Figure 1, the proposed framework for grouped multi-task learning consists of four consecutive steps: (1) joint training all tasks with hard parameter sharing to obtain trained task specific classification heads; (2) composing task representation matrix E based on weights in obtained classification heads from previous step; (3) performing cluster analysis to identify task groups using E as input; and (4) joint training tasks within each group from scratch. The high scalability of the proposed framework is mainly due to only one time training of all tasks is needed for the task grouping and also importantly there is no extra storage that is needed.

B. Study data
To validate the proposed framework for grouped multitask learning and obtain models for future elucidation of functional impacts of genetic variants identified in GWASs of psychiatric disorders, we compiled a dataset that consists of a total of 367 profiles of epigenetic events of three different modalities: chromatin accessibility (CA), histone modification (HM), and transcription factor binding (TFB) in varying cell types related to human central nervous system and tissues in different brain locations. The majority of the profiles were obtained from ENCODE project [3] and the rest were downloaded from NCBI GEO with accession numbers: GSE113483, GSE96615, and GSE96949. For profiles from ENCODE, we directly downloaded files that contain narrow peaks; while for those from NCBI GEO, we downloaded respective FastQ files from SRA and followed the procedures described previously by us [32] to obtain peak files.
This dataset includes chromatin accessibility profiles by two different techniques: DNase-seq and ATAC-seq. The later is a newer and much more popularly utilized in recent years due to its much relaxed requirement on the number of cells comparing to DNase-seq. According to the cell type, we assigned each of the profiles into six groups: (1) neuron group including profiles with labels of neuron, bipolar neuron, and excitatory neuron (in the source); (2) glial cell group including profiles with labels of astrocyte and glial cell; (3) progenitor group including profiles with labels of neural progenitor cell, neuronal stem cell, radial glial cell, neurosphere, and ecto neural progenitor cell; (4) (cerebral) circulation group including profiles with labels of epithelial cell, endothelial cell, and pericyte; (5) cancer group including profiles of cancer cells: SK-N-SH, PFSK-1, SK-N-MC, BE2C, D721Med, D341Med, Daoy, M059J, SK-N-DZ, H4, and A172; and (6) mix group including profiles of bulk of brain tissue that can contain multiple cell types.
The distribution of profiles across the three event modalities and the six cell groups is provided in Table I. Histone modification has the largest number profiles (N = 191) among the three event modalities followed by transcription factor binding (N = 90) and chromatin accessibility (N = 86). Among the chromatin accessibility profiles, there are 41 obtained with ATAC-seq and 45 with DNase-seq. In the cell group end, cancer is the largest group containing 107 profiles followed by mix, neuron, progenitor, glial cell, and circulation in sequence.

III. Experiment setup A. Network architecture
We adopted a network architecture that has been used previously for predicting functional genomic events with success [7]. As illustrated in Figure 2, the feature extractor consists of three stacked convolution blocks and the classification head includes two consecutive fully connected layers. All convolution filters in the network has a uniform size of 8 with a step size of 1. The number of filters in convolution layers and that of hidden units in fully connected layers are indicated in Figure 2. The window size of max pooling is also uniform throughout the network, which is 4 and with a step size of 4. The network takes as input an one-hot coded 1001bp long sequence and outputs the predicted probability of the input example being positive.

B. Compared clustering algorithms
Besides k-means, we explored in our experiment the utility of following clustering algorithms for cluster analysis in the proposed framework.
• Hierarchical clustering. There are two different types of hierarchical clustering,: top-down (devisive) and bottom-up (agglomerative). The later with the Ward's method to define inter-cluster similarity is the most popularly used in the literature and is used in this study. The time complexity of hierarchical clustering is O(T 3 ), thus is difficult to scale to large problems.
• Spectral clustering [33]. This algorithm is based on spectral graph theory and identifies clusters by searching for the minimum cuts of a graph into separate components, without strong assumption on the shape of the desired clusters, which is different from k-means.
• DBSCAN. This is a density based clustering algorithm that is immune to noise in the data, leading to clusters that do not necessarily have a globular shape.
• Sparse subspace clustering [34]. This algorithm does not utilize a pairwise similarity matrix that are relied on by all methods discussed above including k-means. Instead, sparse subspace clustering assumes the data sit in a union of subspaces and attempts to group data points from the same subspace into one cluster.

C. Compared baseline methods
To demonstrate the effectiveness of the proposed method of task grouping in the enhancement of model performance, we compared several baseline methods as listed below. The three previously developed methods for task grouping in deep multi-task learning [23]- [25] were not compared due to their poor scalability.
• Single task learning (STL). The model of each profile was trained separately, i.e., no sharing or any type of joint training at all among models.
• Simple multi-task learning (SMTL). Models of all profiles were trained jointly using hard parameter sharing, i.e., with a common feature extraction component but task (profile) specific classification head.
• Event-grouped multi-task learning (EMTL). Learning tasks were grouped by the modality of respective epigenetic events, resulting in three groups of tasks, i.e., histone modification, transcription factor binding, and chromatin accessibility. This is a domain knowledge based grouping, assuming that negative transfer happens mainly among events of distinct modalities.
• Cell-grouped multi-task learning (CMTL). Learning tasks were grouped by respective cell types, resulting in six groups of tasks (Table I). This is another domain knowledge based grouping, assuming that negative transfer happens mainly among different cell types.

D. Network training and model evaluation
To compose the dataset for training, we tiled the reference genome using 200bp bins, followed by labeling each bin according to called peaks in each profile. More specifically, a bin that contains an overlap with any of the peaks in a profile for at east 50 bps were labeled positive in the respective profile or otherwise, negative. To reduce dataset size while minimizing loss of discriminative signals carried in the data, we excluded bins that have no positive label in any of the profiles. These lead to a dataset consists of a total of 10,386,311 examples from 24 chromosomes, with 367 sets of binary labels corresponding to each of the 367 epigenetic profiles. Networks were trained to take the 1001bp long DNA fragment centered around the 200bp bin of each example to predict their respective 367 binary labels. The dataset was partitioned into three disjoint sets for training, validation, and testing, respectively. Specifi Each model in this study was trained up to 1,920k steps with an early stopping mechanism in place to minimize the training time. One round of forward and backward passing of a mini-batch of training data was considered a step. We used mini-batch size of 64 throughout our experiment and calculated the validation loss for every 16k steps. The early stopping works to terminate the training after completing certain number of more validations (denoted by δ, commonly referred as patience) from the step where the minimum validation loss was observed. To determine a safe δ to use, the full course of training were carried out for 15 representative profiles, five for each of the three event modalities with varying proportion of positive examples: small, medium, and large. The analysis of the collected validation loss values along the courses of trainings ( Figure  3) indicates that δ = 25 was safe to use for all profiles.
All perform very fined learning rate tunning for each profile in single task learning. Instead, fine tuning was performed for those representative profiles used above, leading to a fixed learning rate: 0.01 to use in all single task trainings. For all joint trainings, a tunning strategy including two phases were used: an initial search in the range from 0.05 to 0.35 with a step size of 0.05 followed by searching around the best value found in previous phase with a step size of 0.025. There is very large class imbalance in the dataset used in this study, with the vast majority of the profiles having a proportion of positive examples well below 2% ( Figure  4). It is known that accuracy is biased in the case of large class imbalance and cannot reflect the amount of intrinsic patterns (other than the class label distribution) that a model learned from the data for the prediction. Also, due to such large class imbalance, an inflation in area under ROC (AUC) was observed in our results, failing to differentiate the performance of compared methods. Therefore, in this paper, we used F1 score to evaluate the performance of all models.

A. Task grouping with k-means
As illustrated in Fig. 1, a single network with a common feature extraction component and task specific classification heads was trained jointly for all 367 tasks to obtain task representation for cluster analysis. To visualize the distribution of tasks in space, we performed principal component analysis (PCA) on the obtained task representation matrix E with rows representing tasks. The embeddings of all tasks in the two-dimensional space spanned by the first two principal components are shown in Figure 5A. According to this figure, there is clear clustering tendency among the tasks. As shown in Figures 5B and 5C, neither the event modality of profiles nor cell type align well with the clustering tendency. However, the event modality does correlate with it better than cell type. Most of clustering algorithms including k-means do not automatically determine the number of clusters (K) in a dataset and acquire a specific value for K by an input argument when they are used. K-means were run multiple times with different Ks as input, ranging from 2 to 10 with step size of one. For each obtained cluster solution, we trained models by grouped multi-task learning and explored how the model performance varies along with different Ks. The performance of all trained models are summarized in Figure 6A, which suggests the existence of substantial negative transfer in the joint training when K is below 4. This is evidenced by the significantly improved performance in models for predicting chromatin accessibility and transcription factor binding when K going from 2 to 4. Figure 6A also indicates when K goes above 4, the drop in cluster size (i.e., number of tasks in joint training) generally hurts the performance, especially among models for predicting transcription factor binding. Another important note is that among the three types of functional genomic events, chromatin accessibility is the easiest one to predict with an average F1 score above 0.42 in contrast to only around 0.34 for histone modification and transcription factor binding ( Figure 6A).
The four cluster solution contains two relatively large groups: Group 1 (N = 130) and 2 (N = 151) and another two groups in smaller size: Group 3 (N = 34) and 4 (N = 52). The distribution of all profiles across the four task groups is illustrated in Figure 5D, which shows the task grouping generally follows the clustering tendency in the two-dimensional space spanned by the first two principal components of matrix D. Table II provides the distribution of profiles by event modalities and cell types across the task groups. As shown in this table, Group 1 is primarily a histone modification group with the majority of the tasks (71.54%) to predict histone modification. Interestingly, all other tasks in this group are those to predict chromatin accessibility profiled with ATAC-seq. Group 2 is the largest group and contains tasks to predict events of diverse modalities, with over a half (56.67%) of the tasks predicting transcription factor binding. This group contains all the 45 tasks that predict chromatin accessibility profiled with DNase-seq. Group 3 is the smallest group among the four, containing only histone modification tasks. Similar to Group 3, Group 4 is another histone modification group with all but five tasks predicting histone modification. It is interesting to note that tasks predicting chromatin accessibility were very cleanly split into two groups: Group 1 and 2 by profiling techniques being used, i.e., ATAC-seq vs DNase-seq. This may suggest the existence of systemic but distinct technical bias in the two profiling techniques.
Comparing to the event modality, the concentration of cell types in the identified task groups is much less clear (bottom half of Table II). There are two groups with noticeable concentration of specific cell types. One is Group 2, over half (52.98%) of the tasks in which are for predicting varying epigenetic events in cancer cells. Group 4 is the other one, containing primarily tasks predicting events in neuron given that a large portion of the cells in bulk of tissue for sequencing are neurons. In terms of the distribution of cell types, Group 3 looks quite similar to Group 4, except that it is slightly less concentrated by tasks associated with neuron.

B. Comparison of clustering algorithms
To explore the utility of other clustering algorithms for task grouping, we ran each of the compared algorithms on the same matrix D to obtain four clusters (i.e., task groups). Models were subsequently trained for each group of tasks separately by multi-task learning as illustrated in Figure 1. The performance of all obtained models from the use of each clustering algorithm (including kmeans for easy comparison) is summarized in Figure 6B. Among the compared algorithms, k-means has the best overall performance. It is interesting to note that even  though all other algorithms perform more or less equally well on overall, there are significant difference in their performance when considering the prediction for different event modalities. A good example of this would be the sparse subspace clustering (SSC), which perform the worst for transcription factor binding but the best for histone modification among all compared algorithms including kmeans. These results suggest heterogeneity in the natural of optimal clusters (e.g.., not necessarily globular shape), demanding for new clustering approach that can effectively handle such heterogeneity.

C. Comparison with baseline methods
The performance of models resulted from each of the compared baseline methods is illustrated in Figure 6C, which also includes the performance of models obtained with the proposed method (specifically, those from the four-cluster solution with k-means, KMTL) for easy comparison. To further facilitate the comparison of the proposed method to baselines, we computed its improvement in model performance over those baselines and provided the results in Table III. KMTL has the best performance on overall, with high level of improvement comparing to single task learning (STL, 6.9%), simple multi-task learning (SMTL, 2.8%), and cell-grouped multitask learning (EMTL, 2.3%). By event modality, KMTL helps the most the tasks predicting transcriptional factor binding (TFB), enjoying 30.1%, 10.1%, 2.3%, and 2.4% improvement over STL, SMTL, CMTL, and event-grouped multi-task learning (EMTL), respectively. It is interesting to note that even though CMTL has the worst overall performance among the grouped MTL methods, it has the best performance for predicting histone modificaion (HM). This result suggests there is still room to improve in model performance by introducing flexibility into the task grouping, for example factoring cell type information. More detailed pairwise comparison between SMTL and STL and that between KMTL and STL are presented in Figure 7. Even though SMTL is generally helpful through positive transfer, there is clear indication of negative transfer in the joint learning, especially among tasks predicting transcription factor binding ( Figure 7A). Such a negative transfer is largely reduced in KMTL ( Figure 7B) where jointly training was applied only to tasks in the same group and importantly, without sacrficing postive transfer ( Figure 7C). These results demonstrate the effectiveness of the proposed method in identifying task grouping to address negative transfer in joint training.

V. Conclusion
In this paper, we developed a highly scalable framework for grouped multi-task learning to address negative transfer when simultaneously training deep neural networks for massive amount of epigenetic profiles. Our results demonstrate the effectiveness of the proposed framework and its superiority over baseline methods. Even though our framework was developed in the setting of predicting epigenetic events, it is a general multi-task learning approach and certainly applicable to other domains. One immediate future research would be to exploit obtained models in this study to help elucidate the functional impact of genetic variants identified in GWASs of psychiatric disorders.