{'fraction_non_alphanumeric': 0.041637667564378006, 'fraction_numerical': 0.009894998033104367, 'mean_word_length': 4.813192612137203, 'pattern_counts': {'":': 0, '<': 3, '<?xml version=': 0, '>': 1, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Elucidation of time-dependent systems biology cell response patterns with time course network enrichment

Advances in OMICS technologies emerged both massive expression data sets and huge networks modelling the molecular interplay of genes, RNAs, proteins and metabolites. Network enrichment methods combine these two data types to extract subnetwork responses from case/control setups. However, no methods exist to integrate time series data with networks, thus preventing the identification of time-dependent systems biology responses. We close this gap with Time Course Network Enrichment (TiCoNE). It combines a new kind of human-augmented clustering with a novel approach to network enrichment. It finds temporal expression prototypes that are mapped to a network and investigated for enriched prototype pairs interacting more often than expected by chance. Such patterns of temporal subnetwork co-enrichment can be compared between different conditions. With TiCoNE, we identified the first distinguishing temporal systems biology profiles in time series gene expression data of human lung cells after infection with Influenza and Rhino virus. TiCoNE is available online (https://ticone.compbio.sdu.dk) and as Cytoscape app in the Cytoscape App Store (http://apps.cytoscape.org/).


Introduction
Over the last decades, thanks to major advances and heavily decreasing costs in high-throughput experimental technology, we witnessed a sheer explosion in the amount and type of so-called OMICS data. The Gene Expression Omnibus (GEO) database, just to name an example, stores expression data for almost two million samples (Edgar, et al., 2002).
Amongst them we find an increasing amount of time-series data sets measuring features (here: gene expression) over multiple time points and under different conditions. At the same time, it becomes increasingly apparent that the measurements of activity of individual biological entities, such as transcription of genes or protein expression, are more helpful for understanding complex biological systems when analyzing them not only over time (Bar-Joseph, et al., 2012), but also in the context of their molecular interactions ( Barabasi and Oltvai, 2004;Bracken, et al., 2016). The collection of high-quality interaction networks is one of the main goals of systems biology, and substantial improvements in technologies lead to a rapid growth of interactome databases (Lehne and Schlitt, 2009), such as BioGRID (Chatr-Aryamontri, et al., 2015) or I2D (Brown and Jurisica, 2005). (TiCoNE) is a novel unsupervised computational approach to enrich experimental time series data for overrepresented patterns by combining human-augmented clustering with methods for unraveling interacting systems in biomolecular networks enriched with temporal expression patterns. We developed tailored statistical models to assess enrichment significance at different steps of data analysis (p*).

Figure 1. Overview. Time Course Network Enrichment
Suitable approaches to analyze either of these data types separately exist: (1) Integrating computational approaches with time-series profiling have proven successful in discovering novel biological insights (Chylek, et al., 2014;Reddy, et al., 2016). Discovery of enriched temporal patterns in time-series data sets is usually based on time-behavior similarities and mostly identified using standard clustering methods (Warren Liao, 2005). Only few dedicated time-perspective centered tools have been developed: One of them is STEM , a clustering approach for time-series data. Other approaches, such as SAM (Tusher, et al., 2001) or EDGE (Leek, et al., 2006), identify temporally differentially expressed genes. (2) In addition, de novo network enrichment methods can find subnetworks enriched with object (e.g. genes) that are significantly differentially expressed between two different experimental conditions. Examples for such approaches are DEGAS (Ulitsky, et al., 2008) and KeyPathwayMiner (Alcaraz, et al., 2014). Some methods exist that bridge between the two areas to a limited degree. VANTED ) is a systems biology framework, tackling a larger set of common bioinformatics tasks. This includes data visualization, and basic cluster analysis using Self Organizing Maps. It also allows to map cluster objects to a network and provides network visualizations. However, it does not provide more sophisticated network analysis functionality, such as de-novo network enrichment. NeAT  is a collection of programs that allow for basic network integration of cluster results. However, it does not provide integrated clustering functionality but only allows for the import of clustering results derived with external tools. NEAT  allows for the connectivity analysis of clusters on a network level. However, it does not provide any clustering facilities and expects pre-calculated clusters as inputs.
Overall, none of the existing approaches allow for the integrated analysis of differential time series expression clusters, with molecular interaction networks to illuminate systems biological response patterns of temporal resolution (see Supplementary Table 1 and 2).
Apart from the necessity to provide integrated approaches for time-series data with interaction networks, we firmly believe, that bioinformatics tools can only optimally support researchers in carrying out their analyses, if results are presented in an easily comprehendible way, and if researchers can incorporate their expert knowledge into the computational process.
Some approaches provide pre-defined visualizations for their results (see Supplementary Table 1 and 2), for instance NeAT , STEM , and VANTED . Clustrophile (Demiralp, 2016) also allows the user to interactively modify the two-dimensional data projection that is used to visualize the clustering result. However, almost none of the existing approaches expose interactive interfaces to the user and they derive their results in a purely automated manner. Few, so-called human-augmented clustering approaches exist, that allow the researcher to incorporate their knowledge into the clustering process. For instance, VISTA  allows for the interactive merging and splitting of clusters in the current clustering solution.
To fill these gaps, we have developed Time Course Network Enrichment (TiCoNE), a combination of a novel human-augmented time-series clustering method and a new kind of biological network enricher to identify putative biologically relevant regions in the network. Note that Ti-CoNE works with most kinds of biological entities (genes, proteins, RNAs, etc.) and most types of molecular measures acquirable for them, for instance, expression (transcriptomics, proteomics). As we focus on transcriptomics data later, and to increase readability, we simply refer to genes and gene expression in the following. We have summarized the most important features of TiCoNE in Supplementary Table 1.

Methods
The goal of any hard, partitional clustering method is to partition a given data set = { , … , } with objects into a clustering = { … }, with clusters ⊆ , where ⋃ ∈ = and ⋃ ( ) ≠ ∈ = ∅. We write instead of , if the data set is clear from the context. We denote the cluster of an object ∈ in clustering as . Since we focus on genes as objects in this work, we often refer to an object as a gene in the main text.

Input and Preprocessing
A time-series data set consists of time-dependent measurements for a collection of objects (here: genes), where replicates are measured for each object. We formalize the objects ∈ of a time-series data set as a collection of replicates = , … , , with each replicate being defined as a time series over time points: = , … , , , . In the remainder, the term "object" includes all of its replicate time-series.
The preprocessing of raw data is a highly setting-specific task, and it should thus be performed by the user prior to clustering in TiCoNE. Nevertheless, we provide two basic data preprocessing operations:

•
Remove baseline objects, i.e. = { ∈ |∃ ∈ … : < }, where is a user-given threshold for the average standard variance of objects to keep.

•
Remove objects with low agreement between replicates: where is a user-given threshold for the replicate agreement of objects to keep, and the replicate agreement of object as defined in the next section.

Similarities
The objective of clustering is to group the genes in such a way, that the time courses in one cluster are more similar to each other than to the time courses of other clusters. We denote the similarity of two genes as We support two measures to assess the similarity , between time courses of two genes , ∈ : (1) the (negative) Euclidean distance , and (2) the (scaled) Pearson correlation , . We define the negative Euclidian distance of two vectors , ∈ ℝ as , = − , = −√∑ − = and the scaled Pearson Correlation as , = ρ , + ⁄ .
The former assesses the similarity of the time courses according to the magnitudes, the latter according to their shape. We denote the case , = and call it the agreement of object 's replicates. The similarity of an object to a prototype can be defined as , = ⁄ ⋅ ∑ , = .

Initial Clustering & Iterations
TiCoNE performs iterations ∈ { … }, where F denotes the last performed iteration. We denote the clustering in iteration as = { , … , | | } , , where we refer to as the final clustering. If a clustering is denoted without iteration, we assume that = . TiCoNE integrates the following clustering methods to find an initial clustering with a user-specified number of clusters | |: CLARA Rousseeuw, 2008), k-Means (MacQueen, 1967), PAMK , STEM , or Transitivity Clustering (Wittkop, et al., 2010). For all clusters ∈ new prototypes are calculated.
can then be inspected and iteratively refined in later iterations resulting in clusterings , , … , .

Prototypes & Clustering Optimization Iterations
TiCoNE follows a prototype-based approach (Borgelt, 2006;Tan, 2006) similar to k-means (MacQueen, 1967). With such approaches, each cluster ∈ is represented by a prototype , which is a representation of objects (i.e. genes) ∈ . Usually, is derived from all ∈ using some aggregation function . We define = ( , , … , , ) with being the number of time points. We provide three aggregation functions in TiCoNE, where ∈ { … } and ∈ { … }: • The mean: At any stage during the clustering process, the user can let TiCoNE optimize the current clustering by either (1) performing one optimization iteration or (2) optimization iterations until convergence. Assume, we are in iteration with clustering . Execution of one optimization iteration then consists of performing the following two steps, where ∈ : (1) Reassign genes to the most similar prototype, i.e. + = argmax c∈C S O, c , where we break ties randomly.
(2) Recalculate prototypes + based on the new genes. a.
b. Clusters with identical prototypes are merged.
Iterations until convergence stops as soon as the following condition holds: + = + , i.e. either the clustering was not changed in the iteration, or we performed iterations.

Prototype Discretization
To give the user control over the tradeoff between sensitivity, i.e. detecting more and smaller clusters, and selectivity, i.e. finding less clusters of random noise, cluster prototypes are discretized and can only assume values values, and + is a set of + = | + | positive discretized values. We define the sets of discretized negative and positive values as follows: Where and denote the minimum and maximum input values observed for any gene at any time-point and replicate.

Human Augmentation Operations
With TiCoNE we offer a human augmented clustering approach, which implies that the user may (but does not have to) manually influence the clustering outcome. In contrast to fully automated clustering approaches, TiCoNE makes better use of field-specific knowledge that a fully automated procedure is not aware of. Human-augmented algorithms have been presented in several disciplines, such as robotics (Voyles, et al., 1999), linguistics (Chaiken and Foster, 2002), and also bioinformatics (Laczny, et al., 2015). In TiCoNE the user can modify a clustering in any of the following ways, where is the current iteration:

Merge clusters
A set of selected clusters ̂⊆ can be merged together into a new cluster . The new cluster is defined as = ⋃ ̂ and the new clustering as + = ( ̂) { }. The cluster's prototype is initialized appropriately.

Split clusters
A cluster ∈ can be split into a set of clusters ̂ where + = { } ̂. This can be done in two different ways: (1) use objects ≠ ∈ with argmin , , to calculate prototypes ̂ = and ̂ = for two new clusters ̂= { ̂, ̂}, or (2) objects ∈ can be clustered into a user-specified number of smaller clusters using one of the available clustering methods.

Keep a prototype
A cluster can be marked as "keep", here denoted as ⃗. If a cluster has been marked as keep in iteration it holds that ⃗⃗⃗⃗⃗ = + ⃗⃗⃗⃗⃗⃗⃗⃗⃗ and ⃗⃗⃗⃗⃗ ∈ ⇔ + ⃗⃗⃗⃗⃗⃗⃗⃗⃗ ∈ + , even if ⃗⃗⃗⃗⃗ = ∅. The "keep" flag can always be revoked in later iterations.

Add cluster with predefined prototype
A cluster with a user-specified prototype can be added to the clustering. Initially it holds that = ∅. To prevent immediate removal of from we set it to keep, i.e. = ⃗.

Delete cluster or its objects
The user can delete a cluster ∈ and (1) delete its genes from the data set and its prototype from the clustering. (2) Delete its prototype, but keep its genes. (3) Delete its genes but keep its prototype or (4) delete its leastfitting genes, i.e. + = { ∈ | , < }, where is a userdefined similarity threshold.

Handle least-fitting objects
The user has the possibility to remove genes from the clustering that are least similar to their respective prototypes, i.e. it holds that ∀ ∈ C: + = { ∈ | , < }, where is a user-defined similarity threshold. The deleted genes can either be (1) completely removed from the clustering or (2) clustered into a user-specified number of new clusters.

Clustering History & Reproducibility
Throughout the clustering process TiCoNE keeps a history of all performed steps. This includes automatically performed iterations, as well as human augmentations. At any time, the user can inspect clusterings , … , and, if desirable, revert the current clustering to a previous stage.
The history can be exported for publication documenting the clustering process.

Network Enrichment of Clusters
We provide two ways of identifying subnetworks ̂= ( ̂, ̂) ⊆ = , enriched in genes of a selected set of clusters ̂⊆ : (1) Extracting node-induced subnetworks with ̂= ⋃ ̂ and ̂= { ̂× ̂∈ }. Such networks are algorithmically easy to compute. However, this approach will fail to find connected networks if the genes of a cluster are not directly linked in the network; (2) In a biomedical setting it is reasonable to assume, that not all functionally related objects show a very similar time behavior and end up in the same cluster. Thus, we may want allow for a certain number of exception nodes that are not in the selected clusters but connect other objects that are. We use KeyPathwayMiner (Alcaraz, et al., 2014) to perform this task. It extracts a maximal connected subnetwork consisting only of genes from the selected clusters but a user-given number of exceptions.

Statistics
TiCoNE makes use of different fitness scores in different contexts to calculate empirical p-values, for example for clusters. If multiple fitness scores should be used to derive an overall p-value, a p-value is calculated for each fitness score independently, which are then combined into an overall p-value (see below).
For an observed instance (e.g. cluster), the p-value for one specific fitness score is calculated by comparing the original fitness against fitness scores of instances from a background distribution. The p-value is defined as the number of times, we observe fitness scores at least as extreme as the original one.
Let be a fitness score, let = , … , ) be the permuted instances, and let be the indicator function, defined as follows: In a one-sided setting, the p-value for and is then defined as: And in a two-sided setting as: We combine p-values of multiple fitness scores , … , for an instance by using a function : = ; , … , ; . TiCoNE combines p-values with = ∏ ; , , where PSEUDO refers to a pseudo count constant (default: 0.0001).

Generating background distributions
TiCoNE provides multiple ways of generating background distributions for p-value calculations.
1. Permuting Clusterings: We provide several ways of creating random clusterings: (PC1) Take a given clustering, shuffle the values of the cluster prototypes over time, reassign objects to shuffled prototypes. (PC2) Generate random prototypes and assign objects to those. (PC3) Take a given clustering, and randomly assign objects to the cluster prototypes while maintaining the cluster sizes. (PC4) Take a given clustering and randomly assign objects to the cluster prototypes while trying to maintain the sum of the node-degrees in each clusters, given the network. We sort all objects according to the node degrees in the network (descending). We then greedily assign the objects from top to bottom to random cluster, but only those that have not reached the sum of degrees of the original cluster. Note that this method yields comparable p-values to methods permuting the network using edge cross-over but with substantially reduced computing time.

Significance of Clusters
TiCoNE calculates empirical p-values for clusters based on the permutation scheme outlined above. The fitness scores for cluster p-values are the number of objects, = | | and the average object-prototype similarity, = = | | ⁄ ⋅ ∑ , ∈ . Thus, cluster p-values approximate the probability of observing a cluster at random with | | | | and . In our example study we use permutation method PC2 to generate random clusterings.

Time Course Network Co-Enrichment
TiCoNE allows for analyzing which pairs of clusters are connected more (less) strongly than expected in a given network (temporal cross-talk enrichment). We use our permutation scheme to calculate empirical p-values (either one-sided or two-sided) as well as the expected number of edges between the elements of two clusters in the network. We use the fitness score = → , in the case of directed networks and = ↔ , for undirected networks, where → , is the directed number of edges from cluster to cluster and ↔ , is the undirected number of edges between cluster to cluster .
We calculate the expected number of edges between clusters ↔ , (undirected) and → , (directed) as the average numbers observed across all permutations. We then calculate a log-odds-ratio for two clusters , ∈ as , = log → , → , ⁄ . Using this formula TiCoNE compares the expected number of edges with the observed number of edges for each cluster pair. In our example study we utilized method PC4 to generate a background distribution for co-enrichment p-values.

Phenotype Comparison
If biological entities have been analyzed under more than one experimental condition and are given as two data sets and , the first step is to cluster the objects for each setting separately and then compare the resulting clusterings and for differentially clustered object (e.g. gene) sets. TiCoNE can identify such objects by comparing each cluster ∈ with each cluster ∈ . It then presents these clusters with their normalized number of common objects , = , min | |, | | ⁄ and the similarity of their prototypes , . We assume that a cluster pair , from two different phenotypes is interesting, if and have more objects in common than expected by chance for a pair of random clusters with a prototype similarity at most this high. We calculate p-values for each pair of clusters , ∈ × using our permutation scheme and the fitness score = , c S c , . For our example study we generate random clusterings using permutation method PC2.

Network-Enrichment of Differential Phenotype
Once phenotypes have been compared, TiCoNE can identify subnetworks that are enriched in genes with a differential time behavior. Again, we employ the KeyPathwayMiner approach (see above) to identify all maximal connected subnetworks where all genes but a certain number of exceptions show the selected differential behavior. In our example study, we set the number of allowed exceptions to 0.

Synthetic Data
We generated a synthetic data set for illustration purposes, that consists of (1) genes belonging to three clusters with a distinct predefined time behavior (see Supplementary Figure 1) and (2) background genes with random time behavior. The time series signals of the cluster prototypes lie in the range of [− , ]. For each cluster we generated 80 objects, each with = time-series (replicates) and = time points. We required , . and ∀ ∈ .. , ∈ … : | − | .8 for all objects (i.e. genes) ∈ . Thus, the value range of the objects is [− .8, .8].
For the 120 background objects we generated time series with a random behavior, in the same value ranges and with the same number of samples.
The network describes interactions of the 240 genes of the three clusters and of the 120 randomly behaving background genes. To demonstrate how our co-enrichment feature works, we generated two clusters with similar node degree distributions, but converse connectivity to genes of the other clusters. See Supplementary Figure 1 for an illustration and its legend for details regarding the concrete patterns.

Clustering
We clustered the data set into 10 clusters using the CLARA clustering method, 15 negative and positive discretization steps, the Pearson correlation and cluster aggregation function = .

Influenza and Rhino Virus Data
We applied TiCoNE to human gene expression data measured with an Affymetrix Human Genome U219 Array containing expression levels for 49,386 probes over 10 time points (baseline and at 2hrs, 4hrs, 8hrs, 12hrs, 24hrs, 36hrs, 48hrs, 60hrs, 72hrs after infection) under three different experimental conditions: BEAS-2B lung cells have been infected with (1) Influenza virus, (2) Rhino virus, or (3) co-infected with both. The data set contained five biological replicates for all time points except the first one, for which it contained six. See ref. (Kim, et al., 2015) for more details. It is an ideal scenario for TiCoNE's phenotype comparison feature. Here, we focus on comparing (1) Influenza vs. (2) Rhino virus infection.
We mapped the probe set IDs of the data set to Entrez gene IDs. For genes with multiple probesets measured we kept only the one with the highest variance over time. The input values have already been log2-transformed such that we only normalized them against the control. Furthermore, we removed genes from the data sets, which were not present in the I2D network (see below).

Clustering
We clustered both, the expression data of the Influenza and the Rhino virus infection experiments with CLARA and 20 negative/positive discretization steps into 500 initial clusters. We used the Pearson correlation as similarity function, cluster aggregation function = , and removed genes not present in the I2D network (see below). We then ran automatic iterations until convergence.

Biological Network
We extracted the human interactome from the I2D protein-protein interaction (PPI) database (Brown and Jurisica, 2005), which integrates various other databases of known, experimental and predicted PPIs. We mapped Uniprot IDs to entrez gene IDs to be able to integrate the network with our example time-series data. After removing duplicate interactions between the same pair of genes the resulting network contains 199,025 interactions and 15,161 nodes (genes). Note that our TiCoNE approach works with any kind of graph loaded into Cytoscape or to the TiCoNE online platform. The use case determines the most appropriate network. Here, we aimed for finding human response protein complex formation candidates; hence our choice of using the I2D interactome. Figure 1 illustrates the structure and unique capabilities of Time Course Network Enrichment: 1) One may find subnetworks significantly enriched with genes of a similar temporal expression behavior. 2) The same can be applied to several such clusters of genes, i.e. multi-temporal network expression enrichment. 3) Furthermore, the crosstalk between genes over time can be analyzed on a systems level by computing the likelihood of observing more (or less) network interactions between pairs of time patterns by chances; a procedure we call network co-enrichment analysis. 4) The TiCoNE software computes empirical p-values and cross-talk graphs. 5) Finally, one may analyze multiple condition-specific time series experiments to identify overrepresented temporal patterns responding differentially to conditions in multiple experiments.

Results
To illustrate Time Course Network Enrichment we first utilize a synthetically generated data set, which we use for illustrative purposes only (Supplementary Figure 1, also to be found at the project web site). Afterwards, we will demonstrate its full power by analyzing whole-genome time series transcriptomics data of lung cells after infection with Influenza or Rhino virus together with the human protein interactome. This led to the identification of interesting overrepresented time-series patterns, for which we analyzed the interplay in the network. Finally, we compared the systems biology response between the two infection types.
The TiCoNE software is publicly available as a Cytoscape app (Shannon, et al., 2003), downloadable from the Cytoscape app store, and a feature-reduced web-version hosted at https://ticone.compbio.sdu.dk together with tutorials and screencasts.

Illustration on Synthetic Data
We illustrate TiCoNE's capabilities on artificial time course data and a corresponding synthetic network: We generated a time-series data set containing three object clusters and a set of random objects, RND. The three clusters exhibit a pre-specified time-behavior (prototypes P1-P3, see Supplementary Figure 1). Correspondingly, we create an interaction network of all objects with pre-specified connectivities between the clusters and random objects (Supplementary Figure 1). Initially, we clustered the data set into k=10 clusters, pretending not to know the real number of clusters. See Methods section for a detailed description of the data set and the used parameters. In the resulting clustering, the three hidden clusters are each split up into several highly similar subclusters. This is expected, since k=10>3. We used TiCoNE's humanaugmented merge function (see Methods) to reduce redundancy, and ended up with eight final clusters (Supplementary Figure 2a and Supplementary Table 1). For each of the clusters, TiCoNE computed a consensus pattern that we call prototype (Methods).
Three of the identified clusters show a high object-prototype Pearson correlation as well as a significant p-value. The other five clusters show lower similarities as well as less to non-significant p-values (see Supplementary Table 1). A manual investigation of the prototypes confirms that clusters 5, 9, and 11 correspond to the three gene clusters that we have hidden in the network and aimed to recover. However, several background objects have been clustered into our three foreground clusters: Cluster 5 (7), cluster 9 (11), and cluster 11 (14). Also, 4 objects of cluster 9 are misplaced into random clusters.

Differential Interactome Time-Response to Influenza and Rhino Virus Infection
After illustrating that Time Course Network Enrichment can retrieve artificially hidden temporal expression patterns in gene sets and networks, we apply it to a real data set in the following. We utilized human whole-genome gene expression data of lung cells after infection with Influenza virus and Rhino virus 22. The data set contains expression levels for 49,386 probes over ten time points (baseline and at 2hrs, 4hrs, 8hrs, 12hrs, 24hrs, 36hrs, 48hrs, 60hrs, 72hrs after infection) with five replicates for each time point. We concentrated on two experimental conditions: BEAS-2B lung cells that have been infected (1) with Influenza virus and (2) with Rhino virus. See Methods for more details. We clustered both, the expression data of the Influenza infection (shortly "Influenza data" in the following) and the Rhino virus infection experiments (shortly "Rhino data") into 500 clusters. Since our method iteratively merges clusters with equivalent prototypes we ended up with 173 Influenza clusters (88 with p≤ 0.05) and 225 Rhino clusters (71 with p≤ 0.05) (see Supplementary Figures 3a and 3b for selections of significant clusters). All clusters have a time course behavior distinct from each other and most prototypes peak multiple times into different directions.
For each data set we performed a time course network co-enrichment analysis. When comparing the distributions of undirected edge counts between cluster pairs (see Supplementary Figure 7), the Influenza clusters (i.e. clusters in the "Influenza data") represent gene groups that are more tightly connected in the network (more undirected edges) than the Rhino clusters (i.e. clusters in the "Rhino data"). For both data sets, log-odds scores are slightly left skewed normally distributed, while overall cluster pairs of the Rhino data tend to be more depleted in edges than Influenza cluster pairs. We arranged clusters for both data sets according to their most significant connectivities (p≤ 0.02) on a timeline (see Supplementary   Figures 4 and 5). Generally, Influenza clusters with significant connectivities (either depleted or enriched) form a large connected complex along the timeline. In contrast, such significant Rhino cluster pairs seem to be more fragmented and form multiple separated cluster complexes. Out of the 61 significant Influenza data cluster pairs, we see more often an enriched of the number of edges (39/61) than for the 34 significant Rhino data cluster pairs (19/34).

Discussion
We developed Time Course Network Enrichment analysis, the first method to seamlessly identify regions in biomolecular networks that are enriched in genes (or proteins, metabolites) with significantly overrepresented time-series behavior or time-series co-expression behavior. Our TiCoNE implementation offers a human-augmented cluster optimization strategy and allows the user to iteratively refine the clustering automatically or visually by applying operations such as adding, deleting, merging or splitting clusters. Our approach can compare different phenotypes and identify differentially behaving network complexes. TiCoNE computes empirical p-values for clusters, phenotype comparisons and network coenrichment based on different kinds of permutation tests (see Methods). We illustrated the principle of Time Course Network Enrichment on synthetically generated data. The power of TiCoNE was then demonstrated by processing time-series gene expression data of human host lung cells, infected with Rhino virus or Influenza. Here, we use our co-enrichment analysis approach to construct complexes of clusters along a timeline that are biologically meaningful and may explain the systemic unraveling of host immune response to Influenza virus infection. We find specific properties of co-enrichments of Influenza and Rhino data clusters, which may help explain the large difference in severity of the two viruses on a systems biology level.
Additionally, we discovered de novo groups of genes that behave consistently but show temporally different behavior under the two conditions. By integrating these candidates with the human protein-protein interaction network we discovered a complex of 50 genes, out of which 30 have been previously associated with Influenza virus infection. We consequently hypothesize that the several of the remaining genes are promising candidates of also being involved in the Influenza-specific response of the human immune system.
In conclusion, with Time Course Network Enrichment we close an important gap in systems biology research by providing the first method to integrate time series data directly with molecular biological networks in a sophisticated manner. In addition, it can compare identified temporal systems biology patterns of cellular responses between two different conditions or phenotypes. It also implements a novel human-augmented approach to clustering time series data into patterns. Finally, note that TiCoNE cannot just handle time-resolved data, but in principle is directly applicable as well to multi-condition (instead of multi-time point) expression data. It is publicly available in the Cytoscape app store and can be applied to any kind of expression data and molecular network loaded into Cytoscape. At the project web site (ticone.compbio.sdu.dk) we also offer access to a feature-reduced version of TiCoNE as well as access to example data and screen cast videos describing the TiCoNE usage click by click.

Funding
Conflict of Interest: none declared.