{'fraction_non_alphanumeric': 0.028893125223400453, 'fraction_numerical': 0.023144286905754794, 'mean_word_length': 4.833709817549956, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 1, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Beyond the Twilight Zone: Automated prediction of structural properties of proteins by recursive neural networks and remote homology information

The prediction of 1D structural properties of proteins is an important step toward the prediction of protein structure and function, not only in the ab initio case but also when homology information to known structures is available. Despite this the vast majority of 1D predictors do not incorporate homology information into the prediction process. We develop a novel structural alignment method, SAMD, which we use to build alignments of putative remote homologues that we compress into templates of structural frequency profiles. We use these templates as additional input to ensembles of recursive neural networks, which we specialise for the prediction of query sequences that show only remote homology to any Protein Data Bank structure. We predict four 1D structural properties – secondary structure, relative solvent accessibility, backbone structural motifs, and contact density. Secondary structure prediction accuracy, tested by five‐fold cross‐validation on a large set of proteins allowing less than 25% sequence identity between training and test set and query sequences and templates, exceeds 82%, outperforming its ab initio counterpart, other state‐of‐the‐art secondary structure predictors (Jpred 3 and PSIPRED) and two other systems based on PSI‐BLAST and COMPASS templates. We show that structural information from homologues improves prediction accuracy well beyond the Twilight Zone of sequence similarity, even below 5% sequence identity, for all four structural properties. Significant improvement over the extraction of structural information directly from PDB templates suggests that the combination of sequence and template information is more informative than templates alone. Proteins 2009. © 2009 Wiley‐Liss, Inc.


INTRODUCTION
The three-dimensional (3D) structure of a protein provides important information about its ligand binding, catalytic sites and protein-protein interactions. Compared with over 7 million known protein sequences, as of December 2008 there are only about 50,000 proteins of known structure deposited in the Protein Data Bank (PDB). 1 As experimental determination of a protein's structure is difficult, expensive, and time consuming the gap between sequence-known and structure-known proteins is growing.
Structural genomic (SG) projects aim to narrow this gap by obtaining useful 3D models of all protein families by a combination of experimental structure determination and comparative model building 2 in a high-throughput manner. 3 This makes prediction methods that incorporate structural information increasingly important. If the SG goal is achieved, template structures will be available for most proteins.
Structure prediction methods can be divided into two groups: those that use similarity to proteins of known structure to model all or part of the query protein and ab initio prediction methods where no similarity to a protein of known structure can be found. If a close homologue can be found (e.g., a protein of known structure with a sequence identity greater than approximately 30% to the query), then a model can be produced with a high degree of confidence in its accuracy. 4 However, many proteins share similar structures even though their sequences may share less than 15% sequence identity. 5 Finding these remote homologues is a much more difficult task with two major issues needing to be addressed: finding the best true homologue and building the correct alignment. Many methods have been developed to address both problems, most of which specialise on different degrees of sequence identity between query and template.

ABSTRACT
The prediction of 1D structural properties of proteins is an important step toward the prediction of protein structure and function, not only in the ab initio case but also when homology information to known structures is available. Despite this the vast majority of 1D predictors do not incorporate homology information into the prediction process. We develop a novel structural alignment method, SAMD, which we use to build alignments of putative remote homologues that we compress into templates of structural frequency profiles. We use these templates as additional input to ensembles of recursive neural networks, which we specialise for the prediction of query sequences that show only remote homology to any Protein Data Bank structure. We predict four 1D structural properties -secondary structure, relative solvent accessibility, backbone structural motifs, and contact density. Secondary structure prediction accuracy, tested by five-fold cross-validation on a large set of proteins allowing less than 25% sequence identity between training and test set and query sequences and templates, exceeds 82%, outperforming its ab initio counterpart, other state-of-the-art secondary structure predictors (Jpred 3 and PSIPRED) and two other systems based on PSI-BLAST and COMPASS templates. We show that structural information from homologues improves prediction accuracy well beyond the Twilight Zone of sequence similarity, even below 5% sequence identity, for all four structural properties. Significant improvement over the extraction of structural information directly from PDB templates suggests that the combination of sequence and template information is more informative than templates alone.
Sequence-based methods compare a query sequence with a database of target sequences. If there is a close homologue to the query these methods can generally find it and align it to the query, fast. Two of the most popular of such methods are FASTA 6 and BLAST. 7 Profile-sequence methods, which use a positionspecific scoring matrix (PSSM) or profile hidden Markov models (HMM) to search a sequence database are more sensitive at detecting remote homology if the pairwise sequence identity is in the Twilight Zone (between approximately 20% and 30%). Examples of the more popular profile-sequence methods are PSI-BLAST, 7 HMMer, 8 and SAM. 9 Profile-profile methods such as prof_sim, 10 HHSearch, 11 and COMPASS 12 use a profile to search a database of profiles, enabling them to identify even more remote evolutionary relationships.
The combination of sequence and structure has been shown to improve fold recognition and alignment quality especially in the case where pairwise sequence identity drops below the Twilight Zone but how best to include this information remains uncertain. Methods that include secondary structure, 13,14 solvent accessibility, 15 or both 16 have been developed. The addition of other information has also been shown to be of benefit, for example, more-detailed alphabets of local protein structure. 17 The prediction of 1D structural properties of proteins (i.e., properties that may be represented as a string of the same length as the amino acid sequence) is an intermediate step toward the prediction of protein structure and function. The incorporation of such local structural properties can improve the performance of alignment algorithms and can lead to the detection of more remote homologues, thus improving the accuracy of structure prediction methods. Until recently, predictors of 1D structural properties have generally been ab initio. However, it is clear that homology information can contribute to more accurate 1D predictions. 30,31 Previously 31 we have described a system (Distill_H) based on bidirectional recursive neural networks (BRNN), 32 which relies on homologues detected by PSI-BLAST to predict a protein's secondary structure (SS) and relative solvent accessibility (RSA). Here, we develop a system to deal with situations where no clear homo-logue can be found, that is there is no template exceeding 25% sequence identity to the query sequence. The system has two stages: one in which four 1D structural properties are predicted ab initio and the predictions are combined into a profile, which is used to search for remote homologues; the second stage, in which a specialized machine learning system harnesses information from remote homologues and the query's sequence to output a refined prediction for the same four 1D properties. The properties we predict are SS; RSA; an alphabet of 14 backbone structural motifs (SM) 33 ; and contact density (CD). 34 We first compare three different methods for homology detection: PSI-BLAST, COMPASS (a popular profileprofile method), 12 and SAMD, the novel algorithm for remote homology detection we present here. We search for templates by these three methods and investigate the improvements they yield over ab initio predictions when used as input to Distill_H. Next, we develop a novel system, Distill_RH, trained on SAMD templates, and compare it with its state-of-the-art ab initio counterpart, Distill, 35 and to a baseline that copies the SS, RSA, SM, and CD directly from the best template, for each of the four 1D predictors. Finally, we compare Distill_RH SS prediction accuracy with that of PSIPRED, 36 Jpred 3, 37 and PROTEUS, 30 the only other publicly available SS predictor that we are aware of which exploits similarity to proteins of known structure during the prediction process.
We show that SAMD templates greatly improve prediction accuracy over ab initio, well beyond the case in which clear homology is available (essentially, down to any level of sequence identity), and substantially more than PSI-BLAST and COMPASS templates, while significantly improving over the best PDB template. We demonstrate that Distill_RH predictions, incorporating these SAMD templates, are significantly more accurate than Distill, Distill_H, PSIPRED, Jpred 3, and PROTEUS.

Datasets
The dataset used to train and test our predictors is extracted from the January 2007 25% pdb_select list. 38 We assign each residue's SS, RSA, and / and w dihedral angles using DSSP. 39 We remove all sequences for which DSSP does not produce an output. The final set (S3129) contains 3129 proteins and 461,633 amino acids. SS is mapped from the eight DSSP classes into three classes as follows: H, G, I ? Helix; E, B ? Strand; S, T, . ? Coil. RSA is mapped into four roughly equal classes: completely buried (0-4% exposed), partly buried (4-25% exposed), partly exposed (25-50%), and exposed (more than 50%). SM 33 are constructed by mapping tetra peptides, represented as vectors of / and w angles, into 14 conformational clusters determined by Sims et al. 40 CD 34 is defined as the principal eigenvector of a protein's residue contact map at 8 Å , multiplied by the principal eigenvalue, and is assigned to one of four roughly equal classes, corresponding to very low, medium-low, medium-high, and very high CD. 34 We use this set to train all systems, in five-fold cross-validation. A second dataset (for unbiased testing) is extracted from the October 2007 25% pdb_select list. None of the 555 sequences in this dataset have more than 30% sequence identity to any sequence in S3129. S555 is then processed in the same way as S3129.
Multiple sequence alignments (MSA) for S3129, S555, and every sequence in the October 2007 25% pdb_select list are extracted from a redundancy reduced version of the NR database containing 1.05 million sequences. The alignments and PSSM are generated by three runs of PSI-BLAST with parameters b 5 3000 (maximum number of hits), e 5 10 23 (expectation of a random hit) and h 5 10 210 (expectation of a random hit for sequences used to generate the PSSM). MSA for the October 2007 25% pdb_select list become the target database for COM-PASS alignments after preprocessing by that method.

Predictive architectures
To learn the mapping between our input space I and output space O, we use two-layered architectures composed of BRNN 32 of the same length N as the amino acid sequence. Similar to Pollastri and McLysaght 41 and Vullo et al. 34 we use BRNNs with shortcut connections.
These networks take the form: Input i j associated with the j-th residue contains primary sequence information and evolutionary information, and direct structural information derived from PDB templates: where e units are devoted to sequence and evolutionary information, and t to structural information.
In the case of SS prediction, we use t 5 10 for representing structural information from the templates. The first eight structural input units contain the average eight-class (DSSP style) SS composition in the PDB templates, whereas the last two encode the average sequence identity of the template column to the query and average coverage of the query by the templates. The averages are weighed by the cubed sequence identity of the query and the PDB template, and by the inverse of the quality of the template, measured as X-ray resolution 1 R-factor/ 20, as in Hobohm et al. 38 Template information for RSA, SM, and CD is encoded similar to SS, except that 4, 14 and 4 units are adopted, respectively, to represent average RSA, SM, and CD from PDB-derived templates. The two units encoding the profile quality are the same as in the SS case. When no template is available for a residue, the template section of the input is left blank.
Each sequence-to-structural feature BRNN is cascaded with a structure-to-structure BRNN. 41 Both BRNNs are trained at the same time, but supervised independently. For each prediction task, five two-stage BRNN models are trained independently and ensemble averaged to build the final predictors. Training SE005 after 1000 epochs are performed for each model; the learning rate is halved every time we do not observe a reduction of the error for more than 50 epochs. The number of free parameters per model ranges between 5800 and 8000. Template-based models are only on average 7% larger than the corresponding ab initio ones.
All 1D predictors are trained and tested using five-fold cross-validation procedure on S3129. The five folds are of roughly equal sizes, composed of 625 or 626 proteins, and ranging between 91,049 and 93,474 residues.
We train in identical conditions (training/test set splits, training strategy, network architectures) three different systems: Distill, in which the template-based part of the input is left blank; Distill_H, in which templates are obtained via PSI-BLAST; Distill_RH, in which templates are obtained by a novel remote homology detection algorithm, SAMD, which we describe below.

Building alignments based on structural information
We combine SS (3 classes), RSA (4 classes), SM (14), and CD (4) structural properties into a set of 672 features (3 3 4 3 14 3 4), which we call SAMDs (Secondary Structure, Solvent Accessibility, Structural Motif and Contact Density). The SAMD are predicted ab initio for the query, and then aligned against the SAMDs in the target set (the PDB, or part thereof) to detect homologues. We use a basic local alignment dynamic programming 42 approach to align query and target sequences. Similar to Gong and Rose 17 the alignment score between two residues i and j is composed of two parts: Pði; jÞ ¼ PSSMði; jÞ þ SAMDði; jÞ ð 2Þ The sequence penalty for aligning amino acid i against amino acid j is obtained from a position-specific scoring matrix (PSSM) (see Datasets section). The penalty for aligning the structural properties, SAMD i against SAMD j, is obtained from a structural (SAMD) substitution matrix. We add SAMD and PSSM scores without any relative weighing. The gap penalty for the SAMD matrix is fixed at -15 and the penalty for the PSSM is the minimum penalty value of the matrix. The total gap penalty is the sum of the two.

SAMD substitution matrix
The choice of substitution matrix greatly affects the quality of any pairwise alignment method. 43 The most popular matrices, Pam 44 and BLOSUM, 45 have been used successfully to identify homologues when sequence identity exceeds approximately 30%. 7 More specialized matrices have been shown to improve the rate of detection of more remote homologues, for example the environment-specific substitution tables of Fugue, 19 matrices based on backbone dihedral angles, 17 matrices for different sequence-structure contexts 46 or matrices created from structurally aligned protein pairs. 47-49 To create an SAMD substitution matrix, we use the formalization of Henikoff and Henikoff 45 and construct a BLOSUM-style matrix. We start with the structural annotation of the alignments in the BLOCKS database 50 (version 14.2, March 2006). We label each sequence in a block with its SAMDs, extracted from the corresponding PDB file. We count all possible pairs of SAMDs in each column of every block. The result is a frequency table listing the number of times each SAMD pair occurs. This table is used to calculate a 672 3 672 matrix that represents the log odds ratio between these observed frequencies and those expected to occur randomly.

Estimating statistical significance
To determine if a score does in fact suggest that two sequences share a common ancestor it needs to be compared with a background of random scores and supported by some measure of statistical significance. 51 Much work has been done to develop and improve such methods. 52-55 These methods assume that gapped local alignments scores follow the same extreme value or Gumbel distribution 56 as gap-less local alignments: In this case, where the substitution matrix is very large (672 3 672), an extreme value distribution is used to approximate the distribution of alignment scores. The e, or expectation, value is a measure of the reliability of the alignment score, or how likely it is for a score equal to or greater than the given score to occur by chance in the database being searched using a given scoring system: where x is the score and m and n are the query and target sequence lengths. K and k are parameters that depend on the amino acid compositions of the sequences and on the scoring system, and therefore need to be re-estimated for every new scoring system used. 53,54 The key to using these two equations is the accurate calculation of K and k. For this purpose, we use the method of Bailey and Gribskov 54 based on maximum likelihood estimation and the values are recalculated for ever sequence based on the size of the target database.

Template comparison
We use S3129 to train Distill_H in five-fold cross-validation. Templates are obtained by running a round of PSI-BLAST against the PDB (available on March 25, 2008) using a PSSM generated against the NR database (see Datasets section) with an expectation cutoff of 10. To train template-based predictions in marginal similarity conditions, we exclude all hits that have a PSI-BLAST hit exceeding 20% sequence identity to the query sequence. We compare the performance of PSI-BLAST-based templates, COMPASS-based templates, and SAMD templates on Distill_H for SS prediction.
To obtain unbiased results, we use S555 as our test set. S555 is a subset of the October 2007 25% pdb_select list 38 and none of the sequences in S555 share more than 30% sequence identity to any sequence in S3129. We use this pdb_select list as the target template database. Since we exclude a protein's own PDB file from its list of templates, we do not expect to find any templates showing more than 25% identity to the query, except for short sequences for which the 25% threshold is relaxed.
As the choice of the e value cutoff is critical to the performance of any alignment method, we compare the three methods with seven different values of e and test the effect this has on SS prediction accuracy. We start with a very low e value cutoff of e 5 0.0005. This gives each of the three methods an opportunity to find good templates first. If at least one template is found we stop searching, otherwise we gradually increase the e value six further times (e 5 0.001, 0.01, 0.1, 1, 10, and 50) until at least one template is found or the e value cutoff reaches 50. After these seven rounds COMPASS finds templates for 554 of the 555 sequences, PSI-BLAST finds templates for 545 sequences and SAMD finds templates for all of the sequences. This results in 73.28% of the 81,141 residues in S555 being covered by a COMPASS template, 51.37% are covered by a PSI-BLAST template and 53.15% by an SAMD template. Table I shows the accuracy of the best template for each template type compared with the accuracy of Distill_H when these templates are used during the prediction process. Although COMPASS covers more than 20% more residues than PSI-BLAST the template accuracy and Distill_H SS prediction accuracy is similar. In contrast, SAMD templates cover a similar number of residues to PSI-BLAST but the templates are over 10% more accurate and predictions using these templates are more than 4% more accurate than either COMPASS or PSI-BLAST. Figure 1 shows the distribution of SS prediction accuracy for S555 as a function of sequence identity to the best COMPASS, PSI-BLAST, and SAMD template for each query sequence. For predictions using COMPASS templates with less than 15% sequence identity between the query sequence and the template sequence, ab initio predictions are more accurate than template-based predictions. For PSI-BLAST templates, ab initio predictions are more accurate at just under 10% sequence identity. However, for SAMD templates, the Distill_H predictions are always more accurate demonstrating that the SAMD templates carry informative structural information that can be exploited by the Distill_H predictive system even with little or no sequence identity between query and template.
In the 0 to 10% range of sequence identity SAMD finds nearly five times as many templates as either PSI-BLAST or COMPASS; however, these templates are on average less than half of the length of the COMPASS ones. This suggests that, at least in some cases, we are finding local fragments of templates that have similar 1D features to the query rather than finding a template that is overall correct. As the sequence identity of the templates increases so does template length for both SAMD and PSI-BLAST, however, COMPASS remains constant at an average template length of 87.46 residues.

Distill_RH vs. Distill
We have shown that low-sequence identity SAMD templates can be used to improve on ab initio SS prediction accuracy and that these gains in accuracy are greater than either PSI-BLAST or COMPASS templates under similar conditions. To investigate whether the SAMD templates can yield similar improvement for other structural property prediction (RSA, SM, and CD) as for SS,

Figure 1
Distribution of secondary structure prediction accuracy for S555 as a function of sequence identity to the best COMPASS, PSI-BLAST, and SAMD template for each query sequence. The broken line represents predictions using templates, the solid line template-less, or ab initio, predictions.

Protein Structure Prediction with Templates
and to investigate if we could further increase gains over ab initio SS predictions with additional specialization of Distill_H, we trained Distill_H with SAMD templates but with a relaxed sequence identity threshold of 25% and a reduced expectation cutoff of e 5 0.1 as follows.
We retrain Distill_H in five-fold cross-validation on S3129 with SAMD templates and term it Distill_RH. The target database for template searching is the PDB available on March 25, 2008. To train template-based predictions in marginal similarity conditions, we remove from the PDB all sequences that have a PSI-BLAST hit exceeding 25% sequence identity to the query, before searching for SAMD templates, we then search the PDB for templates with an expectation cutoff of e 5 0.1. We relax the 20% sequence identity threshold used previously for training Distill_H to 25% -given the different way the templates are obtained (redundancy reduced beforehand, or after the PSI-BLAST search), this results in very similar distributions of template identity in the two cases. We test both predictors with identical sequence identity threshold and expectation cutoff.
We test the performance of Distill_RH on S555 and compare the results to ab initio (Distill) predictions. Templates for S555 are created using three rounds of SAMD template generation with increasing e value cutoffs (e 5 0.00005, 0.1, 50), stopping as soon as we find at least one template using the October 2007 25% pdb_select list as the template database. Figure 2 shows the distribution of the best and the average sequence identity of the SAMD templates for S555. For over a third of these sequences the average sequence identity between the query and the template is 5% or less. Figure 3 shows the distribution of SS, RSA, SM, and CD prediction accuracy for S555 as a function of the sequence identity of the query to its best SAMD template, for both Distill_H and Distill_RH. For SS and SM, Distill_RH outperforms Distill even as the sequence identity between the query and the top-ranked template approaches zero. RSA and CD are the least responsive to template input, possibly due to their less conserved nature. Predictions do improve by 1.35% and 2.91% on average in the 10% to 30% range, this represents a real gain in accuracy of 2.42% and 5.29%. If the sequence identity between the query and the best template is less than 10%, the ab initio predictions are marginally more accurate. For SS in the 0% to 10% range, predictions with SAMD templates exceed ab initio predictions on average by 3.12% and are nearly 4.54% better in the 10% to 30% range, representing real gains of 4.04% and 5.67%. SM SAMD predictions improve on ab initio if the sequence identity between query and template exceeds on average 3%. In the 10% to 30% range, they improve on ab initio predictions by 3.79%, a real gain of 6.41%.
When we consider only the residues that are covered by a template (73.55% of 81,141) the Distill_RH improvement over ab initio Distill predictions is substantial. Here, SS predictions are on average over 4.3% more accurate than ab initio, regardless of the sequence identity between that query and the template. Note that sequence identity may be as low as zero and the e value as high as 50. SM and CD show gains of 2.48% and 1.32%, respectively. Again RSA shows less improvement at just 0.26%.
We also compare Distill_RH with a baseline that copies the SS, RSA, SM, and CD directly from the best template (Table II). In every case, Distill_RH predictions for these same residues outperform the baseline. Distill_RH is 7.2%, 11.47%, 8.44%, and 3.1% more accurate than the baseline for SS, RSA, SM, and CD, respectively, a real improvement in accuracy of 8.99%, 25.16%, 13.36%, and 6.04%. The largest gain over the baseline is for RSAeven where the templates are substantially less accurate than the ab initio prediction Distill_RH can harness their information to improve on ab initio prediction accuracy.

Comparison to other methods
We compare SS predictions by Distill, Distill_H and Distill_RH with SS predictions by PROTEUS, 30 PSIPRED, 36 and Jpred 3 37 using S555 as our test set (Table III). We create two sets of templates, PSI-BLAST for Distill_H and SAMD for Distill_RH, as previously described using the October 2007 25% pdb_select list as the target database for the first set and a full version of the PDB supplied with PROTEUS as the target database for the second set. Although Distill_RH incorporates template information, only templates showing less than 25% sequence identity to a query are allowed for the first set. Since this is as strict as the usual separation between training and test set for ab initio predictors, we consider Sequence identity between query sequences and the best, and the average of all, SAMD templates for S555.

Figure 3
Distribution of SS, RSA, SM, and CD prediction accuracy for S555 as a function of sequence identity to the best SAMD template for each query sequence. The broken line represent predictions using templates, the solid line template-less, or ab initio, predictions (Distill_H on the left and Distill_RH on the right). The error in the 0 to 10% range is 0.18 to 0.26%, in the 10 to 20% range it is 0.23 to 0.32% and in the 20 to 30% range it is 0.3 to 0.53%. All improvements for Distill_RH over Distill are statistically significant except for RSA and CD in the 0 to 10% range. Distill_H's improvements over Distill are not statistically significant in the 0 to 10% range for any predictor, and in the 10 to 20% range for RSA. the results on the first line of Table III to be a fair comparison.
PROTEUS is designed to incorporate structural information into SS prediction, but only when a homologue with more than 25% sequence identity or an e value of e < 10 27 is found, hence, when we use the pdb_select as the target database, its predictions are generally ab initio. Proteus predictions using the full version of the PDB are more accurate, 81.86%, than our ab initio predictions but still less accurate than either Distill_H (86.12%) or Dis-till_RH (87.60%) when they use this version of the PDB. This again highlights the fact that a soft combination of sequence profiles and structural templates greatly improves prediction accuracy compared with directly extracting structural information from PDB templates as in the 3D-to-2D mapping method of PROTEUS.
We also show that Distill_H and Distill_RH perform well compared with other methods that do not incorporate structural information, PSIPRED and Jpred 3. PSIPRED is slightly more accurate than the ab initio Distill but Distill_RH is 2.71% more accurate using only remote homologues. This increases to 7.78% when we use the full version of the PDB (we exclude the query sequence from its list of templates). Jpred 3 is also slightly more accurate than our ab initio Distill (80.12%) using their ''easier'' DSSP eight to three class conversion: H ? Helix; E, B ? Strand; G, I, S, T, . ? Coil. Again, both our Distill_H and Distill_RH outperform Jpred 3 using either the ''easier'' or ''hard'' SS assignment.
The overall correct prediction of SS for Distill_RH is 82.53%, 3.01% more accurate than the ab initio Distill, an improvement of 3.78%. To estimate the statistical significance of this result, we measure the standard deviation of the error distribution by sampling with replacement N residues from the S555 set M times. In our case, M 5 1000 and N 5 81141 (the size of the set). Standard deviation can be estimated from N samples x i and from their average x. We obtain nearly identical standard deviations of 0.14% and 0.13% for Distill and Distill_RH, respectively, for the error of both predictors. Given these deviations the observed difference of 3.01% is significant at p ( 0.01.
In Table IV we show that Distill_H and Distill_RH both perform well when compared with other well known state-of-the-art RSA prediction methods. Our ab initio predictor, Distill, is 1.72% better than the next best methods 57,58 ; however, this increases to 3.5% and 4.18% for Distill_H and Distill_RH, respectively. The only RSA prediction method that incorporates structural information from the PDB into the prediction process that we are aware of is ACCpro. 59 In this case, BLAST is used to identify homologs with high sequence identity in the PDB to improve ACCpro predictions. When highsequence identity templates are used ACCpro prediction accuracy is > 77%.

CONCLUSION AND FUTURE DIRECTIONS
1D structural properties of protein residues are useful intermediate representations between the primary sequence and the full 3D structure. Traditionally, predictors of 1D structural properties have been ab initio. However, as the universe of known folds expands, any detectable degree of similarity to proteins of known structure needs to be fully exploited. In turn, improved predictions may feed into, and help to further improve, comparative modelling, and fold recognition systems.
We have developed a high-throughput system for the prediction of 1D structural properties of proteins that takes advantage of similarity to proteins of known structure. We retrieve template information by a novel structure-based algorithm that we have developed in this work, and obtain improved predictions of 1D structural properties well below the Twilight Zone of sequence identity, down to sequence identities of less than 5% in some cases. This suggests that our approach is very robust with respect to template noise, and may glean information beyond the case where templates represent genuine homologues. Predicted 1D properties are consid-  Accurcy measured for residues covered by the best template. erably more accurate than those directly derived from the best template, showing that sequence and evolutionary information can help correct errors associated with lowsequence identity templates. The methods we presented also deal naturally with predictions in areas not covered by templates. We are currently looking into a number of directions of further research, namely under which conditions SAMD templates are genuine homologues and when they are fragments of local structure similar to the query but not from an overall similar fold; whether SAMD-based predictions may be fed back into the loop to refine SAMD templates; whether SAMD-based predictions may be of help in 3D modelling, that is whether the refinement of local structure they yield is also captured by fold recognition systems or not.
All the systems are automated and are publicly available at http://distill.ucd.ie/. When appropriate templates are available they are automatically used in the prediction process.