{'fraction_non_alphanumeric': 0.023333333333333334, 'fraction_numerical': 0.012222222222222223, 'mean_word_length': 5.78643216080402, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Deduplication Improves Cost-Efficiency and Yields of De novo Assembly and Binning of Shot-Gun Metagenomes in Microbiome Research

Metagenomics has in the last decade greatly revolutionized the study of microbial communities. However, the presence of artificial duplicate reads mainly raised from the preparation of metagenomic DNA sequencing library and their impacts on metagenomic assembly and binning have never brought to the attention. Here, we explicitly investigated the effects of duplicate reads on metagenomic assembly and binning, based on analyses of four groups of representative metagenomes with distinct microbiome complexity. Our results showed that deduplication considerably increased the binning yields (by 3.5% to 80%) for most of the metagenomic datasets examined thanks to improved contig length and coverage profiling of metagenome-assembled contigs. Specifically, 411 versus 397, 331 versus 317, 104 versus 88 and 9 versus 5 metagenome-assembled genomes (MAGs) were recovered from MEGAHIT assemblies of bioreactor sludge, surface water, lake sediment, and forest soil metagenomes, respectively. Noticeably, deduplication reduced the computational costs of metagenomic assembly including elapsed time (by 9.0% to 29.9%) and maximum memory requirement (by 4.3% to 37.1%). Collectively, it is recommended to remove duplicate reads in metagenomic data before assembly and binning analyses, particularly for complex environmental samples, such as forest soils examined in this study. Importance Duplicated reads are usually considered as technical artefacts. Their presence in metagenomes would theoretically not only introduce bias in the quantitative analysis, but also result in mistakes in coverage profile, leading to negative effects or even failures on metagenomic assembly and binning, as the widely used metagenome assemblers and binners all need coverage information for graph partitioning and assembly binning, respectively. However, this issue was seldomly noticed and its impacts on the downstream key bioinformatic procedures (e.g., assembly and binning) still remained unclear. In this study, we comprehensively evaluated for the first time the impacts of duplicate reads on de novo assembly and binning of real metagenomic datasets by comparing assembly quality, binning yields and the requirements of computational resources with and without the removal of duplicate reads. It was revealed that deduplication considerably increased the binning yields and significantly reduced the computational costs including elapsed time and maximum memory requirement. The results provide empirical reference for more cost-efficient metagenomic analyses in microbiome research.