license:apache-2.0 size_categories:10K<n<100K format:json modality:text library:datasets library:dask library:mlcroissant region:us
---
license: apache-2.0
---
# Biorxiv Abstracts 
* We collected biorxiv abstracts using biorxiv public API under permitted licenses[1] 
* This dataset is intended to evaluate federated learning algorithms.There are 57k clients (i.e. abstracts) for date range 2023-04-01 to 2024-08-31. Since there are more speeches that are later added, there will be more clients as we increase the date range.
* Hugging Face repo: https://huggingface.co/datasets/hazylavender/biorxiv-abstract
* Github: https://github.com/hazylavender/fl_llm_benchmark_dataset

## Dataset statistics
(cutoff 2024-08-31)
* Number of abstracts: 57k
* Token distribution (by nltk tokenizer)
<img src="https://cdn-uploads.huggingface.co/production/uploads/66c104f1ef1f45cd71d1896c/ctXI8FRVSRheVG8RaYWBR.png" width="500">


 

[1] The dataset we collected are under 'cc_by_nc_nd', 'cc_by_nd', 'cc_by_nc', 'cc_by', 'cc0' licenses. Source: https://www.biorxiv.org/about/FAQ 
