{'fraction_non_alphanumeric': 0.023230598248895395, 'fraction_numerical': 0.010300615326231331, 'mean_word_length': 4.805192761605036, 'pattern_counts': {'":': 1, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
The benefits of prefetching for large-scale cloud-based neuroimaging analysis workflows

To support the growing demands of neuroscience applications, researchers are transitioning to cloud computing for its scalable, robust and elastic infrastructure. Nevertheless, large datasets residing in object stores may result in significant data transfer overheads during workflow execution. Prefetching, a method to mitigate the cost of reading in mixed workloads, masks data transfer costs within processing time of prior tasks. We present an implementation of"Rolling Prefetch", a Python library that implements a particular form of prefetching from AWS S3 object store, and we quantify its benefits. Rolling Prefetch extends S3Fs, a Python library exposing AWS S3 functionality via a file object, to add prefetch capabilities. In measured analysis performance of a 500 GB brain connectivity dataset stored on S3, we found that prefetching provides significant speed-ups of up to 1.86x, even in applications consisting entirely of data loading. The observed speed-up values are consistent with our theoretical analysis. Our results demonstrate the usefulness of prefetching for scientific data processing on cloud infrastructures and provide an implementation applicable to various application domains.


I. INTRODUCTION
Many fields of science are experiencing increases in the volume of datasets available to researchers. Neuroscience in particular is experiencing a rapid growth in data due to technical advances, scientific breakthroughs, and sociological trends towards open data sharing. This increase in data is providing the basis for new discoveries about brain structure and function, but it also presents technical challenges. To deal with the deluge of available data, neuroscientists are increasingly adopting cloud platforms for data storage and processing. However, inefficient use of cloud services can lead to needlessly longer processing time and cost. We aim to investigate methods to reduce the impact of data transfers for neuroimaging workflows on cloud services.
Data prefetching is a well-established technique for the reduction of data access-related costs [4], [11], [9]. Traditionally, prefetching was used to reduce memory latency, as memory accesses were significantly slower than CPU processing. However, since the rise of Big Data, prefetching has also been shown to be beneficial to the processing of large datasets located on remote storage [12]. During the execution of an application, data required for future tasks are copied from the remote storage device to compute-local storage, such that when the application requires the data, it can read it from local storage.
A recent example of the effectiveness of prefetching on the cloud is Netco [8], a prefetching extension integrated into the Hadoop Distributed File System (HDFS). Future data is prefetched based on two measures: 1) size of the data to be processed and 2) task deadline. Netco demonstrated superior performance compared to other file systems, which importantly motivates our study. However, it remains tightly bound to HDFS while cloud applications generally use different file systems. A more versatile solution is needed that would broadly apply to cloud data analysis and storage.
The present study focuses on neuroscience data that describe long-range connections between different parts of the human brain, a central research topic in contemporary neuroscience [1]. The three-dimensional trajectory of the major neural pathways, composed of millions of neuronal axons, are inferred from measurements of diffusion MRI and processed using computational tractography algorithms. These algorithms generate "streamlines": 3D curves that approximate the trajectories of the major pathways. A single human brain measurement may contain several millions of these streamlines, with their coordinates assessed at sub-millimeter resolution. Subsequent analyses of these streamlines usually access streamlines sequentially and entirely within the files in which they are stored. Such an access pattern creates an excellent opportunity for prefetching. This paper investigates the benefits of prefetching for cloudbased processing of neuroscience streamlines. Through both theoretical analysis and experimentation, we characterize the speed-up provided by prefetching compared to sequential data transfers for neuroscience data processing deployed on the Amazon Web Services cloud. More specifically, this paper makes the following contributions: • Formalization and performance analysis of a "rolling prefetch" data scheme for cloud-based applications; • Implementation based on the S3Fs Python library to access data on Amazon S3; • Experimental evaluation in the Amazon cloud with a 500-GB dataset of streamlines derived from dMRI data.

II. MATERIALS AND METHODS
Our implementation of prefetching, known as Rolling Prefetch, is a Python library implemented as a layer on top of S3Fs. The code is available under the MIT license at https://github.com/ValHayot/rollingprefetch. S3Fs is a Python library, based on FSSpec, for interacting with directories and files located on Amazon S3. To offset the cost of latency on S3, S3Fs leverages the on-demand caching mechanisms provided by FSSpec.
Unlike S3Fs which has distinct data transfer and compute phases, prefetched data transfers happen concurrently with compute ( Figure 1). In both cases, data is transferred from cloud storage in blocks of configurable size. While local I/O overheads occur during prefetch, the resulting overhead is in general minimal since memory disks are expected to be used for this purpose. Most importantly, prefetching requires knowledge of the application's data access pattern, such that transfers can be correctly anticipated. In our case, we assume that data is read sequentially, as is commonly the case with tractography data.

A. Algorithm
Rolling Prefetch combines prefetching with data eviction. By evicting data post application processing, it ensures a reduced footprint on local storage. Rolling Prefetch consists of three threads: (1) the reading thread loads data blocks from local storage and marks them for eviction, (2) the prefetching thread transfers data blocks from cloud storage to local storage, and (3) the eviction threads deletes the blocks that have been marked for eviction.
1) Reading: The reading thread is triggered by the application to read data directly from cache or to wait until it is prefetched, if not found in the cache. By waiting for the data to be cached, we ensure that performance is comparable to S3Fs in a worst case scenario. Furthermore, as all data reads are considered to be sequential, if the current block is not in memory then it is the current block being prefetched. Whenever a prefetched block has been read fully, it is up to the read function to flag it for deletion.
2) Prefetching: Throughout the lifetime of the application, the prefetching thread continuously requests blocks from cloud storage, so long as there remain blocks that have not been prefetched (Algorithm 1). Each block will be written to an appropriate cache location while not exceeding user-defined space limits.
Initially, the used variable for all cache locations is set to zero and the file_index variable is set to point to the first file index in the sequential ordering. The algorithm then enters a while loop whose termination is dictated by the main program. If all files have been prefetched, the prefetching threads terminates regardless of the status of the main thread. Within the loop, the algorithm iterates between all available cache locations in priority order. The amount of available storage space remaining is computed, and if it is insufficient to write the next block, the algorithm iterates through the file list total f iles (verif y used()) and queries the filesystem to determine which blocks have been evicted. Based on how many blocks have been removed, we can update the amount of used space, and by extension, the amount of available space. Should sufficient space be released, the next block is fetched from cloud storage and is written to local storage. If there is insufficient space, the next local storage location is tried. 3) Eviction: Similar to prefetching, the eviction thread only terminates when the main thread has completed processing. To avoid additional costs caused by continuously querying the filesystem to determine which files can be evicted, we determine the names of all the blocks that should be evicted (get all blocks), and verify whether they exist in the filesystem at time of removal. We then update the list to ensure that we do not attempt to remove this file more than once. Between each loop, we sleep for 5 seconds to ensure that sufficient time has elapsed between evictions.
The eviction thread ensures deletion of all remaining files prior to terminating.

B. Performance analysis
We consider the cost of processing a file using sequential transfers (S3Fs) to be the sum of three components (Equation 1): 1) latency-associated costs, 2) bandwidth-associated costs and 3) application compute time. The cost of latency is experienced every time a chunk of data is requested from cloud storage. That is, if we read a full file in a single call to cloud storage, we will only pay for latency once. If we issue multiple calls to obtain file data on cloud storage, we will pay latency for each call. In contrast, bandwidth is not affected by the number of calls. Whether we request all the data at once or in chunks, bandwidth remains fixed, and time to transfer the data is the quotient between the total amount of data transferred and the bandwidth. Compute time is assumed to be proportional to data size, as is frequently the case in neuroimaging.
where n b is the number of data blocks, f is the total size of the file to transfer, l c is the cloud latency, b cr is the cloud read bandwidth, and c is the compute time per byte consumed. Rolling Prefetch contrasts Equation 1 in that the compute and data transfer times mask one another (Equation 2). However, Rolling Prefetch has a slightly higher performance penalty than that of sequential transfers when there is no compute, due to reading and writing the data to local storage. Furthermore, we must consider the initial read from cloud storage, where no compute can occur concurrently, and the last compute, where no data transfer can occur concurrently.
where T cloud is the time to download a block from the cloud and write it to local storage, and T comp is the time to read a block from local storage and process it: where l l is the latency of local storage, b lw is the write bandwidth to local storage, and b lr is the read bandwidth to local storage. This simple model provides several insights. First, if we neglect local transfers (l l =0, b lw =b lr =+∞), a reasonable assumption when local storage is in-memory, then we have: and therefore the speed-up provided by Rolling Prefetch compared to sequential transfers is: Rolling Prefetch is therefore expected to provide a speed-up of at most 2x. This upper bound is approached when T cloud ≈ T comp , which requires that cloud transfer and compute times are of similar magnitude. In sequential transfers, using a single block (n b = 1) leads to the shortest transfer time. In practice, block size is of course constrained by available memory. In contrast, with Rolling Prefetch, an optimal block sizen b exists under the reasonable assumption that l l l c : It suggests that on a given cloud infrastructure, the number of blocks should increase when the application compute time or the total data size increases. Finally, as the number of blocks increases, T seq and T pf become equivalent to n b l c and n b (l c + l l ), respectively, resulting in parallel asymptote lines.

C. Data
To demonstrate the utility of Rolling Prefetch in neuroimaging, we analyzed data derived from a dMRI experiment conducted in a single subject. These data are special, in that a Super-Resolution Hybrid Diffusion Imaging (HYDI) method was used to achieve an effective spatial resolution of 0.625 mm 3 , much higher than the typical ∼2 mm 3 commonly used. The details of the HYDI method and parameters of data acquisition were previously described [5]. The source MRI data were 14.94 GB. The data were processed using a probabilistic tractography algorithm [2], implemented as part of DIPY [6] and accelerated using CUDA to work on GPU [3]. This generated ∼498 GB of streamline data. Tractography streamlines are usually stored in the neuroscience-specific .trk file format. In this case, streamlines were sharded into 464 .trk files stored in an S3 bucket using the highavailability Standard storage class.
Each .trk file is comprised of a 1,000-byte header and a body of variable length consisting of a series of streamlines. Each streamline section contains 4 B that denote the number of points in the streamline, followed by a series of floating point values detailing each coordinate and ends with a series of values representing properties of the streamline. We used Nibabel, a Python library that reads and writes neurosciencespecific data formats, to read these files. For .trk files, Nibabel can return individual data points via a generator. This is known as "lazy loading" and can be turned on or off prior to reading the file. As a result of the data representation format in .trk files, Nibabel reads may incur significant overhead: because it issues a total of three read calls for each streamline in the file. In addition, an affine transform is stored together with the data, and used to bring coordinates from different measurements into register. For example, when Nibabel reads .trk files it automatically applies an affine transformation to the coordinates of each streamline stored in the file. This means that some amount of compute is always executed when data is read from file.

D. Experiments
To evaluate the performance Rolling Prefetch, we conducted 4 experiments: 1) varying the number of files; 2) varying the blocksize; 3) parallel processing of files; and 4) neuroimaging use-cases. For experiments 1-3, the pipeline consisted of reading the file with Nibabel. As Nibabel performs a minimal amount of computing, Rolling Prefetch is believed to have an effect even here. Additional compute required by real analysis should only increase the benefit of using prefetching.
1) Varying the number of files: We vary the number of .trk files read to determine when Rolling Prefetch will be favourable relative to S3Fs. We expected that for small amount of data, there should be no significant discernible difference between S3Fs and Rolling Prefetch. This is because compute time is expected to be minimal and the blocksize large compared to the total dataset size. In other words, there will be less opportunities to prefetch data with very small datasets, unless the block size is proportionally smaller, in which case latency will be penalizing both reads from S3Fs and Rolling Prefetch, extending processing time. With larger files, there will be more computation occurring during the reads, and therefore, more opportunities to mask the data transfers within the compute.
We benchmarked the time it took to lazily read 1, 5, 10, 15, 20 and 25 files and extract the streamlines in Nibabel. The total data size for each of the file increments are 1.1, 5.8, 11.9, 18.0, 24.2 and 31.2 GiB, respectively. The blocksize was set to 64 MiB and the prefetch storage was located on tmpfs with a limit of 2 GiB of storage space. We performed 10 repetitions.
2) Varying the blocksize: In this experiment, we aimed to quantify the effect of the number of blocks (n b ) on Rolling Prefetch and S3Fs. According to our previous analysis, we expect that S3Fs will reach its best performance with a single block, and that the runtime will linearly increase with n b from that point on. For Rolling Prefetch, we expect the runtime to decrease to a minimum, and then increase linearly with n b .
Block sizes of 8, 16, 32, 64, 128, 256, 512, 1024, and 2048 MiB were used to load the files into Nibabel and extract the streamlines. We use the largest-sized block (2048 MiB) to determine the overhead of using the Rolling Prefetch algorithm, as no file in the HYDI dataset is larger than 1.7 GiB, and therefore, there are no opportunities to prefetch.
Five files pertaining to the HYDI dataset were used to generate the results. These files ranged from 793 MiB to 1.5 GiB in size each. Since only Rolling Prefetch is capable of treating a list of files as a single file, a new Nibabel object needed to be created for each file in the case of S3Fs.
The experiment was executed on both S3Fs and Rolling Prefetch for each block and repeated 10 times. Tmpfs was excluded from this experiment as the S3Fs blocksize would have no effect on its performance. Prefetched storage was set to be tmpfs and configured to have 2 GiB of available space such that the largest block size could fit in prefetch storage.
3) Parallel processing: Since perceived throughput may be affected by the number of threads used, we aimed to determine if the performance difference between S3Fs and Rolling Prefetch remains proportional to the amount of data being processed per thread.
If throughput is reduced by the number of active threads, it is expected that the cost of data transfers will outweigh the cost of computing, such that the difference between S3Fs and Rolling Prefetch will be very small. Furthermore, since S3 is a scalable distributed file system and local storage may not be (or may be more limited in terms of scalability), local storage throughput is expected to be reduced significantly with the addition of multiple threads.
We used a total of four parallel processes to evaluate the effects of parallel processing. We increased the number of files per thread between each condition from 1-20, with the last condition loading a total of 108 GiB into Nibabel. The blocksize was set to 64 MiB and we set the prefetch storage location to be on tmpfs, with a limit of 1 GiB of storage per process. 10 repetitions were performed.

4) Neuroimaging use cases:
Our previous experiments evaluate the value of Rolling Prefetch when the application performs the most basic and necessary action: converting the data from binary to array representation. We expect that the benefits of Rolling Prefetch can be maximized with an increased compute time, due to greater opportunities to prefetch data during compute. However, we also expect that there is a peak ratio between data transfer time and compute time (data-to-compute ratio), where the benefits of Rolling Prefetch will be highest. If compute time is significantly greater than data transfer, regardless of all the time saved by prefetching data, the total runtime will predominantly be compute. This motivates further profiling to determine how to shard large datasets, like the HYDI dataset, such that the benefits of Rolling Prefetch can be maximized.
Since the benefits of Rolling Prefetch may vary according to the ratio between data transfer and compute time, we have selected two use cases with varying computation times: 1) histogram distribution of streamline lengths; and 2) bundle recognition.
The distribution of streamline lengths is used in order to assess the performance of tractography algorithms. It may also aid in determining if the files are good candidates for compression. The histogram computation consists of loading each streamline within the dataset lazily, gathering the lengths of each individual streamline and generating a histogram of 20 bins from their lengths.
The human brain is composed of multiple different functional regions, and these regions are connected through major anatomical pathways. One of the first steps in the analysis of streamline data from human brain dMRI data is to classify the streamlines according to their three-dimensional trajectory into these different pathways. This is referred to as "bundle recog-nition" [7]. To demonstrate the benefits of Rolling Prefetch in this use-case, we ran a software pipeline that determines whether streamlines within the tractography file belong to either of two different bundles (the corticospinal tract, CST and the arcuate fasciculus, ARC), or to neither [10]. Currently, the pipeline loads the data all at once (i.e. no lazy loading) and then performs the bundle recognition task. Thus, it is not possible to read only fragments of a large file which does not fit in memory, in this case, and as a consequence of the separation of data loading and processing, there will be no opportunities to keep the prefetched data transfers masked within compute operations.
We chose not to modify the pipeline, in order to determine what is the speedup provided by Rolling Prefetch without additional optimization, and in order to be able to speculate on the possible benefits of adapting existing pipelines to support larger-than-memory files and allowing reads and computations to occur together.
Varying parameters were used for the bundle recognition and histogram application. For instance, recognition was executed on two different compute instances (c5.9xlarge and r5.4xlarge). The experiment executed on the c5.9xlarge instance consisted of a 1 GiB file with a 64 MiB blocksize. The histogram experiment executed on the r5.4xlarge instance used 10 files totalling 12 GiB and used a 32 MiB blocksize.
We also considered the speedups obtained from bundle recognition in smaller files. We split up the previous 1 GiB file and split it up into 9 shards containing the same number of streamlines. File sizes ranged from 73 GiB to 165 MiB. We compared the speedup provided by Rolling Prefetch on a 165 MiB shard to the processing of all 9 shards. These experiments were run exclusively on the r5.4xlarge instance with 10 repetitions.
To compare the benefits of Rolling Prefetch with different data-to-compute ratios, we executed the histogram computation and the bundle recognition algorithm on an r5.4xlarge instance using the 1 GiB file. Blocksize in both cases was fixed at 32 MiB. 5 repetitions were performed.
In all cases, 2 GiB of cache storage was allocated to Rolling Prefetch. The scientific applications required significantly more memory than available on t2.xlarge instances. As a result, the scientific applications were executed on a c5.9xlarge instance and an r5.4xlarge configured identically to the t2.xlarge instance, but located in the us-west-2d availability zone. c5.9xlarge instances consist of 64 vCPUs and have 72 GiB of available The data used for all experiments originated from the HYDI tractography dataset stored on the Amazon S3 object store. This dataset is broken down into 464 files ranging from 700 MiB to 1.7 GiB each. This dataset was also located in the us-west-2 region.

E. Infrastructure
To get an idea of the cost of data transfers, we measured the time it would take to read data sequentially from S3 to the t2.xLarge instance. We varied the filesize from 1 KiB to 4 GiB and measure the time it took to read the files from a us-west-2 S3 bucket and from tmpfs. This method was favoured over more standard benchmarking libraries to ensure the two filesystems were benchmarked in the same way. Results were then compared to benchmarking results produced by the s3-benchmark library to ensure validity of our benchmarking method. The results can be seen in Table I.

A. Varying the file size
As we increase the file size we observe that the disparity between S3Fs and Rolling Prefetch increases (Figure 2), with Rolling Prefetch significantly outpacing (∼1.7x faster) S3Fs at 31.2 GiB (25 files). This is expected as per our theoretical evaluation. With a large amount of blocks, the pipeline duration is determined by the maximum between the time it takes to compute the data T comp and the time it takes to transfer the data T cloud . On the other hand, S3Fs runtime is the sum of the time it takes to transfer the data and the compute. In this case, the compute time is large enough such that using Rolling Prefetch significantly reduces runtime with large amounts of data.
As the size of the data decreases, the benefit of using Rolling Prefetch also decreases. In this particular use case, S3 data transfer time is much larger than that of compute time, thus the only time we can save is that of T comp . T comp like T cloud is proportional to the number of blocks used, and thus the amount of time saved increases with file size. In the worst case, Rolling Prefetch performance equals that of S3Fs.

B. Varying the block size
At a very large number of blocks, the performance of both S3Fs and Rolling Prefetch is degraded due to increased impacts of both S3 storage latency and additionally, in the case of Rolling Prefetch, local storage latency (Figure 4). S3Fs starts to surpass the speed of Rolling Prefetch, although not substantially (approx. 1.03 X) at 748 blocks. Furthermore, while we read from S3 at certain blocksizes, the block size on local storage is dictated by filesystem and kernel readahead size. In our specific case, the data would be read from tmpfs, and thus we do not see a significant added cost to Rolling Prefetch. It could be inferred that the overhead of Rolling Prefetch may be more significant with a large number of blocks due to Nibabel's access patterns, which consist of many small reads.
Conversely, when decreasing the number of blocks we start to see an increase in performance in both S3Fs and Rolling Prefetch. This is because both benefit from decreased latency to access data on S3, by generally making fewer read calls to it. Rolling Prefetch is found to be faster than S3Fs with a peak speedup of 1.24 X at 187 blocks (i.e. 32 MiB blocks). The speedup obtained from Rolling Prefetch did not vary significantly between 24 and 187 blocks, averaging at a speedup of approximately 1.2 X. Rolling Prefetch outpaces S3Fs at less than 748 blocks as the cost of local latency has diminished sufficiently with the larger blocks. As the blocksize increases, Rolling Prefetch is able to begin masking S3 latency and bandwidth within its compute. S3Fs and Rolling Prefetch performance converges again when the blocks reach a certain size, placing more weight disk bandwidth than latency. S3Fs exceeds the performance of Rolling Prefetch at a single block, where no actual prefetching can take place and we pay the cost of implementation overheads, such as writing the data to local storage and reading it from local storage rather than directly from memory.
Generally speaking, variations in blocksize did not lead to major speedups between Rolling Prefetch and S3Fs. Since the total files size and compute time was fixed, there was a fixed upper bound to how many data transfers could be masked by compute time. By minimizing the amount of latency through a reduction in the number of blocks, the overall application time was shortened, in both S3Fs and Rolling Prefetch, and thus a larger percentage of the total runtime could be masked by compute.

C. Parallelized workloads
When we parallelize the reads up to four concurrent processes (Figure 3), we notice that the trends are still consistent despite increased contention on local filesystems. This is likely due to the fact that tmpfs speeds would remain sufficiently high even with the increased contention caused by having at minimum 4 and potentially even up to 8 concurrent threads accessing the filesystem at the same time. We speculate that the same pipeline running with data read from a single disk would have had worse performance due to its lack of scalability.
The maximum speedup achieved, on average, during this experiment was 1.86× at 24.2 GiB. The average speedup was around 1.52 x altogether, with the minimum average speedup being 1.37× at 80.6 GB. Due to the high variability in results within both Rolling Prefetch and S3Fs, we expect that these speedup differences may be a result of bandwidth and latency variability across runs. We do not expect that conditions are more favourable for reading the data with four threads processing 24.2 GiB; since they are processing the same amount of data concurrently, the overall speedup should be constant.

D. Neuroimaging use-cases
The results obtained from our neuroimaging use cases indicate that, in all cases, the use of Rolling Prefetch can speed up analyses ( Figure 5). The extent of this speedup is, however, dependent on a number of factors, such as instance type, application and number of shards.
The greatest observed speedup with Rolling Prefetch was found when executing the bundle recognition application with a greater number of shards (1.64× faster). This result echoes what was observed in Figure 2 with even a more computeintensive application like bundle recognition. We do not, however, observe a speedup with a single shard, as the size of the shard was so small it did not incur many reads from S3.
Due to the compute-intensive nature of the bundle recognition pipeline, it comes as no surprise that overall speedup in the unsharded case is minimal (1.14×). The ratio of datato-compute was found to be approximately 1/7, where the compute took, on average around 9000s. Thus, even if Rolling Prefetch significantly speeds up reads, the data transfer time is minimal to the overall compute time of the application. With a highly data-intensive workflow, such as a histogram computation, we observe that the speedup is more significant with a speedup of 1.5×. Although our model dictates that the upper bound of speedup that can be obtained with Rolling Prefetch is 2×, the observed speedups obtained by these two applications never reach that boundary. A possible explanation may be that the data-to-compute ratio of histogram computation was too large to mask many of the reads within the compute, whereas the data-to-compute ratio was too small in the segmentation application, such that any speedup obtained through the Rolling Prefetch algorithm was offset by the computation time. The ideal algorithm for Rolling Prefetch would lie somewhere between the data and computational requirements of the histogram and segmentation applications.
Although the execution parameters differed between the two instance types thus not making them directly comparable, we observed a greater speedup on the c5.9xlarge (1.6×) than on the r5.4xlarge (1.1×) instance. We suspect that the speedup obtained on the c5.9xlarge instance was relatively high as a result of the increased parallelism obtained from the increased number of CPUs, resulting in a more data-intensive application. While the r5.4xlarge instance did process a larger number of files, suggesting that speedup should be greater, the actual file sizes varied, and with it, the streamline features, potentially resulting in a more compute-intensive execution further exacerbated by the decrease in available vCPUs.

IV. DISCUSSION
Our theoretical analysis and experimental results demonstrate that there is a substantial processing time gain to be obtained from using Rolling Prefetch, particularly in the case of mixed workloads, where there is a significant cost associated with time spent on compute and data transfers. This works   Avg. Speedup well with typical use cases in neuroimaging, where tasks vary widely ranging from very short tasks to long ones necessitating hours to complete and where datasets are large enough to incur transfers of similar magnitudes. Moreover, to save time with data transfers, researchers may opt to transfer their data over the compute instance, and perform the computation exclusively with data stored locally. While this does effectively give the optimal performance during processing, researchers are left to manage the data themselves. Since local storage on compute can become quite costly, researchers must decide between processing only a subset of the data, paying hefty fees related to storing large amounts of data or incorporating data management logic directly into their workflows. There are also natural limitations to Rolling Prefetch. For instance, in the case of parallel workloads, in certain instances S3 would be preferred to the available local storage. This is a consequence of the fact that S3 is designed to be scalable and is capable of processing many requests simultaneously, whereas unless configured to do so, local storage will not scale adequately to increased contention.

A. Benefits of Rolling Prefetch
Rolling Prefetch is an added layer on top of S3Fs that replaces its built-in cache strategies to intercept application read calls and ensure that data is preloaded. The implementation ensures that filesystem space requirements are not exceeded during prefetching through limits set by the user. With the built-in eviction mechanism, users are not required to do any form of data management should they be processing datasets larger than available storage. Furthermore, the library allows configuration of multiple storage device as cache for prefetching. Currently, files are written to specific storage devices based on available space and storage priority in the list.
With just a simple implementation performed on a computation based on reading, we have observed up to a 1.8× speedup and a maximum overhead of 1.03×. These observed speedups were obtained when we set the cache location to a tmpfs device and may naturally decrease with a slower device such as an SSD. In our particular case, the speedups meant saving 20 min of processing time on loading nearly 100 GiB of data with a maximum runtime of approximately 71 min. Moreover, this was achieved on a small instance with only 1 GiB of prefetch storage allotted to each parallel process, indicating that we can achieve good speedups even when resources are constrained.

B. Parallelism-related overheads
While our experiments demonstrate that Rolling Prefetch continues to provide a performance improvement to parallel applications running on S3, we do expect performance to decrease if we continue to scale horizontally on a single instance, or use a slower device as cache. Our implementation consists of two threads actively reading and writing to local storage. Each time the number of Rolling Prefetch instances increase, we double the amount of threads writing to local storage. Although it is standard to have multiple DIMMs on a single machine, it may not be necessarily true of other storage devices. That being said, attached cloud volumes may also be sufficiently scalable such that processing time remains unaffected by an increase in processes.
To reduce the load of prefetching data to local storage, we can add a third component to Rolling Prefetch that periodically tracks cache bandwidth. Using such a parameter, the algorithm could take into consideration filesystem load in addition to user-specified priority.

C. Task granularity
S3Fs was designed to be used within distributed processing frameworks. With such frameworks, we must take into consideration task granularity and how that would affect scheduling. Rolling prefetch becomes beneficial with larger files. Assuming a workload where the task is simply to load the streamlines, we would need a few GBs of data to start noticing a speedup. In cases where resources are ample enough to run all tasks in parallel and the amount of data processed by a task is minimal, Rolling Prefetch and S3Fs would probably behave similarly, with S3Fs potentially performing a bit faster.
The risk with passing large amounts data to the library within a task is that it is less robust to failures as any failed task would have to resume from the beginning. This could take a significant amount of time, depending on how much data had been processed by the task prior to failure. It is understandable that implementations such as Netco are adaptations to persistent filesystem, since then prefetching could work without limiting scheduler logic. For our implementation to be efficient within parallel processing frameworks, we would need to decouple it from the task itself and allow it to communicate with the task scheduler to determine which files to prefetch and where.

V. CONCLUSION
Overall, we conclude that Rolling Prefetch would be a valuable addition to large-scale neuroimaging pipeline executing on the cloud, particularly in instances where data transfer and compute time are similar. Future work should focus on the following improvements: The loading part of Rolling Prefetch should be performed outside of task themselves to ensure that it does not interrupt any form of task scheduling and to avoid tasks which take too long to restart if lost. The library should also make sure to be able to communicate with the schedulers to help determine where tasks should be scheduled give the location of the data.