{'fraction_non_alphanumeric': 0.030529014215800513, 'fraction_numerical': 0.008855744581682591, 'mean_word_length': 5.4577376473539, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 18, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Simons Collaborative Marine Atlas Project (Simons CMAP): An open‐source portal to share, visualize, and analyze ocean data

Simons Collaborative Marine Atlas Project (Simons CMAP) is an open‐source data portal that interconnects large, complex, and diverse public data sets currently dispersed in different formats across different Oceanography discipline‐specific databases. Simons CMAP is designed to streamline the retrieval of custom subsets of data, the generation of data visualizations, and the analyses of diverse data, thus expanding the power of these potentially underutilized data sets for cross‐disciplinary studies of ocean processes. We describe a unified architecture that allows numerical model outputs, satellite products, and field observations to be readily shared, mined, and integrated regardless of data set size or resolution. A current focus of Simons CMAP is integration of physical, chemical, and biological data sets essential for characterizing the biogeography of key marine microbes across ocean basins and seasonal cycles. Using a practical example, we demonstrate how our unifying data harmonization plans significantly simplifies and allows for systematic data integration across all Simons CMAP data sets.

Oceanography is a dynamic and diverse discipline that investigates biological, chemical, and physical processes of the world's oceans via theoretical and experimental approaches. The advent of satellite technologies and autonomous instrumentation in conjunction with ubiquitous high-performance computational resources is resulting in an explosive growth in the volume, variety, and velocity of compiled oceanographic data sets. In principle, cross-disciplinary analyses of these data collections promise new scientific advances. And yet, the sheer size and complexity of available data frequently hinders the ability to fully capitalize on this collective potential. Moreover, sharing the generated data sets remains a fundamental cultural and technical challenge (Pendleton et al. 2019) within scientific communities.
Current public oceanographic data sets are typically dispersed across numerous data repositories and are stored in a wide array of data formats turning the data integration process into an arduous undertaking. For example, satellite products and numerical model outputs are housed in diverse data repositories and typically delivered in netCDF format (NetCDF 2021) following CF-conventions (Eaton et al. n.d.). In situ measurements derived from time-series observations, long-term research expedition programs, or climatologies are generally found at both dedicated websites (e.g., HOT [http:// hahana.soest.hawaii.edu/hot/], BATS [http://bats.bios.edu/], AMT [Robins and Aiken 1996], ARGO [Argo GDAC 2019], GO-SHIP [https://www.go-ship.org/], World Ocean Atlas [Boyer et al. 2013]) and at different repositories (e.g., BCO-DMO [https://www.bco-dmo.org/], PANGAEA [https://www. pangaea.de/]) and are commonly available in a variety of formats, including flat files. Recent efforts link expeditionspecific environmental measurements with contemporaneous organism-based molecular data sets from the same expeditions (Planet Microbe [Ponsero et al. 2020], TARA [https://oceans. This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes. taraexpeditions.org/en/]). However, systematic expansion of these linkages beyond the expedition data sets remains as a significant technical challenge.
Data integration across data set types is thus becoming increasingly difficult as marine data repositories become stockpiled with varying file formats and naming conventions. Moreover, the absence of a cohesive data distribution strategy that covers the different sectors of oceanography has contributed to the inconsistent data exposure across all research disciplines. Data architectures for more straightforward access to highly heterogeneous data in Earth science is an active research area. Ongoing research efforts in industry include IBM PAIRS (Lu et al. 2016) and Google Earth Engine (Gorelick et al. 2017). Academic efforts include the PANGEO project (Eynard-Bontemps et al. 2019) and efforts within the SciDB community (Appel et al. 2018). In this article, we present the approach that Simons CMAP is taking to tackle these challenges for the ocean marine-microbes community.

Identification of community needs
Designing large-scale knowledge infrastructures meant to serve multidisciplinary, multi-organizational, and geographically distributed scientists has been and will continue to be a critical challenge (Atkins et al. 2003). The Simons CMAP design principles were guided by a 12-month study of the workflows of a selected number of domain researchers using an approach informed by Participatory Action Research methodology (McTaggart 1997;Kemmis and McTaggart 2005). We employed a mix of ethnographic methods (Marcus 1995), user research methods (Rohrer 2014;Baxter et al. 2015), and Design Thinking (Dorst 2011;Plattner et al. 2012). This mixed-method approach provides a nonlinear, iterative process to understand community needs, define the parameters of the problem, and identify new solutions through a prototyping and testing process. Ethnographic methods were employed to document and analyze the coevolution of stakeholder needs and Simons CMAP. Semi-structured interviews were conducted over the course of a year with 51 participants based in 22 academic research laboratories across the United States, Canada, and the United Kingdom. Data captured from meetings and a subset of initial interviews were analyzed and used to design a survey to evaluate how researchers share and preserve their data, which methods and tools are commonly employed, what factors limit use of publicly available data, and specific content or functionalities that would better support research endeavors. A total of 124 participants received and completed the survey.
Survey results revealed obstacles to the synthetic analyses of publicly available data (Fig. 1). Key factors included differences in the temporal and spatial resolution of different data sets; difficulties in finding, retrieving, and subsetting data from different sources; difficulties in integrating data of various formats; and lack of sufficient documentation and metadata associated with the data sets. Desired functionalities included enhanced abilities to share unpublished data among collaborators. The community-engaged research process also identified and helped prioritize which core public data sets should be ingested in the near term into Simons CMAP.

Data harmonization
A key attribute of Simons CMAP is the transformation of publicly available heterogeneous data sets into a common data and metadata structure. To ensure that data sets are Findable, Accessible, Interoperable, and Reusable (FAIR) (Wilkinson et al. 2016), the transformed data sets are annotated with a semantic layer that consists of contextual keywords about the data set variables. The annotation step is commonly carried out by the data provider or alternatively, by a knowledgeable end-user. The augmented semantic layer makes it possible to identify similar variables registered with different naming conventions. For example, search capabilities are enhanced by the annotation of a variable such as "nitrate" with keywords such as "NO3" or "N + N," two commonly used alternative designations for nitrate. A web-based validation tool allows individual data providers to confirm their data set adheres to the appropriate formatting requirements and helps to identify potentially erroneous or outlier data points. All annotated data sets are subjected to a final, human-curation step to ensure the proper structure of the data and metadata; a DOI is required for all user-provided data sets as Simons CMAP houses data set copies within its data portal and is not intended as an official master repository. The curated data set copies are ingested into a scalable database system of organized data collections with each data entry indexed by a hashed location and time coordinate. This indexing scheme constitutes the bedrock for efficient data retrieval, visualization, and systematic crossreferencing of the underlying data sets regardless of size or resolution.
The current Simons CMAP database encompasses data sets from a wide array of oceanography sectors selected to help achieve an initial, overarching goal of understanding the structure, function, and ecology of marine microbial assemblages. A core component of Simons CMAP is a suite of diverse data sets collected since 2014 on research expeditions conducted through the Simons Collaboration on Ocean Processes and Ecology (https://www.simonsfoundation.org/life-sciences/microbialoceanography/simons-collaboration-on-ocean-processes-andecology/); a component of the Microbial Oceanography program of the Simons Foundation. These data sets include chemical (e.g., trace elements and organic and inorganic macronutrients), biological (e.g., productivity, particle stoichiometry and metabolites, flow cytometry and cell counts, and optical measures), and physical (e.g., temperature, salinity, light) measurements derived from analyzed on-station samples, and from underway and autonomous instruments. The larger oceanographic context is provided by inclusion of global multi-decade remote sensing products (e.g., satellite temperature, chlorophyll, altimetry), several decades of global biogeochemical model estimations (e.g., MIT Darwin [Ward et al. 2012;Dutkiewicz et al. 2015; http://darwinproject.mit.edu/]; Mercator-Pisces [Aumont et al. 2015]), and decades of field measurements (e.g., Argo floats [Argo GDAC 2019]; World Ocean Atlas [Boyer et al. 2013]; Hawaii Ocean Time series [http://hahana.soest. hawaii.edu/hot/]). Numerous additional field expeditions and their associated cruise trajectories conducted over the past decades have been cataloged and ingested in the database, a process that will be actively continued. Figure 2 illustrates a selected list of public data sets that are harmonized and ingested in the Simons CMAP database. The most updated list of datasets can be found at the Simons CMAP website (https:// simonscmap.com).
The Simons CMAP database accepts public and open-source data sets with explicit time and location annotations. We invite the Oceanography research community to submit their data sets via our data submission portal (https://simonscmap. com/datasubmission) where the data harmonization and ingestion process is initiated through a series of step-by-step operations. Once ingested, the data set can be visualized using the plotting services and can be integrated with any other archived data set.

System overview
Over the past decade, Earth science data management systems have progressively adopted advanced technologies underpinning the Big Data ecosystems. While these new advancements open the door to the efficient handling of massive data collections, they also raise the technical skills required to access and analyze the amassed data. Simons CMAP is designed to lower the required technical competencies by abstracting away the details introduced by the underlying technologies. Figure 3 depicts a simplified schematic of the Simons CMAP process-flow illustrating how the complexities associated with data preparation and management technologies are decoupled from the application layer where the end-user interacts with data.
The data pipeline begins with the "collection" step where the collected data sets are curated and harmonized by location and time prior to being transferred to the database systems (https:// github.com/simonscmap/DB). Each data set is populated in a table where columns represent the data set variables. The database acts as the central component as it hosts both the data and the data extraction logic. Integrating the extraction logic into the heart of the database enables two key features: (1) the technical process of data retrieval or co-localization of two different data sets is substantially simplified as it is handled internally and (2) Simons CMAP becomes a language-agnostic product allowing researchers to retrieve the data with their (programming) language of choice by invoking these predefined database-level services. A cloud-based REST Application Programming Interface (API) (Fielding 2000; https://github.com/simonscmap/cmapnode-api) operates as a gateway to the database resources and handles data requests issued by the downstream applications. The outermost component of the architecture is comprised of an application suite developed to expose the Simons CMAP services to the end-users. To date, these applications include an interactive web interface that allows for data submission, access, and visualization (https://simonscmap.com) plus a series of programming interfaces supporting Python (https://github. com/simonscmap/pycmap, https://doi.org/10.5281/zenodo. 3387777), R (https://github.com/simonscmap/cmap4r), Matlab (https://github.com/simonscmap/matcmap), and Julia (https:// github.com/simonscmap/CMAP.jl) languages. These documented programming interfaces (https://cmap.readthedocs.io) provide easy access to the Simons CMAP services and make it possible to conduct controlled and reproducible research studies. Listing 1 shows a simple script that uses the Simons CMAP Python package (pycmap) to retrieve a subset of salinity measurements (variable='argo_merge_salinity_adj') stored at the Argo data set (table='tblArgoMerge_REP'). Data is returned in form of a DataFrame object; a widely-accepted data structure within the scientific computing community with extensive built-in analytical features (McKinney 2010). In addition to extracting subsets of data, the programming interfaces present methods for data visualization, analytics such as aggregation along time and space axes (e.g., time-series, depth profiles), computing dataset-specific climatology with custom time-frames (e.g., weekly, monthly, quarterly climatologies), and co-localization services (see documentation for more details and examples; https://cmap.readthedocs.io/en/latest/ user_guide/API_ref/api_ref.html).
To extract a variable slice using the programming interfaces, one needs to know the exact variable name and the name of the table where it is hosted. The latest list of variable and table names is accessible at the website's catalog page (https://simonscmap.com/catalog) which offers an interactive alternative for data retrieval. Moreover, the website provides the user with an intuitive plotting toolset for creating a range of data visualizations without prior knowledge of the exact variable and table names. Including multiple interfaces is a feature incorporated in response to user interviews, which identified a wide range in technical expertise among users. The combination of a public web interface and a documented with miscellaneous formats are collected and transformed into a common structure. The transformed data sets along with their associated metadata are loaded into database systems where data is indexed by location and time. A cloud-based API exposes the database contents to the end-user and external applications abstracting away the details associated with the underlying technologies. The applications provide graphical and programming interfaces to retrieve, visualize, and analyze the data. public API is designed to map to these different needs identified in community discussions.

Data integration example
Marine observations reflect an intrinsically complex system that is under the influence of interwoven physical, chemical, and biological dynamics. Arguably, one of the most effective approaches to understanding these processes is conducting synthetic analyses where the field observations are joined with independent ancillary data sets such as environmental parameters to leverage their collective insight. However, collecting and integrating multiple different massive marine data sets can be a substantially challenging and time-consuming process as they are obtained from various sources with inconsistent data structures. Here we describe a multivariate analysis example where we compile observations of specific phytoplankton groups archived within Simons CMAP and colocalize them with the adjacent environmental conditions derived from independent data sets. Since all Simons CMAP data sets are uniformly harmonized, the entire process of collecting and integrating multiple massive global data sets with different dimensions and resolutions can be summarized into short code blocks developed using the Simons CMAP programming interfaces.
The planet's oceans are teeming with diverse microbial communities that drive the biogeochemical processes delineating the global biogeochemical cycles (Falkowski et al. 2008). Marine phytoplankton account for approximately 50% of global net primary production and therefore have a critical impact on the production and cycling of nutrients and organic matters (Field et al. 1998). The most abundant members of the phytoplankton are the picophytoplankton with cell diameters less than ≈ 3μm. They are distributed throughout lighted waters of the global oceans. The picophytoplankton are composed of Prochlorococcus and Synechococcus, two important cyanobacteria, and the pico-eukaryotes, a mixture of different eukaryotic picophytoplankton groups. Prochlorococcus is the most abundant photosynthetic organism on the planet and is the numerically dominant phototroph in tropical and subtropical oceans, with key contributions to the ecosystems and global biochemical cycles (Moore and Chisholm 1999).
The Simons CMAP database hosts more than three decades of global measurements of picophytoplankton characteristics in a unified format. These data sets are derived from multidecade research programs such as the Atlantic Meridional Transect (AMT) (Robins and Aiken 1996), global compilations of cruise expeditions and time-series programs (Buitenhuis et al. 2012;Flombaum et al. 2013) and high-resolution observations recorded using SeaFlow, a continuous shipboard flow cytometer (Ribalet et al. 2019). The compiled data set for this case study consists of more than 163,000 flow cytometrybased measurements of volumetric cell abundances of Prochlorococcus, Synechococcus, and pico-eukaryotes from the top 5 m of the global oceans during an approximately 35-year period (Fig. 4). The complete list of gathered data sets can be found in the supplementary materials (Table S1).
The Simons CMAP database is structured to handle the process of co-localizing in situ measurements with environmental variables from diverse data sets, based on user-defined temporal and spatial tolerance parameters (Fig. 5). Here, each flow cytometry- The collected data sets are results of more than 35 years of field expeditions involving 16 cruise expeditions conducted by the Atlantic Meridional Transect Program (Robins and Aiken 1996), 50 cruise expeditions by the SeaFlow Program (Ribalet et al. 2019), and decades worth of time-series and cruise expeditions compiled by (Buitenhuis et al. 2012;Flombaum et al. 2013). The samples are restricted to the surface waters ( ≈ 5 m depth).

Fig 5.
A schematic illustration demonstrating how a cruise trajectory is co-localized with a gridded product such as satellite sea surface temperature (SST) using user-defined spatiotemporal tolerance parameters. Satellite temperature measurements are linked with the cruise data set if they fall within the spatial tolerance parameter δ r ! (zonal and meridional tolerances). If more than one temperature data point match the conditions, their mean value is linked with the cruise data set. The green dots represent the matched grid data points with the cruise trajectory according to the tolerance parameters. Similar tolerance parameters can be defined along the depth and time axes (not shown in the figure).
based abundance measurement for the picophytoplankton (Prochlorococcus, Synechococcus, pico-eukaryotes) was co-localized with a selected list of 20 contemporaneous environmental parameters including temperature, salinity, nutrient concentrations, altimetry, and surface geostrophic currents derived from multidecade global remote-sensing and numerical model data sets (see supplementary materials Table S2). The tolerance parameters used for the co-localization procedure varied depending on the resolution of the underlying environmental data set, but typically they were set to AE 0:25 for the meridional and zonal directions, AE 5 m for the vertical direction, and the temporal tolerance was generally set to AE 1 day or to AE 4 days for weekly averaged data sets. A monthly climatology value was used when observation data points fell outside the temporal coverage of a given environmental data set. The outcome of this procedure is a large compilation of archived sea surface Cyanobacteria cell abundance observations that are integrated with contemporaneous environmental variables. This data set can then be used to quantify the correlation between the observations and environmental factors, and consequently to conduct more detailed statistical analyses. As a simple example, Fig. 6 illustrates the large-scale correlation between Prochlorococcus cell abundances and environmental variables such as temperature and dissolved phosphate concentrations (panels a and b), highlighting Simons CMAP capabilities in integrating data sets from various sectors of Oceanography (panel c).  (Aumont et al. 2015). Panel (c) presents pairwise Pearson's correlation coefficients computed between the integrated environmental variables and the Cyanobacteria cell abundance observations. For more details about the environmental data sets refer to supplementary materials (Table S2). Uniform data harmonization schemes applied across the Simons CMAP database make it possible to interconnect in situ observations with satellite data and model outputs.
In the absence of Simons CMAP, one would instead need to first identify and collect decades of global observations of picophytoplankton abundances, a substantially time-consuming process as these data sets are scattered across the web and research literature. The collected data sets would then need to be co-localized with ancillary data sets (e.g., decades of global satellite data) with varying resolutions and file formats, a laborious and challenging undertaking. In contrast, due to the local storage of these data sets in the Simons CMAP database and harmonization of the data by location and time, the entire task of collecting and integrating the organism abundances with the environmental factors can be accomplished using short and simple scripts saving a substantial amount of data preparation time. The scripts to collect and integrate data sets and the final compiled data set are publicly accessible (Askezari et al. 2020).

Conclusion
Simons CMAP is designed to alleviate current barriers to utilizing oceanographic data sets that can be difficult to find, may be massive in size, and are typically released in a variety of file formats and resolutions. Our strategy is to collect, curate, and construct a database that hosts a wide spectrum of data sets from all sectors of Oceanography in a unifying data structure, combined with simple interfaces to retrieve and analyze the data. The uniform harmonization of data entries by location and time substantially facilitates the process of integrating data sets which in turn can lead to discovery of intriguing scientific patterns. Many ideas in environmental research involve characterizing the spatiotemporal relations among diverse properties. Simons CMAP creates a unified system that accelerates the ability to explore ideas and hypotheses against highly heterogeneous and diverse mixes of data. Key to this is a unified database architecture and a focus on an interface that aligns with the target research community.
An essential component of the genesis of Simons CMAP was surveys and interviews of diverse researchers to gain insights into how, and under what circumstances, domain scientists across various disciplines of Oceanography discover, acquire, share, and use data as part of their collaborative research work. Such iterative and ongoing involvement of the larger community is essential to creation of data research infrastructures appropriate to support cutting edge research. Development of Simons CMAP will continue to be informed by ongoing usability tests and user interviews.
The Simons CMAP project is envisioned to progressively increase in size and diversity into the future, with a focus on organizing the currently widely dispersed data from research cruises and placing these data within the larger context of satellite products and output from numerical models. The longer-term goal is to develop the infrastructure necessary to collaborate with other marine data initiatives to construct a unified network of interconnected and distributed databases. Development of standard APIs that interconnect these databases with compilations of environmental data such as those found in Simons CMAP will significantly contribute to the progress of multidisciplinary studies.

DATA AVAILABILITY STATEMENT
More than three decades of Cyanobacteria cell abundance observations are compiled and co-localized with a variety of environmental variables. The code developed to collect and integrate, as well as the final generated data set is publicly available (Askezari et al. 2020).