Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RHIZOBACTERIAL COMMUNITIES OF FIVE CO-OCCURRING DESERT HALOPHYTES
Review round: 1
Reviewer: 3

Output:
Basic reporting: The study by Li and collaborators presents results of a study of the bacterial communities in the rhizosphere associated with 5 halophyte plant species and bulk soil taken at a site with high soil salinity in northwest China. This descriptive study details the communities found in these rhizosphere soils and determine how these communities are either similar or different according to plant species and how they compare to bulk soil. The authors study the results at different taxonomic levels and look at which genera are enriched in either soil type and how they are influenced by the plant species. The work is well written although some errors can be found (see below for some examples). Sufficient background on the topic is provided. The figures are adequate although perhaps the rarefaction curves in Figure 1 could be moved to supplementary material.
Experimental design: The experimental design is adequate for the study proposed by the authors. Salinization of soil is becoming an increasingly important problem for which solutions are needed. PGP rhizobacteria may offer a solution to aid plant growth under these adverse conditions therefore studies like the present with halophyte plant species may help to identify the most interesting bacterial species. With this in mind it would have improved the impact of the study if such plant growth promoting species would have been isolated in the study. Nevertheless, one of the points that need to be improved in the work is a better description of the techniques used. It is not clear from the study how bulk soil was collected. Was it associated with the plants sampled by the author’s or taken close to the sampled plant? If it were associated with the plant roots how did they distinguish between bulk soil and rhizosphere soil? Another problem is that the authors do not give any information about how soil chemical analysis (line 132) was performed.
Validity of the findings: The conclusions reached generally are supported by the results presented except for the role associated with effect of salinity (Abstract line 44; conclusions line 413). No data is given which supports that salinity is an important driver in saline soil. In fact it is in contrast to data discussed on lines 277-279 where soil salinity does neither correlate nor appears to cause differences in bulk soil communities. If the authors wish to determine the effect of salinity additional samples in non-saline soil with the same plants would have to have been studied as well. Alternatively, since it is clear that the EC differences do not appear to affect bulk soil communities a closer look could have been taken only at rhizosphere communities by perhaps performing CCA incorporating EC data. Nevertheless it is surprising that the authors did not present the Pearson correlations in the Results section and only mention it in the discussion as supplementary data. The data mentioned in the text as significantly different (lines 186-195) do not agree with the significance given in Table 2. This needs to be made clearer. Another point that needs to be clarified is the results given in the Venn diagram in figure 6. What do the OTU numbers mean? Were these OTUs shared by all three replicate for each plant species? Probably not because the average number of OTUs for the five halophytes was 1692 (line 186) while the total number of OTUs for each halophyte in the Venn diagram is in the order of 4000 OTUs. This point is important because an OTU found in only one replicate sample is more likely to be shared with that from one rather than by all three from another plant species thereby inflating the number of shared OTUs (line 40; line 261, line 393) which could lead to confusing conclusions (lines 406-409).
Additional comments: 1) Fig 1 no figure legend is given to indicate which curves belong to with samples.
2) Examples of errors and sentences which need to be improved:
-Line 92 change was to is
-Line 158 physicochemical
-Line 160 Graphpad
-Line 164 show units for electrical conductance
-Line 173 verb tense
-Line 186, 188 The number of OTUs
-Line 311 separate genusin
-Line 324 change ‘retention’ for reduction
-Line 344 change verb tense
-Line 347 better may be than is
-Line 349 improve ‘matters’
-Line 355 sentence not clear
-Line 406 ANOSIM


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RHIZOBACTERIAL COMMUNITIES OF FIVE CO-OCCURRING DESERT HALOPHYTES
Review round: 2
Reviewer: 1

Output:
Basic reporting: The manuscript is well written, clear and fluid. Nothing to mention here.
Experimental design: The authors corrected the issues that I pointed in the previous version.
Validity of the findings: Since the potential mistakes were corrected I don’t see anything to mention here.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RHIZOBACTERIAL COMMUNITIES OF FIVE CO-OCCURRING DESERT HALOPHYTES
Review round: 2
Reviewer: 2

Output:
Basic reporting: The resubmitted version of the manuscript by Li and collaborators generally has succeeded in addressing the reviewer’s comments and is much improved with regard to the previous version. However, at this point the major concern is the quality of the English. Although the manuscript has apparently been checked by a language editing service and possibly a native speaker, with the new writing many new errors have been introduced which cause more confusion instead of helping to clarify notions. The use of ‘co-occurred’ in the title, abstract, and throughout the text as a verb, adjective or without determiner (which or that) is confusing and unnecessary. This needs to be changed, improved or eliminated throughout. Poor or unclear sentences can be found in the Introduction (lines 63-65, lines 68-72, lines 101-103, lines 104-107, lines 116-118), Results (lines 240-241) and Discussion (lines 339-340, lines 444-446), and Conclusions (lines 487-489, lines 489-491). The Discussion section suggested in the author guidelines of PeerJ for the Abstract is missing. In the Abstract and Conclusions it is not mentioned whether ‘much diverse’ (lines 35 and 472) was more or less.
Experimental design: The experimental design is now much clearer and complete.
Validity of the findings: The conclusions reached generally are supported by the results presented. However there are a few minor questions which arise. Now the authors indicate OTUs shared by all three replicates in their Venn diagrams and discuss them briefly (lines 426-432). The conclusion on lines 430-432 is not clear. Also it would be interesting to know more about the high abundance shared OTUs. Which genera do they belong to? Are some of these OTUs included in the taxa found by LefSe (fig 5)? These could be interesting PGPR candidates as successful colonizers of halophyte plants in saline-arid land. On the other hand, although the study is centered on the rhizosphere of halophytes, it would have been interesting to know more of the bulk soil communities with respect to the shared OTUs, LefSe analyses, and the effect of soil properties on these soil communities (CCA). This could have provided some interesting additional information about arid saline soil bacterial communities.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RHIZOBACTERIAL COMMUNITIES OF FIVE CO-OCCURRING DESERT HALOPHYTES
Review round: 3
Reviewer: 1

Output:
Basic reporting: The resubmitted version of the manuscript by Li and collaborators generally has succeeded in addressing the reviewer’s major and minor comments and is much improved with regard to the previous versions. The improved English allows for more fluid reading of the manuscript and a better understanding of the results and conclusions. Some very minor typos remain such as extra spaces, commas or misplaced or lacking periods, non-italicized ‘H’ in ‘Halostachys’ but these as well as the change of ‘leguminous Rhizobium species’ (Lines 403-404) to ‘the Rhizobium-legume interaction’ will surely be changed during type setting.
Experimental design: The experimental design is clear and complete.
Validity of the findings: The conclusions reached are supported by the results presented.
Additional comments: No comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AWARENESS AND KNOWLEDGE OF CHIKUNGUNYA INFECTION FOLLOWING ITS OUTBREAK IN PAKISTAN AMONG HEALTH CARE STUDENTS AND PROFESSIONALS: A NATIONWIDE SURVEY
Review round: 1
Reviewer: 1

Output:
Basic reporting: .
Experimental design: .
Validity of the findings: .
Additional comments: Awareness and knowledge of Chikungunya infection following its outbreak in Pakistan among health care students and professionals: A nationwide survey
(#25826) by Mallhi et al.,

General comments
The authors is addressing interesting research question of Chikungunya Awareness among Healthcare workers and students which currently is a global health problem.

The manuscript has been well written and all the analysis done appropriately.
However, there are few comments.

Data Collection
Line 127, page 12,…….all authors were asked to contact HCSW!. Did all authors participated in data collection?

Line 135, page 12
Please provide the manufacturers name.
Figure 1
The author should explain why figure 1 is important!.  Text explanation is enough.

Line 196-197
The level of awareness reported by Mansoor et al. was higher as compared to our
findings, which might be ……
How did you measure level of awareness?
How much did you got?
Awareness and Knowledge do they represent the same thing in this study?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AWARENESS AND KNOWLEDGE OF CHIKUNGUNYA INFECTION FOLLOWING ITS OUTBREAK IN PAKISTAN AMONG HEALTH CARE STUDENTS AND PROFESSIONALS: A NATIONWIDE SURVEY
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: Add in methodology the criteria to measure and evaluate awareness about the infection
Validity of the findings: no comment
Additional comments: Line 45 correct the words Adese mosquitoes by Aedes mosquitoes


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AWARENESS AND KNOWLEDGE OF CHIKUNGUNYA INFECTION FOLLOWING ITS OUTBREAK IN PAKISTAN AMONG HEALTH CARE STUDENTS AND PROFESSIONALS: A NATIONWIDE SURVEY
Review round: 1
Reviewer: 3

Output:
Basic reporting: The abstract should be reviewed and edited by a native English speaker as it is difficult to read and understand at this stage.

There is a lack of information about the epidemiological context. Was the magnitude of the epidemic estimated? Were there any prevalence survey conducted in the country?

There is no review of literature about the knowledge of chikungunya in the region prior to the discussion. As much as I know, several KABP studies were conducted in the Indian Ocean in the late 2000ies.
Experimental design: How were the participants slected and recruited? There is no information about this...
Validity of the findings: Conclusion are well stated, and linked to original research question.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AWARENESS AND KNOWLEDGE OF CHIKUNGUNYA INFECTION FOLLOWING ITS OUTBREAK IN PAKISTAN AMONG HEALTH CARE STUDENTS AND PROFESSIONALS: A NATIONWIDE SURVEY
Review round: 1
Reviewer: 4

Output:
Basic reporting: The authors Mallhi et al. have presented their findings on Awareness and Knowledge of chikungunya infection in health care students and workers in Pakistan. The text needs significant improvement and clarification and must not repeat sentences from already published articles.

For instance: the authors have copied text from an already published manuscript.
"First chikungunya outbreak in Pakistan: a trail of viral attacks"
New Microbes New Infect. 2017 Sep; 19: 13–14.
Published online 2017 Jun 1.
doi:  10.1016/j.nmni.2017.05.008.

In the Introduction, 1st Paragraph:
"The history of chikungunya in Pakistan dates to the early 1980s, when Darwish et al. [1] in 1983 detected antibodies against the chikungunya virus from sera of four rodents and one human. However, virus spillover did not occur on that occasion. Later, in 2015, three cases of chikungunya were identified in children during a 2011 dengue outbreak"

2nd paragraph:
"Pakistan has experienced a quadruple burden of viral infections in 2016, including Crimean-Congo haemorrhagic fever and outbreaks of dengue and chikungunya. These three consecutive viral infections have created much concern among health authorities and the World Health Organization".

The above mentioned statements are exactly copied from an article on Chikungunya published by authors, with some of them sharing both manuscripts. As a rule, even the manuscript on the similar subject by the same authors should not be replicated as such.
Experimental design: The authors have claimed to present their findings based on a "nationwide" survey. However, the data is biased with over-representation from a province "Punjab" with 229 respondents, compared to only a single respondent from "Azad Jammu & Kashmir (AJK) and less than ten in each of region FATA, GB, Balochistan. This ambiguity poses a serious challenge to target areas where the disease control interventions need to be initiated.
Validity of the findings: The authors should analyze data based on the hospitals known to receive and treat chikungunya patients compared to those without attending such patients. Without this, the true assessment on knowledge and awareness may not be revealed effectively.

The authors should work on data uniformity across all study groups, for example, among the 319 respondents, there were 179 (56%) pharmacist and the total number of nurses, who usually are at much higher risk, were just 6 (2%).
Additional comments: The authors should significantly improve the draft as per comments given in above three sections: Basic reporting, Experimental Design and Validity of Findings. The manuscript must be reviewed by an expert in English language.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AWARENESS AND KNOWLEDGE OF CHIKUNGUNYA INFECTION FOLLOWING ITS OUTBREAK IN PAKISTAN AMONG HEALTH CARE STUDENTS AND PROFESSIONALS: A NATIONWIDE SURVEY
Review round: 2
Reviewer: 1

Output:
Basic reporting: A brief on the epidemiology of recent outbreak of chikungunya in Pakistan can be added to the Introduction section, quoting the two recently published articles:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5587410/pdf/cureus-0009-00000001430.pdf
https://link.springer.com/content/pdf/10.1007%2Fs12250-017-4077-5.pdf
Experimental design: No Comment
Validity of the findings: No Comment
Additional comments: The authors have significantly improved the manuscript draft.

Major Revision:
The Introduction section should be updated by quoting recent outbreak event of chikungunya in Pakistan as presented in the below-mentioned two articles:

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5587410/pdf/cureus-0009-00000001430.pdf
https://link.springer.com/content/pdf/10.1007%2Fs12250-017-4077-5.pdf

Minor Revisions:

The format of references should be corrected and unified as per journal's standards.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WALKING AND HYPERTENSION: GREATER REDUCTIONS IN SUBJECTS WITH HIGHER BASELINE SYSTOLIC BLOOD PRESSURE FOLLOWING SIX MONTHS OF GUIDED WALKING
Review round: 1
Reviewer: 1

Output:
Basic reporting: Clear and concise reporting throughout. The manuscript is generally well-referenced and structured appropriately. The figures and tables are presented professionally. The standard of English is good and suitable for publication.
Experimental design: The authors have presented important 'real-world' data regarding the efficacy of walking activity on resting BP in hypertensive patients, stratified according to baseline BP.

The methods are well-described and the research questions has been sufficiently outlined.
The experimental design has several important limitations. One suggestion might be as follows:
1. This is an uncontrolled study, absent of any control group not participating in a walking intervention. It would be interesting to compare these findings between participants who achieved a certain threshold of weekly walking versus those who remained relatively sedentary. Although this analysis would contain several biases, I believe it would strengthen the findings of this study.
Validity of the findings: I encourage the findings of this study, but we should be carful not to push these findings too far. This study has some important aspects that should be addressed, that will ultimately strengthen the conclusions.
1. Hypertensive patients are rarely free of comorbidities that may alter their participation or benefits from walking. Preferably, the authors should clarify these.
2. The authors suggest that medical therapy remained unchanged through the study, yet baseline medications are not reported in detail. It would be an advantage to include these perhaps in table format for clarity. Presumably, those patients with higher baseline BP may be those on more aggressive therapy (i.e. multiple agents)? The authors should consider whether a suitable adjustment can be made for this?
3. Please include mean and SD walking duration across all participants, rather than just stating that they walked up to 300 minutes per week (i.e. line 105+106), Did the mean walking duration differ according to baseline BP group?
4. Please try and avoid over-dichotomising. For example, figure 2 would benefit from including all patient data rather than just a group mean. This provides much more valuable information on the response according to baseline BP (including 95% CI of the regression to quantify variance).
Additional comments: The authors should be commended on an important analysis. I am confident that with some relatively minor changes, this will be an important study.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WALKING AND HYPERTENSION: GREATER REDUCTIONS IN SUBJECTS WITH HIGHER BASELINE SYSTOLIC BLOOD PRESSURE FOLLOWING SIX MONTHS OF GUIDED WALKING
Review round: 1
Reviewer: 2

Output:
Basic reporting: Yes. Well-structured article. English is fine but I have made some recommendations to restructure sentences or rephrase things in places. Please see my minor revision section.
Experimental design: Please add following declaration of Helsinki when talking about informed consent process. Also please see my comments in major revisions section regarding additional analyses for comparing subgroups based on SBP.
Validity of the findings: I think that this work is impactful and novel in that it shows the importance of baseline systolic blood pressure on improvement. All hypertensive patients should be encouraged to exercise to achieve BP reductions but patients with SBP >150 mmHg in particular should incorporate moderate-intensity PA into their daily life as they see the biggest benefit. Statistics are fine but I would like to see additional analyses once the authors split into subgroups to ascertain whether changes are different by subgroup.
Additional comments: Major comments
1.	I like how the statistics are done to compare pre-post changes in each subgroup but I would like to see additional analyses done to compare each of these subgroups. Should be pretty easy to do. You could run an ANOVA to compare the delta change between sub-groups. Right now you are saying the greatest improvement occurs in the higher baseline SBP group but you haven’t actually run stats to show that improvements in these groups were significantly greater than other sub-groups. Only that the value is higher.
Line 48: Please add that study procedures were carried out after obtaining written informed consent in accordance with the Helsinki Declaration. This is required by the journal.
Line 60: Did you control when and dosage of hypertensive drugs taken prior to each measurement? Just need to be consistent for example if patients took them a few hours prior to measurement at baseline needs to be the same or at least similar when you are re-evaluating them at 6 months.
Discussion
Line 142-147: Are these changes in body weight, BMI and waist circumference similar to other studies that have used walking interventions? Please discuss other studies and also how changes in these variables may reduce CVD risk.
Line 146-147: Are increases in walking speed associated with reductions in CVD risk? Improvements in fitness and functional capacity? Tons of work by Blair, Myers, etc on this topic I believe please cite some of this work here.
Line 157-159: BP lowering is much larger in the higher SBP groups yet body weight is reduced uniformly across subgroups therefore BP lowering is not entirely driven by changes in body weight as you correctly point out. Why do you think the improvement is so much greater in the higher SBP subgroups? Feel free to speculate with a few sentences and use the literature to guide you here. Differences in baseline PA levels in addition to higher SBP maybe?
Line 197-201: Unless this is part of the journal instructions, I recommend moving this limitations section to the last part of the discussion. So it will now be the last paragraph of the discussion.
Line 199-201: Please insert citations for these previous randomized controlled trials you mention in this sentence.
Conclusion:
I really like the conclusion. You took your findings and applied it clinically. Keep as is.
Minor comments/revisions
Abstract
Line 2: Change to the following… “The aim of the study was to assess the effects of walking on blood pressure in sedentary adults with differing degrees of hypertension.”
Line 6: Change to the following… “Walking sessions were supervised by an exercise physiologist.”
Line 7: Change to the following… “The average weekly walking time during the six month study was xxx ± xx. Please insert mean ± SD for number of minutes walked per week where you see xxx ± xx.
Line 7-17: You don’t need to start a new line at line 8 or line 11 in the results section of the abstract group all sentences together in the abstract results section.
Line 12: Change this sentence to start as “Following 6 months of supervised walking, systolic blood pressure reduced significantly in each subgroup with lower of…” Also please add a p-value (p<0.001 or p<0.05) at the end of the sentence in parentheses.
Line 15: Change this sentence to “Diastolic blood pressure, body weight, body mass index and waist circumference were also significantly reduced following the walking intervention.” Also please add a p-value (p<0.001 or p<0.05) at the end of the sentence in parentheses or after each variable.
Line 17: Remove this sentence “Walking speed increased significantly and uniformly in all groups.” This is an interesting finding and definitely belongs in the main body of the paper but it is not needed in the abstract, as your main variables of interest are blood pressure.
Line 18: Consider rephrasing to as follows: “In a large group of sedentary adults with varying degrees of hypertension, 6 months of supervised walking elicited a marked reduction in systolic blood pressure with the largest reductions in pressure occurring in individuals with higher baseline systolic pressure.”
Introduction
Line 38: Rephrase to read as follows “Taking into consideration the 2017…”
Materials and Methods
Line 46: I would change to “The study was advertised through local newspapers…”
Line 58: Change to “No physical activity was performed <3 hours prior to this evaluation.”
Line 89: Please add blood pressure here. It is your main outcome variable.
Line 100: Excellent job on stats very clean and easy to follow. See comment about additional analysis in the Major revisions section.
Results
Line 104: Change to “six month walking program”
Line 105: Did everyone do 300 minutes per week? Would be great to see Mean ± SD for minutes per week if you have that data.
Line 108: Thank for including this information about dosage of drug not changing during the intervention.
Table 1, Table 2 and Table 3: please add units of measure for the variables measured. For example body weight (kg), body mass index (kg/m2), etc…
Table 2 and Table 3: please add standard deviatons.
Line 127: Please consider rephrasing to “Diastolic blood pressure was also significantly reduced following six months of walking in all subgroups (Table 1).”
Line 128: See my comment about adding additional analysis to assess differences between subgroups.
Line 133: How do you know this? Are the statistically higher? Consider analyzing and simply adding to Table 2 if significant differences exist at baseline for anthropometric variables.
Line 135: Please consider rephrasing to “Following six months of supervised walking, body weight, BMI and waist circumference decreased significantly in all subgroups.”
Line 138: consider adding data not shown at the end of this sentence. I totally believe that there was no correlation but just add it in.
Line 138: consider rephrasing to “Finally, walking speed increased significantly in all subgroups”.
Discussion
Line 142: Move line 148 to here. Start your discussion with your main findings sentence. Rephrase it start the discussion with “The main finding of the current study is that six months of supervised walking elicits significant reductions in systolic blood pressure a large group of sedentary hypertensive adults with varying degrees of hypertension.”
Line 142-147: move this paragraph to be the 2nd paragraph of the discussion. Keep the 1st paragraph only about your main findings and blood pressure.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WALKING AND HYPERTENSION: GREATER REDUCTIONS IN SUBJECTS WITH HIGHER BASELINE SYSTOLIC BLOOD PRESSURE FOLLOWING SIX MONTHS OF GUIDED WALKING
Review round: 1
Reviewer: 3

Output:
Basic reporting: In the main, the manuscript has clear professional English. The article is brief and to the point, which this reviewer appreciates. However, I will point out that the introduction (and discussion) requires further 'balancing' of literature. There is an extensive body of research in physical activity and blood pressure (particularly epidemiology) which should be drawn upon to allow narrowing of your focus to a hypothesis (at the end of your introduction).
Experimental design: The manuscript presents original research, with ethical clearance that falls withing the Scope of PeerJ. The research question is relevant but lacks a meaningful definition. For instance, there is no hypothesis to which the article can be held accountable to. This has important ramifications for methods/results/discussion. However, this can be fixed.

Technical aspects:
https://doi.org/10.1161/HYP.0000000000000066 Hypertension. 2018;71:1269-1324

blood pressure diagnosis, using the criteria above, uses an average of ≥2 readings obtained on ≥2 occasions to estimate the individual’s level of BP. At the first visit, record BP in both arms. Use the arm that gives the higher reading for subsequent readings and separate repeated measurements by 1–2 min.This does not appear to have been adhered to in the present manuscript. Further, patients/subjects should avoid caffeine, exercise, and smoking for at least 30 min before measurement. Investigators should use a BP measurement device that has been validated, and ensure that the device is calibrated periodically. Similarly, investigators should use the correct cuff size, such that the bladder encircles 80% of the arm, and note if a larger- or smaller-than-normal cuff size is used, thus ensuring that proper measurements needed for diagnosis and treatment of elevated BP/hypertension

Validity and reliability of the walking test is not presented. The measurement of blood pressure requires further clarity (see below for more detail). These aspects may seem trivial but the authors need sufficient detail as to allow the study to be replicated.

The narrative dialogue of the present exercise programme requires further details of consistency. There is no reporting of subjective / objective determination of activity level prior to enrollment to the study. This is normal for physical activity studies. The methods section It is lacking in detail normally expected. For instance, exercise AND counseling is not an exercise program per se, it is a combined program. It is critical for these points to b acknowledged.
Validity of the findings: Data do not match that reported within the study. Specifically, 520 recruited, 284 dropped out and analysis was performed on 296 ‘hypertensives’. This appears to be a miscalculation.

If we forget the miscalculation, there remains two important problems for the authors to resolve;

             1. There is an inherent sample bias in selecting to analyze those that ‘survived’ a 6 month walking program. It would be of significant value if you had intermediary data in those that dropped out. This would allow you to employ a quasi ‘intention to treat' analysis.

             2.  Under the 2017 AHA (et al.) clinical guidelines specified in your introduction, ‘hypertension’ begins at SBP>130mmHg, whereas SBP:120-129mmHg is classified as ‘elevated blood pressure’. The authors should address this important nomenclature throughout the document. This becomes critical in this work because 'elevated blood pressure' requires the combination of SBP and DBP to be presented. This is of major importance to improve this article.

In my view, the statistical analysis of data requires some focus. I will leave this with the authors apart for indicating that group x time comparison of multiple data requires a multivariate analysis of variance. Similarly, where categorical analysis is being used (ie...movement from categories of hypertension), particular (non linear) statistics should be employed.  That is, you should allow for multiple categorical comparisons.

It is difficult for me to proceed with interpretation of this study as a reanalysis that addresses the reporting bias (ie..those who dropped out) is important to make a meaningful interrogation of your discussion and conclusion. However, I would suggest that this should be done for any re-submission.
Additional comments: Dear Author team,

Please take these comments in the spirit that they are meant. In my opinion, I believe that there are important limitations that have not been acknowledged and some key methodological issues that you will need to address. Encouragingly, it seems that many of these can be fixed and in doing so, you will have a manuscript that provides some value to the scientific literature.

Best wishes.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WALKING AND HYPERTENSION: GREATER REDUCTIONS IN SUBJECTS WITH HIGHER BASELINE SYSTOLIC BLOOD PRESSURE FOLLOWING SIX MONTHS OF GUIDED WALKING
Review round: 2
Reviewer: 1

Output:
Basic reporting: Yes. Well-structured article. English is fine but I have made some recommendations to restructure sentences or rephrase things in places. Please see my minor revision section.
Experimental design: No comment.
Validity of the findings: I think that this work is impactful and novel in that it shows the importance of baseline systolic blood pressure on improvement. All hypertensive patients should be encouraged to exercise to achieve BP reductions but patients with SBP >150 mmHg in particular should incorporate moderate-intensity PA into their daily life as they see the biggest benefit.

All of my comments in follow-up review are to improve the readability of the published paper.
Additional comments: Great job on the responses and revisions. Please strongly consider making a few minor changes to the final version that will appear in print as I feel these will greatly enhance the readability of the paper.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WALKING AND HYPERTENSION: GREATER REDUCTIONS IN SUBJECTS WITH HIGHER BASELINE SYSTOLIC BLOOD PRESSURE FOLLOWING SIX MONTHS OF GUIDED WALKING
Review round: 2
Reviewer: 2

Output:
Basic reporting: Sound.
Experimental design: Sound.
For clarity within the abstract, I recommend replacing "Of the 296 subjects completed the project" with "56% of participants completed the 6 month intervention".
Validity of the findings: Sound
Additional comments: The manuscript has been significantly improved and worthy of publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: BIOWORKBENCH: A HIGH-PERFORMANCE FRAMEWORK FOR MANAGING AND ANALYZING BIOINFORMATICS EXPERIMENTS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The article has clear and unambiguous language, explaining well the methods and background of the work.

The coverage of related scientific workflow systems, while extensive, seem a bit dated or HPC-centric; not including popular bioinformatics workflow systems like Nextflow, Common Workflow Language, Toil or Galaxy. Relevant reviews could be https://doi.org/10.1093/bib/bbw020 and https://doi.org/10.1007/s41019-017-0047-z

The article fails to describe clearly what it the new contribution to the field, as it suggests a BioWorkbench "framework", but it is unclear if this means a general methodology (which should then be further explained as such) or a piece of software (which should then be made available).

Buried in the text is potentially interesting research on using machine learning for predicting execution times and analysing provenance, but as this is not the focus of the article (submitted as "Bioinformatics Software"), this gets a bit in the way of introducing the BioWorkbench framework. I would welcome a change of focus (perhaps separate paper) exploring this approach in detail, but then it would need to relate to other research on predicting computational parameters.

My main challenge with the paper is that it lacks focus and covers several aspects, seemingly proposing new bioinformatics software which is only partially made available.
Experimental design: The authors propose reusing the Swift language for writing bioinformatics workflows because of its scalability benefits and support for high-performance compute backends.

The paper explains in detail the layers of the proposed BioWorkbench architecture which extends Swift to analyze the provenance using R scripts and a web UI. The BioWorkbench workflows reuse existing bioinformatics tools like picard and bowtie, some of which are wrapped in Python scripts. The scripts are then composed within Swift which provides parallelism between the chained tool executions.

The internal provenance model of Swift is explored and exercised with several example queries, however this has not been related to existing approaches to standardizing workflow provenance, such as PROV, OPMW, ProvOne, D-PROV and Research Object. Extensive literature on workflow provenance exists, see for instance the recent IPAW conference proceedings.

It is not well justified why BioWorkbench should use a custom Swift-specific provenance model rather than one of the existing models, other than that this is what was already given by the Swift.

The article uses three bioinformatics pipelines as use-cases, and explains well how these were implemented with the Swift language.  Two of the workflows were provided as GitHub links
https://github.com/mmondelli/swift-phylo and https://github.com/mmondelli/swift-gecko - while perhaps the most complete workflow Rawflow is only provided as a manually composed Docker image. I inspected this to find the third source reference https://github.com/mmondelli/rasflow

All of these URLs are subject to change in availability and content; I would request the authors to use the Zenodo-GitHub integration
https://guides.github.com/activities/citable-code/ to make Software DOI citations with permanent availability.

For the Docker image perhaps a Zenodo upload of the `docker export` tar, as well as a DOI citation also for (apparently) the underlying Dockerfile recipe https://github.com/mmondelli/docker-masters/ (I had to browse GitHub to find this)

It is not specified which Swift version is used (citation Wilde2011) - the above Dockerfile hints of http://swift-lang.org/packages/swift-0.96.2.tar.gz (latest version from 2015). Similarly explicitly declaring the version of each tool used is generally important for bioinformatics pipelines - the BioWorkbench does not seem to handle installation of bioinformatics software (e.g. from BioConda) and these are instead added manually to the Docker image or directly as binaries in the GitHub source code repository, but missing their attribution and open source licenses.

The authors are asked to improve their BioWorkbench distributions to comply with the open source licenses of the tools and software they redistribute.

I will commend the authors on providing a Docker image for reproducibility, however this seems to be incomplete and lack instructions for usage. Some care seems to be needed by users, setting up paths and shell environment variables for running the workflows, but this process is not well explained. The image was still valuable for me in reviewing this paper, but I believe in its current form it is not reusable as "Bioinformatics Software", and so I cannot recognise this as the proposed "BioWorkbench framework".

I am generally unable to replicate any of the results of the paper, even when the authors have a provided Docker image. The main reason for this is that the image lacks documentation, and is incompletely setup - e.g. I had to perform several manual installation steps in the shell of the docker image, but was still unable to generate provenance information.  For instance the workflow seems to require a postgres database, which is neither set up or correctly configured from the workflow.

Some of the workflows contain brief documentation, and I managed to run those inside the Docker image. I did not assess their execution beyond the fact that they produced bioinformatics-like result files.

I must admit I am unable to locate the described "BioWorkbench" software, neither on GitHub or (to my knowledge) in the Docker image, so I am unable to verify any of the claims on the provenance browsing or web interface.

The paper describes workflow execution with bacterial genome sequences, but these datasets are not cited or otherwise made available.

The Weka configurations, customization source code, source datasets and result datasets have not been made available, and are not present in the Docker image. The authors do not specify which version of Weka was used. I am unable to verify any of the claims around machine learning.
Validity of the findings: I have not reviewed the validity of the bioinformatics pipelines or the machine learning approach.

Some of the data are only made available as graphical figures (e.g. Figure 5, 6), and others as tables or inline in text. If this paper was to focus on benchmarking or machine learning approach, the actual results data should also be made available, e.g. on Figshare or Zenodo.

The paper discusses speed-ups in parallelism of the Swift workflows compared to sequential execution. As this would be the case for most "embarrassingly parallel" pipelines in bioinformatics, commonly using distributed and scalable grid/cloud workflow systems, I don't think there is much information gain in providing the % speed-up here, at least without comparisons to other existing approaches (even if this would just show a comparable scaling factor).
Additional comments: Hi, I am Stian Soiland-Reyes http://orcid.org/0000-0001-9842-9718 and believe in open reviews.

The complete review is also available at the (secret) URL
https://gist.github.com/stain/eedf7c3a355e347863a2381a2f55e957
for easier reading and more hyperlinks.

This review is licensed under a
Creative Commons Attribution 4.0 International License
http://creativecommons.org/licenses/by/4.0/

This paper seem to show an area of interesting research by the authors, but tries to cover a bit too many things at once.

Covering large aspects of a piece of software might be appropriate once it is in a mature, reusable state or have an established user-base - but as this stands I get more of the impression that the proposed BioWorkbench is mainly a snapshot of one person's research methodology.

In the guidelines for PeerJ submissions of Bioinformatics Software it is strongly recommended that the software is made available open source -- this is so that it can be reused and expanded by other researchers. The proposed BioWorkbench is either incomplete or insufficiently described and packaged. Further work would be to mature the software so it is a tangible piece in its own right (e.g. with its own GitHub repository and reproducible installation instructions), rather than just a usage pattern of the Swift language.

Achieving scalability with distributed and parallelized pipelines is common practice in bioinformatics, and while http://s.apache.org/existing-workflow-systems currently lists 200 workflow systems (including Swift), there is of course room for BioWorkbench as well, particularly with focus on provenance and predicting computational parameters with machine learning.

Where this paper goes interesting is in the machine learning approach, in particular in relating computational parameters with the domain-specific input parameters like sequence length -- this is in particular a challenging area for metagenomics when trying to predict good job allocations in distributed computing (e.g. required memory is not known in advance).  As this is just one of the many areas of this paper, this does not get the space it deserves.  This is also the part of the research where all data should be easy to share and the post-workflow provenance analysis results easy to reproduce, had they been made available as datasets together with this paper - sadly here most details were missing.

I would recommend the authors to:

* Split out machine learning approach to a new paper. A citation to a first draft at arxiv can be used as placeholder here.
* Improve BioWorkbench as a separate piece of software (git repo, install instructions)
* Make sure the software and workflows can be re-run (and hopefully re-used) by other people
* Make sure the Docker image is complete and runnable out of the box. Include detailed step-by-step instructions for each workflow as well as for how to do post-run provenance analysis.
* Relate or convert custom provenance to existing PROV standards. This could for instance be an export script to one of the PROV formats.
* Refocus this paper to have a single focus -- this might be  BioWorkbench provenance, the bioinformatics workflows, or  machine learning.
* The paper needs to give a stronger claim to what is the proposed contribution, and add a more thorough Discussion section that can  then compare this to other existing work. This might require updates to the Background section.
* Ensure all contributed software and datasets are provided with the article, preserved (and given DOI) using services like Zenodo or FigShare; in addition to the GitHub/DockerHub approach.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: BIOWORKBENCH: A HIGH-PERFORMANCE FRAMEWORK FOR MANAGING AND ANALYZING BIOINFORMATICS EXPERIMENTS
Review round: 1
Reviewer: 2

Output:
Basic reporting: Overall the article is well written and easy to follow.
Experimental design: Overall the approach is sound, and the proposed architecture shows real benefits.
The remaining questions (see detailed comments below) can be addressed in a minor revision.
Validity of the findings: The findings are very plausible. But some extra care should be taken to address questions raised in the detailed comments below.
Additional comments: * INTRODUCTION
- P2, L50: life-cycle of a scientific workflow: design, execution, and analysis
=> What is meant here by 'analysis'?
Is it the analysis of the workflow outputs/results, or the analysis of the workflow itself?
It seems to be the former. Maybe simply call it "result analysis" to avoid a possible confusion?

- P2, L72: You refer to BWB as a 'framework' but also as a 'tool'.
Please elaborate and clarify what it is: it could be both, but there are different expectations associated with those names. For example, if BWB is a tool, then is it something I can download, install, and run in my own environment (at my institution)? Or is it a service/webapp, and if so, who can use it and how? Or is it framework that you prototyped and that you're reporting on, but there is no expectations for other to be able to use it?
(Later in the paper, this becomes more clear, but I suggest to use consistent wording / positioning of BWB throughout the paper.)

- P2, L77: A reduction of 98.9% sounds really good, but the maths seems unnecessarily complicated.
Why not just say something along the lines of "in the case of SwiftPhylo we achieved a speedup of about 100x, i.e., a 13.35 hours execution time was reduced to 8 minutes"

* RELATED WORK

- P2, L88: ".. Pegasus .. enables the specification of workflows through the XML format."
I don't think this comment is particularly helpful as it seems to involve a kind of category error:
Is it really worth mentioning  what exchange *syntax* is used for a  workflow (XML, JSON, YAML, ...)?
What about the specific XML (JSON, YAML) schema being used?
In fact, the schema itself is also one level of indirection away from understanding a more important aspect of the system, i.e., the underlying model of execution employed by Pegasus:
Is the Pegasus model similar (or identical?) to the model of Condor/DAGMAN? Does Pegasus use partially-ordered task dependency graphs (DAGs), where a tasks can only start after all its upstream dependencies have completed?
Or does Pegasus support other execution models, e.g.,  pipeline parallelism (maybe with promises and futures)?

- P2, L90: ".. provenance .. can be queried using the SPARQL language".
Similarly to the point above, the use of SPARQL seems somewhat secondary. What is the provenance model that is being employed? What *kind* of questions are supported by the underlying provenance model?
For example, you advertise that in BWB you have two kinds of provenance that can be employed, i.e., monitoring  and improving performance, and to query domain data dependencies. Is this also the case for Wings/Pegasus? If not, what is the nature of the provenance queries they support?

- P3, L99-103: In contrast to some of the "formal/syntactic differences" mentioned above, the following are examples of the kind of more meaningful differences that I think readers will appreciate when combing through the Related Work section:
-- "[SciCumulus] .. is distinct in that it allows provenance queries .. during the execution .."
--  ".. we use the Swift SWfMS because it transparently supports the execution of workflows in different HPC environments"
-- " .. it supports workflow patterns such as conditional execution and iterations that not supported by .. "

- P3, L123-126: "We  show that it is possible to combine provenance and domain data for more detailed analysis. Also, we demonstrate that these analyses can benefit from machine learning  techniques for extracting relevant information and predicting computational resource consumption that  are not readily detected by queries and statistics from the provenance database."

These findings and results seem to be buried here in the wrong (Related Work) section. They should be moved (or restated) prominently elsewhere in the paper, e.g., under "Contributions" or "Results".

- The Related Work section could also benefit from a discussion of/comparison with the following:

-- Kanwal, Sehrish, Farah Zaib Khan, Andrew Lonie, and Richard O. Sinnott. 2017. “Investigating Reproducibility and Tracking Provenance – A Genomic Workflow Case Study.” BMC Bioinformatics 18.

-- Altintas, Ilkay, Jianwu Wang, Daniel Crawl, and Weizhong Li. "Challenges and approaches for distributed workflow-driven analysis of large-scale biological data: vision paper." In Proceedings of the 2012 Joint EDBT/ICDT Workshops, ACM, 2012.

* MATERIALS and METHODS

- SPECIFICATION AND EXECUTION LAYER
-- Nice introduction to Swift. However, I wonder how one should think of Swift as a model of execution? Is it elements of a functional scripting language.  But there is also this (P4, L160):
  "All expressions in a Swift script whose data dependencies are met are evaluated in parallel. "
 This seems to indicate that Swift also has dependency driven features similar to MAKE or Condor/DAGMAN.
 Can you clarify/elaborate a bit?
 (later in the paper you talk about "activity chaining" .. so maybe mention it earlier or make a forward reference?)

- P4, L158: something missing here: ".. applications outside Swift should be executed, their inputs and outputs."

- P4,L175: something missing here: ".. analyzing the data derivation process of the experiment possible."

- Figure 2 is useful to get an idea of the provenance model. However, I noticed a symmetry w.r.t. the cardinality constraints of stage_in and stage_out, where I would have expected an asymmetry: It seems a file can be consumed by many app_execs but should probably only be produced by a single one (to avoid a write conflict) - or am I missing something? Please elaborate!

- While the model in Figure 2 is nice and simple, there must be a lot more going on in terms of the "complete provenance model".
You emphasize essentially two kinds of uses of provenance information, i.e., for monitoring (and then improving) execution provenance, and for querying science data.
Can you say a bit more about how the "dependency model" in Figure 2 is connected with other provenance information and metadata needed to answer the two different kinds of provenance use cases?

- P6: You describe the use of Weka and ML techniques in support of provenance analysis.
This section is a bit disconnected from the rest of the story.
And while I understand from the rest of the paper that BWB is essentially a framework and "demo"/proof of concept (not meant as a production tool), I still wonder what the challenges were in bringing together so many different system components: Swift, R, Weka, ...
For example, would you anticipate  a production-level version of BWB to also use that many different systems?
What are the repercussions in terms of system maintenance, with all constituent tools evolving independently?
This might be worth mentioning or discussing..

*  RESULTS AND DISCUSSION

- P6, L268-270: "It is worth mentioning that the computational resource is shared with other users and was not dedicated to these executions. This can be considered as one of the factors influencing the performance gains. "

First and foremost, I commend you for including this frank comment!
However, it raises and important question: the 100x speedup reported elsewhere in the paper: how "real" or robust is it?
If some of it might be attributable to differences in system load, it seems something has to be done about it. There seem to be a number of options, for example:
-- Couldn't you use some of your provenance data, especially on execution monitoring, to do a careful accounting of total cycles spent?
(in that case, it would be interesting to see whether/how cpu-time, user-time, wall-clock time etc. are measured and accumulated)
-- Or what about repeating the experiment many times, to see what the variation across runs (and supposedly different load situations) is?

I understand that meaningful benchmarking is a non-trivial exercise all by itself, but it seems that a bit more could be done here to sort this out. After all, provenance collection and reproducibility are a major theme of the paper, and it would be ironic to not be able to reproduce your findings.

For example, how about re-executing some of the experiments, maybe at a smaller scale, on a dedicated system to see what happens with the 100x speedup?  Since you have a docker container for the system/demo available, wouldn't it be easy to rerun some experiments?
But I fear that in practice things are quite a bit more complex, not just because of the limited availability of hardware (say the 160 cores, 2TB RAM machine; again maybe a smaller machine would do for a replication study), but also because of having a docker container probably isn't quite the silver bullet for computational reproducibility as we're often led to believe.

If it is practically feasible to do a little replication experiment to see whether you can get a similar 100x speedup, then I encourage you to do so and report on the findings. On the other hand, if that's not practically feasible, I suggest to make clear that the 98% reduction / 100x speedup has to be taken with a grain of salt. After all, based on this paper someone else might try and replicate your findings and might have a hard time doing so, despite all the right elements being brought into play (use of detailed provenance information, docker containers etc.)

In particular, I think this sentence needs to be reworded:
"Through the provenance data, we show that the framework is scalable and achieves high performance, reducing up to 98% of the case studies execution time."

While the "up to" makes it clear that one cannot  expect this speedup in all case studies, it doesn't rule out that for the particular case study (SwiftPhylo) this result is replicable (incidentally, you report 98.9% elsewehre in the paper, which is closer to 99% or the 100x speedup).
On the other hand, Listing 1 (SwiftPhylo spec.) seems to exhibit an "embarassingly parallel" use case, in which such a speedup isn't all that surprising (especially on corresponding hardware). However it is not clear why one would need provenance data to establish this speedup.

- P6, L272: "The Docker container was built for reuse purposes only, to encapsulate the components that compose the framework."
This begs the question: where any of the use cases run with the Docker container at all? If not, why not?
How would one know that the container "works" if it isn't tested with (at least downscaled versions of) the use cases?

- P7, L277: "The construction of phylogenetic trees allows understanding the evolutionary relationship between organisms, determining the ancestral relationships between known species"
I suggest to reformulate since the output of this analysis is a hypothesis (or adds some evidence to an existing hypothesis). The resulting tree doesn't imply that an actual relationship has been determined with certainty (although it might increase the confidence that certain relationships exist). Maybe something along the lines of ".. allows to study the evolutionary relationships between organisms.." could be used to clarify that the nature of a result of a phylogenetic analysis isn't as clear cut as, e.g., the prime factorization of a large integer.

- P7, L281 SwiftPhylo also uses Perl (adding to the already complex software dependencies).
You mention elsewhere in that paper that you make a container available.
Do you also share the docker recipe? It would be illustrative to include it among the supplementary material of this article to give the reader a full appreciation of the "rich" dependency structure and complexity of putting this application together.

- Figures 5 (in particular) and 6 increase the confidence of the reader that your near 100x speedup for SwiftPhylo on the appropriate hardware is "real" (again, Listing 1 seems to hint at an  "embarassingly parallel" situation, which is nice to have indeed).
However, it still would be helpful to have an understanding how large the (missing!) error bars would be.
It seems that even with only 10 cores, you get in the range of (wall-clock?) runtime of about an hour.
Have you rerun the experiments a few times and are those runtimes available in your provenance store?

- P10, L 345-347: "The execution time decreased from ∼2.1 hours to ∼6 minutes, representing a reduction of ∼96.6%. Also, the workflow was ∼30 times faster when executed in parallel, using 160 processors."
This makes it sound as if the 30x speedup is a measurement separate from the 96.6% reduction, but both refer to the same reduction from 2.1 hours to 6 minutes, correct?
Please reformulate.
For example, instead of  "Also ..."
rather something like : ".. this corresponds to a roughly 30x speedup"
Incidentally this seems to be in terms of wall-clock time (it can't be cpu-time, right? After all, the total cpu-time would be approximately the same, when costing is based on total time spent across all cores..)

- P10, Table 1: Just confirming - is the table sorted in "workflow order"? (might be worth mentioning)
Or how about sorting it by duration (descending) to make it easier on the reader?

- P11, Equation (1): Can you explain this equation a bit more? It's a bit hard to parse..
Is this an arithmetic equation? Then what is the meaning of "v" (looks like a logical disjunction)
After spending a bit of time with it, I think I understand it now, but still encourage to try and make it clearer.
For example, how about something like this:

  - 931.4025 * (word_length \in {10,12})
- 2704.5534 * (word_length = 10)
              + 0 * total_written
- 1006.3049 * (total_genomes \in {20,30,40})
  - 547.6854 * (total_genomes \in {30,40})
... etc

When computing this regression, how many data points did you have?
From how many different runs?

- P12, L384: ".. these experiments demonstrate that is possible to generate simple models able to efficiently predict the computational time demanded by the workflow execution, both in terms of continuous or discrete times. "

Is this really demonstrated by these experiments? To test a prediction, wouldn't you have to extrapolate (or interpolate) to a scenario that you haven't yet measured, and then make the measurement / benchmark to see how close your prediction was? If I understand your analysis right, you didn't do that. Instead you had a set of data points (again: how many? Across how many runs? How many of those were "replications"?) and you fit e.g., a regression model to it. Is that correct? In that case, I suggest you reword the statement on "demonstrating... to efficiently predict..."

What I think your experiments show nicely though is that you can get some insight into the performance bottlenecks and better understand the execution behavior.

- P13, L405: ".. between *the* Bioinformatics Laboratory .." (insert article)

- P13, L410: "... a study that counts on the processing of the sequences ... "
Not very clear what "counts on" means here. Does the study make use of the sequences ?

- P13, L420: "Therefore, the workflow checks for the existence of BT2 files in the filesystem where it is executed."
This sounds like active polling? (repeated checking whether the previous step finished)
How often is this check performed? Are there trade-offs and possible race conditions?

- P14: Can you clarify whether the merge at Step 11 in Figure 8 is such that *either* the samtools branch *or* the GATK output is processed? That is, there is no comparison or merge of the results from the left and right branch?
If that's the case, could a comparison be used, e.g., to compare and double-check the quality of the different branches?

- Figure 9 has some of the additional information I was curious about earlier (so maybe make a forward reference?)

- P15, Listings 3, 4: Although a bit cryptic, it is nice to see that user/scientist's questions become executable SQL queries. Together with the narrative for "Query 1" and "Query 2", the reader gets a good idea how this works.

- P16: Just like above: For the various speedups reported (e.g. 4x, 5x), how many times did you run these experiments? If it was only once, it should be mentioned. If it was many runs, why not report the averages (which I assume these numbers are) and the standard deviation?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: BIOWORKBENCH: A HIGH-PERFORMANCE FRAMEWORK FOR MANAGING AND ANALYZING BIOINFORMATICS EXPERIMENTS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The text is well-written and has a good structure.

I think the code citations should be moved to References instead of footnotes (they are now first class citable), but I understand if the GitHub hyperlinks should remain as footnotes so that the PeerJ typesetters can add them as inline hyperlinks.

I have attached some Detailed Comments line by line. In summary:

* Described usage is not "HPC" -> use "HTC"
* Fixes in Taverna details
* Fixes in Nextflow details
* How are required tool binaries installed on compute nodes?
* Shiny library vs HPFS-Prof vs new "workbench" code?
* Need deeper NCBI links
* Some grammar and phrasing suggestions
Experimental design: While the Docker container containing the Workflow script is great for reproducibility (as I partially verified under Validity), it seems hard for workflow development. In the way this is described, BioWorkbench is not a framework for writing any scientific workflow, but a framework for capturing a particular workflow, e.g. the provided Docker container captures 3 specific workflows and the Dockerfile is similarly specialized to add their dependencies.

As such I think you can say you are providing a Methodology rather than a Framework. If I am to create my own BioWorkbench that does a slightly different bioinformatics workflow I would need to make my own Dockerfile with my particular dependencies, create a brand new Swift script using the same patterns, build a new Docker image and test it inside the Docker container (or use vim inside the container to temporarily fix something).

Such a development methodology is however not mentioned in the paper, and it can be confusing to readers that won't know if BioWorkbench is just something you did (and we can learn from it and play with your demo), something we can repurpose (changing workflows, etc), or something general that can be reused (it can consume any workflow). I think now the truth is somewhere in the middle.

The text should more clearly make the point what maturity level BioWorkbench is at, and show what is the actual contribution. Now the text still gives the impression that BioWorkbench is presented for users, but I would argue that it is this methodology that is the contribution at this stage. I would therefore still be a bit careful about calling it a "framework", perhaps "prototype framework"?
Validity of the findings: Thanks to the updated README I was able to successfully build the Docker image from the bioworkbench recipe. The build takes a while to complete as various R packages are compiled.

I was unable to retrieve the (now automatically built) image from the Docker hub on my laptop, as I ran out of diskspace when writing swift_provenance.db.

I was also unable to retrieve the hub image on a spare server I had available (2x2 TB disk), due to the larger image size (15 GB) and that I was using the Docker LVM production configuration, which had a default max of 10 GB. Setting dm.basesize should improve this; unfortunately I did not have time to reconfigure my server.

This is unlikely to affect desktop Docker users which don't seem to hit this limit, but the authors are still advised to try to split into two Docker images, one smaller base image (tools, BioWorkbench, SwiftProf); and an extension image (using Dockerfile FROM that adds the larger pre-populated database.

Within the shell of the Docker image I was able to run the embedded swift-gecko and swift-phylo workflows according to their README files. As in my previous review I have not assesed the bioinformatics validity of these workflows, but confirm that they produced expected kind of outputs.

While intermediate files are kept separate in directories like run001, the workflow outputs are added directly to the working directory, which makes it hard to distinguish inputs, workflow, binaries and outputs. This may be improved by restructuring the workflow directories or workflow script.

As not enough reference data (e.g. for SwiftGECKO) was included, I was unable to verify any of the % speedup claims, both workflows finished rather quickly, but I did not find any obvious way to modify the number of concurrent jobs when testing on a multi-core server.

I was NOT able to run the embedded rasflow workflow as it did not contain any example sequences. Attempting to read the Swift workflow "loss_all.swift" is quite confusing, as it is a single flat linear script file that mixes shim scripting, variable declarations, control calls like snpCall() and actual code calls. For instance, the 6-step GATK sub-workflow of Figure 8 is 20 lines in the middle of a single large Swift file. The step names in the Figure 8 do not match the code, I would have expected the Figure steps to appear as functions or so. The authors are recommended to refactor the RasFlow workflow script for greater readability.

The authors argued in the rebuttal that RasFlow inputs are missing due to patient confidentiality. I think readers who try BioWorkbench would be grateful if some smaller example sequences and GFF/GTF files could be added instead, so that the RasFlow workflow also can be tested out of the box, even if it would not be able to reproduce the particular provenance data included.

I was able to access the embedded HPSW-Prof profiling web server. The user interface is somewhat confusing as there are no instructions (what goes into "Script Name"?), but I was eventually able to lower the "Date Range" to 2015 to see older runs and - after a wait - access the embedded provenance traces of executions of RASFlow "loss_all.swift". This showed figures similar to the paper's Figure 3 and Figure 6.

Unlike the Figure 6 the UI's "Level of Parallelism" for RASFlow seems to indicate only 4 concurrent executions (the figure is for SwiftPhylo). The number of cores or configured concurrent activities are not shown, but the table gives detailed execution stats per task of the workflow, e.g. "avg_duration" (unit is seconds?), percent CPU used, file system reads (unit is bytes?), and other parameters like "avg_max_rss", "sys_percent" and "user_percent" that remain unexplained.

As these are features/critizism of the previously published (and cited) HPSW-Prot this review does not go further in detail, but note that the consistent colour scheme is helpful to cross-reference tables and graphs.

The included provenance database do demonstrate the Domain Info added to the database after parsing , which includes 94,651 mutations from the 6 patients (anonymized as S1, S2, etc). I have not assessed if making these mutations public are compatible with patient confidentially, as no Ethical Approval assessment or similar data management plan for the RASOpathies study was cited.

The provenance database did not include data from the SwiftPhylo workflow, which was used for the execution speedup discussion in the paper, so I tried to add it:

    swiftlog -import-provenance run001

as referenced in swift-gecko README. As detailed in attached, this did however fail for both swift-gecko and swift-phylo with a ValueError.

As mentioned above I was also unable to run RasFlow, and therefore unable to import any fresh workflow runs into the provenance database to visualize them in the web interface.

While the authors have taken significant steps to improve the packaging and documentation of the software, I am still under the impression that this is more of a demonstration snapshot that illustrate a methodology of developing and inspecting bioinformatics workflows with Swift Language and the previously published HPSW-Prof, rather than a General Availability framework (software or workflow package) for bioinformatics researchers.

From the Docker image I was able to run the new Weka instructions; after setting CLASSPATH to the weka.jar; the weka GUI does not work inside Docker. I got the same linear agression algorithm constants as in the paper's listing (1). As Weka output is quite verbose and outside my expertise I did not verify the remaining Weka calculations, but confirmed that they all executed well.

The rebuttal explains that Swift provenance model predates OPM and its successor W3C PROV, and mentions a swift_provenance-to-opm.sh script to generate OPM, which I tested; however this is not mentioned in the manuscript at all, which I find odd given that several pages are about provenance. It can be listed as future work to update this script to generate PROV instead of OPM.
Additional comments: Hi, I am Stian Soiland-Reyes https://orcid.org/0000-0001-9842-9718 and believe in open reviews.

The complete review is also available at the (secret) URL https://gist.github.com/stain/eedf7c3a355e347863a2381a2f55e957 for easier reading and more hyperlinks.

This review is licensed under a Creative Commons Attribution 4.0 International License http://creativecommons.org/licenses/by/4.0/

This revision has improved significantly on most of my previous concerns. I am particularly pleased with the new completeDocker images, improved instructions and Zenodo code citations. Software used by the workflows (e.g. samtools) is now rigourisly versioned and cited, this is also a good improvement.

There are still a couple of issues with the software, I was unable to actually import the provenance to the database, which would seem central to the argument of the paper. This seems to be a bug in the log parsing, perhaps some sensitivity to how the workflow is invoked. Following the README step by step within the Docker image should work.

I was also not able to run the RASFlow workflow as no examples are provided, yet this is the prime workflow of the paper and the subject of the machine learning approach.

While the manuscript now makes it clearer that the Docker image is for demonstration purposes, I think it can still come across like if BioWorkbench is a piece of software to be used by bioinformaticians. I think the manuscript can be honest about the current maturity level of the prototype framework, and rather present the main contribution as a methodology for building and retro-inspect paralellizable workflows using Swift and provenance database.

See https://gist.github.com/stain/eedf7c3a355e347863a2381a2f55e957 for details.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DETECTION AND PHYLOGENETIC ANALYSIS OF ADENOVIRUSES OCCURRING IN A SINGLE ANOLE SPECIES
Review round: 1
Reviewer: 1

Output:
Basic reporting: This is an interesting manuscript about the detection of novel adenoviruses by PCR in wild-living anole lizards. The aim was to find out if individuals of the same species (Anolis sagrei), originating from different, well-separated geographical regions, may harbour identical, closely related or rather divergent adenoviruses. Secondly, the authors wanted to examine if the somewhat increased mortality that had been observed in the newly established captive lizard colony, could be the consequence of adenovirus infection or not. The paper in general is well written, the majority of the methods used to clarify the questions are adequate and this is true for the conclusions as well. A minor to medium-level revision would help increasing the scientific significance of the paper. Specific points are listed below.
First of all I would suggest changing the title to a more modest one, such as e.g. “Detection and phylogenetic comparison of adenoviruses occurring in a single anole species”, because no real evidences or conclusions regarding the evolution of adenoviruses are thoroughly discussed.
Experimental design: The authors began to screen the dead lizards a bit late, so that only a portion (less than one third) of them could be examined. Nonetheless, I think the overall positivity rate of 26% can be considered high. Adenoviruses tend to establish persistent infection, i.e. after the resolution of the disease, caused by the primary infection, the virus remains in the host without any clinical signs for long periods. In case of decreased immunocompetence of the host, the virus may start to replicate, sometimes resulting in disease again, or just being shed to the environment.
In the described cases, it is hard to comprehend the pathologic role of the detected viruses, since the exact nature of the samples is not specified. The composition the 20 mg of “abdominal tissue” (line 117) might result in huge differences depending on the inclusion (or exclusion) of certain parenchymal organs such as liver, spleen, and kidney. I know that rapid autolysis can prevent the exact determination of the sample, but still it might have a major influence on the outcome of the PCR tests and therefore it should not be left out of consideration when counting the possibility of false negative tests. On the other hand, the presence of virus in the intestinal content (or cloacal swabs) does not necessarily signals active infection, as it may as well originate from a passive carrier status of the host e.g. the virus was ingested and can be detected by nested PCR even in lack of active replication in the given animal. I would recommend a more cautious wording after consideration of the possibilities mentioned above.
Validity of the findings: Unfortunately, the newly-gained sequences are not released, and even the GenBank accession numbers seem not to be assigned yet although it is a prerequisite for acceptance at most journals. I would guess it is so at PeerJ as well. Moreover, the scale of the phylogenetic tree on Figure 2 is missing thus making the comprehension of differences difficult. The majority of the newly detected sequences fit into the genus Atadenovirus and appear close to AdV sequences derived from closely related hosts. The only exceptions are the three sequences originating from the lizards at Staniel Island. Their closest hits in the blast application (if it was run at all) would have been of interest. Mastadenoviruses occurring in more ancient mammals (rodents and especially mice) should have been included into the calculation. This could have helped clarifying the question if possible genus affiliation of the AdV sequences derived from lizards of the Staniel Cay Island.
An explanation for the higher mortality and AdV prevalence, among female lizards compared to males, could be the common caging (instead of individual one) that provides better conditions for the horizontal virus transmission. It would have been interesting to know, how many of the affected animal were kept in shared cages.
The authors seem to be very familiar with the literature of reptilian adenoviruses. Nonetheless, one relevant paper (see below) is badly missing from the list of references. Although this paper is about AdVs found in non-squamate reptiles (namely in different sliders and turtles), yet it presents several genomic variants of the same virus, sometimes present concurrently in the same sample. In this work the sequences of the same PCR fragment from the viral DNA polymerase gene were examined, so that the results can easily be compared to those of the present manuscript.
Additional comments: Typing errors and word usage mistakes:
Line 1: adenovirus is an adjective here neither capital letters, nor italics are needed. The same applies to “adenovirus” in lines 154, 159, 161, 179 & 208.
Lines 72, 104 (twice), 105, 109: I would recommend to use “clinical or disease signs”, as “symptom” refers to a subjective feeling of the patient. This word is not really adequate when speaking about animals.
Lines 117 & 121: The numbers should be separated from the measurement units by spaces.
Line 133: To avoid redundancy, the word “reaction” should be deleted.
In the first lien of legends for Figure 2 (after line 169): “adenovirusus” should be corrected. Also, a scale should be provided.
Figure 2: At least one (out of the three) murine adenovirus should also be included. The vertical lines marking the genera have been mixed up. From the genus Aviadenovirus, only two representatives are shown, whereas four viruses from the genus Siadenovirus. Therefore this latter one should be marked with the longer line. The spelling (regarding capitals and italics) of the genus names is correct. In our experience, the Bayesian analysis is not the optimal choice in case of such short sequences.
Lines 174, 181 & 196: I would recommend changing “isolated” to “obtained” or “derived”. Isolate implies that the virus is available as a strain suitable for in vitro replication in cell culture. (I have replaced the word “strain” with “virus” in the title for the same reason.)
Lines 330 & 331: Please double-check the authors’ list of this reference.
I recommend addition of the following reference:
Doszpoly A et al. (2013) Partial characterization of a new adenovirus lineage discovered in testudinoid turtles. Inf. Genet. Evol. 17:106‒112.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DETECTION AND PHYLOGENETIC ANALYSIS OF ADENOVIRUSES OCCURRING IN A SINGLE ANOLE SPECIES
Review round: 1
Reviewer: 2

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: The manuscript has clear language and is well written. Minimal corrections are suggested. Only as a suggestion for future work, if the research group is interested, inoculate the samples (both positive and negative) in cell line for analysis of infectivity and viral isolation.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DETECTION AND PHYLOGENETIC ANALYSIS OF ADENOVIRUSES OCCURRING IN A SINGLE ANOLE SPECIES
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: I recommend acceptance of the revised and improved manuscript. One tiny mistake still remained in Figure 2. Now the size of the vertical lines marking the genera is correct, but their places should be exchanged. If the Authors check the branches, they will notice that the aviadenoviruses are the Meyer’s parrot AdV-1 and Fowl AdV-5 (just below the sturgeon AdV’s branch). The genus Siadenovirus is represented by 4 viruses, namely those from the tortoise, frog, plum-headed parakeet and the great tit. These 4 viruses form the branch of siadenoviruses. I am sorry for not being clear enough in my original review.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DETECTION AND PHYLOGENETIC ANALYSIS OF ADENOVIRUSES OCCURRING IN A SINGLE ANOLE SPECIES
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: No comment.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RANDOM FOREST AS A GENERIC FRAMEWORK FOR PREDICTIVE MODELING OF SPATIAL AND SPATIO-TEMPORAL VARIABLES
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1.1 Clear, unambiguous, professional English language used throughout.

Yes, just a few colloquial expressions, e.g. a method "blows up".

Abstract: "lower number of" -> "fewer" points...

1.2 Intro & background to show context. Literature well referenced & relevant.

Yes

1.3 Structure conforms to PeerJ standards, discipline norm, or improved for clarity.

No. The Discussion goes much further than the evidence in the Results, it is largely speculative without support from the Results.

1.4 Figures are relevant, high quality, well labelled & described.

Mostly.

1.5 Raw data supplied (see PeerJ policy).

Yes
Experimental design: 2.1 Original primary research within Scope of the journal.

Yes

2.2 Research question well defined, relevant & meaningful. It is stated how the research fills an identified knowledge gap.

Yes. The authors propose a spatial prediction framework based on random forests, including distances to observation points, as a machine learning alternative to model-based geostatistics. They give cogent reasons why this might be desireable.

2.3 Rigorous investigation performed to a high technical & ethical standard.

Not completely. The paper wanders through several examples without a clear formulation of the RPsp model, just a description in words. The examples are diverse and illustrate various techniques without sufficient justification or motivation. This reader had the impression that the paper was pasted together from the tutorial (supplementary material). The later sections of the paper are speculation unsupported by what is presented in the paper. These sections read as the subjective opinions of the authors, and although they are interesting and provocative, they are not supported by any tests in this paper.

In the case studies, difference maps for both predictions and standard errors would add to the reader's understanding of the differences, and could then be discussed.

2.4 Methods described with sufficient detail & information to replicate.

Not completely. The code is missing key parts (e.g., loading of required packages). By digging into the supplementary material at github (linked) it is possible to find the missing pieces, but the presentation in the paper is not self-contained.
Validity of the findings: 3.1 Impact and novelty not assessed. Negative/inconclusive results accepted. Meaningful replication encouraged where rationale & benefit to literature is clearly stated.

This paper could in fact have a large impact, as it provides a machine learning approach to spatial prediction.

3.2 Data is robust, statistically sound, & controlled

All data is from standard datasets used in geostatistical examples, presumably controlled by their authors, not these.

3.3 Conclusions are well stated, linked to original research question & limited to supporting results.

No. See comments under 2.2 above and details below.

3.4 Speculation is welcome, but should be identified as such.

Too much speculation, too far from what is actually done.
Additional comments: The paper has many holes and inconsistencies. The following is a list in line order of some of the issues that should be addressed.

Abstract L31: "the high sensitivity of predictions to input data quality" -- really? how shown?

L33 "close-to-linear relationships" in the covariates?

L53 mention other R packages for RF, e.g. rpart, probably the most commonly used

L60 linear features -> *linear boundaries in the resulting map*

L78 i.e. -> e.g.? If restricted to 2D you should state earlier or even in title. In fact you have a 3D example (NRCS).

L80 spatiotemporal phenomena -> spatio-temporally distributed variables?

L109 omit "basically", adds nothing

Do we need so much background on MLR & RK? nothing wrong with it, but maybe for a text, not a paper?

L149 account for the *spatial correlation* or other non-independence of the *residuals*. Mention REML (Lark)

L160  i.e. -> e.g., can be anything; also not necessary here

L230ff, eqn (19) out of nowhere of a sudden remote sensing is included. This is not inherent in the RFsp idea,  but it is a useful decomposition for situations where RS provides useful covariates. Conceptually X_R  fit into the RF model in the same way as X_P -- there is no mathematical difference. If RS is central to the idea, it should be in the title. But it seems this just limits the approach unnecessarily.

L235ff "Assuming that the RFsp is fitted only using the XG, the predictions will  likely look similar to OK. If all covariates are used (Eq.19), RFsp will likely produce similar results as  regression-kriging" -- this is to be determined by the study, it has not been shown at all to this point. Perhaps rephrase "since the information content is the same as... the results may resemble..."

L247 remove "etc."  Add dIstance to features such as water bodies, cities ...

L251 Yes visibility distances can be calculated, but what theory predicts that these may be useful?

Of this list, which should be more useful? You already rejected (1) above (just use coords) because of linear artefacts, why include it here? One reason to use it would be the analogy to UK using trend surfaces -- if there really is a trend (especially if non-linear or with a threshold) these could be useful. And -- Figure 4 below (and others) show this! So the statement that you only use local buffers is not true. You also use coordinates and then compare the two approaches.

Here we need a mathematical discription of the buffer distances approach. We only see it in action in the examples, but never with any formula. Yes this is simple but needs to be explicit in mathematical form. It is not obvious from the text that the RF is built with all the buffer distances to all the points -- yes, this can be inferred from the code, but it is a major point of the approach, which should be brought out explicitly. Also, the distinction between using distances to all points, and classes of buffer distances, should be explained here.This is used later in the paper with almost no explanation. It's unclear (except by digging into the code at github for that example) how both approaches are implemented.

L264 vague and not supported. "easily"? how? suggest to omit. Concentrate on the value of using distance to points.

L267 X-validation RMSE, but model R^2 does not quantify predictive accuracy, only fitting accuracy. Or do you mean CV R^2?

L270 Spatial bias or residual uncertainty? See Fig. 3(d) caption, seems correct there.

L272 z-scores of what? Fig. 3(c) uses terms "nominal" and "coverage" probability w/o explanation

L293ff. It would be useful to also compare to a classical geostatistical estimate using e.g., gstat or similar

L296 We need some explanation of this dataset -- what are its characteristics, therefore what approaches are indicated? The important fact is that we think the metal contents are related to river flooding carrying upstream sediments.

L299 ff. the explanations from geoR functions are not needed.  "shows that..." just state as a fact (backed up by the references, which are already given)

FIgure 4 also shows using coordinates, but you said above it was (1) not desireable (2) you wouldn't do it. Also you do not show an observed-predicted plot for that case, so why show the map?

Figure 4 "and correlation plots based on the 5–fold cross-validation for OK and RFsp (last row, solid line: lowess scatterplot smoother)."?? nothing there

L332 "we may add global surface water occurrence (Pekel et al., 2016) and  the LiDAR-based digital elevation model (DEM, http://ahn.nl) as potential covariates explaining zinc concentration" This is unmotivated. Why could these be good predictors? In this area elevation may be related to flooding, but the authors do not explain this. How could surface water occurrence explain Zn?

L334 function spc() is not explained -- which package? and more importantly, why used? It is not in any of the loaded libraries (sp, plotKML, geoR, rgdal). Also, some used packages are not explicitly loaded: GSIF, and even ranger!! With some digging I found it in GSIF. But its function is not at all explained in the paper, it appears as if by magic.

L347-355 largely repeats the advantages proposed for RFsp earlier in the paper, this is not specific to the case study

L369 This information of likfit() should be given when the function is first used.

L434 carson1km <- readRDS("data/NRCS/carson_covs1km.rds")  where does this RData file come from? Also the .csv.

L436 In what sense is this a "pragmatic" approach? It seems based on actual lab. precisions.

This section is interesting but adds a complexity that is not reflected in the paper title nor objectives.

L448ff (Geochemical survey) There is a lot of R manipulation here, all very useful, but it reads as a tutorial, not as a motivated procedure. This whole section seems a really wrong way to go about the objective. Yes it can be done mathematically, but you just put all the predictands together, to later disaggregate them with the indicator predictors. I fail to see the advantage vs. fitting separate models.

L489ff (Daily precip Boulder) is an interesting application, but it should be compared with spatio-temporal kriging. L514 "This shows that RFsp connects space and time in a similar way as the model-based geostatistics.", no you didn't show that, what is the "same way as" supposed to mean?

L511 "The single spatiotemporal model can now be used to predict anywhere within the spacetime domain" If you mean only within the space-time region of calibration, yes; but if you mean in general, no, because RF can not predict outside its range of calibration (e.g., daily precip more than any that was used to calibrate the model). Please clarify. L552 mentions this point; but "not recommended" is too weak, it should be "impossible". L590 "serious extrapolation" should be "any extrapolation". What would "serious" be, anyway?

L532-536 repeats previous points, this is the third mention, not including the abstract.

L538 "save" here means what?

L563 "exceptionally simple neural networks can be used to represent inherently complex ecological systems" proof? references?

L566 "Also, we have shown that buffer distances do not have to be derived to every single observation point — for factors it turned out that deriving distances per class worked quite well."

Not true for the first model: "layer.* are buffer distances to each individual point"; but then suddenly L414 "in this case buffer distances are derived to each class"  -- why now and not for the first case? And why not do this in general? How can we decide between distances to points or classified distances?

Also, you did not test this: " For numeric variables, values can be split into 10–15 classes (from low to high) and then again distances can be only derived to low and high values." This could be true but the paper does not show this with any comparative test.

L569ff "Limiting the number and complexity of trees in the random forest models..." Again, you did not test this, so how can you state it for your RFsp approach?

L582 "Also with RFsp, we still needed to fit variograms for cross-validation residuals and derive occurrence probabilities etc." ??? I did not see this. See also the comment for L635.

L592ff "Are geographic covariates needed at all?"  This whole section is speculation and unsupported by the studies in the Results.

L635 Where do these simulations come in? Why do you need to simulate? You have the data points and will use them for the RFsp model, what remains to be simulated? Why do you need to know the spatial structure? You have said throughout you can just use the distance buffers with no need to know the structure, as in variogram modelling. See also the comment for L582.

L662ff "One model to rule them all" Pure speculation, suitable maybe for a session in the pub but completely unsupported here.

L690ff This two-stage framework and the accompanying diagram are interesting, but again unsupported by the results of this study.

L702ff. It seems you have solved all these problems in your paper, even the "large point data sets" (by using class distance buffers).

L707 Acknowledgements: These were not direct helps to your study, they were previous work which you used. It is not common to acknowledge them, unless their authors gave you direct aid.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RANDOM FOREST AS A GENERIC FRAMEWORK FOR PREDICTIVE MODELING OF SPATIAL AND SPATIO-TEMPORAL VARIABLES
Review round: 1
Reviewer: 2

Output:
Basic reporting: See below
Experimental design: See below
Validity of the findings: See below
Additional comments: General comments:
In this paper the authors introduce a random forest for spatial predictions framework (RFsp) where geographical proximity effects are incorporated into the prediction by using buffer distances from observation points as explanatory variables. The approach is illustrated using diverse spatial and spatio-temporal datasets, and results are compared to kriging. The paper is clearly written and the approach is interesting and seems technically sound, with the pros and cons of each approach clearly outlined. A few issues still need to be addressed before the paper can be published:
1.	The authors seem to imply that regression kriging, universal kriging and kriging with an external drift (KED) are the same methods and give similar predictions (lines 140-144). This statement is not correct as KED fits a local regression model within each search window while regression kriging is based on a unique regression model.
2.	The critical step of back-transforming results needs to be better explained. In particular, it is not clear how results can be back-transformed for a Box-Cox transformation with η≠0. For the case of a lognormal transform, Eqs. (13) and (14) are only correct for simple kriging as the Lagrange multiplier is missing. Last, how do the authors compute the kriging variance used to back-transform RK estimates (Line 155)? Do they combine the residual kriging variance with the uncertainty attached to regression estimates?
3.	At several locations across the manuscript, the authors mentioned that some implementations of the algorithms are computationally intensive. It would be useful for the reader to provide some estimates of CPU time.
4.	The space-time example lacks a comparison with space-time kriging.
5.	In any cross-validation of datasets with repeated samples (e.g., time series) one should remove the entire set of co-located observations when estimating a particular location to avoid the under-estimation of prediction errors mentioned on line 643.

Detailed comments:
1.	Line 79. The term “raster map” might be more appropriate than “gridded map”.
2.	Line 168. RK does not assume a linear relationship between dependent and independent variables as any type of regression model can be fitted.
3.	Line 240. Replace “froms the basis” by “forms the basis”.
4.	Line 258. Write “resistance distances” instead of “resistence distances”
5.	Figure 6. The caption should be clarified, e.g. by adding lettering to refer to the different figure panels.
6.	Lines 378-379. Could the homogeneity of the UK prediction error map (Fig. 7) be caused by the use of the same color scheme as for the RK prediction error map?
7.	Fig.11. Why is the shape of the map a rectangle unlike the actual shape of the States of Indiana and Illinois. Does the method impose such constraint? This should be clarified.
8.	Fig. 12. All the maps seem to display a discontinuity (horizontal line), this should be discussed.
9.	Line 614. Write “as long as”.
10.	Line 630-631. This sentence is confusing and needs to be completely rewritten.