Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: BLAST-BASED VALIDATION OF METAGENOMIC SEQUENCE ASSIGNMENTS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Bazinet et al. developed a method for performing targeted strain level classification of sequencing reads from a metagenomic sample. The proposed method uses a less precise taxonomic classification algorithm as a first pass, e.g. Kraken, to identify candidate reads for validation. The authors demonstrate their method using simulated sequence data from four organisms relevant to the biothreat community. Primary issues with the manuscript are:
1. Experimental methods are hard to understand as present,
2. TP, FN, sensitivity, and precision definitions are misleading, and
3. lack of application.

These issues are covered in the experimental design and validity of the findings sections.

Overall the manuscript is well written, though the article would benefit from;  restructuring, additional references, consolidated figures and tables along with more descriptive legends, and a number of sentences throughout the manuscript could be stated more succinctly.

The article could be restructured as text describing the methods and experimental design are distributed throughout the manuscript. Additionally, the section on related methods is out of place. Method description is scattered throughout the manuscript. Moving all methods to the methods section as well as having separate sections describing the blast-validate method (parameter sweep and BLAST result filtering) and method evaluation would help clarify what was done. Additionally, the clade level exclusion method is only briefly described in the methods section without sufficient context for the reader to know how and why the method is used in the study.  Providing scripts used to perform the method evaluation, genome sequences, and seed numbers used to simulate the sequencing reads would facilitate study reproducibility.
I recommend adding references for other taxonomic classification methods developed for strain and species level classification. Additionally, while the proposed method is a targeted approach the introduction would benefit from the inclusion of large taxonomic classification benchmarking studies, e.g. McIntyre et al. 2017, and Sczyrba et al 2017. Including these references would help establish the study motivation, potentially replacing the ‘Evaluation of a “first-pass” taxonomic classifier’ section.

The bulk of the study results are presented in four tables. Due to the overlap of information presented in the four tables, the tables could be combined. Combining the tables would make it easier for the reader to compare results across evaluation experiments. For most evaluations, the near-neighbor and false-negative methods for calculating precision are the same. Only providing results for evaluations where the values differ would make the tables more concise. Combining tables can help highlight show how different genomes (representative and alternative) impacts parameter threshold values. More descriptive figure and table legends would greatly help the reader interpret and digest the material. See experimental design and the general comments section for additional comments on the figures.

McIntyre, Alexa BR, et al. "Comprehensive benchmarking and ensemble approaches for metagenomic classifiers." Genome biology 18.1 (2017): 182.

Sczyrba, Alexander, et al. "Critical Assessment of Metagenome Interpretation—a benchmark of metagenomics software." Nature methods 14.11 (2017): 1063.
Experimental design: The manuscript represents original research that falls within the scope of the journal. Though the research question is not particularly well defined. The authors present a method for BLAST parameter optimization, with no application other than data used to define filter parameters. The study would benefit from processing additional datasets using the proposed optimized BLAST method to validate “first-pass” classifier results. This could be achieved using datasets where the target taxa is present, or simulating different target taxa levels by spiking in reads simulated from the target taxa into a dataset that does not contain the target taxa.

Method description is dispersed throughout the introduction, methods, and results sections making it hard piece together the experimental design during the first read through. Recommend having two methods sections, one describing the validation method, and a second describing method evaluation using simulated sequencing data. Additionally, a diagram for each section would provide additional clarity in differentiating the proposed blast-validate method the approach used to evaluate the method.

Two additional minor points, the clade-level exclusion method needs to be clearly described in the method section. The proposed evaluation figure would help. Additionally, the RefSeq release version should be stated in the methods along with the accession date.
Validity of the findings: Defining parameters to validate taxonomic assignments for specific taxa is a valuable tool relevant to the bioforensic as well as other application areas, e.g. pathogen detection. However, the sensitivity and specificity definitions are not standard and not well described. For example, the sensitivity definition used in the manuscript, TP/read count, is not the standard definition, and the alternative precision definition is the standard sensitivity definition. This leads the reader to question why optimizing for standard definitions of precision and sensitivity yields similar parameter values. Looking at the bast-validate source code, TP is defined as reads classified to the target taxa, and FN is any read not classified as the target taxa, the target taxa lineage, or no database hits. The four truth table values, TP, TN, FP, FN should add up to the total number of observations, here read counts. When the representative genome or alternative representative genomes are analyzed there are only TP or FN. Where TP are reads assigned to the target taxa and FN is defined as reads not assigned to the target taxa. These definitions for TP and FN is consistent with your definition and the standard definition of sensitivity, though inconsistent with both of your precision definitions. Conversely, all reads simulated from the near-neighbor genome can be classified as TN or FP. Using a combined dataset of simulated reads from the representative genome and near-neighbor you could obtain a complete set of truth table values for your parameter sweep evaluation. To address the issue of the near-neighbor not being representative of all potential false positives BLAST results for the representative genome could be used to identify a set of non-target taxa genomes to include as potential FPs and TNs in the dataset used for the parameter sweep.

The simulation results used to define the parameter values for filtering BLAST results to obtain perfect precision are then used to determine method sensitivity. There is a risk of overfitting the data when using the same set of simulated reads for performing the parameter sweep and sensitivity analysis. Expanding on the use of alternative reference genomes by performing a more formal cross-validation to assess parameter robustness as well as an example application as mentioned in the experimental design section would significantly strengthen the study.
Additional comments: - Of the taxonomic systems mentioned in the introduction and figure 1 two are for 16S database references and only one of which is relevant to the database used in the study. I recommend removing reference to different taxonomic systems in the introduction and figure 1 as there is minimal relevance to the study as presented.

- The authors include a number of quoted phrases, make sure new terminology is clearly defined. A section with new terms and definitions might be helpful.

- Provide a more detailed definition of BLAST bit score and E-value as understanding these values is key to understanding the BLAST-validation method.

- State pXO relevance and reference line 207

- The supplemental figures and tables present the same set of results. I recommend using the table to present the optimized parameter values and figures to show the “sensitivity” results. The figures would benefit from a more descriptive figure legend.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: BLAST-BASED VALIDATION OF METAGENOMIC SEQUENCE ASSIGNMENTS
Review round: 1
Reviewer: 2

Output:
Basic reporting: I think the paper was well justified and clear. I strong believe that there is a need for this tool. In our results, we often find false results/assignments and often we need to remove them. I think the figures and table were high quality and useful.

One minor concern: the authors use experimental and simulations in the same lines to discuss their simulations. This is confusing, as I would associate "experimental" with real data, and not simulations.
Experimental design: The work the authors conducted was sound and clear. Their simulated read datasets clearly validate that their methods work the way they designed them. However, I think it is not sufficient to only use simulated reads.

I know their comment in the paper is that "there is no 'typical' metagenomic experiment." However, I think its important to show a few read-data examples. The simulated datasets show that for high-quality reads the approach works well. However, it is also crucial to show that the method can also clarify the results of real examples as well--even if you cannot definitively show that the answers are correct.

In addition, although the authors clearly show their approach significantly improves the Kraken result, it leaves me curious as to how well it improves other methods. (I of course would like to see how well our PathoScope approach fairs on the first-pass, and then how well much improvement the authors). I think they mention MEGAN, PhyloPhythia, Centrifuge in the paper. I would like to see them compare with PathoScope and MetaPhlAn as well.

There is no discussion of run times. I can imagine that the computation would not be prohibitive if the taxa of interest does not dominate the sample. When is this approach reasonable and when would it take too long? For example, a human fecal sample might be dominated by Bacteriodes, so using this approach to validate/refine these taxa would not be reasonable. Also, what is the tradeoff using slower but more accurate first-pass approaches (i.e. PathoScope) versus faster less accurate methods (i.e. Kraken) when combined with the proposed approach? (i.e. is PathoScope+BLAST roughly the same as Kraken+BLAST, but may be faster?)

I took a look at the author's software but I did not run it on a dataset. I think it would be helpful to provide more documentation and a concrete software example/tutorial.
Validity of the findings: I think the results and conclusions the authors make are well justified and sound. I would have just liked to see the authors address the comments in the previous section.
Additional comments: No major additional comments. I enjoyed reading this paper, and I believe it has the potential to make an impact on the field.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: BLAST-BASED VALIDATION OF METAGENOMIC SEQUENCE ASSIGNMENTS
Review round: 2
Reviewer: 1

Output:
Basic reporting: No additional comment.
Experimental design: The NYC subway analysis should be added to the methods section.
Validity of the findings: No additional comment.
Additional comments: Dear Authors,
I commend you on thoroughly addressing the issues raised by myself and the other reviewer.

A few suggestions still remain.
1. For the related work section in the Introduction, a sentence stating that you will describe some related projects before presenting your new method will make the section fit the rest of the manuscript.
2. For the sensitivity and precision definitions, please state early on in the section that non-traditional formulas are used to calculate the metrics but the metric behavior is consistent with standard metrics. You may want to also emphasize that the standard metric equations are not easily applied to metagenomic taxonomic classification analysis and this is an issue that needs to be addressed by the research community.
3. Figure 1 - Greengenes, OTT, SILVA, and RDP are not taxonomies, but rather reference databases with sequences and taxonomic information. Please only include taxonomic systems such as NCBI (https://www.ncbi.nlm.nih.gov/taxonomy) and Bergy’s (https://www.bergeys.org/outlines.html).

Best Wishes


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ANTIOXIDANT STATUS OF RATS’ BLOOD AND LIVER AFFECTED BY SODIUM SELENITE AND SELENIUM NANOPARTICLES
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors of this manuscript describe the antioxidant status in blood and liver of rats treated with sodium selenite or selenium nanoparticles. The aim of the study is to compare two different forms of selenium such as nanoselenium particles with sodium selenite to show whether selenium nanoparticles can increase the antioxidant status of rat metabolism and serve as alternative nutritional supplements for an animal organism.
Experimental design: The results are summarized in the well-arranged schema and graphs that offer detailed information. All experiments were very well designed and executed. The manuscript is clearly written and easy to follow. Therefore I have only a few points.
Validity of the findings: The conclusions of the study are clearly stated but the lack of knowledge on the toxic effect of nanoparticles in animals is an issue.
The hemolysis assay could be added to verify the toxic effect of nano-particles on mammalian cells.
Additional comments: I would suggest a minor correction of the English language style.

The background of the abstract does not have sufficient information about the need for new forms of selenium with lower toxicity as nutritional supplements.

The conclusion needs to be rewritten and it should clearly stay the beneficial effects of nano-particles over sodium selenite.

Thyroxine is mistyped at line 37.

The gender and age of rats should be mentioned in the Materials and Methods section.

The water content in the treated and untreated animals could have a big variation. I would suggest verifying the weight of livers with the protein concentrations in the extracts. The same amount of protein in the extracts should be analyzed and compared.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ANTIOXIDANT STATUS OF RATS’ BLOOD AND LIVER AFFECTED BY SODIUM SELENITE AND SELENIUM NANOPARTICLES
Review round: 1
Reviewer: 2

Output:
Basic reporting: The manuscript is written well.
Experimental design: More experiments need to be done. Please see attached comments.
Validity of the findings: Work is not conceptually novel.
Additional comments: In this paper Urbankova and colleagues synthesized two selenium nanoparticles (Se-NPs), Se-49 and Se-100 and studied their antioxidant status on rat by measuring reduced (GSH) and oxidized (GSSG) glutathione level, and the total selenium content in blood and liver samples. Authors observed differential effects on GSH, GSSG and selenium level in liver and blood—and based on their observations authors concluded that Nano-form of selenium has better antioxidant property compared to sodium selenite, a conventional dietary supplement.

The idea of using Se-NPs as alternative dietary supplements is evolutionary. There is numerous numbers of studies with different Se-NPs to characterize their effects on living organisms and to understand whether Se-NPs could be used as an alternative of conventional dietary Se-supplements. Therefore, the current study is not completely new except using of two new Se-NPs and studying their effects on rat. This reviewer does not think that the work is complete enough to publish in PeerJ in its current shape, however can be consider after answering the following concerns.

1. Several other important experiments must be done in order to characterize a new Se-NP comprehensively, such as 1) activity measurement of Se-dependent enzymes, 2) measurement of markers for liver injury, 3) activity assay of liver GST.

2. Please check the existence of this reference: Sochor J, Pohanka M, Ruttkay- Nedecky B, Zitka O, Hynek D, Mares P, Zeman L, Adam V, and Kizek R. 2012. Effect of selenium in organic and inorganic form on liver, kidney, brain and muscle of Wistar rats. Cent Eur J Chem 10:1442-1451. 10.2478/s11532-012-0064-8.

3. In line 177-179: authors conclude the GSSG level goes up significantly in blood upon selenium treatment. It would be more convincing if authors calculate and provide “p-values”. In addition, it is also suggested to calculate and plot the ratio of reduced GSH to GSSG.

4. This reviewer also recommends replicating the studies with different doses of Se-NPs in diet and after different time intervals from the day of treatment begins.

Please see these comments in the attached PDF


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ANTIOXIDANT STATUS OF RATS’ BLOOD AND LIVER AFFECTED BY SODIUM SELENITE AND SELENIUM NANOPARTICLES
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: The authors have addressed my concerns. The revised paper strengthens the few weak points in the earlier version and is ready for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LINEAR LEPIDOPTERAN AMBIDENSOVIRUS 1 SEQUENCES DRIVE RANDOM INTEGRATION OF A REPORTER GENE IN TRANSFECTED SPODOPTERA FRUGIPERDA CELLS
Review round: 1
Reviewer: 1

Output:
Basic reporting: No comment except that Figs. 4E-4G should be improved as indicated below in the 'General comments for the author'.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: The paper describes molecular events occurred upon transfection of Sf9 cells with JcDV-based linear DNA.
The presented data are rather interesting and useful for future studies of related viruses. However, a minor revision of the presented paper is required before it could be accepted for publication.

Main points:

1. It is not clear what linear DNA molecules derived from pFull, pVP and pNS were used in transfection experiments. No information on this point can be found in Results, Materials and Methods, or Fig. 1. These linear DNA should be described unambiguously.

2. Figures 5E-5G are barely understandable. Another form for presentation of data on rearranged viral sequences should be found. For example, figures 5E-5G would be converted to a separate figure of a bigger size. Such a figure should clearly depict not only the positions of viral sequence fragments relative to the original sequence, but also show how these fragments are interconnected. Positions of non-viral sequences in these clones should be shown. In general, I would suggest avoiding output screens of sequence comparison software.

Minor comments:

Line 43 – Correct the word ‘analysed’.
Line 77 – “0,547 kb” looks strange, would be changed to 547 nucleotides.
Line 111 – What does the word “precise” mean in this context?
Lines 223-227  - should be removed
Line 239 – undetecTable


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LINEAR LEPIDOPTERAN AMBIDENSOVIRUS 1 SEQUENCES DRIVE RANDOM INTEGRATION OF A REPORTER GENE IN TRANSFECTED SPODOPTERA FRUGIPERDA CELLS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The manuscript is well written , conveys well the message.
The objectives are clear as well as the results .
The manuscript is written concisely, without undue details.

However, in the results section, it would be preferred not to repeat some parts of the materials and methods but to describe the outcome, for example:
When  X construct cut with restriction enzymes Y and Z ...(the results were )....
replace with:
 when X construct was cut with restictions enzymes to remove this  (part of the genome) ....  (the results were) proving that ...
Experimental design: Original research, clearly demonstrates that linear viral sequences may initiate integration in host genome.
The researchers had to use several constructs and trials in order to prove their hypotheses
Methods used are scientifically based
Validity of the findings: Data seems robust and conclusions well stated
Additional comments: This research is original, written in a well structure manner and deserves publication
However it may be better understood if the results are rewritten in a clearer manner as suggested above.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LINEAR LEPIDOPTERAN AMBIDENSOVIRUS 1 SEQUENCES DRIVE RANDOM INTEGRATION OF A REPORTER GENE IN TRANSFECTED SPODOPTERA FRUGIPERDA CELLS
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: With all the corrections made, the paper can be accepted for publication in PeerJ.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ESTIMATION OF DIFFERENT DATA COMPOSITIONS FOR EARLY-SEASON CROP TYPE CLASSIFICATION
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1. Authors should conduct a thorough review for several grammatical and incorrect word usages throughout the manuscript. Examples include:  Line 64 (change "An" to "A"), Line 71 (change "quantity" to "quality"), Line 130 (change "composition" to "composite period), Line 225 (change "characters" to "characteristics"), Line 297 (change "procedure's" to "producers"), Line 404 (change "phrases" to "phases") and throughout manuscript change "grape" to "grapes" unless you are using this word as an adjective.

2. Figure 3 should be changed to a Table given it is being presented in table format in the manuscript.
Experimental design: 1.  The authors use several moderate to high resolution remote sensing data sets (e.g., Landsat) to calculate NDVI, but do not clearly state why multiple satellite data sets are needed for the method being presented.  A brief discussion of this point it needed early in the manuscript.
Validity of the findings: No comment
Additional comments: The following are recommended revision to improve the overall quality of this manuscript.

1. In the Abstract, report the classification method that was used in this work.

2. Lines 34-26:  The end of this sentence does not make sense and needed to be rewritten to clearly state the point you are trying to make here.

3. Line 53:  What do you mean by "short image time series"?  Do you mean short historical record or short sequence of composite images during a growing season. Please better clarify.

4. Lines 57-59:  Briefly discuss why Landsat ETM+ data "cannot provide" one cloud-free image in each season globally.  This statement needs brief supporting discussion.

5. Line 61:  State why the image data being discussed in this sentence are "still absent".  Unclear in the current way it is presented.

6. Lines 64-65:  There appears to be something missing from the sentence  of "A simple method....".  I appears it should read that a new simple method "is needed". Check the sentence and revise accordingly.

7. Lines 84-92:  Suggest moving the discussion of the study area to end of this paragraph and stating the study objective to start the paragraph followed by the study area information.

8. Lines 146-147 and Figure 1:  The green patterns showing the spatial distribution of the reference data are difficult to visually identify in the map of Fig 1. Change the color to make them easier to see in this graphic.

9. Line 151 and Figure 5.  What are the bar presented in the line graph?  This needs to be explained either in figure caption or the body of the text in the manuscript.

10.  Line 285-286:  Why do you think the overall accuracy decreased when the entire daily time series was used compared to shorter time series?  A brief discuss of why this might have been would be useful.

11.  Line 426:  What type of "confusion" are you referring to here?  Some discussion of the types of confusion are needed and how they negative impact the classification.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ESTIMATION OF DIFFERENT DATA COMPOSITIONS FOR EARLY-SEASON CROP TYPE CLASSIFICATION
Review round: 1
Reviewer: 2

Output:
Basic reporting: Paper deals with the problem of investigation of optimal satellite derived dataset for early season crop classification. Authors provide qualified literature analysis. Article structure is clear and professional. The idea to use statistical distance to investigate crop separability at satellite images is interesting.
There are some methodological issues that should be addressed.
Experimental design: A lot of experiments have been fulfilled, but the study requires additional justification.
1. First of all, the study area is very small, and the method relies on statistical distribution of different crops.  This distribution could vary in different areas, so the results can be quite different for other regions. Therefore, it is unclear if the results could be extended for wider area.
2.  The JM distance is calculated between a pair of probability distributions. The question is how these statistical distributions of different crops are calculated? Statistical distribution of different crops in different locations can vary.
3. What is the typical field size in the study area? The idea of Modis data downscaling using Landsat images should be justified.
4. It is unclear, how many 300x300m plots were used for training and for validation purposes.
5. What features are used for classification with random forest method? Please, clarify the structure of the model. Statement in Lines 199-200 is unclear. Please, explain, what do you mean by “the number of features to split the nodes (mtry) were defined as 1000 and the square root of the total number of input features”.
6. At  Fig. 10 “Feature” should stand instead of “Feaure”.
Validity of the findings: It is unclear if the results are robust. Experiments were done only for one year for very small territory. It is not enough to provide scientific soundness of the results.
That is why conclusions could not be considered as well stated and justified.
Additional comments: The idea to investigate the optimal satellite derived dataset early season crop mapping is challenging task. Mathematical tools are interesting and promising.
I suggest to clarify the section with methodology description, to extend the study area and to fulfill the experiments for several years.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ESTIMATION OF DIFFERENT DATA COMPOSITIONS FOR EARLY-SEASON CROP TYPE CLASSIFICATION
Review round: 1
Reviewer: 3

Output:
Basic reporting: Language should be improved throughout the text. In particular, I recommend to use shorter and more concise sentences. For instance, line 86-92 I would suggest to split the objectives in bullet points. Line 130-131, “composition” is used four times in two rows. Another example is at Line 104: grape is a tree crop and it is not planted every year. There are a number of these examples that make the paper sometime difficult to read.
The introduction is appropriate and the background information is relevant and in line with objectives and expected results. However, the literature can be improved. There are a number of recent papers that dealt with similar problems (multi-temporal crop-type classification http://www.tandfonline.com/doi/abs/10.5721/EuJRS20164920; https://www.sciencedirect.com/science/article/pii/S0303243417300934) or used similar algorithms (e.g. Random Forest http://www.mdpi.com/2072-4292/8/3/166).
What is also missing in this work, and this is a critical point, is a reference to and use of Sentinel-2 mission data. This constellation of two satellites (S2-A and S2-B) is part of the European Copernicus programme and provides high spatial resolution data (10-20 m) with very high revisit time (10 days in China; 5-days in Europe). There are also tools (http://www.esa-sen2agri.org/) developed by the European Space Agency that support the crop type classification using multi-temporal Sentinel-2 and Landsat-8 data and this should be mentioned in the paper.
The number of figures could be reduced. The quality and the caption must be improved. For instance: Figure 1 should have a reference grid, coordinates and scale for both the region of interest and the study site. Figure 3 should be a table with column names. In the workflow in Figure 6 the arrows should be eliminated (e.g. “Separability saturation points…” is not an input to the “Classification accuracy”). In Figure 10, it is not clear what the Feature Number (x-axis) represents. Describe it in the caption. Perhaps Figure 12-15 could be eliminated. Figure 18 left and right charts seem identical.
Experimental design: The paper reports on a very technical aspect of crop type classification using remote sensing data and I am not totally convinced that it is within the scope of PeerJ. It does not deal with environmental issues relating crop type mapping and e.g. crop management, which would be in line with the scope of the journal.
This research paper could represent a case study but does not bring elements of leading edge innovation that would fill a knowledge gap. There is existing research on the topic as recent literature shows. Moreover, there are some element of criticism in the experimental design. First of all, the authors completely ignore the presence of Sentinel-2 data that has totally changed the data availability scenario and therefore the foundation of the research question and methodological approach. Of more technical relevance, the construction of the reference dataset presents some issues. The distribution of samples amongst the different classes is very uneven (e.g. cotton vs watermelon, 54 vs 7) and limited to four crop types only. There might be also a strong correlation within the ground data. This is because the training/validation samples are taken from the same fields/parcels (as shown in Figure 4c) and used as independent samples. This probably introduces a positive bias in your measures of accuracies.
Validity of the findings: The impact and novelty of the study was not totally assessed and exhaustively discussed in the context of existing studies and datasets. Ground data have limited representativeness being sampled in a limited spatial region and for a limited number of crops (4).
Results and discussion sections could be merged. Many elements of discussion are presented already with the results.
Results and conclusions are sometime presented in a contradictory way and do not answer the key question: is daily data better that aggregated data? I could not appreciate a relevant difference.
The authors report on the possibility to perform “early crop classification”, but also that (line 442) “The features of the green-up stage (Day 60–80) and the NDVI peaks (around Day 210) were the optimal features for identifying the summer crops.” From your chart in figure 5, I read that “Day 210” represent the end of season. Is this correct and how this relates to an early crop classification?
At line 445, the authors mention that “Daily NDVI time series outperformed the other composition strategies because the daily data saturated early (158-day time series)”. The OA accuracy in Table 4 does not show differences (90.23% vs 89.66%) amongst the 4 compositing strategies. When considering daily and composited data, the saturation length of the daily dataset (158-day) is essentially the same as for the 32-day compositing strategy (198-day). Isn´t it? In a real-case scenario, you do not need to wait the end of the compositing period to perform the classification and therefore the daily dataset will provide you a temporal advantage.
Additional comments: The topic of the paper is of interest but it is not fully in the scope of the Journal. I would recommend a major revision and re-submission to a different journal focusing on technical and methodological aspects.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AGRONOMICAL, BIOCHEMICAL AND HISTOLOGICAL RESPONSE OF RESISTANT AND SUSCEPTIBLE WHEAT AND BARLEY UNDER BYDV STRESS
Review round: 1
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: If there is no special reason, please rearrange figure 1. Change position virus bar and control bar.
All figures exept for figure 1 virus bar locate right side.  It is good to maintain consistency for all figures. Please mention meaning of "LISE" at Figure 10-D.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AGRONOMICAL, BIOCHEMICAL AND HISTOLOGICAL RESPONSE OF RESISTANT AND SUSCEPTIBLE WHEAT AND BARLEY UNDER BYDV STRESS
Review round: 1
Reviewer: 2

Output:
Basic reporting: 1.	This manuscript really needs a narrative, and – to my mind – their experimental work deserves it. The introduction and results are both lists of findings, but ideas are not being developed or connected. In this sense the discussion is better. However, because the paper is not framed around a couple of interesting scientific questions/hypotheses, even the discussion does not feel very coherent or focused. Whilst the findings of the study are not earth shattering and I appreciate the sobriety of their presentation, I’m convinced the authors can find a more thought-provoking and stimulating way of present their results.

2.	The paper needs careful proofreading and editing. Articles are often missing, the lit. references are sometimes awkward [… as shown by (X et al., 20NN).] , sentences are incomplete (E.g., second paragraph of the methods section), use of subscripts inconsistent (e.g., OD [subscript] 405).
Experimental design: 1.	The authors state in the methods how they infected plants for their experiment with aphids, but a lot of details leading up to that point are missing. Where did the aphids come from, what exactly is the source virus, how exactly did the aphids acquire the virus? Very relevant details for understanding their work.
2.	How was the cutoff value for the OD405 established? And, given that the ELISA assay has different background levels in the different plants, why would the cutoff be the same?
3.	On a related note, are the authors sure all plants were infected? I.e., for Manning, the SE (!) is rather large in Fig. 4, suggesting that not all individual plants have a significant increase in ELISA values. The authors would really need to do a pairwise test to establish that each point is significantly higher than the mock-infected controls. If that’s not true, the uninfected plant(s) should be removed from the analysis.
4.	Can the authors give a source for using root length as a proxy for root development? It seems obvious, but since I am not familiar with quantifying root development I wouldn’t know for sure.
Validity of the findings: 1.	I find it totally unclear how the statistical analyses were performed. What data were analyzed by one-way ANOVA? This is not explained in the methods, and due to the  incomplete reporting of statistics in the results (only P-values) it never becomes any clearer. State in the methods exactly what data (virus-infected values normalized by mock-infected controls?) were analyzed with what method, and please justify briefly why this method is suitable (are assumptions being met?). In the results, give full ANOVA tables (preferably in the manuscript proper). It might also be that another statistical test is more suitable here, but this cannot be judged form the information supplied.
Additional comments: Choudhury and colleagues present a study the effects of BYDV-PAS infection on two cultivars of wheat and barley, focusing on virus effects on host growth, photosynthesis, the presence of different biomolecule and leaf structure. They find the expected differences between the two cultivars – with different susceptibilities – taken for each species. I don’t see any major problems with the experimental study, although I find the description of the statistical methods and results very unclear. Finally, right now the manuscript is more a technical report than a scientific paper: there are no hypotheses, it is not clear what exactly this whole exercise teaches us, and there is no real narrative. I given some further detailed comments I hope the authors will find helpful.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AGRONOMICAL, BIOCHEMICAL AND HISTOLOGICAL RESPONSE OF RESISTANT AND SUSCEPTIBLE WHEAT AND BARLEY UNDER BYDV STRESS
Review round: 1
Reviewer: 3

Output:
Basic reporting: This paper presents data on the effect of BYDV-PAV infection on resistance or susceptible wheat and barley cultivars. There is no justification of the election of the analysed parameters (biomass, root length, chlorophyll context etc), and it is unclear how or why the effect of infection on these parameters may help understand resistance mechanisms. Resistance mechanisms are better analysed with the tools and approaches of molecular genetics. In addition, the methodology is not properly described nad there are some conceptual errors.
Specific comments:
1.	The Introduction does not lead anywhere nor indicates why the study is made as it is. There is no connection between the paragraphs relating to vector transmission (not analysed) phenolic content or sugar content.
2.	In addition most references are quite old and should be updated. I suggest visiting Volume 17 of Current Opinion in Virology (2016) dedicated t plant virus pathogenesis.
3.	The assertion lines 49-50 is not correct, as it only applies to persistently transmitted viruses.
4.	There is no indication of the source, genotype or isolate of the virus used. Same for the aphids.
5.	The experimental design is not explained: what is considered as a replica? Apparently not a single plant, what then, a pot? With how many plants? Etc
6.	Data were analysed by ANOVA. Were data distributed normally and homoscedastically for this analysis to be valid? How were comparisons between treatments done? No information on sigificnace of differences is given in figures or text.
7.	Figures are miss-numbered. Fig. 2 is unnecessary.
8.	Tolerance and resistance are different defences in plants, But the cultivars here used are sometimes rated as resistant and sometimes as tolerant. What are they? According to Fig. 4, it seems they are resistant, as virus accumulation is decreased.
9.	On which data are assertions on ultrastructure done? Fig. 10 does not show electron micrographs.
10.	Discussion is mostly a repetition of the results.
Experimental design: There is no information on the experimental design.
Validity of the findings: Cannot know, as statical analyses are not presented and the experimental design is not described.
Additional comments: This paper presents data on the effect of BYDV-PAV infection on resistance or susceptible wheat and barley cultivars. There is no justification of the election of the analysed parameters (biomass, root length, chlorophyll context etc), and it is unclear how or why the effect of infection on these parameters may help understand resistance mechanisms. Resistance mechanisms are better analysed with the tools and approaches of molecular genetics. In addition, the methodology is not properly described nad there are some conceptual errors.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AGRONOMICAL, BIOCHEMICAL AND HISTOLOGICAL RESPONSE OF RESISTANT AND SUSCEPTIBLE WHEAT AND BARLEY UNDER BYDV STRESS
Review round: 1
Reviewer: 4

Output:
Basic reporting: See comments below
Experimental design: See comments below
Validity of the findings: See comments below
Additional comments: Comments to the Manuscript PeerJ#22841
* For better understanding please give some information to the terms ‚virus resistance‘ and ‚virus tolerance‘ (e.g. in the introduction).
* Please use these terms clearly (precisely) in the whole manuscript, compare e.g.

L 87 …cultivars with known BYDV resistance…

L 209 …two tolerant cultivars…
* Please give some information about the known effects of the Ryd2 and Bdv2 genes.
* Material and methods
Give some more information to the method of ELISA, e.g. how many positive and negative controls did you use? How did you estimate the detection threshold?
* Please discuss your results more in detail also in relation to prior results (e.g. Jensen 1968, 1969, Orlob and Arny, 1961 …) to bring out the new findings of your studies.
* Please improve the legends of the figures and check Fig. 1 – what is presented?
* The style of professional English has to be improved.
* Please find more comments in the attached file (PeerJ#22841.pdf) of the manuscript.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AGRONOMICAL, BIOCHEMICAL AND HISTOLOGICAL RESPONSE OF RESISTANT AND SUSCEPTIBLE WHEAT AND BARLEY UNDER BYDV STRESS
Review round: 2
Reviewer: 1

Output:
Basic reporting: Given there are still typos in the ms, a final round of proofreading would probably be a good thing.
Experimental design: I previously commented on the experimental design:

“3. On a related note, are the authors sure all plants were infected? I.e., for Manning, the SE (!) is rather large in Fig. 4, suggesting that not all individual plants have a significant increase in ELISA values. The authors would really need to do a pairwise test to establish that each point is significantly higher than the mock-infected controls. If that’s not true, the uninfected plant(s) should be removed from the analysis.”

Compared to Fig. 4 in the original submission, virus bar for “Manning” now has a higher mean and smaller standard error. I can’t follow from the authors’ very brief comment in the rebuttal or the revised manuscript why this is. Were some of the plants indeed not infected and therefore removed? And where these plants then also removed from the other analyses? I think this information should be provided in the manuscript.
Validity of the findings: Under validity of the findings I previously asked about the use of ANOVA. The authors acknowledge this comment, and state they clarified this point in the methods. However, homoscedasticity is not discussed anywhere and the new Table 1 with only F-values is not very helpful. So I’m afraid I have to be very boring and repeat myself: address whether the data meet assumptions for ANOVA and please present full ANOVA results (Tables in Supp. Materials and F test results in paper proper?).
Additional comments: The authors have revised their manuscript and taken on board some of the criticisms. Importantly, the narrative has been improved. However, some aspects are still opaque to me.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AGRONOMICAL, BIOCHEMICAL AND HISTOLOGICAL RESPONSE OF RESISTANT AND SUSCEPTIBLE WHEAT AND BARLEY UNDER BYDV STRESS
Review round: 2
Reviewer: 2

Output:
Basic reporting: 'no comment'
Experimental design: 'no comment'
Validity of the findings: The results of ANOVA (table 1) should be discussed in connection with the different analysed parameters, not in a separate paragraph (line 293-296).

Table 1 as supplement?
Additional comments: Proofreading of the whole manuscript is necessary (see also attached file).


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: REDESCRIPTION OF PHYMOLEPIS  CUIFENGSHANENSIS (ANTIARCHA: YUNNANOLEPIDIDAE) USING HIGH-RESOLUTION COMPUTED TOMOGRAPHY AND NEW INSIGHTS INTO ANATOMICAL DETAILS OF THE ENDOCRANIUM IN ANTIARCHS
Review round: 1
Reviewer: 1

Output:
Basic reporting: This manuscript is an excellent redescription of Phymolepis, a rare and poorly known antiarch from the Lower Devonian of Yunnan, China. The work is perfectly executed and provides a number of new anatomical information which allow to refine previous phylogenetic analyses of this important group of placoderm fishes. I have no major correction to make. Just a small comment concerning Vanchienolepis, from the Lower Devonian of Vietnam: Remember that the skull roof referred to Vanchienolepis langsonensis by Tong-Dzuy Thanh & Janvier (1990, fig.14) was found isolated from any typical Vanchienolepis thoracic armour, and there is no guarantee Yunnae that is belongs to this genus. Its attribution to Vanchienolepis is only based on its small size and the fact that it was found in a block which also contained undoubted Vanchienolepis plates. However, it is thus not ruled out that it is in fact a skull roof of a small, juvenile Yunnanolepis.
Minor corrections:
L. 64: yes, there is no information about the actual neurocranium of antiarchs, except perhaps the posterior part of the brain cavity in Minicrania (Zhu & Janvier 1996). I suspect that, like in osteostracans and galeaspids, the endocranial structures are more often preserved in very small individuals, possibly because of the proximity of osteogenic cells recruited from the overlying dermal skeleton.
L. 77: although their monophyly has not yet reached a consensus…
L.327: …within the nuchal plate, swings laterally when close to the midline…
L.353: Yunnanolepis
L.629 and 639: homoplastic is the original term, but homoplasious is also correct.
The quality of the illustration is outstandings.
Experimental design: not applicable
Validity of the findings: This description provides new key information about the anatomy of yunnanolepidoid antiarch fishes. Very useful in the framework of the question the origin of jawed vertebrates
Additional comments: This manuscript is an excellent redescription of Phymolepis, a rare and poorly known antiarch from the Lower Devonian of Yunnan, China. The work is perfectly executed and provides a number of new anatomical information which allow to refine previous phylogenetic analyses of this important group of placoderm fishes. I have no major correction to make. Just a small comment concerning Vanchienolepis, from the Lower Devonian of Vietnam: Remember that the skull roof referred to Vanchienolepis langsonensis by Tong-Dzuy Thanh & Janvier (1990, fig.14) was found isolated from any typical Vanchienolepis thoracic armour, and there is no guarantee Yunnae that is belongs to this genus. Its attribution to Vanchienolepis is only based on its small size and the fact that it was found in a block which also contained undoubted Vanchienolepis plates. However, it is thus not ruled out that it is in fact a skull roof of a small, juvenile Yunnanolepis.
Minor corrections:
L. 64: yes, there is no information about the actual neurocranium of antiarchs, except perhaps the posterior part of the brain cavity in Minicrania (Zhu & Janvier 1996). I suspect that, like in osteostracans and galeaspids, the endocranial structures are more often preserved in very small individuals, possibly because of the proximity of osteogenic cells recruited from the overlying dermal skeleton.
L. 77: although their monophyly has not yet reached a consensus…
L.327: …within the nuchal plate, swings laterally when close to the midline…
L.353: Yunnanolepis
L.629 and 639: homoplastic is the original term, but homoplasious is also correct.
The quality of the illustration is outstandings.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: REDESCRIPTION OF PHYMOLEPIS  CUIFENGSHANENSIS (ANTIARCHA: YUNNANOLEPIDIDAE) USING HIGH-RESOLUTION COMPUTED TOMOGRAPHY AND NEW INSIGHTS INTO ANATOMICAL DETAILS OF THE ENDOCRANIUM IN ANTIARCHS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper is a detailed description of an important taxon of fossil antiarch placoderm from the Early Devonian of China, with new data revealed through CT scanning, and will be of much interest to workers in this field. However the text can be shortened in places, as we no longer require for every plate to be described in minute detail (similar to Miles 1968 Bothriolepis paper) when figures can now do this clearly. The descriptive text should only used to highlight key main differences in anatomy of the taxon in question as compared with closely related  taxa that might be confused with it. Many statements in the descriptions are unnecessary as they state facts that either apply to all antiarchs or to all yunanolepidoids, so can be deleted (these are listed below for lines 233-286 for the headshield only, but this must be applied to entire descriptive section). The figures are all excellent and show the anatomy very clearly. The discussion and phylogenetic sections are is clear and require little corrective editing.
Indices for plate proportions is of little value if the sample is 1 or 2 specimens eg L356 says lateral wall of Tr/sh has a L/H ration of 2-3. This means there is 50% variation in the ratio, so it is actually taxonomically useful to include  here? I would recommend all the plate measurements and indices for all specimens (only 11 listed- L197-204) be all listed in a table or several tables laid out in a similar style  to how Miles & Dennis (1979) and Dennis & Miles (1979-1984).
The paper would thus benefit from a major rewrite of the descriptive text as it stands and revising how mensuration data is presented.

Specific points to address:-
L59-61 excludes King et al 2016, a major contribution to placoderm systematics that disagrees with the opening statement of this paper as it did not find antiarchs at the base of all placoderms, so a statement about this paper needs be included.
L64 'atlas' is not right word here as little is known of antiarch enodcrania, they are all hypothetically restored. I think "anatomical details of the endocranium" would be better.
L83 "crista transversalis interna posterior" etc throughout the text should be in italics as in Stensio 1948, Zhu 1996 etc.

Description
L236 “suborbital fenestra is smaller than orbital fenestra” is the same in all antiarchs, so not a useful statement. Delete or qualify by adding comparisons to other related yunanolepuidoid antiarchs where it is different, so the statement is more meaningful.
L240 –index of 2.16 i to 2 decimal places means little, with such small sample size,  simplify to 2.1, but should be listed in a table I(as I’ve said above).
L244 “relatively long” compared to what else?. Obstantic margins are only long if the headshield is long in its posterior half-so all yunanolepidoids have this trait. Statements like this need be qualified or they add nothing of value to the description. Qualify or delete.
L249 “elongated’ as above-relative to what taxa?  Qualify or delete.
L250-254 these 2 sentences apply to nearly all antiarchs as far as I know.. Clarify why this needs to be stated here, because it differs in xx closely-related species, or delete.
L256- postmarginal groove short-as in all yunnanolepidoids?  Qualify with more comparisons, or delete.
L257 “relatively broad” relative to what taxa?  Qualify or delete.
L258 “relatively long” relative to what taxa?  Qualify or delete.
L261-B/L index of 1.5 is close enough (one decimal point given such small sample size).
L270 should be ‘anterolateral’ not lateral angles
L271-272: Unnecessary stamen as the nuchal is divided into these 2 halves in all antiarch nuchal plates. Delete.
L275-7: statement about “the X shaped pit lines…” in front of obtected nuchal area applies in all antiarchs where well-preserved,  eg its in Bothriolepis (e.g. Stensio 1948 vol 1 text fig 10) plus Gogo specimens. Delete.
L277-278: “..as seen in other yunnanolepidoids” means it add nothing to the description that is either unique to the taxon being described, or adds new information about yunnanolepidoids,  so qualify with other comparisons or delete.
L278-280 about “the transverse nuchal crista reaching maximum thickness etc” is unnecessary as it applies to all antiarchs. Delete.
You can see from my comments above that the same kind of detailed editing needs be applied for the entire part of the descriptive section to L446, making more use of the figures. A table needs be included to states indices and list all specimens from which the measurements were taken.
 No need to write out the indices in the description or state if the plate is broad or relatively long etc. The suggested new table listing measurements and indices will do this. Just draw attention to how it differs from other closely related taxa and refer to the figure. This will greatly tighten up the descriptive part of the MS.

Other description comments:
L315 'Arenipiscis' is misspelt.
L320  Add ‘the’ before ‘semicircular depression”
L346-4476 need be edited carefully along same lines  s my comments for headshield. Cut out unnecessary statements that apply to all antiarchs or all yunanolepidoids, qualify all statements about being relatively long, broad, elongate,  or robust  as for how these apply to other related taxa.
L348 please add a label for  the ‘pectoral fossa’ in the figures
Discussion
L507 Not Miles 1968 but should be Hemmings 1978 for Microbrachius?
L543 Long & Trinajstic 2015 not in refs and it actually doesn’y exist, so please clarify which paper is meant here. Trinajstic et al 2015 Biological Reviews 90 paper perhaps? Or Long & Trinajstic 2010 Annual Reviews in Earth and Planetary Sciences  paper perhaps?

Phylogenetics etc
L687 “short occipital division” in sinolepids and euantiarchs: or is this a by product of the elongation of the premedian area, making the occiput seem shorter? In essence the occiput area is still long in sinolepidoids, but the premedian plate is also longer than in yunanaolepidoids. The occipital area is only shortened relative to entire postorbital area of the headshield in Bothriolepidoids and Asterolepidoids and certain taxa on their stem as shown in your final figures. Worth adding a short sentence discussing this.

Figures
All excellent. Perhaps revise Fig 14 to include  the entire head shield of Grenfellaspis in Fig 14B ventral view as all the head shield plates of this taxon  are shown in internal view in the paper, so it could easily be reconstructed.
Experimental design: All fine, see my comments above though regarding layout and presentation of mensuration data. Research is meaningful and well defined. Investigation is rigorous, methods OK.
Validity of the findings: validity and interpretation of the findings is  good. My only issue was with the way measurement indices are presented -needs to be in a table showing all speciemens measured, as some indices are based on few specimens, so this needs be clarified to be statistically significant.
Additional comments: This is very good work, but the descriptions follow an older style of presentation that can be greatly tightened up by editing down the descriptive section, letting the excellent figures do the talking. I hope my comments will improve the MS and make it clearer for readers to use.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A GUIDELINE FOR REPORTING EXPERIMENTAL PROTOCOLS IN LIFE SCIENCES
Review round: 1
Reviewer: 1

Output:
Basic reporting: Please see the "General comments for the author" section.
Experimental design: Please see the "General comments for the author" section.
Validity of the findings: Please see the "General comments for the author" section.
Additional comments: This is a review of “The SMART protocols checklist” manuscript.

The work described here is an extensive undertaking and I would like to genuinely thank the authors for the years of work that they have invested into this. Even though I have spent the last five years learning and thinking about life science protocols, I found the paper to be very valuable and educational. The authors’ approach of studying published protocols, journal guidelines, and expert feedback - while laborious, is precisely what this study needed. The final structural components recommended in the guidelines are all very reasonable and important for reproducibility.

I look forward to the publication of this paper and hope that it can help publishers and editors to improve their reporting guidelines for their journals.

Below is feedback that I hope the authors can use to improve their manuscript.

Kind regards,

Lenny Teytelman
———————————————————————
QUESTIONS AND SUGGESTIONS:

1. What is the intended use of the guidelines?

This paper is a thorough study of what ideally should be in a completely-described protocol. I imagine it will be of interest to the publishers. However, is that the intended audience or is it broader?

Is the hope that authors will follow the proposed guidelines? Is this a first step towards establishing standards and working to get journals to adopt them?

2. Charts in figures

The text in “Discourse data elements” is sufficient to describe the expert responses regarding “objective”, “applications”, “advantages” and “limitations”. Figure 2 carries no extra information and can be removed without loss of clarity. At most, leave the table in Figure 2 (but please change “purpose” to “objective” to be consistent with the text). Same holds for Figures 3,4, 5, 6, 7. Also, please note that in Figure 3, the y-axis instead of starting at zero as in Figure 2 starts at 76% and leads to the erroneous impression that the difference between 15/19 and 16/19 responses is somehow significant.

3. Conclusion

The authors write at the end, “Similarly, publishers should adhere to the principle of encouraging authors to have the protocols available; for instance as preprints or in repositories for protocols or journals. Publishers SHOULD NOT enforce the use of a particular repository or journal.”

I understand the sentiment here and I have a clear conflict of interest in commenting on this as the cofounder and CEO of protocols.io, a repository of research protocols. With that said, I must make the following observations:

	•	When publishers recommend the deposition of data into repositories, they are often very explicit in their instructions, naming specific repositories that comply with their requirements (e.g. Dryad and figshare).
	•	If the purpose of the guidelines is to improve manuscripts, compliance by authors is important. Even precise and strict requirements often have very low compliance from the authors (example, data policy from PLOS). Without clear instructions of where and how to deposit protocols, changes to guidelines may become meaningless.
	•	There are additional considerations when publishers or funders suggest or require the use of specific platforms or resources, even at the cost of liberty to the researchers. For example, specifying GitHub as the platform of choice for code-sharing may be somewhat restrictive. At the same time, if the funder wants to encourage collaboration between the grantees, guiding everyone to the same platform is reasonable.
	•	Ideally, recommended platforms and repositories should meet a set of requirements (preservation plans, archiving, backups, open APIs, open access, sustainability), but if they do, as I write above, specifying a given resource or resources may be warranted. Imagine if instead of GenBank, the scientists in the Human Genome Project had the freedom to deposit their sequences in any database of their choice. Chromosome I would be in one place, Chromosome 2 in another, and it would be impossible to query all of the genome from a single source.

This manuscript is nicely referenced, but for the argument above, I do not see any prior evidence or analysis that supports the recommendation of the authors. It is critical to balance authors’ freedom with the needs of readers, the barriers to adoption and compliance, and so forth.

———————————————————————
MINOR ISSUES

a) Grammar/spelling:

While the paper is very clear, accessible, and well organized, it does require a careful editing review for grammar. There are many examples of errors such as:

	•	“Rather that” -> “rather than”
	•	“the SP checklist aim” -> “the SP checklist aims”
	•	“shouldn’t researchers be hold” -> “shouldn’t research be held”
	•	“Methods X” is erroneously “Method X” in references
	•	“This guideline guideline describes” -> “This guideline describes”
	•	“the” should be removed in front of %s, as in “the 100% of the domain experts” should just be “100% of the domain experts”.

b) “SMART” looks like an acronym for something but is never defined. Perhaps “structured” or “complete” or “detailed, structured protocols” should be part of the name of the guidelines?

c) NPE is called a journal here; it is not. Protocol Exchange is a repository, similar to protocols.io. They do not perform peer review, but the authors write on page 5, “Analyzing protocols” that the published protocols in their analyses were all peer-reviewed.

d) Instead of “bad” and “good” example on page 10, consider rephrasing as “complete” versus “incomplete” or “detailed” versus ambiguous.

e) On page 10, under Bibliographic data elements, the authors recommend “50 character” or fewer. That seems much too short. Perhaps it’s a mistake and was meant to be “50 words”? The example of a good title in this section actually has 130 characters.

f) When mentioning “protocols.io” on page 11, you may want to cite the following publication:      https://doi.org/10.1371/journal.pbio.1002538.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A GUIDELINE FOR REPORTING EXPERIMENTAL PROTOCOLS IN LIFE SCIENCES
Review round: 1
Reviewer: 2

Output:
Basic reporting: The article address an interest topic in the context of reproducible science.
while the intent is laudable, the overall read could be easier by avoiding repetitions (e.g in the abstract "xperimental protocols are key elements when planning, doing and reporting research. Experimental
protocols are central to research methodologies")

Also transitions between sentences / idea developed are often abrupt or imprecise (e.g. "Experimental protocols are crucial for the execution and reproducibility of any study, the reporting structure remains highly idiosyncratic. However, there is little consistency in the data elements that should be included; the required data elements to be reported within the same discipline vary from publisher to publisher")

The use of 'However' is not suited. So it may be necessary to have the manuscript proof-read by a native speaker to catch these issues.

The manuscript would be greatly improved if the authors could review their draft to really streamline the reading experiment, paying particular attention to the logical transition between sections and thoughts.

The next sentence moves immediately into a specific observation ("Kilkenny et al. (2010a) evaluated 271 journal articles, they found that 4% did not report the number of
animals used anywhere in the methods or the results sections. ") which somehow conflicts with the previous statement in the fact that this reference indicates that 96% of authors did report sample sizes, which seems pretty consistent.

Furthermore, while the authors pick up on how important sample size is, it does not make to final cut of things to report about experimental procedures.
This leads to my next comment :
It would be great to have a delineation of scope for what a protocol is. In clinical context it has a different meaning that in say 'wet lab/molecular biology' context. This has huge implication into what needs to be reported.

Background / state of the art.
The authors may wish to consider efforts such RRID (https://www.force11.org/group/resource-identification-initiative)
More specifically, they could also identify and clearly name major efforts that effectively aim at normalization description of hardware, antibodies and plasmids. The resources are

https://accessgudid.nlm.nih.gov/
www.antibodyregistry.org
www.addgene.org

while RIP (resource information portal) is mentioned, the authors fail to place in context its importantce
Experimental design: Methodology:
The authors should clarify the inclusion criteria for selecting the resource they have analyzed.

In particular, with regard to the semantic artefacts considered, UBERON and NCBI Tax seem quite remote from anything related to describing protocols.
So it seems that section would benefit from distinguishing resources to use as fillers for properties (UBERON,NCBITax,CHEBI) from resources modeling the domain (OBI,BAO,CHMO,EXPO,...)

Then, not much information is available as to what methods were applied to analyse the corpus of data that had been assembled. Was text-mining used? What kind of metrics the authors devised?
For instance, in figure 1 (which is quite nice), a question is 'were protocols described as required by instructions to authors'. Was it done entirely manually ? were there rule based classifiers? was a croud sourcing effort implemented?

But above all, it would be interesting to have the authors discuss the values of 'checklist' in the absence of machine validation?
It would be more important to provide data reporting templates (metadata profiles, configurations)
Validity of the findings: Results:

line194(9), the link 'https://github.com/oxgiraldo/SMART-Protocols/tree/master/SP' returns a 404 but https://github.com/oxgiraldo/SMART-Protocols/ is accessible
and the following seems to work

Why not presenting this table in the main results rather than sending it to the supplementary section?

https://github.com/oxgiraldo/SMART-Protocols/tree/master/SP%20checklist

Looking at the protocol checklist, I am surprised to find a specific item for 'PCR primers'. while those are critical for any amplification based protocol, would it have been to general it to 'nucleic acid', where primers but also oligo-probes, vectors, antisense RNA,....
Why this selection?

While the table is easy to produce and useful, having a machine readable version for instance in the form of JSON schema, with a JSON LD context file which would provide a semantic markup.
It would be a much more modern representation and above all actionable form of the checklist.
Also, it would allow introducing the ontology produced by the authors and which is burried deep into the text.
Additional comments: Discussion:

Putting Biosharing and initiative such as ARRIVE and STAR is incorrect and possibly misleading.
Biosharing is a catalogue of resources about standard terminologies, policies,formats such as ARRIVE or STAR, but does not constitute a recommendation itself. I would encourage the authors to amend to manuscript accordingly (line 408 but also at the begining line 49)

The discussion would benefit from a rewrite as it feels as if many ideas were thrown in but lack clear focus. For instance, @line 432, the authors state "By the same token, our approach complement the ISA tools effort (Sansone et al., 2012)" but it is unclear how? this is where I would find the major weakness in the work presented. it is always oscillating between 'a simple checklist' or a 'structured specification'. This would require addressing.

Conclusion:
Line 456: "Publishers SHOULD NOT enforce the use of a particular repository or journal."
there seems to be a typographic error with 'should not' appearing in bold capital. This makes the whole sentence appear very loud.  and as it stands, I would strongly disagree with that statement.

Overall, the message is very unclear as what should be done : recommendation/enforcement. The main reason for heterogeneous reporting is the existence of loopholes, the lack of tools to automatically extract and markup entities and that fact in some cases, (patents) exact protocols and methods can simply not be shared. This is not discussed and may be having a few words would not hurt.

minor:
page 1, line 44: s/founders/funders/
page 16, line 412: s/routinary/routinely

Line 458: Please consider exchanging imprescindible with "essential".

the authors should cite MIBBI as follows:
https://www.ncbi.nlm.nih.gov/pubmed/18688244 or 10.1038/nbt.1411


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A GUIDELINE FOR REPORTING EXPERIMENTAL PROTOCOLS IN LIFE SCIENCES
Review round: 1
Reviewer: 3

Output:
Basic reporting: 1)  Clear, unambiguous, professional English language used throughout.
	The manuscript needs to be reviewed for English.  It's not bad, but there are a lot of grammatical errors and non-standard usage.

2)  Intro & background to show context:

I don’t think that the introduction sets up the problem appropriately.  Both the title ("SMART") and the opening paragraph describes the “data elements” of protocols, implying that the authors are aiming for machine-readable protocols, but that is not what the manuscript is about.  Rather, they are laying out essential elements of protocols in the form of a check list.  Why is it called “SMART”?

In the discussion, the authors make it clear why protocols are different than a methods section, but that is not made clear in the introduction.  Why was a check list approach taken?  Why aren’t existing checklists-several of which are mentioned in the introduction-not adequate?

See item 2 in the Experimental design section for suggestions on how to improve the introduction.

3)  Literature well referenced & relevant.

Overall yes.  Other initiatives and previous approaches to improving protocol reporting like KEfED, and process specification language should be discussed and referenced
.
4)  Structure conforms to PeerJ standards, discipline norm, or improved for clarity.

Yes

5)  Figures are relevant, high quality, well labelled & described.

Way too many figures, a lot of which are unnecessary.  Almost every figure needs a far more complete legend. Figures 2-8 reporting results from a survey of 19 domain experts are interesting but do not further the author's arguments or claims.

The actual Smart protocol elements would be better presented in a single table, rather than 21 separate tables, and should each include a good example and a bad example.  The complete table should include things such as the percentage of review protocols that included an 'Application of the protocol' section (for example). More detail can be provided in the text, but the way they are presented now is confusing.

They need to present data about how each part of their checklist is currently represented in the protocols they have reviewed, the author instructions, and the experts.

Table 6:  Why is AR relevant?  What is the difference between Suggested and Optional?  More information needs to be provided.

It looks like some elements, e.g., Description of sample tested are not even considered (NC)  in most guidelines.  If it isn’t there, how did it end up in this list? As will be detailed in the next section, the methods are under-specified (which is ironic given that this manuscript is about better reporting of methods).

6)  Raw data supplied.

The authors provide access to the protocols analyzed via a GitHub repository. Access to data and code is available on github but it needs to be archived via zenodo (or another repository).

The current GitHub repository doesn’t aline exactly with the data presented in Table 2.  The Nature Protocols are labeled inconsistently across the two (no mention of Nature in the Git Hub repository).  The Molecular Cloning protocols are not referenced in Table 2;  the non-polished protocols from CIAT are not in GitHub.

I see a list of instructions to authors, but I don’t see that they have been gathered and made available as a data set.  As instructions to authors change, it would be good to create a stable data set from the versions looked at.

A note on references here: shortened URLs are not acceptable for citation as there are no persistence guarantees (eg. line 511, 522). The citation on 522 should be given a zenodo DOI, and cited accordingly.
Experimental design: Within scope:

Yes

2. Research question well defined:
Needs improvement. The research question is clear to a domain expert and is extremely relevant and meaningful but is not sufficiently articulated in the paper. A simple restructuring of the argument of the paper will make it much clearer. Something like the following could work. "Reporting of research protocols is inconsistent. Despite this variability, we wanted to know if there were elements common across protocols. We want to use those elements as the basis for a set of suggestions for reporting protocols. In our analysis we found a common set of elements that we have used to create a practical checklist to help researchers when they report their protocols.”

3. High technical and ethical standard

As the study involved human subjects who were surveyed to produce data, a statement about whether or not this was reviewed by an appropriate IRB board should be included.  Although surveys may be exempt, a quick review of on-line materials suggests:  “Many survey projects are eligible for exemption.  However, the determination of exempt status (and the type of review that applies) rests with the IRB or with an administration official named by your institution. The determination does not rest with the investigator. “

4. Methods described with sufficient detail.

Needs improvement. The authors describe what was done without providing sufficient detail into how the checklist was actually generated from those steps. The authors repeatedly say that they analysed sources, but it is not at all clear what that entailed. There are a number of black boxes that need to be fleshed out. What were the criteria for including elements on the checklist (where there any)? What were exclusion criteria? Are there examples of elements that were considered but eventually dropped? 18 elements is very large, if you have to reduce the checklist to 10 elements what would they be? The resulting checklist is presented as is, which is fine, but the reasons given for inclusion of certain elements while practical, often do not have any obvious grounding the the various documents that were analyzed. Exact replication is not expected from this kind of work, but the authors need to report more thoroughly the links between their input documents and the elements they propose (see comments on Basic Reporting 4).
Validity of the findings: 1. Rationale and benefit to the literature is clearly stated.  See item 2 above. Benefit not explicitly stated, but the corpus of documents analyzed and the resulting checklist are major contributions in their own right.  But if we have so many of these already and the reporting is still substandard, then why is adding yet another one going to solve the problem?

Validity:  It doesn’t look like there was any attempt to balance the types of protocols or the domains covered.  The list of topics presented for the protocols does not match the expertise of the domain experts.  Also doesn’t look like there was any attempt to validate the results with specific domains that may have not been represented in the expert groups.

2.  Statistics:

Needs improvement. No statistics are reported. Given the number of protocols reviewed, it should be possible to give estimates on the portion of the 'population' of protocols that include certain elements, even if only for the domains investigated. The link to the corpus of protocols is not obvious and should be highlighted (it appears as if it were another publication when it is not).

3. Conclusions:.
Needs improvement. The conclusions section is not a conclusion but starts to delve into additional recommendations which either need to be cut, or need to be developed as additional sections. Questions of licensing (as well as patent vs copyright) are critical and if the review of the author instructions shows that no consideration has been made by journals then that is a reportable finding that should be highlighted! While we agree completely that protocols should be required to be reported (they should be in the paper not the supplement! or perhaps published separately and linked), the authors should cite literature related to replication and and reproducibility issues (since direct evidence that poor protocol documentation is the cause of this is still hard to come by) in support of their claims. In addition we would very much contest the idea the the protocol is treated just as any other publication, in the sense that only those protocols that are published are treated as such -- most protocols are not published at all (!) and methods sections do not count. Discussion of any plans to promulgate the checklist for testing whether it improves reproducibility would be much appreciated.  Again, if we already have a multiplicity of check lists, guidelines, instructions to authors, ontologies, data models etc., then why is adding yet another one going to help?  How will this get adopted?

3. Speculation:

OK
Additional comments: The authors are correct that it is time to treat protocols in a more standard and machine-friendly way and the authors have done a thorough review of the existing practices for reporting protocols in a subset of the life sciences and in that sense is a vital contribution. That said, the paper needs improvement so that it can more clearly communicate how the checklist was developed (specifically a quantification of how the elements included are currently represented across the corpus of protocols), and so that the rationale and motivation are clear to readers who are not also working directly on the problem of protocols.  Discussing why this approach is going to help when all others have not is also necessary.

This review was performed by Tom Gillespie, a PhD student working under the supervision of Dr. Maryann Martone. Dr. Martone has read and fully endorses the review.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A GUIDELINE FOR REPORTING EXPERIMENTAL PROTOCOLS IN LIFE SCIENCES
Review round: 2
Reviewer: 1

Output:
Basic reporting: I am satisfied with the revisions, improvements, and responses in this version.
Experimental design: I am satisfied with the revisions, improvements, and responses in this version.
Validity of the findings: I am satisfied with the revisions, improvements, and responses in this version.
Additional comments: I am satisfied with the revisions, improvements, and responses in this version.