Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FRAGILITY INDEX, POWER, STRENGTH AND ROBUSTNESS OF FINDINGS IN SPORTS MEDICINE AND ARTHROSCOPIC SURGERY: A SECONDARY ANALYSIS OF DATA FROM A STUDY ON USE OF THE FRAGILITY INDEX IN SPORTS SURGERY
Review round: 1
Reviewer: 1

Output:
Basic reporting: As I explain below, argumentation and referencing seem relatively poor to me.
Experimental design: Hard to judge, because so many things remain unclear about the methods.
Validity of the findings: Hard to judge, because so many things remain unclear about the methods and the aim of the study.
Additional comments: I appreciate the general aim and the efforts by the authors of this manuscript. After reading the manuscript, however, I have mixed feelings, partly because there are so many things that I do not understand or that remain unclear. I found many sentences about general issues related to the replication crisis that seem somewhat out of place, sometimes even without a reference. For example, in Line 222, it is said that misconceptions about "these other parameters" in statistics are common. Yes, and why do you say this, at the end of the paper? What are "these other parameters"? In Line 228, it is said that "Robust findings are more likely to be replicated, which is a fundamental scientific principle." Unfortunately, it remains unclear what the authors mean with "robust".

It seems that the authors define a finding as "robust" if chances that a replication is significant are (hopefully) high (e.g., Line 78: "Replicability, which is directly associated with power,..."). However, the presence or absence of significance is a very poor indicator of "robustness" and replicability (see, e.g., Goodman et al. 2016, What does research reproducibility mean? Science Translational Medicine 8, 341, and Amrhein et al. 2017, The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research. PeerJ 5, e3544).

So it seems to me that while the authors are highly skeptical about significance testing (e.g., "The infamous p-value cut-off of 0.05", line 202), they fall themselves trap to overemphasis of statistical significance as the main indicator of robustness. This greatly contributes to my general feeling that the manuscript needs a lot of streamlining, a better focus, and better explanations.

L53. I would delete "even", because it may lead to the misinterpretation that power could also apply if the effect does not exist.

L54. I don't understand this sentence.

L59. There are several names for the phenomenon, and several references could be cited. "This is known as the inflated effect" sounds oversimplified and unclear to me.

L64. I would delete "always". P should be taken as a continuous measure of compatibility between the data and the model, yes, but this is often not the case. Some references should be cited here, e.g. to the recent papers by Sander Greenland.

L68. I don't understand why p-hacking should lead to a dearth of replication studies.

L104. Unclear how recalculating p-values can be used to assess robustness of a finding. To judge robustness of an effect, replications would be needed.

L108. Please explain why you did this. The description sounds as if someone wants to push p below 0.05 by all means.

L115. Explain where the FI was taken from.

L144. Reword the sentence. "Both were seen on one" is not understandable.

L147. What do you mean with "significance was higher"? The entire procedure used in this study is rather unclear to me. Why was all this re-testing done? Were the original p-values all <0.05, and you showed that you found p>0.05 when using a different test? I can hardly see any sense in that so far. The sizes of p-values often change when different tests are used, but this does not argue against robustness of results, it argues against making strong decisions based on p-values or other statistics.

L151. Again, please explain where you took the FI from. Did you calculate it yourself?

L172. Again, please explain why you think that different p-values, obtained from re-analysis of data using different tests, is a "statistical inconsistency". Perhaps I missed something?

L203. "All non-significant results show a post-hoc power of less than 50%". I see no sense in making such a statement. Did you find this relation yourself? If not, give a reference. It has long been shown that observed power is a 1:1 function of the p-value (see, e.g., Hoenig & Heisey 2001, The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician 55, 19-24). Observing low power in non-significant studies shouldn't be a cause for concern.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FRAGILITY INDEX, POWER, STRENGTH AND ROBUSTNESS OF FINDINGS IN SPORTS MEDICINE AND ARTHROSCOPIC SURGERY: A SECONDARY ANALYSIS OF DATA FROM A STUDY ON USE OF THE FRAGILITY INDEX IN SPORTS SURGERY
Review round: 1
Reviewer: 2

Output:
Basic reporting: Clear and accurate (but see comments)
Experimental design: n/a it is a theoretical study
Validity of the findings: OK
Additional comments: This paper provides a bit more information than Khan et al, and should be of interest to people in the field.  The frequency of statistical inconsistencies is alarming and it's important that they are published.

The fragility index is an ingenious idea. It would certainly cast doubt on the strength of conclusions if only one or two people had to me moved into the non-responder category to change statistical significance.

The problem is that it is already known (and is shown again here) that the fragility index is strongly correlated with the p-value. To that extent it is just a surrogate for the p-value and suffers from all the problems of p-values.  These are pointed out in my 2014 paper (among many other papers), to which the authors kindly refer. However  they refer to it only in respect if the inflation effect.  A more up to date version is at http://rsos.royalsocietypublishing.org/content/4/12/171085 and its associated web calculator: http://fpr-calc.ucl.ac.uk/
I suggest that might be a more relevant reference.   I would take the view that what really matters is the false positive risk -the probability that your result is a false positive.

I'm not suggesting that the paper should be rewritten in terms of false positive risks, but it might be worth noting that the fragility index still uses the much-maligned p = 0.05 threshold. Since p = 0.05 corresponds to a false positive risk of at least 26%, the fragility index must exaggerate the reliability of the conclusions   If this is mentioned it will add more strength to the conclusions of this paper concerning the lack of robustness of findings in sports medicine and arthroscopic surgery RCTs.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FRAGILITY INDEX, POWER, STRENGTH AND ROBUSTNESS OF FINDINGS IN SPORTS MEDICINE AND ARTHROSCOPIC SURGERY: A SECONDARY ANALYSIS OF DATA FROM A STUDY ON USE OF THE FRAGILITY INDEX IN SPORTS SURGERY
Review round: 2
Reviewer: 1

Output:
Basic reporting: Interesting concept, but the article does not feel that organized. Interestingly, it also feels somewhat post hoc driven, which is ironic given the authors main message.
Experimental design: Adequate. There has been some similar work looking at the FI and its relationship with p-values (e.g., https://academic.oup.com/eurheartj/article/38/5/346/2422087).
Validity of the findings: The review of the literature and validation of the previously published studies was not reviewed carefully. It is quite interesting and a case study with review with the authors on what might have happened could also be quite telling. Hopefully the authors have been notified so that an erratum can be published.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FRAGILITY INDEX, POWER, STRENGTH AND ROBUSTNESS OF FINDINGS IN SPORTS MEDICINE AND ARTHROSCOPIC SURGERY: A SECONDARY ANALYSIS OF DATA FROM A STUDY ON USE OF THE FRAGILITY INDEX IN SPORTS SURGERY
Review round: 2
Reviewer: 2

Output:
Basic reporting: There needs to be some additional English language review.
The pivotal paper by Khan is not adequately referenced.
Experimental design: See comments below
Validity of the findings: See comments below
Additional comments: Thank you for asking me to provide a post-revision review of this interesting paper looking at the Fragility Index and other measures of study strength in sports medicine and arthroscopic surgery. As far as response to the initial comments of reviewers the authors have attended well to the details and give good explanation of the changes they have made. However, I have further comments on the revised paper to make. The study as presented here is a rework of data previously published by Khan et al in 2016 (Au note: the paper does not have the correct reference for this article), and thus the major fidning of this current paper is not in the fragility of the previous 48 studies but in a) further analysis of the ‘strength’ of the studies, b) an analysis to investigate the relationship of the fragility of the studies with other factors, c) the new statistical analysis of the 48 studies.
Considering a), The results are interesting, tend to confirm previous findings, after all fragility index is not ‘new’ it is just a different way of emphasising the power problems that affect small sample sizes. Why therefore have the authors added a level of confusion by starting their discussion with a redefinition of ‘robust’, this distracts from the study and is confusing. It also makes a discussion which is already overlylong longer still.
Considering b), I understand the reason that the authors use the studies previously analysed by Khan, but that paper is 2 years old and there may be more. The authors should have taken the opportunity to add more recent papers (or state that there aren’t any) found by an appropriate search methodology to the current analysis. In my copy of the study spreadsheet in the supplemental material I am only able to see the study reference when opened in excel. We need more data on these studies, particularly considering point c below.
Considering c). This alone is a worthy of publication, the authors are stating that their analysis shows that almost 20% of papers in this literature claim significant findings which do not stand up to simple statistical reanalysis. This is a major finding and the authors are right to ‘view this as a serious concern’. Firstly this finding could be very serious for the authors of the studies, their institutions, the wider credibility of sports medicine and orthopaedic surgery, not mention PeerJ itself and its editors, as well as the authors. The Editors need to be absolutely confident in this data prior to publication, I strongly suggest a second independent statistical analysis at the minimum. It is a potentially important finding and should not be suppressed, but the reader also needs to be fully assured that it is itself correct and able to be replicated . I myself have not individually checked the original data, but it did occur to me that it might also be wise to contact the original senior or first authors to ask for data clarification. It is also vital that the reader can make their own judgement as well, an excel supplement is in my view inadequate; we should be able to see straightaway while reading the paper, and suitable data in tabular form, to allow us to understand what is happening here. At the minimum we need to see the numbers used in the analysis, drop outs, statistical tests used, p values (etc) quoted, and the results of the new analysis. As I have noted above the file I downloaded did not contain this data. Lastly on this point, the authors should be more factual in their opening paragraph of the results, the words like ‘claimed’, ‘disclosed’ or ‘..one study even..’ are better left to others, here just the facts should be reported the reader will add the rest.
General points: The paper needs English language editing


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FRAGILITY INDEX, POWER, STRENGTH AND ROBUSTNESS OF FINDINGS IN SPORTS MEDICINE AND ARTHROSCOPIC SURGERY: A SECONDARY ANALYSIS OF DATA FROM A STUDY ON USE OF THE FRAGILITY INDEX IN SPORTS SURGERY
Review round: 3
Reviewer: 1

Output:
Basic reporting: See below
Experimental design: See below
Validity of the findings: See below
Additional comments: Thank you for asking me to re-review this paper concerning Fragility of studies in Sports Medicine. A number of points before my comments: I was not able to access the Raw Data Excel file which produced corrupted text; there appears to be no point by point comments or rebuttal on my earlier referee comments, there is for the Editor and Reviewer 1 and 2. The last observation makes it hard for me to follow thorough how the authors have responded to my comments as you can imagine.

Overall the paper is improved, but I still have significant concerns.
I am very confused by Results Para 1. I think the message might be better shown as a flow diagram (as is done with a study recruitment to analysis flow figure). It is very difficult to follow the authors logic here:
 ln 176 What happened when corresponding authors were contacted and responded? Need a detailed breakdown.
Ln 178 are the P values for reg Fischer test the original data or re-worked data? The same comment goes all through this paragraph
Ln184 should the word inconsistent be replaced with incorrect?
How were corresponding authors contacted and how long did they get to respond etc
We desperately need a table of original data and re-worked data, referenced to the original studies.
The authors found new studies to analyse (19 I think Ln 34), but these are not mentioned again outside the abstract, and are not referenced in this paper, and are not discussed in the Results section, nor are commented on in Discussion.
The supplementary files are not referenced in the text, will they be available to the reader? They need to be signposted.
Supplementary file ‘Studies with inconsistencies’ only refers to 3 studies, however Results para 1 suggests another 13 more Ln 177, or 8 Ln 40, explain. Very confusing. Also as in my earlier review it would be useful to have the original author comments seen with the description of the inconsistencies.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: APPLICATION OF THE URBAN EXPOSOME FRAMEWORK USING DRINKING WATER AND QUALITY OF LIFE INDICATORS: A PROOF-OF-CONCEPT STUDY IN LIMASSOL, CYPRUS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors have presented a good background on this topic. The language use is professional, and the paper is structured conform journal guideline.
Experimental design: The manuscript is within the journal’s scope. It has well-defined research questions considering the current knowledge gap in the literature, followed by a rigorous investigation. Methods are well described.
Validity of the findings: Data and statistical analysis are robust and controlled.
Additional comments: Authors of this manuscript have presented a study concerning the application of the urban exposome framework using drinking water and quality of life indicators. The authors have found that urban drinking-water quality using conventional indicators of chemical (disinfection byproducts-trihalomethanes) and microbial (coliforms, E. coli, and Enterococci) quality did not raise
particular concerns. Also, The general health and chronic health status of the urban participants were not significantly (all >FDR corrected p-value of 0.1) associated with any urban life variables. The topic is interesting, an important issue and generally well written, well structured and contributes to the existing knowledge. However, there are still some occasional grammar errors through the manuscript especially the article ‘’the’’, ‘’a’’ and ‘’an’’ is missing in many places, please make a spellchecking.

The results and discussion section needs further improvement, compare your findings with the other author's conclusions.
•	In general, the manuscript needs to revise again concerning technical and grammar errors.
•	Please provide more deep discussion about your results, compare your findings with the other author findings.
•	Please clearly state the novelty of this work.
•	Please check the reference style, some of the references are not according to the journal style, especially the journals abbreviations.
•	As said, it could be a failure to comprehend the main argument; and lightening the discussion would enhance comprehensibility and make the authors’ point clearer and stronger.
•	Therefore, the reviewer recommends to further improve the manuscript before accepting it for publication.

Concluding Remarks
The work presented in this manuscript is an interesting topic, it needs some more efforts to improve it further. Reviewer recommend minor revision of this manuscript and publishing it only after specific improvement of the current version are made.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: APPLICATION OF THE URBAN EXPOSOME FRAMEWORK USING DRINKING WATER AND QUALITY OF LIFE INDICATORS: A PROOF-OF-CONCEPT STUDY IN LIMASSOL, CYPRUS
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper is generally well written and well-organized. As a ‘proof of concept’ paper, it is not meant to draw strong conclusions but rather to demonstrate the process the study went through as well as show ways that similar studies could be done in the future with more details and more specific goals and outcomes. In this sense, the paper does a good job of demonstrating that quality of life indicators can be paired with qualitative assessment/perceptions from study participants to map potential relationships between quality of life and both internal and external factors.

Despite an overall fit-for-purpose design, for a proof-of-concept study, there are some opportunities for improvement as follows (and discussed in more detail in the remaining sections)

1.	In the qualitative perceptions study, it is often not clear from the paper what specifically was assessed and therefore what to think of the associations between that perception and other perceptions or qualitative measurements. Especially for relationships that are specifically discussed in the paper, it should be clear what was measured based upon information in the text and main tables. This is described in more detail in the line-by-line comments to the author.

2.	There are several statements that are unclear, especially in the introduction and the materials/methods section. The paper is generally in clear, understandable English, with a handful of unclear statements. I have mentioned each of these statements in the line-by-line comments for the author.

3.	As described in the paper, the “urban exposome can be defined as the totality of indicators (quantitative and qualitative) that shape the quality of life and health of urban populations (Andrianou & Makris, 2018)”. Reality dictates that only some subset of relevant items can be measured as indicators, and to be useful in this type of analysis they will need to have some known public health or quality-of-life link in the quantities seen. There’s no major issue with the methodology of measuring of THMs (minor comments are included later in this review), but I am struggling to understand the use of variation of low levels of THMs as an indicator for public health or quality of life. The paper makes a presumption that there is a link between say, the difference between 20 ug/l and 50 ug/l and some health outcome. WHO does have established recommended standards for each of the four major THMs individually and an equation as a suggestion for an additive measure for potential regulation without a single additive value (see https://www.who.int/water_sanitation_health/dwq/chemicals/THM200605.pdf). The EU also has a parametric TTHM value of 100 ug/l, and US EPA uses 80 ug/l with a locational running annual average.  The TTHMs measured in this study are a single point in time, whereas the WHO and US EPA values are meant for chronic exposure. Within the WHO guideline, page 41 states “… neither clear evidence of a threshold nor a dose-response pattern of increasing risk with increasing concentration of THMs has been found (Reif et al, 2000).”  There is considerable concern around THMs, including both regulatory and best practice literature on the reduction of THMs, but at the same time there is also a lot that is not known as well as some controversy around THMs. These unknowns and controversy are not mentioned.  Therefore, I struggle to understand how the evaluation of differences in values below regulatory thresholds contributes to understanding of the exposome in instances where there is a lack of clear health-based values. Of 132 samples, all but 3 were below the EU value. I did not attempt to measure against the WHO values but suspect new or possibly none are above those guidelines. I recommend reading Cotruvo & Amato (2018) at https://doi.org/10/1002/awwa.1210 for a history of the use (and potential inappropriate causation) of THMs and certain health endpoints (although this article is not itself peer reviewed, it does link to many peer-reviewed, regulatory, and other authoritative matters on the subject that can be used for background). At a minimum, clearly stating why THMs were chosen considering not having a clear dose-response relationship is vital to demonstrating the value of this study and recognizing the controversy over the value of comparisons of THMs at below-threshold values is essential to the value of this work.  One possible solution is to state that there is no intended health endpoint (especially at values less than the regulatory thresholds). Another is to code the TTHM results as above or below the regulatory limit and study the relationships with other factors as an ordinal variable (above/below) as opposed to an interval variable. Explanation as to the selection of THMs, acknowledgement of the unknowns and controversies will add more credibility to the paper. Variations in THMs does serve as a good proxy for water age in chlorinated systems, so to the extent that this is the intent it should be clearly stated. I recognize that being a proof-of -concept study this isn’t as important as it would be as if specific conclusions about the area were being derived, but regardless this should be addressed to avoid unnecessary controversy and focus the paper on the main findings.
Experimental design: As a proof of concept study, the selection of respondents and other controls are not as vital to the validity of the findings.  The distribution of the perceptions survey was not very random (much of the activity was through a university, which lends itself to more highly-educated persons than average (and there may be other ways in which they aren’t necessarily representative of the population at large. However, since the paper’s conclusions are not contingent on a representative sample, this is not a fatal flaw (although mentioning this in the text would improve transparency).  There are minor comments within the comments to authors.
Validity of the findings: This paper is a proof of concept study.  It demonstrated the collection and correlation of data for the purposes of an “urban exposome framework using drinking water and quality of life indicators” as described in the title.  With acknowledgement of the limitations of the data, the findings (which are essentially that the process is feasible) are appropriate.

The main non-process finding is that there’s a correlation between THMs and cleaning activities. It isn’t clear what cleaning activities specifically measures, and no plausible connection between the two is even mentioned. This finding requires more explanation.
Additional comments: -	Overall, the data provided looks to be helpful and supports the paper.
-	Overall, the paper appears well written and organized.
-	Lines 50-51: It is not obvious why increasing city populations would make access to safe and affordable water more important than it would have been with a smaller population.  It would seem that access to safe and affordable water was always (and will always) be extremely important.
-	Line 65-68: The sentence starting with “to present how the urban….” Is unclear, especially the phrase “of the urban exposome framework that often relies upon a hypothesis-generating scope, instead”.  Please clarify, and possibly split the sentence into two or more.
-	Lines 72-73: The phrase “half of the urban population of Limassol resides within the municipality of Limassol” is unclear as it seems to refer to itself (half of area A lives in area A). Reword for clarity.
-	Lines 119-123: This is the area that indicates that the perceptions survey is strongly weighted to the university, as it was “distributed via email among staff”.  It is also unclear the method of reaching the public through “mailing lists”.  Assuming these were email lists, generally such lists are for a specific interest or purpose. It would be helpful to know more about the distribution forum.
-	Line 126: The methods section only states that participants “were recruited” but doesn’t clearly state now they were recruited.
-	Lines 128-129: Since urine samples were not used in the study, why are they mentioned? If the reference is kept, please define “biomonitoring purposes”
-	Lines 133-134: Recommend changing “small in area and population size quarters” with “Quarters that are small in area and population” for clarity.
-	Line 137: Isn’t “street” and “household” redundant, in that if only one participant can be chosen in a street, doesn’t that automatically preclude two in a household? Also, what constitutes a “street”? What is a road crosses multiple quarters?
-	Line 148: The beginning of the “water sampling and analysis” would be a good place to put in some discussion on how these indicators were chosen, what they mean, etc.
-	Line 152: Was there a known minimum or maximum stagnation time? Since both THMs and chlorine residual are good proxies for water age, this value is potentially important. Was any information obtained about whether the fixture used was or was not used frequently in previous months? Was anything about building complexity (a, whether the sample comes from a single-family home, a small apartment building, a large apartment building, or something else)
-	Lines 178-180: The statement here about samples with filters present here is confusing, as lines 165-166 state that those with filters were removed from the analysis.
-	Line 198: “BrTHM” is used without introducing what it is and why it matters.
-	Line 275: the statement that “most of the respondents (84%) were highly educated” provides evidence for my previous comment that the sampling methodology favored the university.
-	Line 278-279:  One cannot be “severely exposed to air quality”. I think this is supposed to be “severely exposed to poor air quality”.
-	Line 303: Are the listed health conditions all of the ones that count as “chronic diseases”, is there a list elsewhere, or is it open-ended?
-	Line 310 “consensus not using them” is unclear – is it meant to be a “consensus for not using them” or something else?
-	Line 324: There are a handful of edits in the document that appear to be in track changes, such as the change on this line from “to” to “for”. Please remove all track change edits from the paper (accepting or rejecting as appropriate).
-	Line 335: There’s a “Error1 Reference source not found” error on this page, likely for a broken reference link or maybe a link to a table/figure?
-	Lines 379-381: The statement “besides for THM; for example, strong correlation was observed between household cleaning variables… and certain health conditions”.  The sentence structure makes it sound like the correlation is between cleaning variables and health conditions.  However, from elsewhere in the paper, the correlation is discussed as between THMs and other indicators (health, for example).  Please clarify.
-	The sentence “the correlation analysis revealed the association of water THM with cleaning activities”.  Cleaning activities is one of the perception items that is unclear. The supplemental files include number of days dishwashing, amongst other things, which is not clear because dishes alone are not really a cleaning of the household.
-	Line 512-513: the statement that no specific concerns about the drinking water quality were found is very helpful for clarity, but also leads to the question of why THMs?
-	Figure 1: The relationship between human and urban is not clear in the figure.  The columns do not align with each other.  In the “application” column, it isn’t clear whether or not the “applications” are meant to align with the top line items.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: APPLICATION OF THE URBAN EXPOSOME FRAMEWORK USING DRINKING WATER AND QUALITY OF LIFE INDICATORS: A PROOF-OF-CONCEPT STUDY IN LIMASSOL, CYPRUS
Review round: 2
Reviewer: 1

Output:
Basic reporting: There is no any scientific or technical issue.
Experimental design: There is no any scientific or technical issue. The experimental design is correctly made.
Validity of the findings: There is no any scientific or technical issue. The data and findings are valid and meaningful.
Additional comments: Authors of this manuscript have adequately addressed all comments and suggestions made by reviewers; I would like to congratulate them for the effort. This version of the manuscript is significantly improved, therefore I recommend publishing the manuscript as it is.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: APPLICATION OF THE URBAN EXPOSOME FRAMEWORK USING DRINKING WATER AND QUALITY OF LIFE INDICATORS: A PROOF-OF-CONCEPT STUDY IN LIMASSOL, CYPRUS
Review round: 2
Reviewer: 2

Output:
Basic reporting: This paper remains as well written and well-organized. The authors have adequately addressed my previous concerns in this area.
Experimental design: My previous concerns around participant selection have been adequately addressed by the authors.  The remainder of the experimental design is also well-constructed and reported.
Validity of the findings: The only concerns I previously had around the validity of the findings have been appropriately addressed by the authors.
Additional comments: Previously reported general and line-item comments for the authors have been addressed in this revision. I have no additional suggestions for revision, and I believe the paper is in good shape to move forward towards publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WE NEED TO TALK ABOUT RELIABILITY: MAKING BETTER USE OF TEST-RETEST STUDIES FOR STUDY DESIGN AND INTERPRETATION
Review round: 1
Reviewer: 1

Output:
Basic reporting: This is an interesting and mostly useful paper. The declared aim is to use test-retest data and insights into experimental design to build meaningful power analysis. There is nothing in the paper this reviewer disagrees with and I found the article very well structured and written.
Experimental design: This is a methodological paper that relies on previously published data.
Validity of the findings: As a researcher working in PET,I find the author's  quantitative insights matching with experience and I can confirm that indeed endogenous release studies with PET are notoriously underpowered (except for dopamine) and a general waste of money. One note of warning is on the TSPO (PBR28) data. There is a lot of literature on the biological validity of Vt with these tracers; the true concentration of reference for the calculation of Vt is the free plasma concentration of the radiotracer in blood. Hence both plasma radioactivity and free plasma fractions are required. Now, free plasma fractions do not usually change much across subjects/patients and are generally ignored except for these tracers as 1) there is a lot of TSPO in blood cells and 2) TSPO radiotracers seem to have affinity for plasma reactive proteins. However since free plasma fractions for these tracers are very small, they cannot be measured reliably enough; hence the excess variability of VTs and the pragmatic choice of within brain normalizations
Additional comments: No further comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WE NEED TO TALK ABOUT RELIABILITY: MAKING BETTER USE OF TEST-RETEST STUDIES FOR STUDY DESIGN AND INTERPRETATION
Review round: 1
Reviewer: 2

Output:
Basic reporting: This is a solid article on a very important topic. My main concern is that the ideas, technical terms and equations are difficult to digest, mainly because of a lack of illustrations. I think the impact of the paper could be greatly increased by providing detailed illustrations, to help readers digest all the information in the methods section, and to give them an intuition for the relationships among variables/measurements. A first illustration could show the  different sources of variability with the matching symbols used in the equations. Illustrations of actual data from different experimental designs/statistical analyses are also needed.

In the introduction, the standard concerns about false positives and true positives are contrasted with the more unusual one of reliability. In addition,  measurement precision is considered an important factor by some researchers. You can see references about accuracy in parameter estimation here:
[[https://garstats.wordpress.com/2018/08/27/precision/]]
In particular:
Peters, G.-J.Y. & Crutzen, R. (2017) Knowing exactly how effective an intervention, treatment, or manipulation is and ensuring that a study replicates: accuracy in parameter estimation as a partial solution to the replication crisis. PsyArXiv. doi:10.31234/osf.io/cjsk2.

It would be worth articulating how these notions relate to each other in the introduction:
- power
- reliability
- accuracy in parameter estimation
You do mention all 3 aspects in later sections but the point should be clarified in the introduction.
Experimental design: I would suggest adding R code in the text, following each example (tutorial style) or at least as an appendix.
Validity of the findings: Illustrations are also needed to let readers assess the match between real data and the proposed methods. As far as I can see, all the tools implemented in the R package assume normality. Do we have to worry about this reliance on non-robust stats? For instance, skewness and outliers tend to inflate variance estimates and lower effect sizes. Is that a problem when assessing results from relatively small n studies, especially in the absence of scatterplots in the original articles? Are there robust versions of ICC? I’m not expecting a robust version of all the tools presented here, but at least the lack of statistical robustness should be acknowledged.
Additional comments: ## other points
Figure 1: I do not understand the description in the right panel. How can there be a true sample ES? This sentence does not clarify the point and the “respectively” structure is hard to read: “The population (Popn.) effect size (ES) and sample ES are of the underlying distributions from which the data are sampled and of the obtained sample respectively.”

“In psychometrics, reliability is often assessed using internal consistency” - can you provide a reference?

## typos
“USD 10 000” -> 10,000
“little consideration can given to” -> can be given to
“as an estimate the reliability/relative/absolute” -> “as an estimate of the...”


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: WE NEED TO TALK ABOUT RELIABILITY: MAKING BETTER USE OF TEST-RETEST STUDIES FOR STUDY DESIGN AND INTERPRETATION
Review round: 2
Reviewer: 1

Output:
Basic reporting: Fine, only some typos and potential writing issues:

49: there there has been
107: “True scores can never be directly observed, but only observed with different degrees of error.” -> but only ESTIMATED
149: “scaled to the variance of the sample as an estimate the reliability” and subsequent points -> as an estimate OF the reliability / as a reliability estimate / TO estimate reliability...
203: “when performing power analysis” -> a power analysis / power analyses
375: “larger than what what was considered”
376: “insignificant” -> non-significant
491: “reaching the the data”
566: “an equal equal radioligand“
Experimental design: Excellent.
Validity of the findings: Excellent.
Additional comments: The new version is much clearer. Your article presents a tremendous amount of work and I hope it will contribute to researchers spending more time thinking about statistical issues before they run their studies. The revised discussion provides a rich description of the multiple angles to consider when designing an experiment, with plenty of important references for curious readers - great job!

Thank you for adding the figures and the code.
I only have a few minor comments, the main one about the caption of figure 2.

Figure 2 is very nice, but the caption needs to be improved. For instance, this sentence is very difficult to understand: “Low reliability within groups is increased for the total sample in a comparison, given a sufficiently large effect size, due to the larger variance of the combined sample.” One suggestion would be to refer to the 2 panels Group 1 and Group 2, then mention their combination, pointing to the third panel (needs a name), and explain what happens to the ICC. For the right panels, I still find the True/Measured dichotomy hard to understand in relation to the Popn/sample one. Instead of True/Measured, would the distinction be better described as without/with measurement error? Also, a general title is missing.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DOUBLE DECOUPLING EFFECTIVENESS OF WATER CONSUMPTION AND WASTEWATER DISCHARGE IN CHINA’S TEXTILE INDUSTRY BASED ON WATER FOOTPRINT THEORY
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: •The way the authors analyze the double decoupling effectiveness of water consumption and wastewater discharge in China's
 textile industry is very specific. The results of this research can be used to constitute a practical tool in the realization of water decoupling in the textile industry.
•My only suggestion is to verify the English; minor errors were detected.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DOUBLE DECOUPLING EFFECTIVENESS OF WATER CONSUMPTION AND WASTEWATER DISCHARGE IN CHINA’S TEXTILE INDUSTRY BASED ON WATER FOOTPRINT THEORY
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: General Comments:

The paper mainly analyzes the water consumption decoupling, wastewater discharge decoupling, as well as the double decoupling of water consumption and wastewater discharge of China’s textile industry based on water footprint and decoupling theories. This work were conducted and analyzed extensively. However, some serious shortages must be revised.

Specific Remarks:
- This paper analyzed three sub-industries from 2001 to 2014. Why select those data in the years instead of the lasted data?

- Table 1. Emission limits for wastewater pollutants in the textile industry. Where did the data come from? Please give the cited papers or documents.

- Water Footprint Accounting Method had been reported. In this work, the method was used in textile industry by the authors. Can you illustrate the difference?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DOUBLE DECOUPLING EFFECTIVENESS OF WATER CONSUMPTION AND WASTEWATER DISCHARGE IN CHINA’S TEXTILE INDUSTRY BASED ON WATER FOOTPRINT THEORY
Review round: 1
Reviewer: 3

Output:
Basic reporting: This manuscript uses decoupling theory to study the relationship among the development of textile industry, resources consumption and environmental emissions. With the clear thoughts, appropriate approach and the credible results, this article provides a valuable reference to reduce resources consumption, decrease pollution emissions, and promote the green development of textile industry. Meanwhile, it will also make contribution to some relative research for other industries. However, there are still some shortcomings in this manuscript, which need to be further thought and improved.

The literature review needs to be modified. You may accept this following writing framework: 1) The current status of resources and environment research in China should be introduced in the first part (the research background). Then the situation of resources and environment in textile industry can be mentioned. In this part, you may briefly describe the current situation of Chinese textile industry, emphasis with the relationship changes between textile industry and water resources environment, and appropriately add some relevant studies on the textile industry and pollution emissions both in China and aboard. 2) The second part is about the decoupling theory and its application. In order to clarify the inherent meaning and application of decoupling theory, you need to focus on the judgement from decoupling index and to summarize the rule of industry development, then put forward some polices for future development.
Experimental design: The technical method of this paper is fitted, but based on outdated statistics. It will be better if you analysis with the past three years data from 2015 to 2017. The reason is that after 2015, China has vigorously promoted the construction of ecological civilization. Affected with taking ecological construction and industrial restructuring seriously, China's major pollutant emission has shown a downward trend.
Validity of the findings: The results basically match with the facts. But the reason analysis for decoupling or negative decoupling in textile industry needs be improved.

Countermeasures and suggestions are not deeply enough. You need to put forward some more pertinent suggestions, based on further analysis of the development rules and mechanism in textile industry in recent years.

Try to revise and improve relevant charts. For example, the results could be displayed in the form of radar maps to make the annual evolution of the research results more clearly.
Additional comments: A proof reading by a native English speaker should be conducted to improve both language and organization quality.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DOUBLE DECOUPLING EFFECTIVENESS OF WATER CONSUMPTION AND WASTEWATER DISCHARGE IN CHINA’S TEXTILE INDUSTRY BASED ON WATER FOOTPRINT THEORY
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The manuscript can be accepted.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DOUBLE DECOUPLING EFFECTIVENESS OF WATER CONSUMPTION AND WASTEWATER DISCHARGE IN CHINA’S TEXTILE INDUSTRY BASED ON WATER FOOTPRINT THEORY
Review round: 2
Reviewer: 2

Output:
Basic reporting: The manuscrip has improved the reference.
Experimental design: The experimental design is reasonable and the data has been updated.
Validity of the findings: The conclusion is correct. The suggestions will have reference to management.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SERUM HEPARAN SULFATE AND CHONDROITIN SULFATE CONCENTRATIONS IN PATIENTS WITH NEWLY DIAGNOSED EXFOLIATIVE GLAUCOMA
Review round: 1
Reviewer: 1

Output:
Basic reporting: line 25: typo: exraocular - should be extraocular
line 158: typo: filed - should be visual field

no further comment
Experimental design: Yes: original primary research within Aims and Scope of the journal
Yes: Research question well defined, relevant & meaningful; fills a knowledge gap
Yes: Methods described with sufficient detail & information to replicate pending answers to below:

Suggestion for improvement: Overall great study
line 106: For the authors - please clarify: was POAG defined present with the typical glaucomatous disc WITH OR WITHOUT glaucoma visual field changes (or is it and/or glaucoma visual field changes, as is written.  Same question for PXG, line 108-109.

The reason to ask for this important clarification is "The vast majority of of patients with POAG have disc changes or disc and visual field changes, but there may be rare cases where there may be early visual field changes before there are detectable changes to the optic nerve."  Preferred Practice Pattern: Primary Open Angle Glaucoma; American Academy of Ophthalmology 2016.

Please clarify: were patients with diabetic retinopathy excluded?
Validity of the findings: Results:
line 234: what was the p value for IOP in the XFS group vs NC group?

line 255: while that between XFG and POAG was NOT significant (rather than borderline significant (p = 0.099)

limitations: consider including small sample size

consider further commentary on how Diabetes could have affected the results and average HgA1c if available
Additional comments: Overall this is a good study and idea, that could generate future studies and investigation into therapeutic measures and early disease detection.  Please see the comments above


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SERUM HEPARAN SULFATE AND CHONDROITIN SULFATE CONCENTRATIONS IN PATIENTS WITH NEWLY DIAGNOSED EXFOLIATIVE GLAUCOMA
Review round: 1
Reviewer: 2

Output:
Basic reporting: In my opinion the overall strength of manuscript can be improved by explaining the differences between the XFG, XFS and POAG in the introduction for common audience.
How many subjects did authors excluded based on exclusion criteria?
Most of the documents in supplemental folder are written in Serbian language. The documents must be translated and verified.
Measurement range and detection limit for Human chondroitin sulfate should be written in methods section.
In line 195 change “Tuckey” to “Tukey”
Experimental design: Even though XFS is a systemic disease, XFG and POAG are ocular diseases so shouldn’t tears be more ideal biological fluid to look for biomarkers?
Did the authors observed any gender differences in the HS and CS expression levels?
Did the authors agree that ocular characteristics like IOP, BCVA seems to be better biomarkers than CS and HS? If yes, then what is the rational behind looking for HS and CS in serum?
Validity of the findings: In the discussion the authors note the importance of finding sensitive and specific biomarkers to accurately predict the presence and progression of disease however in the current study they did not provide the specificity and sensitivity values of HS and CS. Without good sensitivity and specificity, the significant differences observed by the authors cannot be used as a good biomarker.
In Fig’s 1-4 if the bars represent min and max values how can they have outliers and extreme values shouldn’t they be the parts of bars they are min and max values?
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SERUM HEPARAN SULFATE AND CHONDROITIN SULFATE CONCENTRATIONS IN PATIENTS WITH NEWLY DIAGNOSED EXFOLIATIVE GLAUCOMA
Review round: 1
Reviewer: 3

Output:
Basic reporting: No coments
Experimental design: No comment
Validity of the findings: No comment
Additional comments: Dear authors,
Thanks for the opportunity to revise your manuscript. I found it very interesting. As I work and investigate in exfoliation since many years ago, I think we need to find biomarkers to identify the disease and to identify the patients in which the disease can led to blindness.
Before acceptance certain changes must be done.
The introduction is too long, please keep around 40%.
In lines 193-195, about statistics, please explain. It seems like you have used both parametric and no parametric models. Why? In which cases?
The discussion is not so focused in your findings. Lines 334-336, I cannot understand what you mean. The whole paragraph (lines 337-364) has no meaning.
Please try to hypothesize the relationship between your findings and exfoliation. Common chemical pathways? Do you think that heparan and condroitin sulfate come from the eye? If not, from which part of the body do you think they are coming?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SERUM HEPARAN SULFATE AND CHONDROITIN SULFATE CONCENTRATIONS IN PATIENTS WITH NEWLY DIAGNOSED EXFOLIATIVE GLAUCOMA
Review round: 2
Reviewer: 1

Output:
Basic reporting: line 72, GAGs, are (not as)

lines 393-412: much improved, recommend tightening up and organizing (ie, no need write, on the other hand, in other words..)

lines 466-468: IOP and BCVA are not better biomarkers for XFG;  IOP and BCVA are routine parameters checked in standard eye exams.  I recognize that the authors addresses IOP and BCVA due to a reviewers comments; however,  IOP and BCVA are nor better biomarkers for XFG;  I recommend remove lines 466-468
Experimental design: good
Validity of the findings: can improve clarity/organization of conclusions, tighten up
Additional comments: Great work and much improved manuscript.  This is novel work and can stimulate further investigation.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SERUM HEPARAN SULFATE AND CHONDROITIN SULFATE CONCENTRATIONS IN PATIENTS WITH NEWLY DIAGNOSED EXFOLIATIVE GLAUCOMA
Review round: 2
Reviewer: 2

Output:
Basic reporting: I have no further comments.
Experimental design: I have no further comments.
Validity of the findings: I have no further comments.
Additional comments: The authors answered all questions. So, I recommend the publication of the manuscript.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SERUM HEPARAN SULFATE AND CHONDROITIN SULFATE CONCENTRATIONS IN PATIENTS WITH NEWLY DIAGNOSED EXFOLIATIVE GLAUCOMA
Review round: 2
Reviewer: 3

Output:
Basic reporting: No comments
Experimental design: No comments
Validity of the findings: No comments
Additional comments: I'm pleased by the changes made in the manuscript.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: GENE EXPRESSION IN TRIBOLIUM CASTANEUM LIFE STAGES: IDENTIFYING A SPECIES-SPECIFIC TARGET FOR PEST CONTROL APPLICATIONS
Review round: 1
Reviewer: 1

Output:
Basic reporting: In general paper is good written and all necessary background is provided. But there are some ambiguities:
1) From manuscript is it unclear what RNA-Seq data were generated for this study and what have been already published. As far as I can understand life stage data were published previously, while RNA-Seqs for CPG knockdown were generated in this study. But there is no information about whether (and where) the latter data made public.
2) Authors should clearly explain what expression threshold they used. "RPKM 2>8 log2" is completely meaningless for me.
3) Authors should clearly explain how they define significant GO terms. "reduced to the most specific" is not enough.
4) From the method section it seems that authors didn't use multiple texting correction when they analysed RNA-Seqs for CPG knockdown. Authors should either explain how they corrected for multiple testing or implement such correction if it was not done yet.
5) Authors are encouraged to illustrate results of differentiall expression analysis using volcano plots or by similar visualizations methods that shows both: significance and effect size.
6) It is unclear why authors think that best targets should have low and stage-specific expression. Naive view suggest the opposite... Authors should clearly explain their motivation to select target in that way
7) Target selection procedure should be clearly explained including all sequentially applied threshold and filters altogether with number of genes passed given threshold. Authors should provide motivation for all thresholds used (for example histograms of parameters used for filtering)
8) On line 412 Authors state that they found not alignment of part of CPG gene in mRNA of other species. But it may be caused by differences in annotation, probably this part of gene is not annotated in other species. Search should be performed against whole genome (probably only syntenic region), not just mRNA.
Experimental design: While I'm not a specialist in experimental biology, I believe that Mock should contain not only dye but also dsRNA with shuffled sequence. In the current design it is not possible to distinguish effect of CPG knockdown from "any double RNA ejection"
Validity of the findings: Taking into account my comments on experimental design and reporting, for now it is unclear whether results are well supported.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: GENE EXPRESSION IN TRIBOLIUM CASTANEUM LIFE STAGES: IDENTIFYING A SPECIES-SPECIFIC TARGET FOR PEST CONTROL APPLICATIONS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The language of this paper is mostly clear and unambiguous, with only a few exceptions, which are noted below. Professional English is used throughout.

The references used are adequately extensive. That said, the authors reference the iBeetle Project, which has aimed to collect phenotypic data from RNAi knockdown of all Tribolium genes. However, the authors did not address if the iBeetle Project had also targeted CPG. I think the authors need to address this prior to publication. Even if the iBeetle Project had tested knockdown of CPG, the results are worth repeating, particularly with the addition of the authors’ RNA-seq data, but what, if any, findings the iBeetle Project got from CPG should also be discussed, lest readers suspect the authors of hiding something. If the iBeetle Project has not examined CPG, this should be state clearly.

The structure, figures, and tables of this article were predominantly professional and well-managed, with a few exceptions: Figures 1 and 3: Is the intensity of the grey color in these figures significant? If so, this should be explained in the legends. This work is self-contained and coherent.

As mentioned above, I found several very minor issues with the language and organization that I think would improve the readability of the paper. I will list them in the order they were found in the paper:

A: Abstract:
Lines 29-30: the list of genes do not appear to offer any contribution to the abstract, especially considering none of these genes is discussed in much greater detail in the paper. The abstract would benefit from leaving them out.
Line 33: the phrase “284-fold” makes this sentence awkward, and can easily be left out.
Line 35: “RNA” should be plural. There were several other instances throughout the paper where plural noun forms were not used consistently. I encourage the authors to double check for appropriate and consistent use of plural/singular.

B: Introduction:
Lines 75-76: Prior to this sentence, the authors mention the importance of using less conserved genes when planning RNAi for pest control. They then describe how results from the iBeetle study, which was carried out in Tribolium, were used as the basis of an RNAi study in D.v.virgifera. What is not clear is if the authors are suggesting the D.v.virgifera study proves their point about genetic conservation resulting in off-targets, or if they are lifting up the iBeetle project as an example of how research in Tribolium can be applicable to other species. This sentence should be rephrased, or the paragraph reorganized to clarify this issue.
Line 87: the word ‘resistance’ would probably be clearer than the word ‘resistant’ when describing genes that help insects escape the effects of pesticides.
Lines 97-99: This sentence is worded awkwardly. Replacing the phrase “reducing the expression of CPG” with something simpler, like “CPG RNAi” could improve the readability. Also, I believe the word “connection” would make more sense to most readers, rather than “connectivity”

C: Materials & Methods:
Line 129: I believe the authors intended “RPKM<8 log2” not “RPKM 2>8 log2”

D: Results:
Line 237 ff: The authors are inconsistent in their use of plurals when referring to life stages. I encourage them to be consistent. “Eggs” and “Adults” seemed particularly prone to loss of the plural form.
Lines 255-262: this paragraph seemed out of place – out of the flow of the narrative the authors had been establishing with their results. I just want to encourage the authors to reexamine the placement of this paragraph, to ensure that it is in the right place. If so, they may benefit from the addition of a transition to help readers follow their line of thought in this matter.
Line 299: the authors need to add a close parenthesis, “)”, after the word “genes”
Line 313: The authors should add “p=” to the beginning of the second parenthetical.

E: Discussion:
Line 391: “male” and “female” should probably be plural.
Line 422: the arrangement of the phrase “non-injected Control and Mock” could confuse readers into believing that both “Control” and “Mock” are “non-injected.” I would recommend that the order be swapped; alternatively, the phrase “non-injected” could be dropped, since “Control” have already been defined as “non-injected” elsewhere.
Experimental design: The authors do a good job of explaining the questions they are trying to address with this research. Their described methods seem thorough, with only a few exceptions describe below. Their experiments were well designed to address their questions in a rigorous way.

Line 162: the authors describe one PCR step as being “anneal at 68C.” I believe they mean “extension” here, not anneal. They also describe their conditions as “standard”, but I believe 72C for extension is more typical. Unfortunately, the authors do not reveal the polymerase they used. Is 68C the required extension temperature for their polymerase, or are they using an unconventional temperature for reasons they then fail to explain? Either way, more detail is needed for this part of the M&M

Line 223: The version of MEGA used is several versions old. While the authors did not get any unusual results in their trees, my experience is that more recent versions are more accurate. While I wouldn’t expect the tree to change significantly, a more recent version could help with the shockingly low bootstrap values seen near the bases of the tree. I encourage the authors to explore this possibility, although I do not think publication of this paper should depend on them doing so.
Validity of the findings: The authors describe their results thoroughly, and do a great job of not over-selling their conclusions. The data and analyses appear to be very sound.
Additional comments: SUMMARY of the PAPER: The authors of this paper report on the use of RNA-seq to identify gene expression changes among the major life stages of the beetle, Tribolium castaneum. They found several gene expression changes that may be of interest to the larger Tribolium research community, particularly for those interested in gene-based methods of pest control. Indeed, as a proof of principle of their methods, the authors identified a fairly unique gene, CPG, with a low, larval expression pattern as a target for RNAi. Reduction of target expression enhanced lethality in injectees. RNA-seq data was then collected from RNAi individuals to examine gene expression changes resulting from knockdown of CPG; these results hint at a mechanism for escape from the lethal effects of CPG knockdown.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: GENE EXPRESSION IN TRIBOLIUM CASTANEUM LIFE STAGES: IDENTIFYING A SPECIES-SPECIFIC TARGET FOR PEST CONTROL APPLICATIONS
Review round: 2
Reviewer: 1

Output:
Basic reporting: Paper is good written and all necessary background is provided. Unfortunately, I'm still cannot understand notation authors used to denote expression thresholds: "RPKM greater than 2 less than 8 log2". Does it mean that log2(RPKM) should be between 2 and 8?
Experimental design: In revised version authors didn't address two of by concerns:
1) Authors didn't use shuffled RNAi control. In rebuttal they said that use of such controls "caused a lot of problems", that they can (but did not) use "scramblase" control, and that later should be very similar to Mock. I do not understand the difference between shuffled RNAi and "scramblase" as well as I do not understand why first should "cause problems" - to my knowledge it is widely used. I believe that this issue should be reviewed by someone with strong experimental background.
2) Authors didn't prove species-specificity of sequence used as RNAi target, in rebuttal they told that they run some searches (presumably using  some web-services) but unsuccessfully because of time out. Just to note, it is just a technical problem - blastn (as well as other tools used) can be run locally. This issue is not so important, since species-specificity of designed RNAi is not a main topic of the paper.
Validity of the findings: With exception of one concern mentioned above, I'm satisfied by experimental design an think that conclusions are supported by shown results
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: GENE EXPRESSION IN TRIBOLIUM CASTANEUM LIFE STAGES: IDENTIFYING A SPECIES-SPECIFIC TARGET FOR PEST CONTROL APPLICATIONS
Review round: 2
Reviewer: 2

Output:
Basic reporting: The language of this paper is mostly clear and unambiguous, with only a few exceptions, which are noted below. Professional English is used throughout.

- Lines 72-73: The authors say "secondary primers were designed to avoid off-target effects using OfftargetFinder." Because the authors had already mentioned OfftargetFinder, this sentence was confusing. I believe the authors mean that they designed the secondary primers to amplify the region of minimal off-target effects that they had identified with OfftargetFinder, but this sentence could be edited to be clearer about this.

- In lines 301-302 and 308, the authors appear to list multiple GO terms, but list all the terms within a single pair of quote marks. If these terms are indeed separate, each term should be given its own pair of quote marks.

- Line 341: the authors say "fold-change", but I think they mean "expression"

The references used are adequately extensive. The structure, figures, and tables of this article were predominantly professional and well-managed, with a few exceptions:

- In figures 2 and 4, the authors say "The number in parentheses indicates the number of genes in each category." However, no parenthetical numbers were visible in the actual images.
Experimental design: The authors do a good job of explaining the questions they are trying to address with this research. Their described methods seem thorough, and their experiments were well designed to address their questions in a rigorous way.
Validity of the findings: The authors describe their results thoroughly, and do a great job of not over-selling their conclusions. The data and analyses appear to be very sound.

That said, in lines 342-344, the authors discuss the significant mortality caused by CPG RNAi. They discuss the significance only relative to Control, but also mention the Mock mortality, only to say it was "not significant." Presumably, they are stating that mortality of Mock injected individuals was not significant relative to Control. However, the do not give the mortality rate of Control, which was clearly non-zero, nor do they give the significance between Mock and RNAi injections. Both of these numbers would be helpful to clarify the degree to which CPG RNAi causes death. It might also be helpful, but is not necessary, to discuss the rate of death, since figure 5 seems to suggest ~half of all RNAi deaths came within the first 24 hrs.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TRANSCRIPTOMIC STUDY OF THE MECHANISM OF ANOIKIS RESISTANCE IN HEAD AND NECK SQUAMOUS CARCINOMA
Review round: 1
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: The author presented a combination of computational and experimental work for the mechanism of anoikis resistance in head and neck squamous carcinoma. This is a good design and the results were validated by experiments. The topic is novel, however I do have a few concerns:

1. Your methods needs more detail. I suggest that you improve the description at lines 97- 100 to explain "How the author obtain the DEGs?"
2. I would also suggest the author make you result subtitle more specific. Like line 187 "Gene Set Enrichment Analysis (GSEA) of DEGs", could you please say what is the main purpose or finding from here.
3. Seems the authors have done many analysis, but not link to each other. It will be better to make a workflow for describe the relationship between different analysis restuls.
4. For Figure 1B, why all the RT-PCR is for normal, how about the AR cases?
5. For Figure 1A, what is the meaning for the color? FPKM?
6. In supplementary table, seems come Chinese charaters to be fixed.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TRANSCRIPTOMIC STUDY OF THE MECHANISM OF ANOIKIS RESISTANCE IN HEAD AND NECK SQUAMOUS CARCINOMA
Review round: 1
Reviewer: 2

Output:
Basic reporting: In this manuscript, the authors did RNA-seq of three CAL27 samples and three anoikis-resistant CAL27 samples induced by suspension culture. They did GO and KEGG pathway enrichment analysis on differentially expressed genes (DEGs), and reported that the down-regulated genes in anoikis-resistant CAL27 samples were enriched in cell cycle, while up-regulated genes enriched in extracellular exosome, and up- or down-regulated genes were accordantly related to VEGF and EGFR pathway. They did RT-PCR experimental validation of some differentially expressed genes, and then focused on validating VEGFA-STAT3-KLF4-CDKN1A signal axis.
In general, I thought this study is good, but the English and some logic may need to be improved. The authors seem to fail to mention some related literature references in introduction or compare with them in discussion, for example PubMed ID 22154512, 16007176. The article structure is professional. Figures and tables are mostly good and self-contained. Raw data can be found in SRA (SRP158985).
Experimental design: This study is within the scope of PeerJ. The research question is relevant and meaningful, but not very sharply defined in the Introduction section. Analysis and Results look good, although some methods may need more details.
Validity of the findings: The RNA-seq experiment was done on three anoikis-resistant CAL27 samples vs three control CAL27 samples. Differential expressed genes (DEGs) are defined with the standard that is not very clear and is changed several times, and I am not sure about the statistical significance of DEGs. Multiple test correction (FDR or Bonferroni or q-value) was not done for the first part of GO enrichment analysis by DAVID. The conclusion part may be a little more than that the results can support.
Additional comments: Some specific comments are listed below.
Line 34, 'though' should be a typo of 'through', in Results part in Abstract
In first paragraph of Introduction (Line 43-54), there should be a sentence to link the relationship between lip, oral cavity, pharyngeal cancer and HNSCC.
The sentence introducing anoikis (Line 51-52) should be moved ahead, before previous sentence mentioning 'cancer cells undergo anoikis'.
Line 52, 'a few cancer cells are prone to development of the ability to resist anoikis' might be better as 'a few cancer cells are likely to develop the ability to resist anoikis'
Line 57, 'allows transformed cancer cells survival' might be better as 'allows transformed cancer cells to survive'.
Line 64, 'mechanism' is a strong word, which cannot be made by bioinformatics analysis of association.
Line 66-67, 'RNA-Seq technique was used to obtain deep transcriptional analysis between…', RNA-seq is a sequencing technique, which is different from RNA-seq data analysis. And I think 'deep transcriptional analysis' looks strange, which might be better as 'deep sequencing data for whole transcriptome analysis'.
Line 70, the word 'meticulously' has fuzzy meaning; I am not sure what 'the key signaling pathways' refers to; 'discovered' might be strong.
Line 95, I am not sure what 'as described previously' refers to
Line 97-98, what are 'Loget' and 'probability'? Are the DEGs statistically significant?
Line 102 & 165, why the standard of DEGs is changed again?
Line 104-105 & 169-186, 'p-value<0.05' & 'p-value=…', why not use FDR or Bonferroni corrected p-value? Do the authors apply multiple test correction to the results of DAVID enrichment analysis. BTW, I am not sure if the scientific notation of 1.01E-21 should be written as 1.01*10^-21 (-21 is superscript) in the text.
Line 106, 'three ontologies of GO were collected picked up separately', two verbs next to each other.
Line 117, 'Different gene sets in the MSigDB were used to detect activated or repressive pathways related to our DEGs.' Why use different gene sets?
Line 123, 'GEPIA' needs citation, which seems to be Line 133-134.
Line 142, Table S1's column names are in Chinese and should be translated into English.
Line 159 & Figure 1 legend, 'had good heterogeneity and repeatability' might be better as 'had low heterogeneity and high repeatability'.
Line 160, 'All the detected 21 genes matches the results of RNA-Seq,' How are these 21 experimentally validated genes were selected? randomly or manually selected? Some of the 21 validated genes cannot be found in Fig S1, although can be found in Supplemental_file_S1.
Line 173, 'p-value = 2,63E-12' might be a typo of '2.63E-12'
Line 182, Supplement Table 2, why some gene length has fractions? why not also list genes in 'nucleoplasm', 'cell division', etc.?
Line 191, 'These signaling pathways included' refers to 'activated signaling pathways'?
Line 196, since cell cycle processes were suggested to be suppressed in CAL27AR cells, do CAL27AR cells grows more slowly?
Line 201-202, one unnecessary line break
Line 220, 'DEGs enriched in … terms', do you mean 'DEGs related to … terms'? because enrichment should be on the level of function/pathway terms, but not genes.
Line 222, 'negatively influenced the cell cycle', do you mean the genes down-regulated by VEGFA overexpression, or VEGFA overexpression may suppress cell cycle?
Lines 254, 'First,' I did not see any 'second' in latter part. And this paragraph seems to be an extension of the signal axis to other cancer types, but not 'To further verify our view,' (Line 253).
Line 357-358, in Conclusions section, 'Abnormal cellular metabolism changes … may both result in anoikis resistance'. There seems to be no experiments to show there are actually abnormal cellular metabolism changes. And the bioinformatics analysis and validation on DEGs may only show association but not causal relationship with 'result in'.
In Figure 1A, the number -2~2 with red~black~green bar in the legend, what is the meaning of the number? z-score? BTW, red-green heatmap might be unfriendly to red-green color blind readers.
In Figure 1B, how are these validated genes were selected? I did not fin KRT80, KRT8, KRT18, KT5, LRRN4 in Fig S1.
In Figure 2B, 'serine-type endopeptidase inhibitor activity' seems to have FDR > 0.05 when I rerun DAVID.
In Figure 2C, the legend for 'Count' is better to have some more levels.
I did not find figure legend for supplemental figures.
As previously mentioned, some related literature references, for example PubMed ID 22154512, 16007176, might be mentioned in introduction or compare with their results in discussion. The purpose and research question might be better sharply defined in introduction section.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TRANSCRIPTOMIC STUDY OF THE MECHANISM OF ANOIKIS RESISTANCE IN HEAD AND NECK SQUAMOUS CARCINOMA
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comments
Experimental design: No comments
Validity of the findings: No comments
Additional comments: The author made some revision, however, some important comments not addressing well. Like my previous comment#3, how could author link different results. The author give a workflow, which is good. My point is how different results can support each other for a whole story, like any interesting genes shared in different analysis results. I would hope the author could add some discussion on this.

For figure 1A, it is still hard to recognise the expression difference, I would suggest change you colour index and make it more clear.

One more question is about Figure 1B, like the ALOX5AP, how could the result is a lot of spot instead of bands. It need more explanation, it is contamination or not.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TRANSCRIPTOMIC STUDY OF THE MECHANISM OF ANOIKIS RESISTANCE IN HEAD AND NECK SQUAMOUS CARCINOMA
Review round: 2
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors have made great efforts to improve the manuscript, and have resolved most of my concerns.

A minor comment is that some answers in the rebuttal letter seem not to have corresponding revision in the manuscript. To make the manuscript clearer, it might be better to make the revision.
1) All supplemental figures might be better to be briefly introduced and cited in the main text, including the new supplemental figure "workflow". And figure legends for supplemental figures still cannot be found (I mean the introductory text figure legends, like the page before each main-text figure).
2) "Loget" and "probability" (of difference) is not well defined or explained or cited to other reference in Methods section.
3) The selection of 21 RT-PCR validated genes (15 keratin-related genes + 6 randomly selected other genes) is not introduced or mentioned at all in either Methods or Results section.
4) The explanation of the color legend in Figure 1A - "z-score" (transformed FPKM) - is still not shown anywhere, which could be either as the title of the color legend or in the text figure legend.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SIMPHONY: SIMULATING LARGE-SCALE, RHYTHMIC DATA
Review round: 1
Reviewer: 1

Output:
Basic reporting: A freely available R package for generating realistic circadian time series is certainly a welcome addition, to assist in testing new rhythmicity and circadian parameter estimation methods. The manuscript is clearly written.
Experimental design: There is a serious flaw in the design in that the length of the simulations is only 24h, one cycle, in length. Rhythmicity testing requires 2 cycles at the very least.

All the time series depicted in the main manuscript and in the supplement are all only 24 hours in length (the vignettes use only 24h as well). How can rhythmicity be tested with only one cycle? Just because a time series goes up and down over a day doesn’t mean it will do it again the next day. I’ve had experimental results where the first day looked like the first cycle of a rhythm, but then the next day was quite different, showing it was a transitory event and not a true rhythm (with a similar pattern across all samples, so multiple samples per time point doesn’t alleviate the problem). All examples should show at least 2 days.

See https://www.ncbi.nlm.nih.gov/pubmed/29098954 for guidelines and best practices, which at least one of the manuscript authors signed onto and is referenced in the text. This article states, "By definition, biological rhythms repeat. We therefore recommend collecting at least 2 complete cycles of data when detecting rhythmicity (i.e., 48 h for collections under constant conditions). The guiding principle behind this recommendation is that when identifying a rhythmic process, one would like to observe both the peak and trough repeat at least once. Simulations show that collecting fewer than 2 cycles in a time series makes the resulting data sensitive to outliers and can dramatically increase the number of false-negatives (see the “Synthetic Data for Benchmarking” section)."

For purposes of phase, waveform, and amplitude estimation, it's true that under LD entrained conditions 24h of data can be used. But rhythmicity cannot be correctly tested with only 1 day of data. How is limma detecting rhythmicity with only 1 day of data? Is it really testing for a sinusoidal waveform?

In addition, in trying out the package, I couldn’t find a way to simulate time series longer than 24h. *It’s really important that there be an easy option for specifying length of the time series simulations.* Note that CircaInSilico does provide this feature. For use in studies after constant conditions or for LD conditions in which entrainment may not be stable (short or long T-cycles, skeleton PP, jet lag experiments, etc), multiple cycles will be very important.
Validity of the findings: Due to the serious flaw of only one cycle being simulated, the rhythmicity results shown in Figure 2 are not valid.
Additional comments: For clarity, please state explicitly in lines 94-95 that the count-dispersion relation is being used to generate the function g_ik; this direct link will help the reader make the connection.

What choices does the user have: # days, time step, noise type and parameters, signal amplitude, phase, baseline, waveform, others? Perhaps include a table listing the options available, as a convenient summary showing the flexibility of the package.

FigS2: just an example to illustrate options? Two simulated series for each condition and gene? Can it accurately detect the amplitudes and phases? The purported goal of the package is to provide large simulated datasets to guide experimental design, so do some examples that mimic large-scale experiments. For instance, demonstrate how to use the simulated data sets to determine how many samples are needed per time point given a certain sampling rate like every 4 hours over 2 days to achieve a desired false discovery rate.

The current text seems a bit thin. Demonstrating its use by applying metacycle to a large simulated data set generated by symphony would nice to see, rather than only limma for the single validation example. Using a package like metacycle will provide an alternative rhythmicity test(s) as well as phase and amplitude estimates and could further expand the scenario in Figure S2 into an interesting test case (like determining what sampling rate with 2 days of data would be required for reliable estimation of phase).

For installing simphony, note that you can also simply type  drat::addRepo('hugheylab')
Into the Console window. I couldn’t get the .Rprofile approach to work, but simply pasting that line into the Console worked fine.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SIMPHONY: SIMULATING LARGE-SCALE, RHYTHMIC DATA
Review round: 1
Reviewer: 2

Output:
Basic reporting: Authors present an R package, called Simphony, to simulate large-scale circadian transcriptome data regarding different rhythmic features measured at multiple time points in multiple conditions. To do so, a measure of abundance is generated regarding a signal-noise model using two distribution families: the Gaussian (idealized experimental scenario) and Negative Binomial (which approximates read counts from RNA-Seq). A github repository contains all package dependencies and data.

This paper uses a suitable language and the structure and goals are clearly exposed. However, some statements require additional references and other times they are too old, see the comments made on lines 24,28 of the .pdf file. Literature review presents several gaps, see comments on the lines  34, 44 of the .pdf file.

The scientific question proposed in this paper, i.e. the simulation of circadian rhythmic genes imitating real data bases is relevant for biologists. However, this question has been already dealt in literature (see the comments on line 34 in the .pdf file); the novelty of the simulation model-based is not clear (see comments on line 60 in the .pdf file); and the procedure is only validated for symmetric shapes, despite that circadian data bases also include non-sinusoidal patterns (see line 143 in the .pdf file). Please, see Experimental Design for details.
Experimental design: As I mentioned above, this paper presents a meaningful research question and methods are described with sufficient details.
However, it presents several controversial points:

A.- This question has been previously studied in literature, both for microarray (Dembelé (2013) among others) and RNA-seq  technologies (Zappia, Phipson and Oshlack (2017) among others). Authors do not refer/compare to this fact in the paper.

B.- The model employed to generate synthetic data is very close to Cosinor model, at least for the default case considered in this paper, i.e. when f is assumed to be a sinusoidal function. Authors should discuss the similarities/differences with this model. In addition, only this default (sinusoidal) function is considered to validate the procedure. It is not enough to replicate the wide variety of patterns that appear in practice.

C.- As expected, results  show a suitable performance of the simulation tool. Note that data were generated from a sinusoidal model and the fittings are made with Cosinor model (based on sinusoidal shape too). However, non-sinusoidal (asymmetric) rhythms are found in circadian data bases (Thaben 2014). Although, authors consider a sawthooth wave, it is again just a symmetric pattern. More arbitrary and extreme patterns are required to validate the procedure.

Please, revise the attached .pdf file for specific details, comments and suggestions
Validity of the findings: The originality of the research question is questionable.
Conclusions depends on the periodic simulation function (which is always symmetric) and on the Cosinor regression model which intrinsically assumes sinusoidal shapes. Thus, those conclusion cannot be extended to circadian data bases where there are many arbitrary rhythmic patterns, not only sinusoidal.
Additional comments: Please, find comments, details and suggestions in the attached .pdf file


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SIMPHONY: SIMULATING LARGE-SCALE, RHYTHMIC DATA
Review round: 1
Reviewer: 3

Output:
Basic reporting: .
Experimental design: .
Validity of the findings: .
Additional comments: Manuscript Title: Simphony: simulating large-scale, rhythmic data

The increasing number of large-scale experimental datasets available for time-course analysis requires computational methods that can distinguish true periodic signal from noise. Simulated data, where truth is known, help to evaluate how well a particular method performs. In response to the lack of freely available (and flexible) tools for simulating periodic data, the authors developed and benchmarked a new framework called Simphony.

Simphony represents a nice addition to the toolkit of any chronobiologist who designs or analyzes large-scale time-course experiments. Advantages over other publicly available tools are simulations that can more closely approximate experimental: 1) feature waveforms (not always sinusoid) and 2) distribution of RNAseq data (not always gaussian). As usual, the technical details of the work are superb. However, there are a some improvements that could improve the impact of the work [see below]. Although more flexible than current publicly available tools (i.e.  CircaInSilico, JBR 2017), it is considerably less user-friendly. And, in terms of performance, it appears to be less flexible than prior published approaches. Improvements to either of these aspects would considerably increase the impact of Simphony as a resource and the paper as a consequence of that.

Major Comments

Different methods for rhythm detection usually perform best when validated against their own simulated datasets. There are few published papers that evaluate multiple methods using the same simulated dataset. In that sense, there is a need for a simulation framework that is both easy to use and generalizable, as the authors point out. Our concern, however, is that Simphony in its present form falls a bit short of meeting these needs.

●	Simphony generates two rhythmic waveforms. Are more necessary? In 2013, Deckard A. et al. reported a simulation generating six different periodic patterns and two non-periodic patterns.
●	While free to download and use in R stats, Simphony is still not as user-friendly as CircaInSilico (Hughes et al, JBR 2017), which can be run as a web-based App (no coding required). This could ease the learning curve and encourage wider adoption.
●	The manuscript lacks explicit description of how Simphony could be used to help guide the design of a typical time-course experiment. The ROC curves (Fig 2) demonstrate nicely how cosinor regression better discriminates rhythmic from non-rhythmic features when the sampling interval is smaller (and amplitudes are higher). This, however, is pretty intuitive, and a conclusion reached in prior work (e.g. Deckard et al., Bioinformatics, 2013, Hughes ME et al., PLoS Genet., 2009). A basic and recurring question for the researcher trying to balance dollars vs. detection power, is, what specifically do I gain by increasing temporal resolution and/or replicates? This should be addressed directly in the manuscript text and/or by package vignette.
●	The authors “plan to extend Simphony to simulate nonstationary trends (e.g., damped rhythms) and to accommodate different periods for different genes within a simulation.” This would make a strong addition to THIS initial publication!


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SIMPHONY: SIMULATING LARGE-SCALE, RHYTHMIC DATA
Review round: 2
Reviewer: 1

Output:
Basic reporting: The text refers to CircaInSilico as “lightweight”, which seems quite dismissive and harsh. Is that intended?
Experimental design: I appreciate that the figures were altered to show 2 cycles. However, I strongly disagree with the authors’ assertion that rhythmicity and period can be detected from a single cycle. There is no way to tell whether the pattern will repeat the next day without actually collecting the data. This computational tool should reflect that fact by only generating simulations that are at least 2 cycles long, and the text should make clear this important condition of requiring at least 2 cycles.

I don’t understand the authors’ statement that simulating one cycle or multiple cycles is the same. The noise can vary substantially from cycle to cycle, and it’s important to see that variation when generating simulations for testing. The presence of noise is what often complicates the detection of rhythms, making each cycle different.

Cosinor can’t fit to 1 cycle of data. It may generate some numbers, but they won’t be reliable.
Validity of the findings: The manuscript still seems really thin. I expect most readers won’t understand the fundamental purpose of this package without a more substantial example demonstrating how to use it in experimental design.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SIMPHONY: SIMULATING LARGE-SCALE, RHYTHMIC DATA
Review round: 2
Reviewer: 2

Output:
Basic reporting: Manuscript has notably improved after review. However, there are still some flaws on it.

First, authors state that Simphony is a tool to simulate large-scale rhythmic data. But the paper focuses on very simple examples for shor time series. I strongly recommend to the authors to extend it (Figures) to large-scale data and compare results with those illustrated in the paper for short time series. Simphony accuracy should be assesed for those examples too. As far as I know, JTK does not detect correctly asymmetric patterns. How does it performs for sawtooth shapes? Does it work for slightly sawtooth shapes? Are you methodology ready for non-equal spaced or misssing data? On the other hand, mathematical functions used in R to simulate the examples illustrated in the paper should be included, at least in the Supplemental material, it may help to non familiar readers. Finally, I am not conviced of the novelty of this work, what are the advantages of this paper with respect to Deckard A, et al. (2013) (https://academic.oup.com/bioinformatics/article/29/24/3174/193410). It should be discussed in the paper.
Experimental design: See 1
Validity of the findings: See 1
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SIMPHONY: SIMULATING LARGE-SCALE, RHYTHMIC DATA
Review round: 2
Reviewer: 3

Output:
Basic reporting: The authors did a great job responding/clarifying to all my previous critiques, and I am now comfortable with publication of this manuscript.
Experimental design: Fine.
Validity of the findings: Fine.
Additional comments: The authors did a great job responding/clarifying to all my previous critiques, and I am now comfortable with publication of this manuscript.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A MATHEMATICAL LANDMARK-BASED METHOD FOR MEASURING WORN MOLARS IN HOMINOID SYSTEMATICS
Review round: 1
Reviewer: 1

Output:
Basic reporting: This manuscript is well-written and very clear.  There is sufficient background and relevant literature is cited.  The authors have succeeded in creating a clear and concise argument.

The raw data are provided; however, it was difficult for me to tell which data was from the mathematically derived method and which data were from the anatomically derived method.
Experimental design: The design was clear and well described.  Given the raw data and description of statistical methods, replication could easily be achieved.  The figures of the landmarks were also well constructed and clear.  Dental wear is an issue with all dental anthropological studies, this research addresses this problem and provides possible concrete solutions to assessing morphology even with advanced occlusal wear.  Their focus was on molars, but an extension to other teeth could be imagined.
Validity of the findings: While the mathematical model did not perform as well in the DFA as the anatomical method, its validity is still presented.  The sample size is small, but general trends can still be identified.  In discussions of the DFA, the authors may want to address sample size and the number of variables input into the model.  There could be an issue of overfitting with the sample.  A step-wise analysis was performed, but the number of variables in the final model was not indicated.  Also, results could be discussed in terms of positive predictive value as opposed to 'correct classification.'
Finally, some discussion of sexual dimorphism could be included.  While teeth were scaled and size was not generally evaluated, there could be sexual dimorphism present within the sample.  Some discussion of dental sexual dimorphism or a simple statistical test of dimorphism could be included.
Additional comments: Thank you for the opportunity to review this manuscript.  It is very well-written and clear.  The methods are well described and could readily be used in future research projects on worn teeth.  There are only minor comments for improvement on what is already an excellent manuscript.