Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SNARE-CNN: A 2D CONVOLUTIONAL NEURAL NETWORK ARCHITECTURE TO IDENTIFY SNARE PROTEINS FROM HIGH-THROUGHPUT SEQUENCING DATA
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comments
Experimental design: No comments
Validity of the findings: No comments
Additional comments: I'm satisfied with the changes made by the authors and they address all my major points.

I would revise the end of the introduction to either exclude or modify the user-friendly web-server suggestion. As far as I can tell, there is no such user-friendly webserver where a potential user can submit a protein sequence for SNARE vs non-SNARE prediction. I don't think this is a particular problem though as I don't think all methods need to have an associated user-friendly server and that the code shared by the authors is a decent start. I know this is in direct conflict with the other reviewer's suggestion, but including this section in the introduction makes it appear that such a server has been provided.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EPYPES: A FRAMEWORK FOR BUILDING EVENT-DRIVEN DATA PROCESSING PIPELINES
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper is in general nicely written, with good use of the English language in a professional manner. However, there are some typos, unclear statements, and formatting that need to be addressed.

Line number: Comment
18: Development -> Developing?
20: To -> to, this paper -> This paper
35: Communication fabric? What does this mean?
38: tools of trade -> tools of the trade
70: Python -> a/the Python
77: only this -> only is this
94: directed acyclic graphs -> directed acyclic graphs (DAG).
101: movement gained -> movement has gained
115: In distributed systems -> In distributed systems,
135: computational graphs' execution -> execution of computational graphs. See also line 79.
140: i.e. -> i.e.,
141: to much space above and below the equation.
142: the word where should not start a new paragraph.
146: indegree and outdegree. rephrase.
148-149: Ugly formatting of In_f and Out_f. Please fix.
153: conform -> conforms
166: It is not really clear what the token sets represent. Maybe reformat and put in some more context.
197: Don't start a sentence with a symbol (func_io).
228: operation the same -> operation is the same.
240: of Pipeline -> of the Pipeline
253: as synchronization -> as the synchronization
264: SE(3) is not introduced or defined. Rephrase.
284: continuously-run -> continuously run
301: available tool set? which tools? rephrase.
400: Hard to read. rephrase.

The paper follows a semi-traditional structure. The related work section is presented after the main body of the text. This is in itself not a problem. However, the related work section is too short and needs to be expanded and the presented work must in a much greater detail be compared to other systems/frameworks. E.g., in robotics, ROS is much used for vision prototyping and development. How does this work relate to ROS nodes, nodelets, etc.?

The figures in the paper are very good.

The paper is self-contained, but needs to be expanded. See comment above.
Experimental design: The problem description presented in lines 57-63 is good.

The experimental results are severely lacking. It is not clear what the experiment presented in section EPYPES USE CASE is meant to convey. The timing results are not placed in context and discussed.
Validity of the findings: The conclusion is written like a summary. Please connect the conclusion to the problem description presented in the introduction. Important findings from the experimental results should be included.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EPYPES: A FRAMEWORK FOR BUILDING EVENT-DRIVEN DATA PROCESSING PIPELINES
Review round: 1
Reviewer: 2

Output:
Basic reporting: - I checked the code and it works well
Experimental design: - The python code should not include inside the text. The authors maybe can replace with algorithms or pseudo code.
- The study contains some similar descriptions and figures (Fig. 6, 7, 9, 10) I am wondering whether a way to merge some of them to make the comparison clearly.
- References are weak, the authors might need to make their work stronger with more related references.
Validity of the findings: - How their work is different from previous work (Semeniuta, 2015). How much better performance is there? What is the statistical significance of improvement? It needs more discussions.
- Authors should compare with other related works.
Additional comments: Although the authors had done a good work with, I recommend the authors should organize their methods and results in a better way. Now it contains a lot of explanations, codes, and figures that make readers be hard to follow. Also, please discuss more your results to make your contributions significantly.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EPYPES: A FRAMEWORK FOR BUILDING EVENT-DRIVEN DATA PROCESSING PIPELINES
Review round: 2
Reviewer: 1

Output:
Basic reporting: There are some typos here and there, e.g., UPD instead of UDP in the conclusion, that should be fixed.

The paper is substantially improved.
Experimental design: no comment
Validity of the findings: no comments
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EPYPES: A FRAMEWORK FOR BUILDING EVENT-DRIVEN DATA PROCESSING PIPELINES
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A THEORY ON INDIVIDUAL CHARACTERISTICS OF SUCCESSFUL CODING CHALLENGE SOLVERS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper offers a clear outline of the proposed research and its underpinnings; although I missed an initial series of hypotheses to be tested, which is typical when correlation analysis is used as a theoretical building framework, the work is well elaborated and offers interesting insights. On the one hand the paper is well connected to state of the art literature but on the other hand some related work in software engineering skills, education, and understanding is still missing, e.g., [1] and [2] off the top of my head. Furthermore, from yet another perspective, several early attempts were made at offering a "goodness index" for software developers (e.g., [3]) - these should at least be mentioned in the scope of the paper as related work.

[1]  Teles, V. & de Oliveira, C. (2003). Reviewing the Curriculum of Software Engineering Undergraduate Courses to Incorporate Communication and Interpersonal Skills Teaching. Software Engineering Education and Training. Proceedings. 16th Conference on (p./pp. 158--165), Los Alamitos, CA: IEEE Computer Society.

[2] Matturro, G. (2013). Soft skills in software engineering: A study of its demand by software companies in Uruguay.. CHASE@ICSE (p./pp. 133-136), : IEEE Computer Society. ISBN: 978-1-4673-6290-0

[3]  Borreguero, F., Nitto, E. D., Stebliuk, D., Tamburri, D. A. & Zheng, C. (2015). Fathoming Software Evangelists with the D-Index.. In A. Begel, R. Prikladnicki, Y. Dittrich, C. R. B. de Souza, A. Sarma & S. Athavale (eds.), CHASE@ICSE (p./pp. 85-88), : IEEE Computer Society. ISBN: 978-1-4673-7031-8
Experimental design: The design of the experimentation appears sound although, as the authors also notice, the experimentation can and should only be considered preliminary, since the sample population is missing some obvious control factors (e.g., age, background, etc.) - that being said, it would be sufficient to elaborate on these aspects in the threats to validity section (perhaps discussing which areas of validity are affected by which sampling assumptions) and in the future work section.
Validity of the findings: the findings were very interesting although visualized in a preliminary and basic way; more advanced visualization plots may have helped further understanding of the work and its implications, perhaps even helping with the discussion.
Additional comments: The title is a bit convoluted - I would choose a new one based on the actual goals that the authors are exploring, e.g., "Developer Characteristics vs. Coding Challenges: an Empirical Study"; in essence, anything more indicative of the actual content of the work would help.

the paper has several imprecisions in English grammar, spelling and some haphazard ways of presenting table entries (e.g., table 3 and following, please use "p-value" instead of "p", explain "n") - these issues should be addressed to maximise paper quality.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A THEORY ON INDIVIDUAL CHARACTERISTICS OF SUCCESSFUL CODING CHALLENGE SOLVERS
Review round: 1
Reviewer: 2

Output:
Basic reporting: This article presents a study of 32 students who solve 3 coding challenges. Several factors, such as personality traits, and exam grades are used to construct a theory for predicting coding challenge success.

Overall, I admire the careful construction of study, detailed discussion of literature, and clarity in writing and presentation of results. However, there are a few issues that stand out in experimental design and interpretation of the results. Some of these issues could be addressed with careful discussion and improved framing of the results. However, some of these issues also lead me to question the validity of the results and purpose of the study itself.

In conclusion: I worry that a really nice and careful study was performed without really moving forward our understanding on this problem and I'm not sure how to fix this without considerable reframing or additional investigations.
Experimental design: ### Single High Performer

Out of 32 participants, only 1 participant succeeded in answering all three questions with the ideal time complexity. The remaining participants are split between answer all or most questions with suboptimal time complexity, or not succeeding at all. There are several interesting consequences/observations:

* Perhaps the questions were simply too difficult for the given population. Students that typically enroll in code challenges may be atypical, or practice/train for competitions. What would have happened if there were more questions like "countPairs".

* Several of the problems involve considerable problem translation (_build a mansion along a road, other side of road are houses, in which a certain number of people live_). These can be quite strange and abstract, leading to misinterpretation and significant time trying to understand what the problem is asking. Interesting enough, the proposed ideal solution is not very complex (a simple for loop), suggesting these are problem statements may be intended to serve as a distractor.

* What was the main reason to count the solution to be completely incorrect by failing on a test case? How would counting partial solutions change the results or conclusions?

* Given the skewed distribution of results (with only one fully correct and ideal set of solutions), it is difficult to conclude anything about what makes a "successful" coding challenge solver. Indeed, if that was truly the goal, then it would have been better to study students who are engaged in the competitions themselves and not a random population.

* From a "game theory" perspective, one might conclude that on average, a candidate should strive to answer all questions with suboptimal times; otherwise, attempting to find the ideal time complexity (and failing) could result in performing worse.

* We could also conclude the "coding challenges", especially the nature used in the study, are unsuitable for interviews since they are unlikely to yield high performers. I would have to interview 32 candidates in order to potentially find one high performer.
Validity of the findings: ### Theory

Four factors were investigated in constructing a theory: Happiness, Personality, Academic Performance, and Experience.

The coding challenges did not exhibit any need to used data structures (trees, graphs, hashtables, hashmaps, linked lists etc.). It is not clear what content has prepared the students for these coding challenges and can simply related to some other latent variable.  One possible alternative conclusion can be that success in Algorithms and complexity course is more representative than success in Data structures and algorithms course --- because students have more exposure to the concept of time complexity and also receive training on code challenges.

Emotional state was measured.
> We believe that there could be a cause-effect relation between sadness and the coding challenge performance, because from the participants who felt sometimes or often sad in the past four weeks, nobody came up with a correct solution to coding challenge 3.

Are there sad because they are not doing well in school? Or because they were incidentally not having a good day, recently experienced a stressful event, or maybe they were just tired [Fucci 2018]. Is this a residual effect or ephemeral effect?

In short, it is not clear how well connected or justified any of these characteristics are to the 3 coding challenges and larger research question being studied. Why are other factors such as sleep, word problem comprehension, typing speed, Java experience, interview experience, also not investigated?

It is not clear we have the appropriate elements for building a theory.

[Fucci 2018] Need for Sleep: the Impact of a Night of Sleep Deprivation on Novice Developers’ Performance
Additional comments: ### Clarifications/Minor

* Explain conscientiousness. What is this. What is the significance and implications of this personality trait. What is the natural frequency of this trait in human populations.

* Do they have the same examiner in Data structures course?

Line: 202 "ideal employable age" sounds like a lawsuit in waiting. Rephrase.
Line 250: How did you made sure that the pilot candidate is a good representative of their whole candidate population?
Line 252: How did you verify candidates were expressing time pressure?
Line 366: Redundant word “found”
Line 379: What are those strong reasons?
Line 611: Did they want to say “junior” instead of “senior”?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A THEORY ON INDIVIDUAL CHARACTERISTICS OF SUCCESSFUL CODING CHALLENGE SOLVERS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The writing is quite clear and precise.  Sufficient literature references are provided. Raw data is shared for replicating the calculations. The paper is self-contained.
Experimental design: The experiments are well designed and described.
Validity of the findings: The authors are generally careful in delineating conclusions based on evidence and speculation.
Additional comments: While there were limited actions the authors could take to address some of the questions raised in the initial review, the revised version of this article has gone through great lengths to improve the limitations, discussion, and add clarity to the paper, nearly 2 more pages of new content in total. In short, I believe this currently paper stands well on its own, even with its limitations. My hope is that these questions will help guide future work.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MAPPING ISO 19115-1 GEOGRAPHIC METADATA STANDARDS TO CODEMETA
Review round: 1
Reviewer: 1

Output:
Basic reporting: The manuscript is well written, appropriately referenced, and clearly structured.   The background supplied is sufficient if somewhat minimal.  This is largely a stylistic choice which leaves the manuscript focused on the details of the research performed, but it could nonetheless be valuable to set these issues more deliberately into their broader context.  The issues discussed here are not unique to the relatively niche "codemeta" description and the ISO standard, but appear to fall into broader distinctions between the underlying, widely used Schema.org standard and many ISO standards.
Experimental design: no comment.
Validity of the findings: The mapping presented here is impressively comprehensive and of obvious immediate value to the relevant communities in this space.  These results are clearly presented, though a machine-readable (e.g. csv file) version of the tables presented in the pdf would no doubt be welcome.

I think the one area I will comment on is the discussion of the distinction between ISO and "codemeta" formats.  I don't think this impacts the validity of the author's conclusion, but perhaps it merits some discussion.  In any event, I include it here in case it is helpful and not merely confusing.

While I agree with the author's insightful observation that a difference between what the author calls "hard types" vs "soft types" is precisely what distinguishes between these, it seems to me that this distinction reflects more fundamental differences in the data model involved.  Schema.org explicitly follows the Resource Description Framework (RDF) of subject-predicate-object triples, often expressed in JSON-LD as in codemeta.  The "hard typing" arises because "author" is the natural predicate in the triple such as: This "software" has "author" "Tim Berners-Lee", e.g.

{
  "@type": "SoftwareSourceCode",
  "author": "Tim Berners-Lee"
}

The equivalent statement in the ISO standard is typically expressed in using an XML structure that does not permit the same simple semantics, in particular, because XML has a notion of attributes that can be used to modify the meaning of a node (e.g. citedResponsibleParty[role='author'] which is not available in JSON and is correspondingly more ambiguous to express in RDF triples.  While we like to think of both standards as abstractions independent of serialization, it is hard not to notice the role of those serializations here.   I am not an expert in these matters, and I suspect it that the ISO standard could be expressed as RDF, but it does appear to me that the difference between 'hard' and 'soft' are really differences between a vocabulary intended as Linked Data and a standard intended to be specified as an XML schema.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MAPPING ISO 19115-1 GEOGRAPHIC METADATA STANDARDS TO CODEMETA
Review round: 1
Reviewer: 2

Output:
Basic reporting: The article proposes a mapping between ISO metadata (from technical committee
211) and CodeMeta metadata. The writing is clear and professional. References
are very scarce, and should definitely be improved. The article is
self-contained and easy to follow.
Experimental design: While, strictly speaking, no "experimental design" is applicable to this
contribution, the main problem of the paper is precisely in its validation.
The proposed mapping seems to be, in essence, the author's opinion of how the
two ontologies should be related (see below to some potential issues with
author's choices, but the point here is more general). Given how CodeMeta
works, and in particular how their crosswalk table is maintained, it seems to
this reviewer that the appropriate course of action to validate author's
choices is to propose these mappings to the CodeMeta community, to gather
feedback (and potentially criticism) on the proposed mapping. A scientific
publication does not seem warranted without that essential validation step.
Validity of the findings: - line 58: I don't understand the dichotomy in requirements between
  discovering/citing software and use/trust software, The classification of
  metadata requirements can be much richer than that and this partition should
  be explained

- line 131: Hard and soft typing isn't a reference to existing terminology,
  would be interesting to know where it comes from. The governance issue for
  soft types seems questionable, because even attribute types (in soft typing)
  can be subject to validation using various XML schema languages

- line 161: print issue where the items aren't found

- line 254: the author should reference the hash of the file and not the
  unstable url provided

- table 1 should be versioned/dated because schema.org can make changes and
  doesn't use versions

Regarding the proposed mappings:

- relatedLink and sameAS properties are mapped to the same ISO property

- contributor is mapped to an ISO property where it excludes the role=author
  from the list of citedResponsibleParty instead of using role=contributor (In
  my understanding from the paper of soft typing, this addition of
  role=contributor shouldn't be a problem)

- producer property is mapped to citedResponsibleParty role=creator - why not
   role=producer?

- creator property is mapped to citedResponsibleParty role=originator- why not
  role=creator?

 - codeRepository property, which is one of the most source code specific
  properties, is mapped to 5 different options, but none is specifically the
  characteristics of a code repository

- programmingLanguage is mapped to descriptiveKeyword[type='theme'], but there
  is a keywords property in CodeMeta- this double mapping and other properties
  that are mapped with the same value should be explained or commented

- softwareSuggestions is mapped to additionalDocumentation but
  softwareSuggestions is refering to optional dependencies and software
  components and not documentation.

- 8 CodeMeta properties are mapped to additionalDocumentation - this should be
  explained or commented.

The mapping is from CodeMeta to ISO and should be specified as a 1 to n
properties, if an automated process took a codemeta.json file and converted it
to ISO, this mapping would be the best way not to loose information, on the
other hand if the process was reversed the data would be duplicated and quite
frankly hard to comprehend.  This mapping results in a very high "dialect
coverage" because of the 1 to n links from CodeMeta to ISO, it would be
interesting to know what is the "dialect coverage" if the other way was mapped
from ISO to CodeMeta.

Also in the mapping the description of the property there is only the CodeMeta
description and it might be helpful to have the description of the ISO
properties.  If the paper's objective is to open the discussion with the
CodeMeta community before adding the ISO vocabulary to the crosswalk, I would
suggest to state it clearly in the abstract.  Otherwise this mapping should be
added with a PR to the crosswalk table before publication.

Final remark, examples of mapping difficulties and resolution would be a great
way to explain some mapping choices.
Additional comments: Nothing else to report.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MAPPING ISO 19115-1 GEOGRAPHIC METADATA STANDARDS TO CODEMETA
Review round: 2
Reviewer: 1

Output:
Basic reporting: The paper "Mapping ISO metadata standards to codemeta" describes the challenges of mapping the ISO 19115-1
 metadata standard to the codemeta project, which defines a vocabulary for software metadata.

As indicated by previous reviews, the paper is well-written and clear. In terms of the background provided, it seems
sufficient for a reader with some knowledge in this area. A bit more introduction about terminology and formats
might be needed for readers not familiar with metadata concepts in some places (e.g. explain the acronym RDF and RDF model)
to make the manuscript self-contained.

This new version of the paper has addressed most comments by reviewers and the paper is improved.

Here a few more comments and suggestions for your consideration.

The title of the paper refers to the "ISO metadata", but I recommend to change it to refer explicitly to the ISO 19115-1 standard
for geographic information (https://www.iso.org/standard/53798.html), making clear that it was designed for geospatial data metadata.
While many of the elements of the standards are generic (as also emphasized in the conclusions), I think it is important to make a
better distinction between the purpose of the creation of ISO 19115 vs the purpose for creating codemeta, in particular. This should
also help in describing the domains in which both standards are actually applied.

In the introduction, the outputs of the Codemeta project are enumerated as the vocabulary and the crosswalks
with other vocabularies/schemas, repositories, registry and archives.  I think it would be useful for
the reader if the paper also highlighted the software tools produced by the Codemeta project (https://codemeta.github.io/tools/).

When comparing the standardization processes of Codemeta and ISO, I think it is important to highlight that Codemeta follows an open process
and it is an open standard, while ISO is not and access to the information about ISO19115 is behind a paywall.

According to PeerJ policy (https://peerj.com/about/policies-and-procedures/#data-materials-sharing), which I commend, "Data should be provided in an appropriate,
machine-readable format. Note: formats such as PDF, Powerpoint, and images of tables etc. are not considered suitable for raw data sharing." Thus,
at a minimum, the mapping tables should be provided as CSV files. Ideally, it should be included in the Codemeta crosswalks (see comments about this later),
but otherwise, I suggest to include the mapping files in a repository such as Zenodo, so that they are versioned and given a DOI for their citation.
Experimental design: The mapping between the Codemeta vocabulary and the ISO19115 standard is presented in a very comprehensive way, with discussions
about the different challenges.

The goal of performing the mapping in this manuscript is presented as "facilitating the creation of codemeta-compliant
descriptions of software that is documented using the ISO standards" (from the introduction) and the mappings was
created "in order for ISO metadata creators and users to take advantage of the codemeta recommendations" (from the abstract).

To make this research question relevance more clear, it would be useful to see what are the use cases for ISO metadata creators to
convert to codemeta. What would be the scenarios of application? Also, if this is the objective, shouldn't the implementation
of such conversion be important and provided as one of the outputs (as one of the previous reviewers pointed out)?
As indicated above, I strongly recommend at a minimum that the mapping tables are made available in a machine-readable format.

Additionally, it is not clear to me why there are difficulties for this particular mapping to appear as a crosswalk file in the
codemeta repository (https://github.com/codemeta/codemeta/tree/master/crosswalks). In the conclusion, the author indicates
"Adding an ISO crosswalk to the codemeta framework was the original goal of this work, but the differences
identified and described here are significant enough to make that impossible." It seems to me that some of the challenges
presented in this paper coincide with the challenges of mapping other schemas to Codemeta. Checking for example the
mapping with DataCite(https://github.com/codemeta/codemeta/blob/master/crosswalks/DataCite.csv) and
considering that DataCite metadata is defined by an XML schema (http://schema.datacite.org/meta/kernel-4.1/), as it is
ISO19115, with some common characteristics with ISO19115 (e.g. relies on soft types rather than hard types),
it is not evident to me why this case is particularly difficult.

The conclusions section states that "the purpose [of the mapping] is different than the primary purpose of the crosswalks proposed
on the codemeta project site which is to facilitate automated translation between different JSON-LD vocabularies and RDF". Again,
this is not evident to me, as e.g. DataCite is based on an XML schema. I am aware of the tool that converts DataCite metadata into
schema.org-based JSON-LD, but the DataCite mapping (as linked above) is done based on the XML schema.

The analysis of the 'Dialect coverage and scope' is also useful for the Codemeta community. Is Figure 1 produced
based on the Codemedata full crosswalk table? Could the process be made available so that the figure could be updated
when crosswalks are revised or new crosswalks added?

The simplified XPath notation including only the role names and no XML-specific information is helpful to improve the readability
of the tables. However, given that there is no current actionable conversion (e.g. XSLT) of crosswalk in this paper,
it would be useful to provide the full XPath expressions as an additional table.

When describing 'Hard types and soft types', the author mentions RDF as an example for 'hard typing', i.e. where specific names
are required as names alone can be used to distinguish between items. However, this is not an RDF model constraint but rather
an implementation issue. In RDF, it is possible to use qualified relationships to work around the limitations of
binary predicates (see http://patterns.dataincubator.org/book/qualified-relation.html). Some of the conclusions on this respect
might need to be revised.

Table 2 presents the 'hard types' and associated ISO19115-1 codelist values grouped by item (e.g Dates, People and Organizations, etc).
Having the hard types and codelists lumped together doesn't allow to see the actual mappings. I suggest modifying this table
to provide more details, making clear that, for instance, Codemeta.datePublished maps to a date having CI_DateTypeCode.publication, while
on the other hand, there is no equivalent in Codemeta for CI_DateTypeCode.validityBegins and CI_DateTypeCode.validityExpires.

In the mapping tables, it seems that when there are no equivalent ISO 19115 terms, then the schema.org elements were not included. For example, in Table 6
representing the mapping from https://schema.org/Thing to ISO 19115, properties such as https://schema.org/alternateName or https://schema.org/image are not considered.
If not relevant, should the mapping table indicate this?

It would be helpful for the tables to include links to the properties information on the web. Please, also
note that some of the links in the Word document weren't converted as links in the PDF.

When referring to the schema.org entities, I suggest to present them without the whole path from Thing to the entity (i.e. schema:SoftwareApplication
rather than schema:Thing.CreativeWork.SoftwareApplication). If the hierarchy needs to be clarified, it can be done on the text rather than
when referring to the entity, as the schema.org namespace is flat and the presentation is otherwise confusing.
Validity of the findings: As indicated by one of the previous reviewers, the best way to validate the mapping would be to include it in the Codemeta crosswalks
(https://codemeta.github.io/crosswalk/). In the previous comments, I indicated that it is not clear why this seems to
be 'impossible' for ISO19115. Otherwise, it would be great to see ISO 19115-1 standard in the full crosswalk table
(https://github.com/codemeta/codemeta/blob/master/crosswalk.csv). Anyway, as a minimum, the mapping tables should be made available
in a machine-readable form.

If not the full code for the conversion, it would also be useful to have example instance data for ISO19115 and its Codemeta equivalent.

Given that Codemeta relies on the schema.org vocabulary, I wonder if the author considered the mappings developed by the W3C Spatial
Data on the Web Working Group: https://www.w3.org/2015/spatial/wiki/ISO_19115_-_DCAT_-_Schema.org_mapping ?

The conclusions provide a good summary of the challenges. I would like to see some discussion on how an RDF representation relying
on qualified relations would modify the interpretation of the mapping w.r.t. hard/soft types.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SOFTWARE ENGINEERING PRINCIPLES TO IMPROVE QUALITY AND PERFORMANCE OF R SOFTWARE
Review round: 1
Reviewer: 1

Output:
Basic reporting: I think for the sake of saving people the need for a dictionary, "autodictacts" should be replaced by "self-taught individuals" and "codified through" should be replaced by "constructed using" or something like that.
Line 69 is missing a full stop.
Line 129 is missing its last word (years)
Line 140 has a misformatted citation (jpresse)
Line 166 has "overtime" where it should say "over time"
Line 167 should say "percentage" and not "percent"
Line 483 has a misformed citation
Other than this, the English used is clear and professional and the citations provide nice
background/context.

The raw data is shared.

The article is self-contained with relevant results to hypotheses.
Experimental design: The research question is well defined, relevant & meaningful. It is stated how this research fills an identified knowledge gap.

Rigorous investigation is performed to a high technical and ethical standard.

Methods are described with sufficient detail and info te replicate.
Validity of the findings: The findings are valid and the authors have gone to considerable and sufficient effort to ensure that the data is robust.

The conclusions are well stated.
Additional comments: I find the article interesting and extremely relevant. In particular I welcome the improved analysis of unit testing in CRAN packages relative to what I did myself a few years ago.

I find the examples of unit testing using the pccc package a little difficult to understand. Sure, they can be understood with some effort, but I think there must be an easier example out there which would permit a more concise code listing. Perhaps the glue package? I know pccc is a niche example and you're probably trying to encourage niche developers to test their packages too, but I think for the sake of example, simpler is better.

The study of packages employing optimization is very interesting. However, with testing I think it's easy to say if a package isn't tested, it should be, but a package without obvious optimization attempts could just be very well written (using only vectorized code from other packages) and hence not need more explicit optimizations of its own. I think this should be stated: a package without obvious optimization isn't necessarily in need of optimization. Having said this, figure 3 is interesting, giving the change in explicit optimization efforts over the years.

In figure 1, I think the "No" colour should be on the bottom and in particular in fig 4, "None" should be on the bottom, not in the middle. "None" should maybe also be given a striking colour to show that it's different to the other results.
It should be made clear in the caption of fig 4 what the n>14 is about.

Consider using bench::mark instead of microbenchmark (I read recently that bench is better and the tidyverse people are pushing it now).

 Thanks for a very nice article.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SOFTWARE ENGINEERING PRINCIPLES TO IMPROVE QUALITY AND PERFORMANCE OF R SOFTWARE
Review round: 1
Reviewer: 2

Output:
Basic reporting: - Professional structure (but I have asked for some reorganization).
- References provided are good, but more are needed to substantiate specific claims (see general discussion below).
Experimental design: - Data collection and analysis is good.
Validity of the findings: - Empirical results on testing and performance optimization seem trustworthy.
- High-level discussion of rules to follow only partly backed up by empirical evidence (see general discussion).
Additional comments: General:

I feel the authors are trying to:

1. Introduce readers to key ideas and tools in software testing.

2. Survey the current state of software testing for R packages.

3. Do both of the above all over again for performance optimization.

The empirical study of how many packages on CRAN do testing and optimization was the most interesting part of the paper for me, but I acknowledge that I'm not representative of likely readers of this paper.  I think that the overview of testing the authors give in lines 180-262 is not detailed enough to serve as a tutorial for people who aren't already familiar with the topic, while being too long for people who _are_ already familiar with it.  (I feel the same way about the discussion of big-O notation starting at L440.)  I therefore recommend that the authors:

1. Put the empirical material on testing and optimization in one section near the front of the paper to show that there is significant room for improvement.

2. Replace the paragraph-length high-level advice on how to test and tune with bulleted lists of rules, each having pointers to longer-form discussions.  This will help experienced readers (who will nod at the lists of rules), while also helping newcomers operationalize those rules (which I think they would struggle to do based on the current brief explanations).

3. Expand the examples to show specific applications of the general rules.  For example, I would like to see the performance results for the PCCC code on line 413 and following, and then see what changes the authors made to the code to speed it up, and a second set of performance figures.  Similarly, in the testing example starting on line 301, I do not know what the bug was that the test found, or how the bug was detected before the test was written and its detection then translated into a test.

---

Specific:

28: "we show that reproducible and replicable software tests are not available" -> as written, this is a very strong claim.

39: Introduction of software engineering seems disconnected with preceding material.

69: missing "." between "maintenance" and "Software maintenance"

71: "chief factor" -> isn't people's time the chief factor in _all_ maintenance, not just that for statistical software?

76: "As research scientists tend to think..." -> Are they wrong?  I.e., if I assert that the majority of software written by researchers exists to solve one-off problems, rather than to be used repeatedly, is there data to show that I'm wrong?

81: Here and elsewhere, I worry that "best practices" is not validated.  There is, for example, no published research showing that the use of version control makes people more productive.  (Believe me, I've looked.)  I think the authors need to present evidence that various practices actually improve productivity and/or reliability before calling them "best".

102: Do the authors have data showing that commercial software and/or open source software are tested any more frequently?  (I've seen a _lot_ of projects on GitHub that don't have any tests...)

140: What is "jpreese"?

149: I'm unclear what is meant by "Grep for...directories" - are the authors working from textual manifests of projects, or are they using "grep" as a synonym for "search for"?

160: should "updated" be "most recently updated"?

166: "over time" rather than "overtime"

172: a table might be a better way to display this summary of "have X but not Y".

184: There is also growing evidence that TDD doesn't actually make developers more productive (see e.g. http://people.brunel.ac.uk/~csstmms/FucciEtAl_ESEM2016.pdf).

185: "A better approach..." Better by what criteria, and what data can the authors cite to support this contention?

194: "In an ideal world...100% test coverage" Do the authors mean line coverage, statement coverage, branch coverage, combinatorial coverage, ...?

208: "Once answers to these questions are known..."  I have found that giving high-level advice like this only frustrates most scientists, because they don't know how to operationalize it.  Can the authors point at concrete examples of how to translate these general rules into specific decisions, priorities, and/or tests for specific software packages?

226: "In addition to unit tests, users should perform..." I believe the authors mean "developers", not "users" (but could be wrong).  I also think that this statement trips over an important distinction between testing what-if contingencies for software tools (which developers of packages should do), and testing specific users of those tools and their inputs for particular analyses (which analysts doing particular analyses should do).  I think that repeatable sanity checks make sense for the latter, but that doesn't necessarily mean use of unit testing frameworks.

233: A notebook demonstrating the use of the software is _not_ the same thing as an acceptance test, though the automatic validation of the expected end result may be.

238: Radcliffe et al have developed a nice framework for thinking about the ways in which data analysis can go wrong, which is summarized in the figure in http://stochasticsolutions.com/pdf/TDDA-One-Pager.pdf.  It seems that most of the discussion in this paper is focusing on Step 2 of that model's 5-step process - if so, the authors may wish to cite that model and make this explicit, and if not, expand their recommendations for other phases.

301-302: some odd indentation.

430: "run 10 timess and find the mean" No - this can easily give a misleadingly high performance result because of caching effects.

437: "There are many optimizations that can be considered..."  This is the kind of advice that my students used to find frustrating, because it tells readers that something exists without telling them what it is or where to find it.

560: the summary of possible optimizations is very condensed - can the authors point at resources that have lengthier and more detailed coverage?  (There are several guides to high performance R...)


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SOFTWARE ENGINEERING PRINCIPLES TO IMPROVE QUALITY AND PERFORMANCE OF R SOFTWARE
Review round: 2
Reviewer: 1

Output:
Basic reporting: See the general comments.
Experimental design: See the general comments.
Validity of the findings: The discussion and conclusions are well stated, they are limited to the results obtained.
Additional comments: The paper is interesting and motivating. The main goal and motivation of the paper are clear. Nowadays, the development of scientific software needs a guide to optimize and test the code and, therefore, to improve software maintenance.

Generally speaking, I consider the work has been significantly improved. The authors have effectively addressed all the comments raised in the previous version.

I think that this work can be accepted for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DUAL NETWORK EMBEDDING FOR REPRESENTING RESEARCH INTERESTS IN THE LINK PREDICTION PROBLEM ON CO-AUTHORSHIP NETWORKS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The structure of the paper is correct. However, the text presents many grammatical and structural errors that make it hard to follow, the English is far from perfection.
Specific comments:
1.	It is noted that your manuscript needs careful editing paying particular attention to English grammar, spelling, and sentence structure so that the goals and results of study are clear to the reader.
	Page 1 line 12: the sentence “We compare existing graph features engineering…”should be improved to better express its meaning.
	Page 1 line 25: the sentence “the authors measuring their influence on…”should be improved to better express its meaning.
	There are misusing between singular and plural. Like Page 1 line 26: “such methods does not” supposed to be “such method does not”. There are many cases like this throughout the text.
	Page 1 line 28: “for e.g.,” is not correct. You can say “for example,” or “e.g.,”.
	Page 2 line 88: “the theory of hidden representations has impacted on…” reads better. The same as page 3 line 106 “Several works cover the node attributes…”, page 3 line 114 “…space of author’s research interests…”, and page 3 line 115 “Based on this information…”.
	Page 3 line 128: The term “author homonimy” is unusual in the literature. The same page line 132 “common neighbors” is a metric not measure.
	Page 4 line 160: What do you mean by the expression "adjust the feature vector based on …"?
	Page 4 line 176: “egde” should be “edge”.
	Page 9 line 289: The incomplete sentence “The applicability of our system for …”.
2.	There are some references suggestions follows:
     Some important recent references are missing, e.g. VOPRec: Vector Representation Learning of Papers with Text Information and Structural Identity for Recommendation, IEEE Transactions on Emerging Topics in Computing, 2018. DOI: 10.1109/TETC.2018.2830698
	Page 1 line 28: The review [48] is a little out-of-date.
	Page 1 line 43: Please check the reference [10].
	Avoid using a reference as part of the sentence by replacing it by the authors' name. For example, page 2 line 70: “In [13], the authors presented…” the sentence should be structured as “Gao et al. [13] presented…”. here are many cases like this throughout the text.
	Page 3 line 102: The wrong format of reference “[Carstens et al.]”.
	What is the different between the reference [23] and [24]?
Experimental design: The combination of these features shows the improvement of recommendation.
Validity of the findings: The authors studied the link prediction problem considering network structure and research interests based on network embedding and manual features in the co-authorship network.
Additional comments: 3.	The keyword “Co-occurrence network” is suggested to appear in the abstract part for readers to understand the key-word co-occurrence network in the following text.
4.	Page 3 line 138: What’s the relation between the description of “the influential persons” and the datasets?
5.	According to the page 3 line 132 the author disambiguation is based on three rules, however the author disambiguation in line 146 is solved by common neighbors. The rules of author disambiguation need to be clearer. How to perform the author disambiguation on these datasets?
6.	Page 5 line 175: What are “specific component-wise functions”? It supposed to explain in the text clearly.
7.	Page 6 line 210: What are “machine learning models”? Are they Logistic Regression with Lasso regularization, Random Forest, Gradient Boosting, and SVM in the following text? The methods description need to be earlier in line 210.
8.	Page 6 line 212: What do you mean by “based on List 5”?
9.	Page 7 line 255: What is the “2.6” in sentence “The suggested by authors in [29] edge embedding from 2.6…”? More explanation need to be given.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DUAL NETWORK EMBEDDING FOR REPRESENTING RESEARCH INTERESTS IN THE LINK PREDICTION PROBLEM ON CO-AUTHORSHIP NETWORKS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The authors propose a method for link prediction in co-authorship based on modern machine learning techniques applied to the network itself combined with information from features of keywords derived from publications.

The text is clear but needs some improvements, I recommend a carefully revised to achieve the quality of the journal. For instance, line 151 and the repetitions of construction We ... (see section 4).

The references cover the most important and related works. However, because peerj is a multidisciplinary journal and this article may be of interest to scientists without a strong background on machine learning, some abbreviations and terms should be explained or at least referenced. For instance: AUC, F1-SCORE (MACRO/MICRO), LOGLOSS, ROC-AUC, XGBoost, SVM.

The proposed problem is clear and very relevant to the area.
Experimental design: The analysis is solid  and the results confirms that there is a considerable improvement on the prediction quality by combining topological information of the co-authorship networks with contextual information from keywords.

The authors provide a set of codes to replicate the findings.
Validity of the findings: Since the employed datasets cover only one institution, the model may be finding collaborations related to a few grants projects or among researchers in the same group. I think a discussion about this specific characteristic should be added to the paper.

As a continuation of the previous point, the method was applied to a very small dataset of authors. However, much larger and partially disambiguated datasets are now available, such as the Microsoft Academic Graph. I'm very curious to see if the proposed technique scales well for a larger and broader dataset. A simple analysis that could be performed would be sampling a fraction of the current datasets (e.g. 20%, 40%, 60%, 80%) and check the classification performance (just one metric is enough to illustrate the effects of the data size).

The method used for evaluating the classifiers needs to be described or at least referenced. Also, if you are using a n-fold cross validation, why there is no deviations for the performance metrics?

There is no information of the number of authors and publications for the considered Scopus subset.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DUAL NETWORK EMBEDDING FOR REPRESENTING RESEARCH INTERESTS IN THE LINK PREDICTION PROBLEM ON CO-AUTHORSHIP NETWORKS
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: All my concerns have been solved.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DUAL NETWORK EMBEDDING FOR REPRESENTING RESEARCH INTERESTS IN THE LINK PREDICTION PROBLEM ON CO-AUTHORSHIP NETWORKS
Review round: 2
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors have revised the manuscript according to the reviewer comments. The paper can be accepted now.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: USING CANTOR SETS FOR ERROR DETECTION
Review round: 1
Reviewer: 1

Output:
Basic reporting: Few sentences can be rephrased::

ine 58:: demonstrate, Cantor set has desirable properties that enable efficient error detection while<with> GLS-decoding.

line 61:: that enable error detection. Repetition codes are shown to lie on a Cantor set<sentence can be framed in a better way>. Inspired by this, we incorporate error detection into GLS-coding using a Cantor set in section 4 and show simulation results of the proposed method. We conclude with open challenging problems in section 5.

< Para can be re-phrased>
line 159 Repetition codes which are error detection/correction codes lie on a Cantor set. How can we extend this
160 idea of placing codewords on a Cantor set for GLS-coding? Here, we establish the connection between
161 repetition codes and GLS-coding.

<Section heading can be re-phrased>
201 4.3 Error Detection while<with> GLS-Decoding

line 202 : If while GLS decoding, the forbidden symbol ‘F’ is encountered
can be rephrased as :
<If the forbidden symbol 'F' is encountered during GLS decoding>

line 215: The amount of redundancy that is added is easy to determine.
can be rephrased to:
<The amount of redundancy that needs to be added is easy to determine>

line 226: this line can be re-phrased
This is because, any error can only affect decoding for subsequent bits and if the error was towards the end-of-file (EoF), not many bits are available to catch it.
Experimental design: 4.4 Simulation Results
The author did not mention the number of iterations, confidence interval
Validity of the findings: no comment
Additional comments: 1) The novelty this paper is in identifying that Repetition codes are closely related to cantor sets. The author has done only preliminary experiments.
Is it possible to do experiments on real workloads.
What are the advantages/disadvantages of using this scheme?
Under what conditions this scheme performs better? Under what conditions it does not perform well?

2) Can the author also estimate (quantify) the probability of undetected errors - they mention that it is non-zero for finite length messages.

3) Looks like the error model - here is a 1 bit error. They did not explicitly mention this.

4) Can the author compare the current technique with other existing techniques which provide the same amount of detection capability. For e.g parity can also detect single-bit errors. So, why should anyone use this particular approach if parity is better.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: USING CANTOR SETS FOR ERROR DETECTION
Review round: 1
Reviewer: 2

Output:
Basic reporting: The description of the author's prior work on GLS coding is needlessly succinct. Unless there are page limit restrictions that need to be satisfied It should be expanded to at least include an example.  The idea is straightforward enough that it really should not need much additional work.

The use of a forbidden symbol was also used in Pettijohn et al. (2001) and should probably be included in section 4.5.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: Nice work!


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: USING CANTOR SETS FOR ERROR DETECTION
Review round: 1
Reviewer: 3

Output:
Basic reporting: The reporting is basically sound, but certain clarifications would be helpful to the reader.
The map from source symbol to interval could be better illustrated and explained.
START and END should be labeled or indicated in Fig. 1 or its caption.  Initial value corresponding to '001' and number of bits to send it should be given.
Why is reserving a width epsilon for a forbidden symbol ensure codewords lying on a Cantor set?   I thought the Cantor set remains when middle third is repetitively removed.
The properties of "totally disconnected" and "isolated points" are true of the Cantor set, where the middle third is repetitively removed ad infinitum.  Clearly, that is impractical and sub-intervals containing points of the Cantor set remain.   So a single bit error may not take the value outside the sub-interval in which it occurs.  The error might be detected if follow a sufficient number of sub-divisions of that interval.     I think an explanation along these lines would better clarify the reason why a later bit error may not be caught if not enough bits remain in the code sequence.
Experimental design: I have no comments.   All aspects are in order.
Validity of the findings: The experimental findings look to be correct and support well the validity and efficacy of the method to detect bit errors.  I am somewhat confused by the first column in Tables 2 and 3.  This column gives ranges of distances of a single bit error from EOF.  Since there is only one corresponding number for Detected and Undetected, it seems that these numbers count the Detected and Undetected errors in that range; and the bit error is inserted prior to the range's beginning point.    If so, I suggest a change of column heading wording.  Otherwise, everything else is clear.
Additional comments: The paper is well-written and presents a novel scheme for continuous detection of transmission errors using GLS decoding.   Please take into account the above comments.    One additional minor matter:
in 3.2, you mention correction of up to (n-1)/2 bit errors.  Only true when n is odd, or change to
 floor( (n-1)/2 ).


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MACHINE LEARNING WITH REMOTE SENSING DATA TO LOCATE UNCONTACTED INDIGENOUS VILLAGES IN AMAZONIA
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This is an excellent piece of work aiming at locating uncontacted indigenous population in Amazonia with non-invasive technology, namely random forest applied on Landsat imagery. I would recommend accepting the manuscript as it is. I identified only an error: line 109- Wessel & Smith, 1986 (1996 in the reference list). While "Strahler numbers" is an accepted name, I would suggest replacing with "Strahler stream order" throughout the manuscript, as the later is less ambiguous.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MACHINE LEARNING WITH REMOTE SENSING DATA TO LOCATE UNCONTACTED INDIGENOUS VILLAGES IN AMAZONIA
Review round: 1
Reviewer: 2

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: There is only a point where I would appreciate a clearer statement.

Experiments are based on the leave-one-out (LOO) paradigm due to the limited data set size. Moreover, it clearly emerges from the analysis that the results in the classification between contacted and uncontacted villages strongly depends on the random forest parameters, in particular on cutoff. Therefore, the procedure applied to estimate this parameter is crucial to evaluate performance:  from the text, it is not clear if two nested LOO paradigms have been used to optimize its value and evaluate performance. If this is the case, please state it clearly. On the other hand, if a LOO experiments has been repeated by using different threshold values, then performance would be  overestimated and there is a risk of overfitting, also because the data set is so small. A similar clarification is also needed for the task of predicting new villages.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MACHINE LEARNING WITH REMOTE SENSING DATA TO LOCATE UNCONTACTED INDIGENOUS VILLAGES IN AMAZONIA
Review round: 1
Reviewer: 3

Output:
Basic reporting: I really enjoyed reading this paper. It is well-written, well-organized and provides sufficient background information.
Experimental design: My only criticism is related to the explanation of the created prediction model.  There is no information on how the training samples were collected. I assume that this information is available on the project website. However, I think that the readers have to be informed in the paper itself about this. Furthermore, the variables used as input for creating the model are not described. For example, where did you get the data on the lights at nigh fromt? Is this variable related to the distance to town variable? Why did you decide to use these eleven specific variables to create the prediction model? How many decision trees did you defined, i.e. ntree parameter? How many variables did you use to split the trees nodes, i.e. mtry parameter?
Do the uncontacted villages share the same characteristics? I am not an expert in this field, but I am curious whether there are specific cultural traits that might influence the relevance of the variables used to predict the presence of these villages. For example, the four variables identified as being the most relevant variables might vary from one village category to another??
Specific comments are available below:
Line 35: How do you define near-real time in your study?
 Line 83: how did you collect the data on the uncontacted and contacted villages?
Lines 106-107: how did you identify the lights-at-night?
Lines 113: which preliminary work do you refer to?

Figures: I am not sure why there is a duplication of the figures’ explanations?  (figures captions)
Figure 2:  please check the leave-one-out (you wrote leave-outout)_
Validity of the findings: The reported results  have a clear and important societal relevance. The impact of the study is well described.
Additional comments: no further comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MACHINE LEARNING WITH REMOTE SENSING DATA TO LOCATE UNCONTACTED INDIGENOUS VILLAGES IN AMAZONIA
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: I think the paper is now ready for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ADAPTIVE AUTOMATION: AUTOMATICALLY (DIS)ENGAGING AUTOMATION DURING VISUALLY DISTRACTED DRIVING
Review round: 1
Reviewer: 1

Output:
Basic reporting: Well written.
Only a few small grammatical comments.
1. line 160 missing a period before "During"
2. line 229-231 "Furthermore, the participant was not allowed to perform the secondary task without looking at the secondary task, to ensure that he/she was not visually distracted when performing the secondary task." I find this sentence confusing, the first part seems to convey that they had to look at the CD player to complete the task, but the second part conveys that they were not supposed to be looking away while completing the secondary task.

Figure 4: Is this "proportion of participants with automation turned on" or "mean proportion of manual driving time"? The first, as it is described in the first sentence of the figure discussion, leads me to believe that the y axis should be proportion of participants. Also, the “automation status of individual participants”, is that indicating when automation was activated (for backup) and deactivated (for forced)?

Figure 5: Why is the magenta line vertically shorter in the forced condition? Does that convey a different experimental condition?

Figure 7: I think "percentage of participants glancing ON-ROAD" (or "OFF-ROAD") would be more insightful than "ON-SCREEN", as this makes it more intuitive which direction on the y axis is safer. Also, on-screen makes it less generalizable, as a screen is usually equated to an in-vehicle screen (e.g., infotainment system).

Figure 10: What is the y-axis? Are these raw scores or transformed? It doesn't really convey what value would be considered "better" as it currently stands.
Experimental design: I found the description of the automation very confusing, and only understood once I read the entire manuscript. This needs to be more clearly described up front, all at once. For example, I think the paragraph in line 576 should be incorporated with line 112, where you discuss the rational for using the "Forced condition". As it is currently written, the forced conditions seems like an unideal system that consumers would not want in their vehicle and that could yield liability issues to manufacturers. I also think it needs to be more clearly discussed that this is essentially cruise control or ACC with a lane keeping system, and that the lane keeping system is being manipulated. The "manual" condition is not entirely manual, as the vehicle is still cruising at 70 kph, correct? It's fine to use the term "manual", but I just think this needs to be more clearly defined - again perhaps earlier on, maybe in the abstract too? This would also help make the results more generalizable, because essentially you are looking at a partial automation condition, with only lane keeping engaged/not engaged, but always an ACC component.
Validity of the findings: line 368: “In summary, the more automation was applied, the better the overall average lane keeping performance.” I’m not convinced that “forced” can be described as more automation, it seems that if the system is forcing the driver to resume lateral control when they need it most (i.e., distracted), then it is not increasing automation.

Sections 4.1-4.3 doesn't really add much discussion, it is just re-summarizing the same information presented directly above. Perhaps incorporate more references to the literature, or discussion of interpretations, and this could be more meaningful.

Section 4.5 doesn't really discuss the generalizability of the findings, but more so is the first time the "forced" condition is rationalized for the experimental design. I think more expansion on how these findings could be used would be more fitting.
Additional comments: This is certainly an interesting topic, with a well thought experimental design and appropriate analysis. I think further clarity up front of the design would be helpful, as well as more consideration on how to best present the data in the figures.

Line 95-97: Usually include page number of reference in citation when you use a direct quote.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ADAPTIVE AUTOMATION: AUTOMATICALLY (DIS)ENGAGING AUTOMATION DURING VISUALLY DISTRACTED DRIVING
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper investigates how a car’s lane deviation is affected by the automation mode. Three situations are compared: manual driving, a car where the automation is the “back-up” that takes over when there is too much distraction (as determined based on eye-tracking), and a system that occasionally forces the human to drive (even when distracted).

The article meets most of PeerJ’s basic reporting criteria. The exceptions are:
1. The article occasionally refers to supplementary files for scripts and additional analyses, yet I cannot see these files in the online submission system. For a couple of claims in the manuscript, the details are essential.
2. I also cannot find files that contain the raw data, which seems to be a requirement of the journal.

I also have some more detailed feedback below to improve basic reporting.
Experimental design: The research question is well defined and is relevant, in the sense that it provides data-driven insights into the success of a “backup” and “forced” automated car system. The investigation is rigorous (looking at various aspects of lane deviation, gaze, and subjective experience).

For the Methods section, I would encourage the authors to improve on the following:
3. In section 2.5 the three conditions are explained. The initial explanation is quite high level, but later (line 200) some nuances are brought to the attention of the reader. I would encourage the authors to provide the high-level description outside of the methods section to the introduction (i.e., to introduce the general concept), and to focus in the methods section on the *exact* implementation of the method (i.e., the exact method that was used; here the details matter as the method is a specific implementation of the general idea). That reduces the chances that readers might overlook these important details.

Again there are also some more detailed comments on this item below.
Validity of the findings: In general, this meets the criteria of the journal, with some detailed comments below to further improve this.
Additional comments: Here are the more detailed comments that I hope can help to improve the manuscript and line of work. I ranked them in what, from my perspective, is relatively more “major” and more “minor”.

Relatively more major:
4. A hidden assumption behind the work is that gaze at the road (or screen) is indicative of attention for the road. This is a very strong assumption (by the way: which many other researchers also make either implicitly or explicitly). Given that gaze is the input for your system, it is important that some nuance is introduced in the paper. Specifically, I would encourage the authors to make more explicit that this is an assumption, and that there can be exceptions to the rule “where you gaze at is where you attend”. In general, the literature on “covert attention” can be useful in this regard.
To state a couple of examples of why the distinction between covert and overt attention is relevant: people with “tunnel vision” might look at the road, but not fully process all relevant information. Similarly, people who are mind wandering, or mentally distracted might gaze somewhere but not really attend to (or more neutral: Process) information at the gaze location.
It would be good to provide some nuance on this topic in the introduction (e.g., around line 69) and the general discussion.

5. The secondary task is introduced as a task that “commonly occurs while driving” (e.g., line 146). This claim is too strong — at least when concerning the specific implementation of the CD change that is used here, where (a) a lid needs to be opened, and (b) a physical stack of CD cases is handled. In particular, I am not aware of in-car CD players that adhere to property (a) above.
More generally, one might wonder whether modern cars are not more focused on MP3s and digital devices (e.g., use of phone connected to car) to play music, instead of CDs.
To avoid making too strong a claim, I think the better claim is that the authors used this task because it involves visual-manual distraction, and because the steps behind the task are detailed reported in another study (cited in line 218). That is, there is a relevant benchmark/reference for it.

6. Line 235: the participant was scored. It does not say explicitly who scored the participant. Was this an automated algorithm or the experiment leader? In general, the details (which should be in supplementary material, but I can’t find them) are needed here to assess whether this metric is valid, as (without knowing more details) it seems to involve both objective and subjective criteria. For the potential subjective component, some better understanding of the standards and training of the marker/experimenter are needed to interpret correctly. (or at the least: an acknowledgment that there is a subjective element to this — as I understand that this might be hard to report in hindsight)

7. Figure 4 now reports a score between 0 and 1, whereas this axis uses binary data (did a participant look yes/no) and then uses that data to assess what percentage or fraction of participants looks at the screen. The y-axis should, therefore, be changed to reflect percentage (or fraction) of users.

8. Line 518: these were “infrequent” and “never appeared to last long enough”. What was the rough criteria to call something “infrequent’ (e.g. “Less than X% of the cases) and what was the rough criterium to call something not “long enough”?

More minor:

9. Counterbalancing: On line 159 it is claimed that the study is counterbalanced. However, there are 31 participants and 3 conditions, and therefore 6 unique orderings (3 x 2 x 1). 31 participants does not allow for a fully counterbalanced design (i.e., that would require a multiple of 6, such as 30), and this is also not claimed by the authors. However, I also do not see in what other way this odd number can be balanced (wouldn’t a balanced design not require an even number?).
In other words: the exact way of counterbalancing should be explained more clearly.

10. Line 178 “inferences from the literature”: can the authors describe more precisely what the claims or data in the literature was and how this lead to the inference? For example, what is the range of values used or mentioned in the literature? And how does this range relate to your criteria?

11. Line 264: “If the calibration was successful”: I assume that this was determined by the eye-tracker system. What are the criteria for calling it successful?

12. This leads to a more general question: what is the accuracy of the eye-tracker system. Note that this can be expressed in terms of what the company defines (which are often relatively ideal values), and in terms of what you found (where the authors already mention some limitations). Do the authors have data on how often there is a “miss” or “False alarm” in terms of whether the participant is looking at the screen?

13. A more personal aspect, that does not need addressing in the paper, but which I hope is of interest to the authors: when I read the paper another hidden assumption seems to be that eventual implementation of such a system simply ‘works’ and that the human driver “knows” when it works. In particular, for the backup system, to function well the system really needs to work well, and the human must be able to rely on it. In practice, I suspect that such a system could lead to “mode confusion” (because maybe it can be turned off, and the human might misinterpret when it is on or off), and if a human incorrectly relies on it, or if the system does not function properly when it should, accidents might happen. I am reminded of this, as such mode confusion (and how to think about it systematically) is something I recently wrote about myself, and that might be of interest to the authors. See Janssen, Boyle, Kun, Ju, & Chuang (in press, International Journal of Human-Computer Interaction) (a preprint is on my website)

Signed,
Christian P. Janssen


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: 20 GB IN 10 MINUTES: A CASE FOR LINKING MAJOR BIODIVERSITY DATABASES USING AN OPEN SOCIO-TECHNICAL INFRASTRUCTURE AND A PRAGMATIC, CROSS-INSTITUTIONAL COLLABORATION
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This is a very exciting paper that outlines a methodology for aligning large biodiversity datasets in a very short amount of time (certainly, less time than it takes to download some of the data). The approach is a significant advance in an area that has struggled to map data across multiple data providers.

I have a few minor quibbles, mostly about how the work is presented. These are listed below, in no particular order.

The phrase "taxon graph" was a little unclear. If I'm correct, it's simply a "star tree" with a node in source database in the centre, and each external identifier that it is linked to is represented by a node connected to that central node. To treat this as a graph seems overly elaborate, why not simply treat it as a set of external identifiers?

The description of overlap and consistency (line 169 onwards) might benefit from a diagram of some examples, rather than require the reader to keep a set of identifiers in their head. It might also be useful to explain why "GLOBI:null@Procladius sp1 M_PL_014" had so many mappings.

The performance is impressive, but the comparison to other efforts (e.g., EOL, BioNames, line 230) suffers somewhat because it relies on anecdotal recollections of how much effort was expended on semi-automated mapping. However, there's not much which can be done about this.

The authors recommend (line 245) that owners of taxonomic databases provide lookup tables to help data integration. The description of a look up table is verbal, and so open to some ambiguity. A diagram of an example would make it very clear what format is expected (I'm reminded of how explicit the "Tidy data" paper is in this regard, see http://vita.had.co.nz/papers/tidy-data.html).

The Venn diagram in Fig 3 is not to scale, which is somewhat misleading. How about using something like a cluster map, see http://iphylo.blogspot.com/2012/06/visualising-differences-between.html I've attached a very crude version to show what I mean. Such a diagram makes it clear to what extent the various sources overlap, without the distortion of the Venn diagram (the circles could be scaled to the log of the number of taxa in each category, for example).

Looking at the github repository for this project (https://github.com/bio-guoda/guoda-datasets/blob/master/wikidata/README.md) I'm invited to go to https://guoda.bio/ to learn more. There's a problem with the HTTPS security certificate for this site, which triggers all sorts of warnings in various web browsers.

DOI and LSID are acronyms and should be capitalised, and perhaps spelt out the first time they are mentioned.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: 20 GB IN 10 MINUTES: A CASE FOR LINKING MAJOR BIODIVERSITY DATABASES USING AN OPEN SOCIO-TECHNICAL INFRASTRUCTURE AND A PRAGMATIC, CROSS-INSTITUTIONAL COLLABORATION
Review round: 1
Reviewer: 2

Output:
Basic reporting: The manuscript is well written and well structured. It is easy to read and to follow the authors' reasoning. The literature references provide sufficient context.

Figure 1 is a bit oversimplified. In addition to product logos, it would be helpful to learn about the functional elements of the architecture.
Experimental design: The use of Wikidata as a source of taxonomic information is highly novel and well executed.

As part of the discussion (line 241-242), the authors mention the social barriers that have prevented the biodiversity community from introducing identifiers for taxa. A paper by P. Nimis (2001) illustrates these barriers quite well and might be included as a reference.
Nimis, P. L. (2001). A tale from Bioutopia - Could a change of nomenclature bring peace to biology’s warring tribes? Nature, 413(6851), 21. https://doi.org/10.1038/35092637

Further on in the discussion (lines 250 f), the authors note that a detection of inconsistencies in the identification of species across systems might also be implemented. It would be of interest to learn why this has not been implemented, even if it was just due to a lack of time or resources. A possible approach to dealing with taxonomic inconsistencies has been demonstrated by Huber & Klump (2009), showing that the validity of taxonomic names can be estimated by graph analysis.
Huber, R., & Klump, J. (2009). Charting taxonomic knowledge through ontologies and ranking algorithms. Computers & Geoscience, 35(4), 862–868. https://doi.org/10.1016/j.cageo.2008.02.016
Validity of the findings: In their paper, the authors stress several times that their solution is very fast. The manuscript would be more informative in this respect of the authors could supply the reader with more information, if available, about where the bottlenecks in processing are and how the system scales.
Additional comments: In general, I enjoyed reading this manuscript. The use of Wikidata and JSON outlines a new and original approach to biodiversity data that will be able to bridge the gaps between existing biodiversity information systems at scale.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: PERFORMANCE, WORKLOAD, AND USABILITY IN A MULTISCREEN, MULTI-DEVICE, INFORMATION-RICH ENVIRONMENT
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This is a very well-designed, -executed, and -written study. My comments are only suggestions for improvement, not requirements for publication.

One factor that you might want to discuss is how a tablet with a touch-screen supports different types of interaction than does a keyboard, mouse, monitor set-up. If the experimental task were to involve operations such as zooming in/out, or other more direct manipulation tasks, then the impact of tablets may be different than what this study found.

Another suggestion is to restate the main objective of the study (line 60-61) at the end of the Introduction. This would help to connect the literature review to the methods.

Lastly - Are there any qualitative results on how the participants used the displays/tools in completion of the task? Did they generally arrange the windows the same way, or was there a lot of variation (within the constraints presented in table 1)?  I realize qualitative analysis was not the main objective of the study, but it is possible it may help explain the performance results.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: PERFORMANCE, WORKLOAD, AND USABILITY IN A MULTISCREEN, MULTI-DEVICE, INFORMATION-RICH ENVIRONMENT
Review round: 1
Reviewer: 2

Output:
Basic reporting: Interesting perspective, with a useful research question on multiple interfaces for accessing complex information.  The reference to medical information use and electronic health records (EHR) seems awkward and disconnected; although the EHR context was previously studied by the authors, the current task (NASCAR pit stops) and analysis flow charting is not directly related to the EHR environment.  The raw data and tables are shared, but the statistical analyses are not.
Experimental design: The research question is well defined, and the task analysis is based on an interesting approach.  However, the sample is constrained to undergraduate students in a single domain of engineering, and is a small sample (only 18 total).  This strongly limits generalizability of the design to a particular set of tasks and user populations.  If the tablets were free to be moved, did the authors actually measure their location when the study participants moved them?  That would be helpful from a perspective of learning how much variation in configuration is desired / expected among a general population of users.
Validity of the findings: The data analysis section seems incomplete, especially compared to the capability of sharing raw data (and thus statistical analysis code or F-ratio tables).  The design of the study is actually a repeated-measures, within-subjects design, but it is unclear if the analyses actually used this design rather than the more common (but incorrect) between-subjects, independent groups design.  (More clear and evident presentation of the ANOVA models would clarify this question.  This lack of clarity / possible confusion in analysis is the primary objection I have to publishing the paper in its current form.
Additional comments: There are a number of tasks for which this study design is relevant and potentially helpful for user experience / usability  / workload analysis.  It is also much appreciated that the authors distinguish monitor size from screen resolution (the largest monitors from 2000 would have much lower resolution, and thus poorer readability, than current monitors with much higher resolution).  However, the reference to EHR tasks seems little more than a chance for the authors to cite their own prior work; there is no shared context (or even explanation of EHR tasks) that seems to link those references to the current study application. Even with that concern, the bigger critique (from a revise and resubmit standpoint) is the lack of clarity, shared information, or explanation of the analysis of variance model between within subjects and independent groups designs.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: PERFORMANCE, WORKLOAD, AND USABILITY IN A MULTISCREEN, MULTI-DEVICE, INFORMATION-RICH ENVIRONMENT
Review round: 1
Reviewer: 3

Output:
Basic reporting: Discussion of related work needs adjustments. For details, see main review below.
Experimental design: Rationale for study design should be improved. For details, see main review below.
Validity of the findings: Statistical reporting should be improved. For details, see main review below.
Additional comments: The authors conducted a lab-based within-subjects experiment (n=18) to evaluate the effect of multi-display environments on performance, subjective workload and usability. Specifically, following three environments were compared: desktop computer with single screen (C1), desktop computer with dual screen and additional tablet (C2), desktop computer with single screen and two additional tablets (C3). No statistically significant differences in efficiency or workload were indicated. C2 did result in statistically significant higher usability ratings compared to C1 and C3 and to significantly fewer errors compared to C3.

The topic is timely and relevant. The paper is written in an understandable manner.
The introduction is brief (1 page), out which half a page refers to a previous paper-based study by the authors themselves. This brief introduction should be expanded and put into context of actual computer-based multi-display environments.

Some (mostly older) related work is listed but the authors miss to discuss how their work differentiates from previous work. I would suggest considering discussing articles and studies about recent multi-display environments (also including second-screen applications in TV and gaming settings) e.g.:

Brown, A., Evans, M., Jay, C., Glancy, M., Jones, R., & Harper, S. (2014, April). HCI over multiple screens. In CHI'14 Extended abstracts on human factors in computing systems (pp. 665-674). ACM.

Carter, M., Nansen, B., & Gibbs, M. R. (2014, October). Screen ecologies, multi-gaming and designing for different registers of engagement. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play (pp. 37-46). ACM.

Grubert, J., Kranz, M., & Quigley, A. (2016). Challenges in mobile multi-device ecosystems. mUX: The Journal of Mobile User Experience, 5(1), 5.

Neate, T., Jones, M., & Evans, M. (2015, April). Mediating attention for second screen companion content. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 3103-3106). ACM.

Vatavu, R. D., & Mancas, M. (2014, June). Visual attention measures for multi-screen TV. In Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video (pp. 111-118). ACM.

Section 2.1 introduces the study design. While some rationale is presented on why the study design was chosen as it was, the authors could add a clear description on the limitations and interdependences of their chosen setup (i.e. multiple factors such as screen type, position and size were varied simultaneously). Otherwise, the experimental description follows scientific standards.

The presentation of results should be improved by including effect sizes and statistical power analysis, given an experiment with 18 * 3 = 54 samples for three conditions. Also, as the authors highlight statistically significant difference for the usability ratings they should report the statistics for the post-hoc comparisons (section 3.3).

Section 3.4 presents some brief qualitative findings, with one being that participants would prefer a dual stationary monitor setup, without further highlighting the role of the additional tablet in C2.

Section 4. (Discussion) seems not like an actual discussion but solely a summary of how the results aligned with or differed from the hypotheses.

Section 4.1 (Key findings) summarizes the inconclusive results of the experiment. Beyond what is discussed in the paper I would have liked to see a discussion about the potential impact of the chosen task on the results.

Again, my hypothesis is, that if the authors would have varied fewer factors at once, the experiment might have resulted in more insights compared to the experiment conducted.

To summarize, the authors tackle an important and timely topic. The paper is overall well written but suffers from a suboptimal differentiation from prior work and a study design with (potentially too) many factors that were varied at once. Still, after addressing the issues (that can be fixed) raised in this review, the paper can add value to the existing literature.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: PERFORMANCE, WORKLOAD, AND USABILITY IN A MULTISCREEN, MULTI-DEVICE, INFORMATION-RICH ENVIRONMENT
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: Improved from prior design, now achieves "Pass"
Validity of the findings: It is still questionable whether findings from 1995-2005 with monitor resolutions of the era can be directly compared to more recent findings, but overall, the authors' edited explanations and conclusions are significantly improved.
Additional comments: The authors have done a very competent job of reviewing and responding to the reviewers' comments, and conducting a suitable revision of the statistical analysis.  The additional commentary that most participants did not adjust the positions of the monitors is also useful in supporting the comments that additional monitors and functionality did not significantly improve participant performance or workload.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN INTEGRATED PLATFORM FOR INTUITIVE MATHEMATICAL PROGRAMMING MODELING USING LATEX
Review round: 1
Reviewer: 1

Output:
Basic reporting: No comment (see reviewed PDF).
Experimental design: No comment (see reviewed PDF).
Validity of the findings: No comment (see reviewed PDF).
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN INTEGRATED PLATFORM FOR INTUITIVE MATHEMATICAL PROGRAMMING MODELING USING LATEX
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper is well-organized, and the use of the English language is generally good.

In line 84, the authors state that “this is the first prototype workable scheme to address how LaTeX could be used as an input language to perform mathematical programming modeling.” This statement is not necessarily true. I encourage the authors to take a closer look at MOSAICmodeling (www.mosaic-modeling.de), which is a modeling and optimization framework based on a LaTeX-style syntax for inputting algebraic and differential equations. A comparison between MOSAICmodeling and the proposed framework would be helpful in terms of evaluating the advantages of this contribution.

In lines 92-97, 22 references are listed with no further details, which is not very useful for the reader. Also, this seems to be a fairly biased list, with contributions mostly from the process systems engineering (PSE) community and many papers co-authored by the authors of this work. I recommend selecting a more representative set of papers from a wider range of fields, considering that the topic addressed here is of interest to more than just the PSE community.
Experimental design: The proposed approach is sound and it makes natural sense to divide the work process of the parser into three parts: objective function, constraints, and variables. However, I have some questions that I hope the authors could answer in their response or address in the revised manuscript:
1. The authors state that the parser can handle Greek letters. How about variables that consist of a Greek and a Latin letter, e.g. \Delta t?
2. How does the parser handle superscripts? Does the parser differentiate between superscripts and exponents?
3. In the literature, summations are often defined over sets instead of explicit conditions, e.g. \sum\limits_{i \in K} with K being a subset of the full-cardinality set I. Can the parser handle this case?
4. Looking at the example in line 347, it seems to be possible to use subsets for constraint generation. How are these subsets defined as these definitions usually do not explicitly appear in the LaTeX formulation of an optimization model? Is this information possibly directly extracted from the .dat file?
Validity of the findings: The validity of the proposed approach is nicely demonstrated in an illustrative example. However, some information on the platform’s ability to detect errors in the LaTeX formulation would be useful. For example, in the illustrative parsing example, what happens if in one of the constraints, the user forgot to write one of the two indices of the variable x?

Moreover, I personally would welcome a discussion on the challenges of extending this platform to the nonlinear case.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN INTEGRATED PLATFORM FOR INTUITIVE MATHEMATICAL PROGRAMMING MODELING USING LATEX
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: The revised article, as well as the authors' comments, satisfactorily address my concerns from the first review.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN INTEGRATED PLATFORM FOR INTUITIVE MATHEMATICAL PROGRAMMING MODELING USING LATEX
Review round: 2
Reviewer: 2

Output:
Basic reporting: The authors have addressed all the reviewers' comments in the revised manuscript.
Experimental design: The authors have addressed all the reviewers' comments in the revised manuscript.
Validity of the findings: The authors have addressed all the reviewers' comments in the revised manuscript.
Additional comments: The authors have addressed all the reviewers' comments in the revised manuscript.