Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EXACT ACCELERATION OF COMPLEX REAL-TIME MODEL CHECKING BASED ON OVERLAPPING CYCLE
Review round: 2
Reviewer: 1

Output:
Basic reporting: This paper develops an extension of the parking-cycle acceleration
technique for timed automata to consider multiple overlapping cycles.
The extension is shown to be exact with respect to reachability
properties, and supporting experiments show that it performs superior
to the standard parking-cycle technique, and also to the
appended-cycle method, and permits application of timed-automata model
checking to systems which hitherto were too complex for standard
timed-automata model checkers.

The paper is reasonably well-written and carefully covers previous
work on accelerating cycles as well as possible applications to
real-world IoT systems.  Except for some minor remarks below, the
paper can be accepted.

l.26: "... of *the* parking-..."

l.30f: I did not understand the sentence starting "For the".  (How
"can [it] accelerate the [...] concurrent states"?)  Please
reformulate.

l.49f: Same here: please reformulate the part of the sentence after
"these methods".

l.62: "... which *can* cause ..."

l.101: "... optimizes *the verification of* the LEGO ..."

l.121: Please remove the line break before "theory".

l.138: "... introduces /the/ timed" (remove "the")

l.139: "..., and *the* theory ..."

l.150: T(C) does not "contain[...] all clock variables", please
reformulate.

l.161: "... is the *set of* initial location*s*. ..."

l.162: "... clock constraint/s/"

l.199: What do you mean "the timed automaton detects a symbolic model
wth cycles"?  Please clarify.

l.248: You put I'(l_0')=\emptyset, but that's supposed to be a clock
constraint.  Do you mean "true"?

l.268f: Same here, both for I'(l_0') and for the guard on the second
added edge.

l.276: "... accelerated automat*a* ..."

l.293f: I didn't understand the sentence starting "The model".  Please
reformulate.

l.376:  Please remove the first "l_reset".

l.376f:  Please clarify your statement by saying "There is a single
acceleratable cycle whose effect is ...".

l.383: The condition "a_j+a_k\ge \forall(a_j, a_k)" is mathematically
meaningless.  Please reformulate.

l.404: Please remove the first "l_reset".

l.415: Please remove the first "l_reset".

l.421f: Again some strange occurrences of \emptyset.  You probably
mean I'(l_reset')=true, but on the edges in l.422 you twice put a
*label* \emptyset.  Do you mean \epsilon here?  (There is one more
\emptyset which should be "true", too.)

l.472: "..., from *the* timed ... from *the* accelerated"

l.503: What do you mean "overcome the parallel composition"?  Please
clarify.

l.566: What do you mean "the number and scale of access number"?
Please clarify.
Experimental design: This paper develops an extension of the parking-cycle acceleration
technique for timed automata to consider multiple overlapping cycles.
The extension is shown to be exact with respect to reachability
properties, and supporting experiments show that it performs superior
to the standard parking-cycle technique, and also to the
appended-cycle method, and permits application of timed-automata model
checking to systems which hitherto were too complex for standard
timed-automata model checkers.

The paper is reasonably well-written and carefully covers previous
work on accelerating cycles as well as possible applications to
real-world IoT systems.  Except for some minor remarks below, the
paper can be accepted.

l.26: "... of *the* parking-..."

l.30f: I did not understand the sentence starting "For the".  (How
"can [it] accelerate the [...] concurrent states"?)  Please
reformulate.

l.49f: Same here: please reformulate the part of the sentence after
"these methods".

l.62: "... which *can* cause ..."

l.101: "... optimizes *the verification of* the LEGO ..."

l.121: Please remove the line break before "theory".

l.138: "... introduces /the/ timed" (remove "the")

l.139: "..., and *the* theory ..."

l.150: T(C) does not "contain[...] all clock variables", please
reformulate.

l.161: "... is the *set of* initial location*s*. ..."

l.162: "... clock constraint/s/"

l.199: What do you mean "the timed automaton detects a symbolic model
wth cycles"?  Please clarify.

l.248: You put I'(l_0')=\emptyset, but that's supposed to be a clock
constraint.  Do you mean "true"?

l.268f: Same here, both for I'(l_0') and for the guard on the second
added edge.

l.276: "... accelerated automat*a* ..."

l.293f: I didn't understand the sentence starting "The model".  Please
reformulate.

l.376:  Please remove the first "l_reset".

l.376f:  Please clarify your statement by saying "There is a single
acceleratable cycle whose effect is ...".

l.383: The condition "a_j+a_k\ge \forall(a_j, a_k)" is mathematically
meaningless.  Please reformulate.

l.404: Please remove the first "l_reset".

l.415: Please remove the first "l_reset".

l.421f: Again some strange occurrences of \emptyset.  You probably
mean I'(l_reset')=true, but on the edges in l.422 you twice put a
*label* \emptyset.  Do you mean \epsilon here?  (There is one more
\emptyset which should be "true", too.)

l.472: "..., from *the* timed ... from *the* accelerated"

l.503: What do you mean "overcome the parallel composition"?  Please
clarify.

l.566: What do you mean "the number and scale of access number"?
Please clarify.
Validity of the findings: This paper develops an extension of the parking-cycle acceleration
technique for timed automata to consider multiple overlapping cycles.
The extension is shown to be exact with respect to reachability
properties, and supporting experiments show that it performs superior
to the standard parking-cycle technique, and also to the
appended-cycle method, and permits application of timed-automata model
checking to systems which hitherto were too complex for standard
timed-automata model checkers.

The paper is reasonably well-written and carefully covers previous
work on accelerating cycles as well as possible applications to
real-world IoT systems.  Except for some minor remarks below, the
paper can be accepted.

l.26: "... of *the* parking-..."

l.30f: I did not understand the sentence starting "For the".  (How
"can [it] accelerate the [...] concurrent states"?)  Please
reformulate.

l.49f: Same here: please reformulate the part of the sentence after
"these methods".

l.62: "... which *can* cause ..."

l.101: "... optimizes *the verification of* the LEGO ..."

l.121: Please remove the line break before "theory".

l.138: "... introduces /the/ timed" (remove "the")

l.139: "..., and *the* theory ..."

l.150: T(C) does not "contain[...] all clock variables", please
reformulate.

l.161: "... is the *set of* initial location*s*. ..."

l.162: "... clock constraint/s/"

l.199: What do you mean "the timed automaton detects a symbolic model
wth cycles"?  Please clarify.

l.248: You put I'(l_0')=\emptyset, but that's supposed to be a clock
constraint.  Do you mean "true"?

l.268f: Same here, both for I'(l_0') and for the guard on the second
added edge.

l.276: "... accelerated automat*a* ..."

l.293f: I didn't understand the sentence starting "The model".  Please
reformulate.

l.376:  Please remove the first "l_reset".

l.376f:  Please clarify your statement by saying "There is a single
acceleratable cycle whose effect is ...".

l.383: The condition "a_j+a_k\ge \forall(a_j, a_k)" is mathematically
meaningless.  Please reformulate.

l.404: Please remove the first "l_reset".

l.415: Please remove the first "l_reset".

l.421f: Again some strange occurrences of \emptyset.  You probably
mean I'(l_reset')=true, but on the edges in l.422 you twice put a
*label* \emptyset.  Do you mean \epsilon here?  (There is one more
\emptyset which should be "true", too.)

l.472: "..., from *the* timed ... from *the* accelerated"

l.503: What do you mean "overcome the parallel composition"?  Please
clarify.

l.566: What do you mean "the number and scale of access number"?
Please clarify.
Additional comments: This paper develops an extension of the parking-cycle acceleration
technique for timed automata to consider multiple overlapping cycles.
The extension is shown to be exact with respect to reachability
properties, and supporting experiments show that it performs superior
to the standard parking-cycle technique, and also to the
appended-cycle method, and permits application of timed-automata model
checking to systems which hitherto were too complex for standard
timed-automata model checkers.

The paper is reasonably well-written and carefully covers previous
work on accelerating cycles as well as possible applications to
real-world IoT systems.  Except for some minor remarks below, the
paper can be accepted.

l.26: "... of *the* parking-..."

l.30f: I did not understand the sentence starting "For the".  (How
"can [it] accelerate the [...] concurrent states"?)  Please
reformulate.

l.49f: Same here: please reformulate the part of the sentence after
"these methods".

l.62: "... which *can* cause ..."

l.101: "... optimizes *the verification of* the LEGO ..."

l.121: Please remove the line break before "theory".

l.138: "... introduces /the/ timed" (remove "the")

l.139: "..., and *the* theory ..."

l.150: T(C) does not "contain[...] all clock variables", please
reformulate.

l.161: "... is the *set of* initial location*s*. ..."

l.162: "... clock constraint/s/"

l.199: What do you mean "the timed automaton detects a symbolic model
wth cycles"?  Please clarify.

l.248: You put I'(l_0')=\emptyset, but that's supposed to be a clock
constraint.  Do you mean "true"?

l.268f: Same here, both for I'(l_0') and for the guard on the second
added edge.

l.276: "... accelerated automat*a* ..."

l.293f: I didn't understand the sentence starting "The model".  Please
reformulate.

l.376:  Please remove the first "l_reset".

l.376f:  Please clarify your statement by saying "There is a single
acceleratable cycle whose effect is ...".

l.383: The condition "a_j+a_k\ge \forall(a_j, a_k)" is mathematically
meaningless.  Please reformulate.

l.404: Please remove the first "l_reset".

l.415: Please remove the first "l_reset".

l.421f: Again some strange occurrences of \emptyset.  You probably
mean I'(l_reset')=true, but on the edges in l.422 you twice put a
*label* \emptyset.  Do you mean \epsilon here?  (There is one more
\emptyset which should be "true", too.)

l.472: "..., from *the* timed ... from *the* accelerated"

l.503: What do you mean "overcome the parallel composition"?  Please
clarify.

l.566: What do you mean "the number and scale of access number"?
Please clarify.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EXACT ACCELERATION OF COMPLEX REAL-TIME MODEL CHECKING BASED ON OVERLAPPING CYCLE
Review round: 2
Reviewer: 2

Output:
Basic reporting: #1 Basic Reporting
As I have already this paper, and my previous comments about the
general structure of the paper and readability  of the paper remains, I will
not discuss too much here.

The paper is still well-structured and remains readable. This said, I
still find Definition 3 hard to read. The paragraph
afterwards does however help in explaining better the window of an
acceleratable cycle.
Experimental design: One of my comments from my last review was that the papers made claims
about the memory consumption, that was not visible in the paper. In
the rebuttal they comment on this, and I believe they are correct: the
problem is that software does allocate memory in chunks from the
operating system , and so the atual actual higher memory
consumption would not be visible between the two techniques. As the
authors are aware of this, I think it would have been adequate to
mention how memory was measured - and that accelerating cycles does
not (necessarily) reduce memory consumption (it could even increase it
based on the rebuttals mentioning that the state space will  increase
due to adding extra locations?).
It does indeed reduce time (no doubt about it), but I keep pushing on
the memory because the authors in the introduction (page 7 line 83)
say Exact Acceleration can reduce required storage space. This is
not clear in the experiments, and they contradict this statement in the
rebuttal. Admittedly, Uppaal (and other model checkers) will usually
not generate the full state space during a search, and so the state
space actually searched with acceleration might be smaller.
Validity of the findings: The paper has made up for the lacking definition of continuous zones -
basically it means that there is an overlap of the visited clock zones in a
given location after repeating the cycle.

Now, In my previous reviews
I said I was unable to validate the the correctness of the paper, as
it hinges on the correctness of Theorem 2 (now Theorem 3). I am
pleased to see, that the authors have updated this proof - but my
objection remains: they set out to prove there exists an integer $n$
such that the clock zones becomes continous afte $n$ iterations of the
cycles. In the proof they start out by stating: "According to the
forward reachability analysis, the problem of  fragments in the
acceleratable cycle will inevitably lead to the overlap of clock
zones, that is the appearance of continuous clock zone. So the point
of the proof is to determine the value of the positive integer, that is,  when
there is a continuous clock zone" and later "At the reset location, we
assume that the clock zone is continuous from the (n+1)th time onward... essentially stating that their
Theorem is correct, and they just have to determine the value of $n$ (hence my comment about assuming the correctness of their own
Theorem).
Additional comments: As you can see, I'm still not convinced about the proof of Theorem 3. I find the result plausible, but would still recommend a revision of the proof.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors compare many different clustering and classifier algorithms for identifying 11 types of cancer on a microarray dataset. Very few references are given to the previous applications of machine learning to microarray data, which is a vast literature. The exact contribution of the study, compared to other works, is also unclear. Many authors have compared different machine learning techniques on microarray data. Deep learning has also already been used. What exactly is new in this study? The main motivation provided in the abstract is that there have been "advances in both hardware and software technologies", which is not enough to motivate the analysis. I have also some concerns regarding how the experiments were carried out (see below). Finally, the language needs some improvement. Due to the aforementioned reasons, I do not recommend publication in PeerJ Computer Science unless there are some major improvements to the manuscript.

- Most of the introduction is spent talking about Deep Learning, supervised vs unsupervised learning and machine learning in general. Prior work on machine learning methods are note discussed, only briefly mentioned. The authors should include a literature review section describing previous works in order to provide context for their study.

- Almost no description is given about the dataset. What does the data represent (besides gene expression)? What is each of the 174 samples? What is the number of objects in each class (this is only provided in table 4 in the results)? Is this data unique in some aspect? Besides, the provided link (http://www.gems-system.org) is not working.

- The authors should provide a brief explanation about each algorithm. Some algorithms (and parameters) are more suited for unbalanced classes, for high dimensional data, etc. The authors need to provide some prior information on what is expected from each algorithm for this particular data.
Experimental design: - The authors mention that they used a training, validation and test set. It is not clear if the test set was used only after tuning all parameters of the methods on the training and validation sets. Actually, looking at the source code it is not clear that a test set was used at all. If the parameters were tuned and the accuracies were quantified using the same data, even with cross-validation, there will be overfitting. The test set should be set aside and not be used in any of the training/parameter tuning steps. They are used for testing the final performance of the methods only after tuning all parameters in the training/validations sets. Alternatively, a nested cross-validation scheme might be used (one for measuring the accuracy during training and another for the final accuracy after parameter tuning).

- The author mention that "the Ward method is the one with the best performance". This is based on which criteria? The Ward method can find "well-separated" clusters even in normally distributed data. Just because the method managed to find clusters it does not mean that they are the correct clusters, or that the data has any cluster at all.

- Why are CNNs being used?  Do the order of the rows and columns of the gene expression table have a special meaning? In my experience with this type of data, usually do not have any particular meaning, that is, either the columns or rows can be shuffled without loos of information. If there is no spatiality in the data, it is difficult to see why CNNs are useful here.
Validity of the findings: - See above for the validity of the findings, which depends on the experimental design.
Additional comments: - The authors wrote

"of each algorithm to find the “real” accuracy. Additionally, "

Please define what is the meaning of "real".

- Figure 2 is mentioned before Figure 1

- The DL architecture used is only presented on the last paragraph of the results section. It should be placed before the results.

- Many English mistakes in the text. Some examples:

- since allows an improved

- and guaranteeing the effectiveness and less side effects

- ML and DL techniques are entitled to learn characteristics of

- using K-means algorithm varying amounts of cluster between one and eleven using all datasets

- a sequential process composed by choosing values of parameters which achieved best accuracy and then used it to found others

-  except RF and DT due to more than one hyperparameters were tuned.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 1
Reviewer: 2

Output:
Basic reporting: The article presents some problems with the English language. There are wrong or incoherent expressions in many parts of the article. These wrong expressions need to be edited, and they can be found as follows:
- Abstract: lines 22, 24, and 33
- Introduction: lines 40, (42,43,44), 65, 72, and (74-75)
- Preparing the data: lines (118,119)
- Unsupervised learning experiments: lines 123, 132, and 134
- Results: 172, 197, (213,214), (218,219)
- Conclusions: 288
- Figures 2 and 6: caption

It would be interesting to explain a little better the motivations of the authors for the task of cancer types classification; in my opinion, the explanation was a bit simple. Also, you should mention the importance of using machine learning and deep learning methods.

You described a few related works about the task you are researching. You should mention more works that developed the task of cancer types classification for both machine learning and deep learning methods. Also, you should mention more works that have used the "11_tumor" database.
Experimental design: You should explain with more detail the database you are using (11_tumors database), maybe with some examples of how a gene is represented in such a database.

In the unsupervised learning experiments section, at line 131, you should explain a little each of these metrics, but the most relevant ones. It would be important to include some references.

In tuning the algorithms section, at lines 155 to 158, you should improve the writing and explanation of the steps for finding the best value of parameters
Validity of the findings: no comment
Additional comments: The article is very interesting and the authors developed many experiments and results for the task of cancer types classification. It was relevant that you addressed several machine learning approaches like supervised and unsupervised classification, and also deep learning methodologies. However, there are some mistakes that need to be taken into account and corrected so that the article can be accepted. These issues are detailed in the Basic Reporting and Experimental design sections.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 1
Reviewer: 3

Output:
Basic reporting: The authors present a comparative study of several supervised, unsupervised and deep learning algorithms to understand which the best algorithms are to deal with microarray 11 tumor database.

The article is well written but the introduction needs more details. The related works that were described in the introduction were superficially approached and many of them had no direct relation to the manuscript research. I suggest creating a new section of related work.
Experimental design: There are related works such as Machine Learning in DNA Microarray
Analysis for Cancer Classification who makes an analysis similar to the analysis presented in the manuscript. What is the contribution of this manuscript to others?

No statistical analysis was mentioned in the manuscript.

Why was chosen PCA instead of applying feature selection techniques? What were the motivations that led to this decision?

What was the reason for choosing the algorithms that were used in the comparative analysis? The text did not mention the reason.
Validity of the findings: The link provided to access the database does not contain the database. The authors should verify that the database is publicly available.

The text describing the result (page 11) is not well detailed. For example, indicate why some techniques obtained a high accuracy when applying PCA and scaling and others not.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors answered all of my queries. Therefore, I consider the manuscript suitable for publication in PeerJ Computer Science.

Nevertheless, there are many grammar mistakes remaining in the text.
Experimental design: No comment
Validity of the findings: It is still unclear if the CNN is overfitting to the data, since the hyperparameters were tuned on the same set that was used for accuracy calculation. Still, the results show that the model have enough capacity to reach high accuracies.
Additional comments: No comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 2
Reviewer: 2

Output:
Basic reporting: In the earlier version of the manuscript, the main contribution of the work was missing. The lack of an adequate bibliographic review describing the related works to contextualize the problem and the poorly detailed way in which the study was approached decreased the relevance of the work, no showing any kind of contribution. In this new version of the manuscript, it is possible to see the contribution of the work indicated in paragraph 3 of page 6 and the last paragraph on page 7. Although the contribution of the work is now visible, the authors have very difficult detailing their explanation. These issues are detailed in the Experimental design sections.
Experimental design: Next, I will explain some issues related to the text: For example, in paragraph 3 of page 6, it is mentioned by the authors that several other works applied machine learning techniques to the 11-tumor dataset, but they only used some machine learning methods, one preprocessing strategy and one learning technique (supervised or unsupervised). The information is very vague. Should be mentioned these methods. What preprocessing technique was applied? What unsupervised learning algorithms the authors use? What did these authors conclude? Is there any relationship between the results obtained from this manuscript and the other works, or not? All of this information is relevant because it will validate the study done in this work.
Validity of the findings: When a comparative analysis is done, it is expected to see the statistical significance of comparing the algorithms used. For example, if one algorithm had an accuracy rate of 90% and another 92%, we could say that this difference was statistically significant or not? I was hoping to find this statistical analysis in the text.
Additional comments: 1. As there were several changes in the text, the abstract must be updated to reflect the clarifications made by the authors on the contribution and the results.
2. About my question of why a feature selection method had not been used I understand that feature selection was not considered by project scope. Only I want to clarify to the authors that PCA is not a feature selection, is a dimensionality reduction algorithm that performs a linear transformation of the data into a lower dimension. On the other hand feature selection select and exclude features keeping the features intact with any kind of transformation.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 3
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: All previous suggestions were covered, therefore  I consider the manuscript suitable for publication in PeerJ Computer Science.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A COMPARATIVE STUDY OF MACHINE LEARNING AND DEEP LEARNING ALGORITHMS TO CLASSIFY CANCER TYPES BASED ON MICROARRAY GENE EXPRESSION DATA
Review round: 3
Reviewer: 2

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: No comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SURVEY ON EXPONENTIAL RANDOM GRAPH MODELS: AN APPLICATION PERSPECTIVE
Review round: 1
Reviewer: 1

Output:
Basic reporting: Is the review of broad and cross-disciplinary interest and within the scope of the journal?

Yes.

Has the field been reviewed recently? If so, is there a good reason for this review (different point of view, accessible to a different audience, etc.)?

Yes, but focuses on the technical perspective, while this emphasizes more on applications.

Does the Introduction adequately introduce the subject and make it clear who the audience is/what the motivation is?

Yes.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: This is a review article on exponential random graph models and their applications. I have a few points for the authors to consider.

As the endogenous factors need to be selected before implementing the model, it might be also helpful to introduce some basic considerations when making hypothesis about the theory. For instance, why would a user of ERGM incorporate covariates, e.g., k-star, edge-wise shared partners, and four-cycles, into their model? What statistical effect, significantly postive or negative, will that covariate have on the outcome network? The authors should emphasize the importance of this precedure when choosing endogeneous network effects because they are inherently different from the control variables used for classic statistical models such as logistic regressions.

Authors should also have a more elaborate discussion on recent advances of ERGM, such as temporal ERGM, weighted ERGM, et al., and their potential applications in science.

A couple of other papers on the application of ERGM:

Ecosystem assessment: Leifeld, Philip, and Dana R. Fisher. "Membership nominations in international scientific assessments." Nature Climate Change 7.10 (2017): 730.

Environment: Campbell, Benjamin W., et al. "Latent influence networks in global environmental politics." PloS One 14.3 (2019): e0213284.

Arms trade: Thurner, P. W., Schmid, C. S., Cranmer, S. J., & Kauermann, G. (2018). Network Interdependencies and the Evolution of the International Arms Trade. Journal of Conflict Resolution, 0022002718801965.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SURVEY ON EXPONENTIAL RANDOM GRAPH MODELS: AN APPLICATION PERSPECTIVE
Review round: 1
Reviewer: 2

Output:
Basic reporting: The survey is not written in professional English so it has to be re written in most parts. References are covering most of the literature but both theoretical and application context can be explained better.

Instead of citing:
- Caimo A, Friel N. 2012. Bergm: Bayesian exponential random graphs in R. arXiv preprint arXiv:1201.2770.

The authors should cite:
- Caimo, A. and Friel, N. (2014), “Bergm: Bayesian Exponential Random Graphs in R,” Journal of Statistical Software, 61(2), 1 – 25.

Regarding missing data. There are a few more references that can be included:

- Krause, R. W., Huisman, M., Steglich, C., & Sniiders, T. A. (2018, August). Missing network data a comparison of different imputation methods. In 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (pp. 159-163). IEEE.

- Krause, R. W. and Caimo, A. (2019), “Missing Data Augmentation for Bayesian Exponential Random Multi-Graph Models.” International Workshop on Complex Networks, 221, 63 – 72. Springer.
Experimental design: The Section "Computation of prior probability" is not clear and correct. I guess that section should be focusing on Bayesian methods where the target is the posterior distribution.

The section Classic methods also is not clear. This should be Maximum likelihood estimation and should come before the Bayesian section. The estimation methods should be explained in more details especially regarding the MCMC-MLE and network sampling algorithms.
Validity of the findings: The survey is timely and potentially very useful but can be improved in terms of explanation of the results obtained in the literature. Model assessment methods are not present, they should be included. Conclusions should include more discussion of the future directions / challenges.
Additional comments: The survey must be improved a lot in order to be publishable. The text is often not clear or very difficult to understand for those who are not expert in this research field. Much more explanations and details are needed about the practical implementation of the ERG models.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SURVEY ON EXPONENTIAL RANDOM GRAPH MODELS: AN APPLICATION PERSPECTIVE
Review round: 1
Reviewer: 3

Output:
Basic reporting: It is suggested to add some tables, to compare each method in detail and make the article more clearer.
Experimental design: It is suggested to cite some studies about engineering. There are some outstanding research by using ERGM on communication, electrical engineering, and so on, done by engineers.
Validity of the findings: There are no clear conclusion presented by authors. It suggested to add some conclusion about what are the appropriate methods for different problems.
Additional comments: The article is clear in logic and accurate in language. It is suggested that engineering-related reviews and citations be added.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A NEW NON-MONOTONIC INFEASIBLE SIMPLEX-TYPE ALGORITHM FOR LINEAR PROGRAMMING
Review round: 1
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: Validity of the findings seems right.
Additional comments: In the future work:
Authors should choose the data set that has the following properties:
1- The benchmark problems must be varied according to size
2- The benchmark problems must be varied according to the sparsity because most of the used problems are dense problems.
3- The data set of the benchmark problems must has a large number of problems.
On the other hand, the CPU time of the proposed algorithm must be manipulates.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A NEW NON-MONOTONIC INFEASIBLE SIMPLEX-TYPE ALGORITHM FOR LINEAR PROGRAMMING
Review round: 1
Reviewer: 2

Output:
Basic reporting: Line 128:  The meaning of the sentence “…should the initial direction be feasible, can be applied directly on the original problem, without having to  first compute an initial feasible basic solution thus completely avoiding Phase I.” is not clear to me. I do not understand what the meaning of feasible direction is. The condition of feasibility applies to points, not directions. Perhaps this sentence should be rewritten.

More on the paragraphs from line 124 to line 150: the intention of these paragraphs is valid. Here the authors establish a quick comparison among the EPSA, the iEPSA and the PDIPSA. The reading is attractive and invites for details. But the writing is not clear. I think these three paragraphs are crucial to begin the explanations of the proposed algorithm. In line 127, for example, the word “another“ suggests a previous advantage was mentioned. It was not. The sentence starting in line 137” The iEPSA is initialized with an infeasible basic solution and with an interior point” seems to be a contradiction as it is written. Or maybe I do not understand the exact meaning the authors want to convey. If there are two point to initialize this algorithm it should be clearly stated. If there is an error in the sentence it should be corrected.

The terms αα and ββ are extensively used, but nowhere is indicates their meaning.
Experimental design: The authors try to explain the method using an example as if an example guaranteed the functioning of a mathematical procedure. They even inappropriately use the term “demonstrate”. This would not be a major problem if changing some words could fix it. But the explanation using this example is poorly organized. I just cannot follow it. Much less understand the method and its rationale. It may even have serious algebraic errors! If you compute A_B  ∙ 〖A_B〗^(-1) using the values presented in the example, it does not yield the Identity matrix I. There are things here that not corresponding to the textual explanation. I suggest explaining the method in a logical and general manner, and then, illustrating it with an example. Moreover, why using eight constraints? Is not it unnecessarily “large” example?

In line 165 you say: “The N set of indexes actually represents the current basic solution.”. But previously you used the letter N precisely to refer to NON-basic solutions. This is impossible to understand.
Validity of the findings: There seems to be an interesting idea about exploiting the behavior of linear programming in the feasible and unfeasible spaces, to accelerate the convergence of the method towards the optimal solution. However, the explanation offered in this manuscript lacks of the organization needed to evaluate the plausibility of the method. In the conclusions the authors signal as the advantage of the method the reduction of the number of iterations of their method, not mentioning the fact that, according to the algorithm presented, they need to compute the insertion of matrix A_B at every iteration.  Thus, it is difficult to believe the offered method actually represents important advantages.
Additional comments: This is not to disqualify the study. I insist, the idea of combining interior and exterior point methods could be promising. But it has to be better organized, evaluated and explained.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A NEW NON-MONOTONIC INFEASIBLE SIMPLEX-TYPE ALGORITHM FOR LINEAR PROGRAMMING
Review round: 1
Reviewer: 3

Output:
Basic reporting: As authors state, the proposed algorithm is comprised of three different parts : i) interior Exterior Primal Simplex Algorithm (iEPSA), ii) Exterior Point Simplex Algorithm (EPSA) and iii) Primal-Dual Interior Point Simplex Algorithm (PDIPSA). The diagram of the proposed algoriithm (iEPSA) is provided in Figure 1 ad Algorithm 1. This algorithm applies EPSA after finding a primal feasible basic solution and applies PDIPSA when moves to a dual feasible partition.

The language is clear and unambiguous, but there are some partial sentences such as ; at line 246 where the sentence lacks some reference to a section. Authors provide sufficient lemmas/theorems. However, numbering of theorems and lemmas should be separated from each other.

The structure of the paper should be improved. It is hard to follow the paper;

The diagram of the iEPSA is given on page 8 and afterwards an example is provided to explain the inner wokings of the algorithm, which is nicely presented. However, the formal definition of the algorithm is given on page 15. This makes it very difficult to follow the algorithm. The diagram in figure 1 and the algorithm 1 should be given close to each other. Also, before moving to the example, the authors should explain the formal definition of the algorithm.
Experimental design: The authors sate that the proposed approach aims to reduce the number of iterations. They also compare it with CPLEX and Gurobi in terms of number of iterations and CPU time. They benchmark it on netlib, Kennington, Mesz´aros.
Hwever, the proposed algorithm is implemented in Matlab. Others are implemented in C. This makes the CPU time comparison not very meaningful.
Validity of the findings: - Since the proposed algorithm is implemented ina different language, then authors should comment on the iteration times of CPLEX, Gurobi and the proposed algorithms. Since, it is not possible to comment on the superiority of the algorithm in terms of the runtime.

- After reporting the number of iterations and CPU time, they only provide the percentage of the experiments the algorithm is better in terms of number of iterations. Some more deeper analysis should be provided such as; the effect of the size or the sparsity of the problem.
Additional comments: The structure of the paper should be improved in order to follow work. Also the algorithm should be explained line by line if possible.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A NEW NON-MONOTONIC INFEASIBLE SIMPLEX-TYPE ALGORITHM FOR LINEAR PROGRAMMING
Review round: 2
Reviewer: 1

Output:
Basic reporting: No problems observed here.
Experimental design: No problems observed here.
Validity of the findings: No problems observed here.
Additional comments: The author has successfully answered and explained all issues I commented in the initial review. After considering these comments, and the others reviewers  comments, this manuscript is in my opinion valid  for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS A BLOCKCHAIN-BASED CERTIFICATE AUTHENTICATION SYSTEM IN VIETNAM
Review round: 1
Reviewer: 1

Output:
Basic reporting: This study aims to solve the problem of forgery of certificates / graduations, a social problem in Vietnam, through integrated business services that authenticate and verify between government agencies, various educational institutions and participants through blockchain technology.
Experimental design: In the data structure, the common data schema that serves as the interface between the client's input data and the blockchain network's assets is designed to secure interoperability that can be linked with other blockchain networks in the future.
Validity of the findings: However, for the performance measurement results, the network performance measurement results according to the message size in the block, the number of transactions, the throughput statistics according to the number of virtual processors and the memory, and the latency statistics were presented and analyzed. It is also necessary to evaluate the performance according to the number of transactions, and there is no analysis in terms of services. In addition, although the consensus model is simple unlike the open blockchain due to the nature of the licensed blockchain, the proposed service does not know how to implement the consensus model by performing the performance evaluation without the specific consensus model. Finally, the proposed service is questionable whether it can be called a blockchain in that the state (Ministry of Education) has control over blockchain network configuration nodes or participants (recentralization) through a licensed blockchain.
Additional comments: Please make revisions as indicated


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS A BLOCKCHAIN-BASED CERTIFICATE AUTHENTICATION SYSTEM IN VIETNAM
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper proposes a blockchain-based certificate authentication system for managing all the educational diplomas in Vietnam. Additionally, this system is deployed using Hyperledger Fabric blockchain infrastructure and the paper presents experimental performance results based on this infrastructure. Apart from this core contribution of this paper, it also summarizes state of the art of blockchain technology in different aspects.

Overall, the paper is very well written and presents interesting and technical sound results. There are only some minor comments that could help the authors to improve the presentation of their paper:

1) In page 6 lines 234-247, the contribution of the paper could be moved in the Introduction section and somewhere at the end of the section (page 2, line 74-75).

2) It is good to mention, possibly in the Conclusion section, that the proposal system is generic and it can be applied to any country and not only in Vietnam.

3) The Figures 7, 8 and 9 should be resized because the text on them is very small for the readers.

4) In Tables 1 and 2, the numbers 2, 4, 8, …, 512 are huge (font size) compared to the rest of the text in them.

5) The authors, if they want, could also add a comment about the cost of Hyperledger Fabric per school or university (or even for MOET) and if it is still expensive to apply a blockchain infrastructure in a real setting.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS A BLOCKCHAIN-BASED CERTIFICATE AUTHENTICATION SYSTEM IN VIETNAM
Review round: 1
Reviewer: 3

Output:
Basic reporting: The paper presents an interesting application of the blockchain technology to provide notarization and transparency for skills certificates, and I enjoyed reading it.

The authors focus on the Vietnamese context and aim at providing a solution to the anticounterfeit of school and academic certificates. One of the most exciting aspects of the paper is the idea of using the blockchain not only to notarize and provide transparency related to the certificate but also include the learning process. Including the learning process permits to provide a logic for automatic issuing of the certificate once all the courses have been passed and certain conditions are met.

The background to show the context is clear, and it is mainly focused on the Vietnamese education system. It is not clear why the authors focused on the Vietnamese case. Although Vietnam has some conditions that make the system more appealing and useful, we consider that the tackled problem is general and can be considered world-wide, despite in some nations is less relevant.

The literature references are sufficient, and there are more than 10 references related to the use of blockchain for skill certification. The context is clear, and there are several references to the specific Vietnamese condition.
Experimental design: As for the results, the information provided by Fig. 7 appears to be completely independent of the application and from the blockchain as it is related to pure TCP performance. Furthermore, it is not clear what are the conditions on which the measures are taken, is there other traffic or it is only the traffic of the network_? What are the TCP parameters? Which TCP version?
Validity of the findings: The main contribution of the paper is related to the proof of concept of the application of blockchain for certifying skills.
Additional comments: The paper describes the steps for designing a blockchain-based system for providing transparency and trust in handling certifications of skills. However, the actual implementation details are not provided. The authors state that one of their contributions is the analysis of blockchain situations in the aspects of technologies, classifications, functionalities, research trends, and applications. Still, the paper has not the structure, nor the broad vision of a review paper, despite it clarifies several exciting elements, especially those reported in Figure 1.

Figure 1 is interesting and provides a proper classification of blockchain technology. Still, some elements are not consistent (e.g. a classification by chain openness contains a single chain, cross-chain and side chain, which is more related to the number of blockchains involved and their connections).
Furthermore, DAGs are reported as a kind of consensus, while they refer to the data structure, and we suggest removing ‘others’ from the figure, except there is a specific element to add.
Under the element ‘signature’, we would recommend considering single and multi-signatures.

Figure 2 reports the four-phase design of the proposed approach. It is not clear what is the trust model, especially regarding the validation phase. If the validation phase is just a formal validation of compliance to a specific data-model, it is ok and easy to perform. How are the skill certificate validated? The scholars can be untrusted, but what about the schools/universities/ private bodies? Should they be trusted as data flow in their database before being written in the blockchain? What it happens if a novel certificate 'appears' on the database? Is that included in the blockchain? What is the validation policy? Are the databases of the schools considered trusted?

Figure 4 reports an ‘issue certificate’ action. Is the certificate in the form of a document that can live outside the blockchain? In this case, where is the trust? I understood the certificate is issued on the blockchain, so what does this arrow represent?

Figure 6 shows only one ordering service. Please, consider showing more ordering services form multiple institutions and the message exchange between them.

Figures are relevant, high quality but are not always well-described. Furthermore, their relevance is reduced by the companion tables, which contain the data that are plotted in the figures. For example, table 1 and 2 provide the same information as Figure 8, tables 3 e 4 present the same results of Figure 9. We advise using tables or figures but not both.

The paper contains the sentence “Since a private blockchain network is used to deploy VECefblock". Private or permissioned? Furthermore, transactions are said to be validated by educational institutions before the Ministry decision. What is the Ministry decision? This part is not clear. If the Ministry can decide what can be written on the blockchain, then it is a central trusted authority, and a database should be used instead of a blockchain. Is the Ministry only able to decide which are the recognized educational institutions that can write on the blockchain?
It is not clear how the designed system prevents the ‘garbage in garbage out’ problem. What if the educational institution is not trusted and ‘sells’ a certification? Please, consider explicitly stating if the educational institutions (including the private ones) are trusted or not.
To ensure the uniqueness of the learnerID, the authors suggest hashing “the learner ID, institution name, timestamp (at the time of registering ID data) is hashed together as the identify key of learner in blockchain network”. So, there is a learnerID before the hashing? How is this id provided? Furthermore, adding the timestamp related to the student registration may result in a duplication of entries for one student. For example, a student is identified today in a school has an id H1 H(studentX | timestamp_today) and another id H2 if it is registered tomorrow H(student | timestamp_tomorrow). This method could provide a situation where one student is identified by multiple IDs, which is not what is generally desired.

The English used in the paper is good, but some sentences could be improved (e.g. the sentences highlighted in purple in the attached document). Some typos (e.g. Crytography) and grammatical mistakes are present, and some sentences are difficult to read. Furthermore, please, consider substituting the term 'blockchain engine', with the blockchain platform. In facts, this wording may be misleading, even if it applies to specific cases (e.g. where the blockchain is based on virtual machines), in general, it is not a valid wording.

As for the results, the information provided by Fig. 7 appears to be completely independent of the application and from the blockchain as it is related to pure TCP performance. Furthermore, it is not clear what are the conditions on which the measures are taken, is there other traffic or it is only the traffic of the network_? What are the TCP parameters? Which TCP version?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS A BLOCKCHAIN-BASED CERTIFICATE AUTHENTICATION SYSTEM IN VIETNAM
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors should go through paper thoroughly and check for spelling and grammar issues for English as none of the authors are native English speakers.

Some key references are missing, please try to add some recent findings in this area from 2019 and/or 2020.
Experimental design: It is improved from the original version
Validity of the findings: Seem fine to me, better than original paper
Additional comments: Overall paper is improved and seems to be better than original. All my concerns were addressed in the revision as I was an original reviewer


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS A BLOCKCHAIN-BASED CERTIFICATE AUTHENTICATION SYSTEM IN VIETNAM
Review round: 2
Reviewer: 2

Output:
Basic reporting: The current version of the paper is ready for publication.
Experimental design: No comment
Validity of the findings: No comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A VITERBI DECODER AND ITS HARDWARE TROJAN MODELS: AN FPGA-BASED IMPLEMENTATION STUDY
Review round: 1
Reviewer: 1

Output:
Basic reporting: The article is well written and contains the appropriate amount of information. However a thorough literature review / background section would be welcomed. This would allow the reader to position the paper within the field and understand what was done before and what are the contributions of the paper.

As a broad example, the work discusses a 3% increase in heat due to the trojan, a brief search on google scholar discusses the detection of trojan using off-the-shelf work to detect trojan using heat cameras.
Experimental design: The paper provides information about  the design and experiment, and while the authors provide information about the creation of the trojan,  other papers have been discussing the creation of trojan for this type of work and it would be good to see the decision process taken throughout the paper.

Furthermore, the authors should clarify their decision of designing the trojans themselves rather than using known trojan as the performance degradation are interlinked with the design and choices made for the trojans.
Validity of the findings: Generally, the paper provides a good overview and description of the work, however the authors should provide a longer discussion on the effects of the performance degradation due to the type of trojan being used to disrupt the initial design.

The conclusion is well written and provides enough information to summarise the work undertaken  However, the authors should identify some future work as well as the limitations of their work.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A VITERBI DECODER AND ITS HARDWARE TROJAN MODELS: AN FPGA-BASED IMPLEMENTATION STUDY
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper presents an FPGA implementation of Viterbi Decoder and tests the effects of hardware Trojans on coded communication systems. There needs to be a solid connection between the introduction of the hardware trojans and the actual trojans that this paper implements.
Experimental design: One question of the experimental design is that in figure 20 to 22, why in lower SNR, the BER is even lower with trojans?

Second is that the zip file provides not much directly useful message. Find another way of delivering the experiment result.
Validity of the findings: One suggestion is that more newer and closely-related works are needed to be explored and the direct comparison is needed.
Additional comments: Has the potential to be accepted if solved the above-mentioned issues.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A VITERBI DECODER AND ITS HARDWARE TROJAN MODELS: AN FPGA-BASED IMPLEMENTATION STUDY
Review round: 1
Reviewer: 3

Output:
Basic reporting: Manuscript is quite comprehensible. References are appropriate.

Background material on hardware trojans is
adequate. There are minor grammatical
issues -

Example: line 35: "A hardware of a communication system" to "The hardware of a.."
line 36: "This will allow.. " to "This allows"
Please add a paragraph to explain Figure 1 that describes
classification of hardware trojans.

Authors also assume familiarity with viterbi decoding.
For example: line 99, it is unclear what
"hard decision metric" and "cumulative branch metrics" refer to.
"The decoder computes the hard decision metric for each branch starting
from the zero initial state. The cumulative branch metrics along with all the possible paths leading to a node or state constitutes the path metric.
Consider elaborating the "viterbi decoder"  section with a simple example to improve clarity.
line 100: At any node the path with the lowest path metric is retained as the survivor path and the
101 rest of the paths are discarded
Please add appropriate punctuation.

Tables and Figures appear adequate.
Experimental design: This work describes a technique to detect the presence of hardware trojans
in communication systems by analyzing bit error rate (BER) of the decoded
data stream. Detecting hardwrae trojans is a difficult problem when
the malicious circuit is designed with small area/power footprint
and therefore, the presented research problem is valid.

The paper assumes trojan is always inserted at the decoder. Why?
Can trojns inserted at the encoder be detected? and if yes how?
Please explain in the introduction why the paper focuses on trojans inserted
at decoder.

Please make it clear how this work is different from your prev. conference submission.

Authors have provided a thorough illustration of experimental setup, decoder block diagram etc. enough to reproduce results.

The main conclusion from this work appears to be that hardware trojans
have a distinct BER signature (high BER). But, the presence
of this signature doesn't guarantee the existence of a hardware
trojan. For example, high noise in the channel can also
result in high BER which raises the question - if
a channel's noise characteristics are unknown,
how do you conclude the presence
of a trojan purely from BER analysis?
Validity of the findings: Results need more data and explanation.

1. line:171 mentions the BER drops to 0 for SNR > 6db. But, the plot in Fig 14
doesn't report BER for SNR > 6dB. Also, BER for 6dB
is shown as 10^-4 which isn't zero. Please provide complete
data and fix description to match the plot.

2. Plots 14 and 15 can be combined to better illustrate that FPGA measurements
match matlab model.

3. Figure 20 is incomplete since it doesn't report BER results without trojan for SNR > 6dB.
Please fix.
Additional comments: Overall, well written paper.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A VITERBI DECODER AND ITS HARDWARE TROJAN MODELS: AN FPGA-BASED IMPLEMENTATION STUDY
Review round: 2
Reviewer: 1

Output:
Basic reporting: Grammatical issues have been addressed. References are appropriate.
Please remove references to supplemental figures, unless they may be included in the final version of the manuscript.
Experimental design: Concerns have been addressed.
Validity of the findings: No comment
Additional comments: My concerns have been addressed. Please remove references to supplemental figures, unless they may be included in the final version of the manuscript.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A VITERBI DECODER AND ITS HARDWARE TROJAN MODELS: AN FPGA-BASED IMPLEMENTATION STUDY
Review round: 2
Reviewer: 2

Output:
Basic reporting: This paper demonstrates the implementation of hardware trojan in a Viterbi decoder implemented by FGPA. They have shown the changes in LUT used and power dissipation with trojan inserted. Meanwhile, they also demonstrated the BER drops with the presence of Trojans.

Strengths:
1.	The author gives an overview of the hardware trojans and more specifically how the trojans work in the channel decoders.
2.	This paper tests the effects of BER dropping with the triggered Trojans.
Experimental design: However, I think the contributions and the novelty of this paper are not very promising and clear to me.

1.	First of all, from the title and abstract, I get the main theme for this paper is discussing how hardware Trojan attacks are vulnerable to the Viterbi decoder. However, the structure of this paper makes the theme not clear. This paper uses more than half of the contents to illustrate the process in designing the Viterbi decoder, which I don’t think it aligns with the main theme of the title.  I admit that to add trojans, one needs to be 100% clear on the RTL structure of the design, however, too much details on the design but with less than 1/3 on the discussion of the hardware trojan makes the structure of the paper problematic. To make this paper logic acceptable, I would suggest the author shrink the part discussing the design of Viterbi decoder but with emphasis on the hardware trojan.

2.	The author demonstrates the effectiveness of the hardware trojan on the designed Viterbi decoder. I wonder whether the trojan the author proposed are fixed to those structures or they can be universally used in the other structure of Viterbi decoder? If the proposed trojans are just applied on the proposed structure, the work of this paper should be improved by proposing some universal structure which works at least for all the Viterbi decoder.

3.	This paper has some evaluation results on the hardware trojan and performance analysis on the BER drops. However, I think the author might need to add more discussion on the security analysis of the hardware trojan. For example, the author would like to discuss how the triggering success rate for each trojan. Also, it lacks to distinguish the influence of the three different kind of trojan on the performance of the BER drops. Moreover, it would be sufficient if the author evaluates the trojan detection method on his proposed trojans to further confirm that the trojans are feasible and not easy to be detected.
Validity of the findings: Some editorial remarks.
The author should proof-read the whole paper and modify some mistakes in the format. A few examples are incorporated but not an all-inclusive list.
1 Please scale figure 11 and figure 14.
2 please use the same font size for the Table 1 and Table 2. The size of the column “shift modification trojan” is larger than the other columns.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN APPROACH FOR SEMANTIC INTEGRATION OF HETEROGENEOUS DATA SOURCES
Review round: 1
Reviewer: 1

Output:
Basic reporting: The introduction and related work sections contain excessive background. As explaining concepts like heterogeneity and mappings are required, I would suggest a specific section for previous concepts, so that the related work is focused on existing integration approaches only. With less concepts, the introduction would better highlight the context, motivation and contribution of the paper.

Similarly, the abstract should be extended to describe the key characteristics of the approach, e.g. global and local views and the use of ontologies, and how such a solution constitutes a progress in the field.

The text of some figures (e.g. Figures 5 and 6) is difficult to read. Please increase size or resolution. Figures should be located close to the text describing them (e.g. Figures 4 and 5).

Table 2 is not referenced in the text.

Avoid the use of contractions (don’t)
Experimental design: The paper does not formulate neither explicit nor implicit research question. The introduction should formally state the research problem and briefly mention how the proposed system and the experiments effectively solve it.

The methods for relational data sources integration, i.e. R2RML and axiom mappings, should be explained in more detail. Similarly, the annotation activity is mentioned as a necessary step in the case study, but I cannot find a subsection devoted to it in section 4.
Validity of the findings: The authors present a case study, which is valuable, but from which general conclusions are more difficult to extract. Besides, some of the decisions taken during the example transformations are not justified: 1) how thresholds are set and why they change when a third source is included (only the value of beta is mentioned); 2) some mappings are manually removing, but it would be difficult for the user to know when or why these relations should be discarded. Due to the focus on a case study, the authors should go into the details. Some guidelines in this regard would be highly appreciated.

Additional questions regarding the experimentation are: 1) how the system solve or detect a query that cannot be translated into one of the sources? E.g. the local ontology does not contain the information asked by the user; and 2) Time overhead (or estimations) in each of the phases of the approach could be reported, since the approach performs multiple transformations. Also, scalability tests could be planned to study the influence of the number of instances in more realistic contexts.
Additional comments: The paper presents a data integration system for heterogeneous data sources that combines the benefits of the two traditional approaches, namely global-as-view and local-as-view. The approach is based on the use of ontologies at both global and local levels. Mediator elements adapt queries from the global view to the local views. After describing the components of the architecture, a case study in the area of medical information is presented.

The paper provides a solid background on data integration and its related issues, but at the cost of late introduction to the scientific contribution. Many technical solutions might exist for data schema and mediation, so the authors should clearly motivate why there is a scientific problem here and why ontology-based solutions are the most suitable way to address it.

A fully operative system implementing the approach seems not to be available yet. Although it should not be a problem to report inconclusive results in PeerJ, I encourage the authors to provide access to some prototype or controlled example to reach a wider audience.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN APPROACH FOR SEMANTIC INTEGRATION OF HETEROGENEOUS DATA SOURCES
Review round: 1
Reviewer: 2

Output:
Basic reporting: The manuscript is overall well written and it has merit for publication
Experimental design: There is no technically a experimental evaluation but a presentation of a case study.
Validity of the findings: Better comparison with related approaches would be required
Additional comments: The abstract is too short and simplistic.
Authors should extend it and highlight the contribution of this paper.
Authors should avoid the excessively use of bullet points in the manuscript.
There is a broad but not too deep coverage of the state of the art and related technologies, a more in depth analysis of existing approaches and clarifying how the proposed methodology overcomes the limitations of the state of the art would be appreciated.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN APPROACH FOR SEMANTIC INTEGRATION OF HETEROGENEOUS DATA SOURCES
Review round: 2
Reviewer: 1

Output:
Basic reporting: The introduction has been greatly improved. The separation between background and related works makes reading easier. The only issue here is subsection 3.1, as no other subsections appear in section 3. Besides, the discussion is focused on a comparison between GaV and LaV, not specific approaches taken from the literature, so maybe it would be more useful at the end of section 2.

The quality of the figures is better now. However, there are missing references to figures in lines 448 and 473 (section 6). Please check that figures/tables are always written with capital letter at the beginning or fully lowercase (see journal format).

I strongly recommend proof-reading. I have found several typos, such as:
is an hard task => a hard task
It exploit => exploits
is ho to => how
Experimental design: The introduction now refers to the research problem and proposed solution.

The explanation of the method has been properly extended.
Validity of the findings: The authors have made changes to address my comments.
Additional comments: The authors have made important efforts to improve the paper. The contribution and reporting of findings are now better explained. I only have some minor comments for authors to fix before publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN APPROACH FOR SEMANTIC INTEGRATION OF HETEROGENEOUS DATA SOURCES
Review round: 2
Reviewer: 2

Output:
Basic reporting: no further comments
Experimental design: no further comments
Validity of the findings: no further comments
Additional comments: Authors have addressed the comments of the reviewers


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DISCRETE TWO DIMENSIONAL FOURIER TRANSFORM IN POLAR COORDINATES PART II: NUMERICAL COMPUTATION AND APPROXIMATION OF THE CONTINUOUS TRANSFORM
Review round: 1
Reviewer: 1

Output:
Basic reporting: This is a clear and well written manuscript. It has sufficient references. It is self contained with relevant results.
Experimental design: The present paper the authors discuss the computations aspects ıof the two-dimensional Dicrete Fourier Transform (2D DFT) in polar coordinates. They demonstrate how the decomposition of the 2D DFT as a DFT, Discrete Hankel Transform (DHT) and inverse DFT sequence can be exploited for efficient code. It is also demonstrated that the new 2D DFT can be used to approximate the continuous forward and inverse Fourier transform in polar coordinates.
Validity of the findings: Nothing to add.
Additional comments: I  recommend the publication of this article.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DISCRETE TWO DIMENSIONAL FOURIER TRANSFORM IN POLAR COORDINATES PART II: NUMERICAL COMPUTATION AND APPROXIMATION OF THE CONTINUOUS TRANSFORM
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper is well organized.
Experimental design: I do not understand what is actually the difference between the "sampled continuous forward transform" and the "discrete forward transform". There should be no difference.
Validity of the findings: I doubt that these findings can withstand a rigorous check from the mathematical point of view.
Additional comments: Looking at this text and the ones previously published I feel like a beginner. Many, many things are not understood. First of all, I do not understand the motivation for the "Polar 2D DFT" and the way it will outperform the standard DFT. Second, the DFT arises very naturally because functions are truncated in time and frequency which means they are periodized in time and frequency. It leads naturally to the DFT which treats functions as if they were periodic in both domains and because periodization in one domain means discretization in the other domain we actually obtain functions which are discrete in time and frequency, simultaneously. Third, the treatment of the Dirac impulse is not done in the generalized functions sense and I saw many mistakes regarding the Dirac impulse. I do not doubt that these results might be usable in some way but I believe they are mathematically not rigorously deduced. Due to the many indices and technical details, I also doubt that these results will indeed be used by researchers who are willing to understand and use it.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DISCRETE TWO DIMENSIONAL FOURIER TRANSFORM IN POLAR COORDINATES PART II: NUMERICAL COMPUTATION AND APPROXIMATION OF THE CONTINUOUS TRANSFORM
Review round: 1
Reviewer: 3

Output:
Basic reporting: see full report attached
Experimental design: see full report attached
Validity of the findings: see full report attached
Additional comments: see full report attached


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DISCRETE TWO DIMENSIONAL FOURIER TRANSFORM IN POLAR COORDINATES PART II: NUMERICAL COMPUTATION AND APPROXIMATION OF THE CONTINUOUS TRANSFORM
Review round: 2
Reviewer: 1

Output:
Basic reporting: The paper is more or less an extensive technical report on some methods to compute a 2D Fourier transform of continuous functions using  a kind of polar grid.

The discrete transform presented in the earlier paper is implemented and tested on a few cases.

The paper may be encouraging to others to study the weakness or strength of the proposed method.
Experimental design: The method is tested on a few cases, which have known Fourier transforms.

However, also e.g. there is a well known formula for the 2D FT of a shifted Gauss function or of a modulated one, and it should perhaps be checked if for those functions the behaviour is equally valid.

The MATLAB code (I prefer to write this program in capitals) provided appears to be quite complicated.
The matrix established by   theta-matrix  could be obtained much more straight (avoiding a double loop) by the simple command sequence
b2 = 2*pi/N2;
progr2= -pi + b2/2 : b2 : pi - b2/2;
theta = progr2(:)*ones(1,N1-1);
The psi-matrix is the same as the theta-matrix, so no separate code is required.
The two  line code
gau = @(x)  exp(- pi * ( x).^2);
f = gau(r);
should be a more elegant way of producing the sampling values, since it allows to apply the function command to a matrix of positions to get the matrix of values.
Validity of the findings: I am missing a claim: When should the new approach me preferable, faster of providing more insight. So I am taking it what it is for me: There are some arguments to try it, and it has been done, and the output is reported.
Additional comments: As a mathematician I would like to see more concrete claims or estimates. To which extent are the two matrices introduced at the beginning really (at least approximately) inverse to each other.

It should not be too difficult to compare the difference between their product and the corresponding unit matrix e.g. by doing an SVD on it, to see from where the largest deviations/errors appear. The Gauss function used for testing is certainly not in the linear span of the bad singular vectors but among the best possible ones.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DISCRETE TWO DIMENSIONAL FOURIER TRANSFORM IN POLAR COORDINATES PART II: NUMERICAL COMPUTATION AND APPROXIMATION OF THE CONTINUOUS TRANSFORM
Review round: 3
Reviewer: 1

Output:
Basic reporting: This is just a revision. The authors have modified critical parts according to suggestions.
Experimental design: The MATLAB code has been adapted.
Validity of the findings: The results are certainly worth being published. The presentation is clear.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: COMPARATIVE INVESTIGATION OF PARALLEL SPATIAL INTERPOLATION ALGORITHMS FOR BUILDING LARGE-SCALE DIGITAL ELEVATION MODELS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Some grammar problems, especially missing articles (the, a, an). Overall however the English is readable, and the flow of the article is logical, although the background and methods sections are probably too terse and could be expanded.
    I’m confused by the references to literature that cite only author initials (e.g., T, C instead of complete last name and initials (for example Smith, J.). Also problematic that some citations include a publication year and others utilize some sort of code (e.g., 018a)  This format for some but not all authors challenged a smooth reading flow, as I had to return to the References and in several cases check online to establish who the author(s) of cited literature actually are.  I suggest a consistent format for citations that includes complete last names for all authors.  Many of the references are not original, for example Shepard’s algorithm (also called Inverse Distance Weighting) was first described by Shepard, Donald (1968). "A two-dimensional interpolation function for irregularly-spaced data". Proceedings of the 1968 ACM National Conference. pp. 517–524. doi:10.1145/800186.810616,  not by Mei G and N 2017.  Curiously, the authors cite Shepard (and it is D. Shepard, not S. Shepard, as cited in the appendix). Likewise, kriging was not first derived in 2013 by an author named T but by Georges Matheron in 1960 based on a thesis by Donald Krige (see for example Cressie, N. A. C., The origins of kriging, Mathematical Geology, v. 22, pp 239–252, 1990). Radial Basis functions are originally attributed to Michael J. D. Powell (1977). "Restart procedures for the conjugate gradient method". Mathematical Programming. 12 (1): 241–254. doi:10.1007/bf01593790.
    Another point about the literature review is that while articles are cited, there is little discussion about them.  For example, to say that Gumus (2013) compared different algorithms seems insufficient in an article about comparing algorithms.  What were Gumus’ findings? Similarly, what did Chaplot or Aguilar discover about the associations between landform types and data density on DEM accuracy? Discussing these articles in more detail would really strengthen the introduction and lead the reader toward the authors’ research questions and significance of this current study. The paragraph in lines 65-66 of the introduction seems misplaced – it should introduce discussion about the impacts of terrain type and data density, rather than follow the end of it.
Experimental design: The authors could strengthen the Methods section considerably by expanding on the discussion of the two 30 m DEMs.  What agency compiled these DEMs originally?  And for readers’ benefit, how do Hebei and Sichuan landscapes differ, given that landscape type is one of the controlling factors in the study?  I would also suggest correcting the legend for area a, which appears to have an elevation range of only 30 or so meters – why does the legend extend to 50 meters? The authors state that the elevation range extends to 48 meters, but there are no red pixels in the image, while the legend indicates that elevations above roughly 35 meters would be orange or red. In the second study area, the DEM apparently has been hillshaded, and the lower elevations (blue) appear to be higher than than the upper elevations (red). Removing the hillshade could eliminate the optical illusion, or possibly the legend is inverted?  I can’t tell.
     I would appreciate more information about the triangular mesh, was this interpolated somehow and if so by what algorithm?  Also, what is the spatial distribution of the random samples? Table 4 implies a regular and irregular distribution, but this is not described explicitly. Were samples constrained to limit the number of samples in each 30 m pixel, to avoid over-fitting?  Also how were the five sampling densities chosen? Finally, why does the number of interpolated points exceed the number of data points? If the samples were triangulated from the original DEM and then stripped of elevation values, presumably some sort of seeding would be required to anchor the estimations (the interpolated values). But this is also not explained.
     The incorporation of the kNN search should also be described in the Methods section, as it figures prominently in the Discussion section. In particular (in the Discussion section) lines 262 -266 could be moved forward to the Methods section and expanded upon.  It’s not clear why the kNN search is needed, is my point.
Validity of the findings: What are the units of Table 4? And are these values showing RMSEs? Residuals, or some other metric? If these are meters, then the errors are on the order of hundredths of a millimeter, which seems too small a magnitude (but again, I don’t have a clear sense of the sampling rate, constraints, or which of the several alternatives for each interpolation method were used, so I might have misinterpreted this table).
     The authors state that interpolation accuracy for MLS is better than RBF or Shepards (IDW). Whatever the metric, the values for RBF exceed the MLS results.  Meaning that either the RBF is more accurate than MLS, or more accurate than IDW.  But neither reflects the statement that the progression of accuracy is MLS – RBF – Shepard.
     I appreciate seeing the improvement of parallel over sequential computing.  It is of course important to balance accuracy against processing speed, especially in situations where modeling inputs are quite large (here, I estimate over 13 million pixels in each DEM). But I wonder if it is necessary to show all of the sequential results, given that one should expect that the HPC computations will complete in a much shorter time.  I suggest eliminating Table 5 entirely and simply report the range of average computing times for all ten of the flat and all ten of the rugged experiments. I also wonder about whether the comparison between single and multiple GPUs is really the top priority in the article, whose main purpose I surmised is to compare differing landscape characteristics and data density.
     In the discussion section, what is the metric used to assess “relative error”?  That is, relative to what? And why show errors for the single GPU, when the multi-GPS solution is faster? Given the article title, I would expect to read a discussion focusing on details of the regular versus irregular sampling in flat and rugged terrain, possibly with a 2-by-2 contingency table, than to see pages of graphs comparing the various parallel solutions.
Additional comments: The authors compare RBF, Moving Least Squares and Shepard’s interpolation algorithms (parallelized) for building DEMs, criteria are terrain type, data density, distribution of errors, processing efficiency. I like the starting premise that specific scene characteristics should be accounted for.  However I am not sure that they maintain this as a primary focus throughout their analysis, which seems to get off-track in prioritizing attention to various parallel computing environments and how much each type improves upon sequential computing.
     There is some ambivalence among researchers about the term “high” resolution (does it mean large pixels or small pixels?).  In this article one assumes it refers to small pixels; but I suggest using the term “fine resolution” versus “coarse resolution” to avoid any confusion among international readers.
     It is unclear what the question marks in the Methods section refer to (line 107).
     Finding number 1 (line 313 in the Conclusion section) runs counter to numerous articles comparing interpolation of elevation, and this should be noted by the authors. See for example Ghandhari et al 2019 in IJRS. This is not the only article that compares elevation estimation, but it is a recent one and has a good review of recent literature.  Finding 3 (line 322-323) claims that the IDW method is only suitable where data points have “smooth trends”. It certainly seems reasonable that IDW will generate higher errors in rugged terrain, but there is uniformly rugged and non-uniformly rugged, and you tested only the former situation. Was a trend surface analysis run on the data to justify this claim?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: COMPARATIVE INVESTIGATION OF PARALLEL SPATIAL INTERPOLATION ALGORITHMS FOR BUILDING LARGE-SCALE DIGITAL ELEVATION MODELS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The background and literature review section is not well written. The citation method is confusing and references are not properly documented because names are abbreviated in the references section. Several references are not cited and overall the paper seems to have been quickly written without much concern to details. Table and figure captions are not complete making them difficult to understand. Additional details about methods must be included.
Experimental design: The experimental design should be improved to include some statistical metric(s) to compare interpolation accuracy of the different interpolation techniques. Methods try to compare the accuracy and efficiency of a several interpolation techniques for elevation data in a flat and a rough section of data.
Validity of the findings: Finding from this work indicate that a regular pattern of sample points produces a more accurate interpolation solution than an irregular pattern, and this result is true for the flat and the rough terrain datasets. Some types of interpolation may be more accurate than others, but statistical testing is not employed to determine significant differences between interpolation techniques. Parallel processing methods using GPUs and CPUs are shown to improve processing speeds when compared to sequential processing methods, but the speedup is not precisely defined in the article.
Additional comments: 1.	References do not appear to be properly formatted because several citations do not include complete last names for all authors. At least full last names should be included for all authors of each citation.  Likewise the reference to each citation in the text cannot have last names abbreviated.
2.	Missing several interpolation methods. Missing reference: Lam, N. 1983 article Spatial Interpolation Methods.
3.	Missing Shepard 1970 article. Shepard, D.S. 1970. A two-dimensional interpolation function for computer mapping of irregularly spaced data. Papers in Theoretical Geography, Technical report 15, Laboratory for Computer Graphics and Spatial Analysis. Cambridge, Mass., Harvard University.
4.	Please expand on line 32. What is meant by spatial structures and types of data. Is this meaning type of remote sensing instrument, data resolution, 3-d point cloud data?
5.	Missing reference: Ghandehari M, Buttenfield BP, and Farmer CJQ (2019) Ghandehari M, Buttenfield BP, and Farmer CJQ (2019) Comparing the Accuracy of Interpolated Terrain Elevations Across Spatial Resolution. International Journal of Remote Sensing 40, 13, pp. 5025-5049. https://doi.org/10.1080/01431161.2019.1577581
6.	HASM is not defined before use in line 58.
7.	Need to expand discussion for single sentence paragraph in lines 65-66.
8.	What is L2 - norm in line 78? These symbols are not defined.
9.	Upper and lower case for basis function p may not be consistently applied in line 78.
10.	Explain what is meant by nodes in line 78. Does this refer to a node in a cluster?
11.	Who is Schmidt in line 81? Do we need a reference?
12.	In line 78: Consider adding a word, such as “And” to the beginning of “w(x−xk) is the compact-supported weight function.” Otherwise the period looks like an extension of the formula.
13.	What does alpha stand for in equations 5 and 6 of line 79?
14.	What is f in equation 7 of line 79?
15.	Also, it is unclear where ai fits into the Othogonal MLS interpolation algorithm. Please clarify.
16.	What is fk in equation 7 on line 79?
17.	Missing some spaces in text within line 84.
18.	In equations 11 and 12, it appears the square (2) outside the absolute value signs need to be superscripted.
19.	Explain what d represents in equation 15.
20.	Missing references in line 107
21.	Define kNN in line 113, and please explain that method in the paragraph.
22.	In line 117, explain the primary contribution found by Mei G (2016). Describe the Structure of Arrays.
23.	It is unclear how the randomly sampled points are being used. Are these the values for fa in the NRMSE computations? Please explain further how the samples are used. Are the samples used to generate the triangular mesh?
24.	In lines 128-130, why are the coordinate values too large and transformed into a local coordinate system?
25.	Line 137 should say “see Table 1”. Currently reads see Table 2.
26.	What do the different sizes in Tables 1 and 2 refer to? Sizes go from 1 to 5. Is this just a different number of sample points?
27.	Equation (20) in line 160 shows the normalized root-mean-square error (NRMSE), but it is unclear how the theoretically exact value is determined for fa. Please explain how the interpolated values are generated at the locations where the exact values are sampled. I’m guessing that somehow the generated mesh is used to interpolate values at the locations of where the exact values are sampled, but it is unclear how that is being done.
28.	In line 170, what do you mean by “not much different”? The scale range is different in Figure 2 for the regularly distributed and the irregularly distributed data. Looks like the regularly distributed method provides better results (lower RMSE) than the irregularly distributed method for both the rugged and flat data.
29.	The colors selected in figures 3 and 4 should be adjusted. There are two shades of red, and two shades of blue/purple that are difficult to distinguish between.
30.	Interpretation described in lines 189-192 seem incorrect. kNNRBF has faster times than MLS methods in all cases, and MLS methods appear faster than other methods. Lines 190-190 should be revised from “computational time from high to low” to “computational time from fastest to slowest. . . “.
31.	Interpretation in lines 197-199 seems incorrect.
32.	Line 203, “inaccurate or even incorrect” is two ways of saying the same thing.
33.	Figure 4 shows regular and irregular has similar maximum frequency of about 100,000, but the text suggests regular distribution is taller. This interpretation seems incorrect.
34.	Interpretation in lines 219-220 is incorrect. Non-uniform distribution generates larger range of errors than uniform distribution as shown in Figure 4.
35.	It is unclear what data the authors are interpreting in lines 230-238 where they compare results from different point densities. Is this information that is in the table, because no figures show results from different point densities?
36.	Captions for figures 6 and 7 should indicate which CPU or GPU sequential computation method is employed for the displayed results.
37.	In figures 10 and 11 and lines 257 to 261, what is meant by speedups? Is this in seconds or percent speedup compared to sequential method?
38.	Figure 12: change “Plate zone” to “Flat zone”.
39.	Table 9 caption is incomplete. Please include what the proportion is compared to, i.e. sequential implementation compared to what? Likewise please include additional information for Table 10.
40.	Line 383 should be the Shepard reference. Author’s improperly abbreviated.
41.	Overall, the paper presents some important considerations for evaluating raster data that may be interpolated from a set of points, or which may be the result of a resampling method. Several interpolation methods are compared using different levels of sampling and processing techniques to discern the more accurate and efficient methods. However, the method of accuracy assessment must be clarified to better describe how the truth data are determined and how the interpolation methods are employed. Without a clear definition of the methods it is difficult to understand the results. Furthermore, it seems that some form of statistical method could be used to compare the accuracies of different interpolation approaches and computation methods. Tabulating such a statistical summary could allow the authors to only focus on a few charts that reveal the most important results, rather than charting all solutions, which is difficult to legibly present and puts too much burden on the readers. It appears that the authors visually interpreted results and, in some cases, the interpretations do not appear accurate or are not properly described.