Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: HIGH-DIMENSIONAL NORMALIZED DATA PROFILES FOR TESTING DERIVATIVE-FREE OPTIMIZATION ALGORITHMS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The prose of the article requires significant revision in order for it to meet the requirement for use of clear and unambiguous English. There are frequent grammatical errors and use of terms which are, to my experience, non-standard with respect to the common language of the mathematical sciences. A non-exhaustive list of some of the errors with proposed corrections is included in the “General comments for the author”.

The structure of the article also requires revision. For example, the first two paragraphs of the Discussion (starting Line 251) introduce the Genetic Nelder Mead (GNM) algorithm and provide the only summary of the author’s Multidirectional Trigonometric Nelder Mead (MTNM) algorithm. The Discussion section should be limited to the interpretation of the author’s work, in the context of previously presented background information, as much as possible.

As the GNM algorithm is a core component of the author’s evaluation of the MTNM algorithm and the proposed data profile (Equation (25)), I believe that it should be more prominently introduced in the article’s introduction. References should also be provided to justify the choice of the GNM algorithm over other Nelder Mead variants for use as a point of comparison. References should also be provided for the sentence starting on Line 220.

The minimisation example starting on Line 49 requires further explnation. While I do find the chosen topic (surgical grafts in cardiology) to be interesting, it is not clear with respect to what variables the objective function is defined.

There are a number of small issues with the Figures and Table presented in this article:
•	Figure 1: Sublabel “A” has an identical to the variable “A” present in Figure 1 A and B. The parameters $d$, $d_l$ and $Th$ are not defined. The axes of the plot Figure 1 A are not labelled.

•	Table 1 performs a comparison which I do not believe to be an accurate communication of the work of  Fajfar et al (see Section 2).

•	Figure 2, 3, 4 and 5: As data profiler variables are defined in the accompanying description, the axes of the plots may use ds(*), W, Y and T (sec)  as the axes labels (suggestion). It is not clear how parameter Y is an “estimate” of the objective function, as the function should be computed up to the limits of numerical precision at each point. I suspect “estimates” refers to the number of “evaluations”.

•	Figure 5 A and B: It is difficult follow the lines for each solver without zooming in considerably.

The authors have provided code for 6 of the 36 functions comprising test set proposed by More´ et al. I was able to execute this code to verify this limited subset of the author’s results. However, code should be provided for the complete suite of test functions.

Finally there are a number of, mostly small, mathematical omissions or errors:

•	Line 51: The domain and range of $f$ is not defined. In the context of the Nelder Mead algorithm, the generalised objective function is typically $f: \mathbb{R}^n \rightarrow \mathbb{R}$ such that for, $f(x)$, $x \in \mathbb{R}^n $.

•	Line 101 presents an ordering of vertex pairs, this is incorrect in light of the ordering described on the previous line. The ordering should be notated in a manner equivalent to: $A=f(v_1) < B=f(v_2) < C = f(v_3)…$

•	Line 152: If the values for $delta_u$ and $\delta_z$ have been arrived at through the authors’ practical experience, they should state so, if there is a formal or historical justification for these parameter values, it should be explained.

•	Equation (25): Parameter “Z” refers to the “number of machines”.  The term “number of machines” is ambiguous. I assume, however, that the authors are referring to the number of CPU cores.
Experimental design: The author’s present two original contributions:

1.	Firstly, a modified version of the Nelder Mead algorithm, the MTNM algorithm, utilizes a set of 5 distinct solvers which work independently to identify the objective function minimum. These simplexes introduce different rotational ‘shifts’ through modification of the functions governing the placement of new simplex vertices at each iteration. An early stopping criterion is used to abort solvers which are failing to converge. The MTNM algorithm is shown to outperform solvers developed via the GNM algorithm in some cases. Of particular note is its performance in high dimensions, for example,  when minimising the Quadratic (24) problem.

2.	Secondly, a higher dimensional data profiler function by which to evaluate the performance of two dissimilar solvers operating on the same problem set. It introduces dimensions associated with wall time, number of simplex evaluations and number of CPUs.

However, there are a number of significant issues with the experimental design:

•	It is not clear why the GNM algorithm has been chosen as a point of comparison. In the referenced paper, Fajfar et al outline a generic algorithm (GNM) for the evolution (or “hyper-optimisation”) of alternative Nelder Mead simplexes. They arrived at 5 candidates and present a single “optimal solver” (solver 1). Conversely, the authors propose an algorithm that uses an ensemble of 5 solvers.

•	 Table 1 implies that the GNM algorithm is an ‘ensemble’ Nelder Mead solver when that is not the case. This also occurs on Line 306 where it is stated that the GNM algorithm requires about 12 hours to complete using 20 CPU cores. These values are taken from the time reported by Fajfar et al for completion of their genetic evolution algorithm, not optimisation of the standard test problems of More´ et al.

•	The data presented by Fajfar et al does not allow for a comparison of their solvers with the MTNM algorithm using the author’s defined data profile.

•	The addition of the T, W and Z parameters require further justification, as it stands, I have the following concerns:

o	The Nelder Mead, and modified Nelder Mead, algorithms presented in these articles require $n + 1$ function evaluations per iteration, where $n$ is the number of objective function parameters. The early stopping test employed by the MTNM algorithms results in a meaningful difference between the number of simplex evaluations and function evaluations, but this is certainly not the case in general.

o	While comparison of $W$ and $Y$ is perhaps useful for exploring how the number of ‘active’ MTNM simplexes changes with respect to the number of objective function evaluations, the overall performance of the algorithm is still almost entirely dependent on the number of objective function evaluations alone. Dependence on $W$ could only arise through the use of simplexes with distinct algorithmic complexity. If there is a measurable difference in the algorithmic complexity of the 5 MTNM solvers, the authors should demonstrate this by way of proofs or numerical experiments.

o	The algorithm wall-time, $T$, is not generalizable due to its dependence on the particular computer hardware used and the state of the host computer.  Furthermore, for $T$ to be independent of $Y$, there must exist a significant (non-constant) overhead associated with the solvers that are independent of the number of function evaluations. However, as any such time overhead will necessarily result from the algorithmic complexity of the simplex construction step (see above), the $T$ and $W$ parameters do not seem to present independent dimensions for data profiling.

o	The number of CPUs, $Z$, was not used by the authors in their evaluation of the MTNM algorithm and I am unsure of how it relates to the NM algorithm in general. The most common target for parallelization techniques in the context of numerical optimisation problems is the objective function as it is (almost always) the most computationally expensive step. For $Z$ to be relevant the authors should present an optimisation algorithm that follows an alternative parallelization scheme.
Validity of the findings: The authors present interesting results with respect to the performance of the MTNM solver, however, exploration of these results is hampered by issues in the experimental design (see Section 2). Unforteuently, it is not possible to easily validate all of the author’s findings and reliance on externally sourced data (from the referenced paper by Fajfar et al) is an additional barrier to replication.
Additional comments: Below are some suggested grammatical corrections and comments relating to the prose of the submitted article:

Line 18: “…through an angle <that> designates…”

Line 37: “<We are> motived by…”

Line 47: “<It seems a fact> that if a problem can be described as a mathematical expression, then at some point we will be <required> to minimize it or tune <its> independent variables at a <minimum>”.

Line 59: “…reflections over the <changing> landscape of <the> mathematical problems until the coordinates of the minimum point <can> be obtained…”

Line 66: “…the simplex becomes <increasingly> distorted <with each iteration>, generating different geometrical formations that are less effective <than> the <original> simplex design.”

Line 69: “…the sequence of simplexes <to> converge to a non-stationary point.”

Line 75: “The rest of this paper is organised as follow<s>.” “presents the theory of <the> sequential design of trigonometric Nelder-Mead algorithms<s>…”

Line 76, 97 Figure 1 A: “…the vector theory” -> “vector theory”

Line 78: “…and presents <the> multidimensional…”

Line 83: “…the theory of <the> Hassan…”

Line 86: “<This is different from the> traditional NM algorithm…” (suggestion)

Line 93: “…the generat<ed> sequence of triangular simplexes are guaranteed not only have different shapes but <to> also they have different directions<.> <These characteristics lead to> better performance than the traditional hyperplanes simplex.” “ Note that <to> find the reflected point D we add the vectors $H$ and $d$<, as shown in Figure 1 A.> ”

Line 106: “Now is we consider two combinations or more”: It is not clear what sort of combinations we are considering.

Line 107: “..multiple components of <the> triangular HNM simplex…”

Line 114: “In fact, the HNM algorithm is designed to deform its simplex in a way
that introduces nonlinear movement, by incorporating a rotation<al>  <shift at> each iteration.” (suggestion)

Line 115: “After all axial components of the Th point are updated to see whether the new Th is better than C to be replaced or not.”

The above sentence and, possibly the one before, would be better positioned before the concluding paragraph as they outline key aspects to the HNM algorithm.

Line 122: “…approximation to a solution, generating different geometrical configurations…”

Line 123: “…build the initial simplex with <edges of > equal length …”

Line 129: “The risk is that<,> if the initial simplex is perpendicular to an optimal solution, then the algorithm…”

Line 132: “…initial simplex with the NM method…” (suggested)

Line 134: “…the generat<ed> sequence of…”

Line 134: “One of the problems is that the generating sequence of  simplexes should be consistently scaled with respect to the best vertex obtained and restart the simplex  around the best observed results.”

This sentence is difficult to parse as “One of the problems” is closely followed by “should be”. The first phase is a negative imperative, while the second phrase is a positive imperative.  A clearer sentence could start “One of the problems is that the generated sequence of simplexes is not scaled with respect to the best vertex…”  or, perhaps, “Ideally the generated sequence of simplexes should be scaled with respect to the best vertex…”.

Line 138: “”Alternatively, the most popular way to initialize a simplex is Pfeffers´ method <(Baudin, 2009).> <This approach scales> the initial simplex <based on the starting point, $X_0$>. The initial vertex is set to  v1 = x0, and the remaining vertices  <are> obtained as follows,”

The paragraph starting Line 141: Please present a brief argument as to why Equation (15) leads to improved starting points. The last sentence does not deductively follow from the Equation definition.

Line 145:  “On contrary, our solution depends on allowing the components of the reflected vertex to  <transform non-uniformly with each iteration of the simplex>.

Line 154: New paragraph at “In this test, we are more interested in launching multiple sequences of trigonometric simplex designs with different…”

Line 158: “This procedure reinforces the standard Pfeffers´ method of creating a simplex  with additional designs.”

The above sentence could be safely removed.

Line 158: I suggested that the sentence starting on this line, and the one following, be reworked.

The first sentence makes a definite claim as to the effectiveness of the multidirectional simplexes without providing justification, it is also not clear what “effective” means in the context. Finally, instead of “mathematical landscape”, I suggest the term “solution space” or “the domain of the objective function”.

In the following sentence, it is not clear (to me) if the proposed methods seek to rotate the simplexes such that they have a unique starting direction or a unique direction with each iteration.

For example, see the following rephrasing:

 “<We will demonstrate that our> proposed multidirectional simplexes <converge to a minimum with a smaller> number of simplex gradient estimates <as compared to the previously discussed methods of simplex design>. <Key to this outcome is the> shifting <of> points associated with the highest, the second-highest, or the lowest values of the objective function with a displacement that <generates simplexes with> distinct directions.”

Line 163: One of the potential designs <multiplies> the odd<-indexed> variables of odd-<indexed> vertices by (-1)

Line 167: “…even components of x0,”

Line 168: “…specified edge lengths and orientation…” All simplex edge lengths and orientations are specified in some manner. Are the specifications those goals outlined in the paragraph starting on Line 145?

Line 169: “…by multiplying (-1) the parameters…” Is the (-1) a typo?

“usual delta” and “zero term delta” -> “$\delta_u$ and $\delta_0”

Line 170: “…the absolute value <of the components of $x_0$ and> and subtracting or adding to adjust the simplex orientation…”

Line 171: “…and can be modified as needed…” By what criteria are the position vectors added or subtracted?

Line 173: “…as well as the slope…” The slope with respect to what function and what point points?

“Therefore, a window of size 10 points…” The choice of 10 points does not deductively follow from the previous sentence, why 2, 4, or 100? If the window size is a hypermeter derived from your practical experience please make that choice clear.

Line 176: “…the direction vector, which designates the direction of the simplex…” -> “the direction of the simplex”

Line 178: “For the particular <case>,”

Line 189: Non-deductive therefore. The phrase "Therefore" should only when drawing a logical link between ideas.

Line 190: “tools” -> “metrics” or “figures of merit”, “similar algorithms,” -> “the considered algorithms.”

“Which are summarized as follows: <the> accuracy of the algorithm compared to the actual minima, <the wall-time to convergence>, the number of function evaluations, the number of simplex evaluations, and identification of the best sequence of trigonometric simplex design.

Line 194 to 200: “The main reason for designing the set of functions”: It is not clear how the set of functions relates to the previously mentioned testing guidelines. I suggest that the authors make the link explicit (e.g. “These guidelines utilize a set of functions…) or place this discussion after the subsequent sentence.

Line 207: What is “short-term” behaviour in this context?

Line: “launch” -> “launches”

Line 209: It is not clear to me how the individual solvers “cooperate” to find an optimal solution. As it stands I do not think they share information beyond their common starting point, $x_0$, is this correct?

Line 233: “All computational experiments were carried out on an i5 CPU (4 Cores at1.8 GHz) and 4 GB of RAM. We used C# to implement the MTNM algorithm and carry out the experiments.”


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: HIGH-DIMENSIONAL NORMALIZED DATA PROFILES FOR TESTING DERIVATIVE-FREE OPTIMIZATION ALGORITHMS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors have addressed all my comments.
Experimental design: The authors addressed most of my comments. However, they did not address the most important two issues:

- The test functions have not been presented in the text. More importantly, all these problems have only a few variables. The authors need to consider much larger problems.
The authors state in their response that they want to be consistent with the literature in terms of the number of variables and use the same total number of variables of in order to be able to compare computational results. It is OK to include these problems in order to compare the results with the literature. However, it would add more value to the paper to also include larger problems.

- Why not compare against efficient DFO algorithms (e.g., NOMAD)?
The authors state that they choose GNMa because GNMa is one of the best algorithms that utilizes the test functions of (Moré et al., 1981) and utilizes normalized data profiles that involves one dimension (simplex evaluations). It is OK to use GNMa for the aforementioned reasons. However, why not use the state-of-the-art DFO algorithms, like NOMAD?
Validity of the findings: The authors have addressed all my comments.
Additional comments: The authors have addressed all my comments.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: HIGH-DIMENSIONAL NORMALIZED DATA PROFILES FOR TESTING DERIVATIVE-FREE OPTIMIZATION ALGORITHMS
Review round: 2
Reviewer: 2

Output:
Basic reporting: The overall prose of the article has improved but significant proofreading and revision of some paragraphs are required in order to meet the criteria of clear and unambiguous professional English. Identified typos and suggested re-phrasing are included in 'Additional comments', however, I am certain that the list is non-exhaustive.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: The experimental design is outlined clearly and the authors' original contributions are clearly defined. The authors introduce the Multidirectional Trigonometric Nelder Mead (MTNM) algorithm, which utilises an ensemble of five simplexes developed through variation to the simplex of the Hassan Nelder Mead Algorithm.

The performance of this algorithm is assessed against the Genetic Nelder Meader (GNM) algorithm, which provides a single simplex derived through genetic programming. To compare their ensemble solver to this algorithm the authors proposed a high-dimensional normalised data profile that takes into account the number of cost-function variables, cost function evaluations, simplex evaluations, wall-time and CPU count.

The authors provide sufficient background and references, have addressed previously identified issues in their diagrams and have shared the code needed to reproduce their original numerical results. The article is self-contained, presents results relevant to the hypotheses and includes clear definitions of all terms.

Overall, I am happy with the authors' response to feedback, in particular, their discussion of the data profile dimensions in "Detailed Analysis of the Five Solvers".

Below are a number of identified typos and small suggestions:

Line 32: it’s -> their
Line 39: We are motivated to notice -> We are motivated by the observation
Line 40: For example, data profiles -> For example, some data profilers

Line 45: Thus, the proposed multidimensional data profiles are more compact and effective in allocating a computational budget for different levels of accuracy.

Instead of 'Thus' perhaps say 'We argue that' or 'Numerical results indicate that'.

Line 75: NAa -> NMa

Line 77: with best -> with the best

Line 80: that is hybridizing -> that hybridizes

Line 81: Instead of using a restricted NM simplex to only one traditional design, the authors evolve NMa genetically to produce  deterministic simplexes that can develop adaptive vertices through generations.

	The sentence needs to be rewritten (simplified).

Line 86: and basically -> and that, basically, the

Line 93: upgrade the data profiles

	Which data profiles?

Line 98: of trigonometric -> of the trigonometric

Line 111: When the next simplexes are characterized by different reflections, the HNMa performs not only similar reflections to that of the original simplex of the NMa, but also others with different orientations determined by the collection of non-isometric features. -> When different reflections characterize the next simplex, the HNMa performs similar reflections to that of the original simplex of the NMa and others with different orientations determined by the collection of non-isometric features.

Line 114: guaranteed to search the solution space of mathematical problems” -> guaranteed to search a higher proportion of the solution space

Line 118: calculated at the vertices -> calculated at vertices

Line 119: in ascending with -> in ascending order with

Line 120: $and$ -> and

Line 122: The need for Th arises when the HNMa performs a reflection in an axial component, it replaces the value of the axial component of the Th.

	This sentence is not clear, please rewrite it.

Line 142: in extraction one -> in the extraction of one

Line 145 Therefore, the optimization solution of the HNMa not only reflects the opposite side of the simplex through the worse vertex, but also leads to implement various reflections that are determined by the collection of extracted features. -> Therefore, the optimization solution of the HNMa reflects the opposite side of the simplex through the worse vertex and leads to the implementation of reflections determined by the collection of extracted features.

Line 152: Solutions to the HNMa help extract the optimal characteristics of the non-isometric reflections in the reflected vertex and produce simplexes that lead to faster convergence rates than the original triangular simplex of the NMa.

Line 177 and 178: “ -> ``

Line 242: (in second), -> (in seconds)

Line 247: optimization classes -> optimization class.

Line 292: A simplex in some cases explores the neighborhood to update its threshold, but do not move only if the threshold is good enough to replace the worst point.

	Grammatically, 'Do not' should probably be 'Does not'. However, the sentence makes more sense if 'but do not move'  is changed to 'but moves'. Please rework this sentence.

Line 296: In this way, the HNMa mimics the amoeba style of maneuvering to move from one point to another when approaching a minimal point. -> In this way, the HNMa mimics an amoeba style of maneuvering from one point to another when approaching a minimal point.

Line 304: “ -> ``

Line 334: using 20 of 2.66 GHz -> using twenty 2.66 Ghz

Line 358: For example,  MTNMa performs better to determine a solution to Quadratic (16) then Quadratic (8) in Table 1. -> For example,  MTNMa performs better when optimizing Quadratic (16) as compared to Quadratic (8) in Table 1.

Line 359: While in other cases, the MTNMa generates fewer number of simplexes to approximate a particular solution for high dimensions than for less dimensions. ->  While in other cases, the MTNMa generates fewer simplexes to approximate a particular solution for high dimensions than for lower dimensions.

Line 371: such environment.-> such an environment

Line 378: provide a complementary -> provide complementary

Line 396: requires the best possible reduction has to equal -> : requires that the best possible reduction has to equal

Line 407: By analyzing those five solvers of multi-directional trigonometric simplex designs, we have conducted  further tests, which reveal that further evaluations are important to make decision on which solver should  be used when there is a limited computational budget. -> We have conducted further tests by analyzing the five multi-directional trigonometric simplex solver designs. These reveal that higher dimensional data profiles are essential to deciding which solver should be used with a limited computational budget.

Line 410: solvers (1 and 5) already catch up, approximately after overtaking 400 simplex evaluations -> solvers (1 and 5) catch up after approximately 400 simplex evaluations

Line 417: This forms a strong argument that how a solver, in some cases, may always require a large number of simplex gradients but may have the potential to take less time to solve 100% of the test problems and improve the performance. -> This forms a strong argument as to how a solver, in some cases, may require a larger number of simplex gradients but may have the potential to take less time to solve 100% of the test problems.

Line 447: In the situation where Solver2 stands out as being the best of the five solvers, because certainly it requires not only less function evaluations but also less computational time than the other solvers. -> In this situation, Solver2 stands out as being the best of the five solvers because it requires fewer function evaluations and less computational time than the other solvers.

The paragraph starting on Line 451:

Regarding the statement on Line 452. While different mathematical problems are parallelizable to differing extents, I would argue that the significance of Z is more immediately dependant on whether the CF is implemented (or perhaps compiled) as a parallel function, whether the solver is implemented as a parallel algorithm and the effectiveness of the chosen parallelisation methods.

I suggest that this paragraph be simplified, for example:

The number of CPUs (Z) was not examined in our evaluation of the MTNMa and is included in formula (22) for completeness.  This dimension is significant if an optimiser is deployed in a distributed environment such as Amazon Web Services (AWS). In such a case, the number of nodes in the virtual cluster is an important aspect of the computational budget and the inclusion of Z assists in the allocation of optimal numbers of CPUs for different solvers and for specific levels of accuracy.

Line 471: rotation property determined -> rotation determined

Line 471: To examine the performance of multiple solvers simultaneously, a linear model with a window of size 10 samples is suggested as the criteria that is used to make decision whether a solver is aborted or continued based on the direction vector of the window. -> When executing multiple solvers simultaneously, a linear model with a window of size 10 samples is suggested as the criteria by which a solver is aborted or continued based on the direction vector of the window.

Line 474: We also showed in this research that using the data profile as a function for allocating the computational budget and examining the relative performance of multiple solvers based only on the simplex gradients (one dimension) is not appropriate when the evaluation is expensive. -> We also showed in this research that using a data profile based only on the number of simplex gradients (one dimension) for allocation of the computational budget and examination of the relative performance of multiple solvers is not appropriate when cost-function (?) evaluation is expensive.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: HIGH-DIMENSIONAL NORMALIZED DATA PROFILES FOR TESTING DERIVATIVE-FREE OPTIMIZATION ALGORITHMS
Review round: 3
Reviewer: 1

Output:
Basic reporting: The authors have addressed all of my comments.
Experimental design: The authors have addressed all of my comments.
Validity of the findings: The authors have addressed all of my comments.
Additional comments: The authors have addressed all of my comments.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MITIGATION OF BLOCK WITHHOLDING ATTACK BASED ON ZERO-DETERMINANT STRATEGY
Review round: 1
Reviewer: 1

Output:
Basic reporting: I recommend to (also) display the revenue of the independent miners in an additional Figure 4(c).

Could mention, at least as outlook / room for further consideration:
Would mining difficulty adjustment (after 14 days) adjust pool revenue facing a block withholding attack?

Section "ZD strategy" (from 7/17, line 225) actually lacks a precise and compact definition of a "ZD strategy":
Lines 245/246 on page 8/17 do provide some conditions ("If the strategy selected by X satisﬁes ... This is the ZD strategy"), yet unclear what set of candidate "strategies" (which then do or do not fulfil the conditions) X can select from.

Conclusions sections very short. Should also summarize restrictions (e.g.: for the case of two pools) and outline further work.

Should define all abbreviations on first mention (TFT = Tit for tat? WSLS=?)

Should use a small space (typically "\," in LaTeX) between number and unit (10mins ->10 mins, 300BTC -> 300 BTC).
"Pool1"/"Pool2" should use subscript für index values ("Pool$_1$").
Experimental design: Usual not to find experiments replicated.
Would expect multiple runs of 100 (or 500) steps each and variance reviewed.
If not, should (at least) argue why - according to the authors - this is not necessary.
Validity of the findings: no comment
Additional comments: no (furher) comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MITIGATION OF BLOCK WITHHOLDING ATTACK BASED ON ZERO-DETERMINANT STRATEGY
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper deduces the calculation formula of the actual income of the mining pool when the block withholding attack is launched. The ensemble strategy, extortion strategy and the proposed adaptive strategy of ZD are studied. Then, the mitigation of block interception attack based on ensemble strategy, extortion strategy and zero-determinant adaptive strategy is theoretically studied. The authors also selected and simulated 30 sets of game strategies to illustrate changes in the pool's returns.
Experimental design: Although this paper is structurally complete and logically clear, there are a few problems in the manuscript, which needs the author to modify for being accepted. The problems in the manuscript are shown below in detail.
1.	In the abstract section, the author should make clear which works are his own, rather than linearly describe the prose structure of the work.
2.	Some formulas are not in the same format as (11) (15). Authors should proofread carefully to ensure that all formulas are in the same format.
3.	In some of the figures, it is difficult to see the specific situation corresponding to Y as shown in Figure 6 and Figure 7. The author should make corresponding changes to make the figure clearer.
4.	In the conclusion part, the author should make a concise summary of each chapter in the conclusion, clarifying his own contribution.
5.	The following major workings are missing in the references: Multiple cloud storage mechanism based on blockchain in smart homes; Novel vote scheme for decision-making feedback based on blockchain in internet of vehicles.
Validity of the findings: Although this paper is structurally complete and logically clear, there are a few problems in the manuscript, which needs the author to modify for being accepted. The problems in the manuscript are shown below in detail.
1.	In the abstract section, the author should make clear which works are his own, rather than linearly describe the prose structure of the work.
2.	Some formulas are not in the same format as (11) (15). Authors should proofread carefully to ensure that all formulas are in the same format.
3.	In some of the figures, it is difficult to see the specific situation corresponding to Y as shown in Figure 6 and Figure 7. The author should make corresponding changes to make the figure clearer.
4.	In the conclusion part, the author should make a concise summary of each chapter in the conclusion, clarifying his own contribution.
5.	The following major workings are missing in the references: Multiple cloud storage mechanism based on blockchain in smart homes; Novel vote scheme for decision-making feedback based on blockchain in internet of vehicles.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MITIGATION OF BLOCK WITHHOLDING ATTACK BASED ON ZERO-DETERMINANT STRATEGY
Review round: 1
Reviewer: 3

Output:
Basic reporting: In this manuscript, the authors investigated block withholding attack in blockchain based on game theory. They consider the situation that only two mining pools exist and one or both pools attack each other. By calculating revenue of each mining pool, they pointed out that choosing whether it attacks the other pool or not is similar to choice of action in the prisoner's dilemma game. Then, they investigated zero-determinant (ZD) strategies in the iterated prisoner's dilemma (IPD) game, and showed that an adaptive ZD strategy promotes cooperation between pools.

Although their idea about applying ZD strategies to blockchain mining is interesting, I think that the current version should not be published. The reason is that, whereas the first part "Mining dilemma analysis" is significant as an application of game theory, the second part "ZD strategy" does not contain any novel results. They just investigated ZD strategies in IPD, which have already been studied by many researchers, and the paper does not advance our understanding at all.

I think that the problem comes from the fact that the authors reduced their model to IPD, which is too simple, although action in their original model is the continuous parameter $r_i$. I recommend that the authors investigate ZD strategies in their mining game without reducing it to IPD. In research on ZD strategies, McAvoy and Hauert extended ZD strategies to continuous action space [PNAS 113(13), 3573 (2016)]. I think that this paper is useful for improving their manuscript.

In addition, as a minor comment, their survey on ZD strategies is not sufficient.
There are many papers which should be mentioned in "Related works". For example,
*ZD alliance [Hilbe, Wu, Traulsen, Nowak, PNAS 111(46), 16425 (2014)]
*ZD strategies in noisy games [Hao, Rong, Zhou, Phys. Rev. E 91, 052803 (2015)]
*ZD strategies in games with a discount factor [Hilbe, Traulsen, Sigmund, Games and Economic Behavior 92, 41 (2015)]
*Extension to alternating games [McAvoy, Hauert, Theoretical Population Biology 113, 13 (2017)]
*Consistency and independence of ZD strategies [Ueda, Tanaka, PLoS ONE 15(4), e0230973 (2020)]
*Extension to memory-$n$ strategies [Ueda, Royal Society Open Science 8, 202186 (2021)]

I hope that the authors resubmit a revised version.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MITIGATION OF BLOCK WITHHOLDING ATTACK BASED ON ZERO-DETERMINANT STRATEGY
Review round: 2
Reviewer: 1

Output:
Basic reporting: Revised paper is much improved and can be accepted now.
Experimental design: Revised paper is much improved and can be accepted now.
Validity of the findings: Revised paper is much improved and can be accepted now.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MITIGATION OF BLOCK WITHHOLDING ATTACK BASED ON ZERO-DETERMINANT STRATEGY
Review round: 2
Reviewer: 2

Output:
Basic reporting: The authors sincerely addressed my comments or explained the novelty of their work in their rebuttal. Now I understand that the innovation of their work is an application of zero-determinant strategies to block withholding attack, and recommend an acceptance.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ALIGNING RESTRICTED ACCESS DATA WITH FAIR: A SYSTEMATIC REVIEW
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1.	To date, it remains unclear how sensitive data should be managed, accessed, and analysed (Cox et al., 2016).
-	6-year-old statement is given to express today’s situation

2.	Is there any solid ground behind the selection of Eligibility criteria?
3.	Condition four of the eligibility criteria is not present in a vast number of articles?

4.	The corpus of publications resulting from the query was exported to Rayyan
-	Why this specific tool is being used?
-	This tool was presented in 2016. Do we have any latest tool available in this context?
-	Comparative analysis is required for the selection of such tool from the list of available tools
-	It is seemed that the proposed work is heavily depending on this tool. What if this tool is removed / replaced with some other tool?
-	Which specific version of the tool is used?

5.	In the context of this research, the cycle has been slightly modified to better suit our results and to bring more awareness to also the stages when the data is processed, as well as the steps required after the data is used.
-	If there is no other reason of modifying data lifecycle management then it seems a bit biased to modify the process.
-	Secondly, what alteration has been made?

6.	TRL levels details are missing.
7.	Line 244 to 251 can be better represented in a tabular form.
8.	What is OWL ontology and why it is being used here?
9.	An RDF ontology was generated to describe the Data Methods subclass hierarchy.
-	Is there any alternate to the above representation?
-	Why do we need this step?

Is the review of broad and cross-disciplinary interest and within the scope of the journal?
-	Yes
Has the field been reviewed recently? If so, is there a good reason for this review (different point of view, accessible to a different audience, etc.)?
-	No, the authors claim is that this is the first study of this type.
Does the Introduction introduce the subject and make it clear who the audience is/what the motivation is?
-	Yes, up to some extent.
Experimental design: Methods described with sufficient detail & information to replicate.
-	No, with the provided details it will be difficult to reduce the result because it is working on the result of google scholar. At different time, google scholar will return different papers. This will affect the result.

Is the Survey Methodology consistent with a comprehensive, unbiased coverage of the subject? If not, what is missing?

Are sources cited? Quoted or paraphrased as appropriate?
-	Yes
Is the review organized logically into coherent paragraphs/subsections?
-	Yes
Validity of the findings: Is there a well-developed and supported argument that meets the goals set out in the Introduction?
-	In this context, goals are not being set out in the introduction.

Does the Conclusion identify unresolved questions / gaps / future directions?
-	There is a limitation heading found in the article in this regard.

Overall, this article is divided into two parts:
-	Domain / problem introduction
-	Proposed Solution

First part is being presented very well. Covering subject area well e.g., introduction, problem statement, background etc. but when it comes to problem solution then some weak technical stuff is being observed e.g., Methods, eligibility criteria etc. The Results section contains nothing but the detailed explanation of the selected dataset. Similarly, the outcome of the paper is heavily depended on the selected dataset. E.g., different input articles will have different results.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ALIGNING RESTRICTED ACCESS DATA WITH FAIR: A SYSTEMATIC REVIEW
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper analyzes various practices being followed in the research community to meet the data sharing proposed by FAIR principles. The manuscript provides a detailed survey of key mechanisms  involved in applying FAIR principles on restricted research data.
Experimental design: The identified research questions  look appropriate however the utility of RQ2 and RQ3 should be further elaborated, i.e., how research community may take advantage of proposed categorization  and whether the mature practices can be converted into some kind of standards or standard operation procedures or not?

The records show that majority of the papers are unable to meet the criteria set by the submitted manuscript and 40 articles are selected for investigation of research questions.  Theses number suggests that research community is either not aware of FAIR or the principles are not believed to improve the quality of research. A survey may be used to contact the authors of the excluded papers to verify the intention of no following FAIR principles.

It is suggested to target papers that have received awards in conferences or journals in the study as most of the conferences consider reproducibility as key condition for such awards. Also it can be found out whether compliance of FAIR principles is a requirement for the such awards or not?

Recently a few journals have appeared that particularity target publication of datasets (Data in Brief). Authors may consider including journal or conference based criteria to evaluate further the research questions.
Validity of the findings: A comprehensive discussion has been provided based on the collected data along with limitations, however authors apparently have relied on the reader for developing the inferences related to the research questions.  It is suggested to provide crisp response to the research questions particularly RQ2 and RQ3.

Authors have discussed the aspect of anonymization of data however it is suggested to investigate the aspect keeping finer granularity in mind, i.e., what kind of anonymization principles should be or already part of FAIR?

Table 2 provides fields of research categories and no example can be found that belongs to multiple fields. As the manuscript is submitted in a computer science  journal, it is therefore suggested to make effort to find a few paper that belong to either core computer science or its constituents field, i.e. communication networks, social network analysis, software project management etc that usually include research works involving sharing of data.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper proposes a novel methodology to diagnose cancer. The abstract should be verified in order to remove the grammatical errors.
Which type of cancer and from which organ has to be explained in detail.
The abstract states that the proposed system can also detect diseases from the heart. This sentence can be removed or the evidence for this sentence could be included.
Table 1 must be shown in the introduction section as it is not related to the literature review. Rather, a summary of the literature can be included as a separate table in this section.
The proposed solution can be written as Proposed Methodology
More details can be included in the Basic ANN Operation as it plays a major role in this paper.
Experimental design: It will be well if the experiments were done for the proposed system.
A table explaining the values of TP, FP, and TN for the input data should be included.
Validity of the findings: Table 3: Comparison of Various Neural Networks in terms of Classification Accuracy has to be rewritten so that it can be easily understandable to the readers.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 1
Reviewer: 2

Output:
Basic reporting: More points have to be included based on the complete methodology and results
The sentence “Moreover, the system is reconfigurable and can be programmed to classify any sample into one of two classes” has to be justified as the classification is nowhere shown throughout the manuscript.
The sentence “The system can predict about 63.5 million cancer samples in a second” has to be justified as the results were not shown in the results and discussion section. Rather, it is shown only in the abstract and also in the conclusion.
Experimental design: An experiment has to be done in order to depict the prediction of 63.5 million cancer samples in a second. Also, a tabulation has to be framed to analyze the prediction accuracy.
Validity of the findings: It will be well if executed results of image samples were shown in the results and discussion section so that the proposed methodology seems to be executed.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 1
Reviewer: 3

Output:
Basic reporting: The paper proposed a hardware implementation for the cancer diagnosis problem. However, the contribution is not clear in the hardware methodologies that are introduced

1- The proposed architecture is very weak and has no enough contribution
as in Figure 3: Proposed ANN Topology
2- The proposed system is not clear from Figure 6: Internal Structure of the Proposed Hardware System.
3- Figures 4,5, and 7  are not clear.
Experimental design: 1-  The experiments should include several cancer datasets and evaluate the performance between them
2- The splitting technique is very weak, I think you should use 10 folds cross-validation to get fair comparison.
3- The training parameters needs to be illustrated why u choose these parameters
The learning rate is kept equal to 1/3 . The momentum is equal to 0.9. The data is processed
in batches in order to achieve high accuracy; the batch size used in the proposed system is 100.
Validity of the findings: The paper did not contribute enough to be published in the journal and needs to high modification level
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 1
Reviewer: 4

Output:
Basic reporting: Authors have proposed a hardware-based deep learning system for cancer diagnosis. The research problem selected by author(s) is timely and important to be addressed. The overall structure and organization of the paper are satisfactory, and the paper qualifies for an above-average up-to-date bibliography. However, there are a few issues that are required to be addressed by the authors.

1)	The main problem with this paper is coherency. In many paragraphs, sentences are not written coherently.
2)	Why the title is specific to Cancer Diseases? Authors have also claimed the ability of the proposed scheme to predict the presence of other diseases. So, the title should be a hardware-based deep learning system for disease diagnosis.
3)	Authors are encouraged to improve the abstract by focusing only on the most significant details that are unique to their proposal. Authors have simply claimed in the abstract that “In this context, we propose a hardware-based neural network that can predict the presence of cancer in humans with 98.23% accuracy.” But they didn’t mention the features of their system that help to gain an accuracy of 98.23%. Authors are advised to write their significant contributions clearly. Moreover, in the abstract, it has been mentioned that the proposed system is about 5 to 16 times cheaper and at least four times speedier than many other contemporary systems? What is meant by many? Not clear.
4)	Authors have several times used the term hardware friendly but have not described it.  For example, in the abstract and other sections, it is stated “this is why scientists have come up with functions that are not only accurate but are friendly to hardware platforms”. What is meant by friendly to hardware platforms? Any reference?
5)	“Conventional activation functions such as sigmoid and hyperbolic tangent (TanH) yield high accuracy but are not suitable for hardware implementations. This is because they involve division and many other hardware-inefficient operations.” Authors have mentioned this fact, but without reference(s).
Experimental design: 6)	It seems there is no need for this sentence “To know more about
efficient implementation of neural networks on edge devices, the reader is referred to [11].”  Is the implementation of neural networks on edge devices is related to the authors’ work?
7)	Authors have claimed “this is the reason why we adopt these functions for implementation in the proposed system. Adopted these functions for implementation of what? in the proposed system. It is not clear here.
8)	A summary or comparison table of proposed techniques discussed in the literature review is not available.
9)	Details in paragraphs 2, 3, and 4 of Section 1 are not coherent. It is highly recommended to re-write these paragraphs to highlight your contribution.
Validity of the findings: 10) The required level of accuracy, speed, etc. depends on the underlying application, as shown in Table 1. Delay is not mentioned and how this line is related to the other lines of this paragraph. Again, authors are advised to be coherent in writing paragraphs.
11)	 Section 3 proposed solution starts with the working of the proposed system but authors must add text about the proposed techniques here. What is the proposed system, components, and working?
12)	 “Data Preprocessing” heading is not required and the text under this heading should be adjusted in section 3.1.
13)	 Figure 4 is not clear.
14)	 Figure 6 should be discussed in section 3.2.
15)	 The proposed algorithm is compared with other state-of-the-art algorithms in terms of classification accuracy, throughput, and implementation cost.  Which state-of the art algorithms? Names?
16)	 The proposed system consumes only 983 slice registers, 2655 slice look-up tables, 234 DSP48 elements, and 33 block random access memories (BRAMs). But why? It has not been mentioned.
17)	 It can be seen from Table 4 and Table 5 that the proposed system is about 5-16 times cheaper and at least four times speedier than most modern systems. What modern systems? Mention features of your proposed solutions that make these results better.
18)	 Future work or extension is not clear. Just one line has been mentioned, “In future, more complex datasets can be chosen for better diagnosis.”  This is not enough to say what is meant by complex datasets?
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 2
Reviewer: 1

Output:
Basic reporting: All requirements have been revised satisfactorily.
Experimental design: All requirements have been revised satisfactorily.
Validity of the findings: All requirements have been revised satisfactorily.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 2
Reviewer: 2

Output:
Basic reporting: All comments have been addressed
Experimental design: All comments have been addressed
Validity of the findings: All comments have been addressed
Additional comments: All comments have been addressed


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HIGH-PERFORMANCE, HARDWARE-BASED DEEP LEARNING SYSTEM FOR DISEASE DIAGNOSIS
Review round: 2
Reviewer: 3

Output:
Basic reporting: Authors revised the paper, so all comments were met
Experimental design: Authors revised the paper, so all comments were met
Validity of the findings: Authors revised the paper, so all comments were met
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ENERGY EFFICIENT SERVICE PLACEMENT IN FOG COMPUTING
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1. Authors are required to improve the English language and some sentence structures need to be updated.
2. Some cited references are very old of year 2012
3. Overall article structure is good.
4. Hypotheses and results are OK and shared with the RAW data.
5. In some sections authors have written too many paragraphs for expressing their views. Need to update the style with some professional writing style.
6. Provide expanded from of abbreviations wherever used for the first time.
Experimental design: Here, authors have proposed an improved version of JAYA approach for optimal placement of modules
that minimizes the energy consumption of a fog landscape. They have analyzed the performance in terms of energy consumption, network usage, delays and execution time.

- Authors need to provide the details about the JAYA approach. As authors have cited that this approach is published  @ref[19] but this publication is not a standard one and there is a doubt that the journal is a predatory journal. Authors are required to update this citation with some published/ cited with some standard publisher.

- Make your Introduction section more technical than in the current form it just focuses on the basics of fog computing.

- What is the worst-case time complexity of the modified JAYA approach and previously published JAYA approach.

- In the literature survey, authors are discussing vehicular communication.... why??

- Kindly refer the following work for further improvement in the literature survey section
10.1109/ISPCC53510.2021.9609479
10.4018/IJKSS.2020100102

- Why authors have not applied any MCDM based approach for the allocation of resources and services in the Fog landscape?

- Equation(3) is a general equation and computes the energy consumption. But how the Eq 2 has been derived?
- In the figure 2, there is a flaw in the flowchart. As the output from the first decision box irrespective of the condition whether it  is true or false is becoming the input for the next level.....Why??

- What is the purpose of Figure 3?/

- Figure 4 doesn't include the unit of energy consumption.

- Provide the latency comparison of all the approaches.
Validity of the findings: Obtained results are valid.
Additional comments: NIL


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ENERGY EFFICIENT SERVICE PLACEMENT IN FOG COMPUTING
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper is generally well-written and reasonably structured.
Experimental design: The authors have explained their system model and the proposed algorithm algorithm very well. However, they need to further clarify some points such as:
-	The complexity of the proposed algorithms are not analysed.
-	Some details about where (on edge-devices, on gateways, in cloud?) the proposed algorithms are run are also needed. Also, it is unclear how these algorithms are triggered to run. The algorithms are run every time a new application appears? Some details are needed;
Validity of the findings: The authors have simulated several representative scenarios. However, they need to compare their results to other relevant examples such as the Edgeward algorithm inherently built within the iFogSim simulator. In addition, they are kindly requested to compare their work to other closely related efforts such as the fowling papers:

https://dl.acm.org/doi/10.1145/3344341.3368795

https://link.springer.com/article/10.1007/s12652-021-02910-w

-	For the results shown in Fig. 4, the authors are requested to show the percent decrease in energy dissipation as compared to each existing algorithm. In addition, it can be notices that the EPSO algorithm has achieved very close numbers to the proposed algorithm.  In other words, they have mentioned that they have achieved 31% average improvement considering the Cloud-only algorithm which is considered as a naïve benchmark in presence of a bulk of recently proposed application placement algorithms. Moreover, they have less that 3% improvement as compared to the EPSO algorithm. In addition, they need to compare their results to other closely related efforts such as the aforementioned two papers.
-	Overall, the performance of the proposed algorithms is relatively close to that of the EPSO algorithm. Hence, the authors need to show what significant advantages their proposed algorithm can achieve.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ENERGY EFFICIENT SERVICE PLACEMENT IN FOG COMPUTING
Review round: 2
Reviewer: 1

Output:
Basic reporting: Authors have improved the language.
References have been updated that includes some more recently published work
Figures are OK.
Experimental design: The authors have carried out the various experiments as per identified problem statement.
They have updated the methodology section accordingly.
Validity of the findings: The proposed solution is novel and supported by results and comparative studies with previous approaches.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IMPROVED GENETIC ALGORITHM OPTIMIZED LSTM MODEL AND ITS APPLICATION IN SHORT-TERM TRAFFIC FLOW PREDICTION
Review round: 1
Reviewer: 1

Output:
Basic reporting: In this paper, author presented an improved genetic algorithm (OGA) to optimize long-term and short-term memory neural network (LSTM). However, there are some limitations that must be addressed as follows.
1.	Some sentences in abstract are very lengthy (for example see sentence 1). These sentences should be changed to make the abstract more attractive for readers.
2.	In Introduction section, it is difficult to understand the novelty of the presented research work. This section should be modified carefully. In addition, the main contribution should be presented in the form of bullets.
3.	Different existing application of traffic events or traffic flow should be discussed. In addition, the most recent work about Deep learning-based traffic events or traffic flow should be discussed as follows (‘Traffic accident detection and condition analysis based on social networking data’, ‘Fuzzy Ontology and LSTM-Based Text Mining: A Transportation Network Monitoring System for Assisting Travel’, and ‘Transportation sentiment analysis using word embedding and ontology-based topic modeling’, and ‘Fuzzy ontology-based sentiment analysis of transportation and city feature reviews for safe traveling’.).
4.	Equations should be discussed deeply.
5.	LSTM is not properly discussed in Section 4. What about Bi-LSTM?
6.	Captions of the Figures not self-explanatory. The caption of figures should be self-explanatory, and clearly explaining the figure. Extend the description of the mentioned figures to make them self-explanatory.
7.	The whole manuscript should be thoroughly revised in order to improve its English.
8.	More details should be included in future work.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IMPROVED GENETIC ALGORITHM OPTIMIZED LSTM MODEL AND ITS APPLICATION IN SHORT-TERM TRAFFIC FLOW PREDICTION
Review round: 1
Reviewer: 2

Output:
Basic reporting: The work is technically sound and has potential merit for publication. However, major revisions are needed to make it worth publishable.
Experimental design: The experimental section, in particular, the setup and parameters being used, needs further explanation.
Validity of the findings: The experiments seem inline, valid, and consistent with previous findings. However, the evaluation section is written well. However, it is not clear how the simulations were performed? Which optimization tool was used? Furthermore, the evaluation metrics should be briefly described.
Additional comments: I have the following comments and suggestions to further improve the quality of the work:

1) The abstract is not concise enough to sketch the entire theme, in particular, the results of the manuscript. Also, there were some serious grammar issues that should be corrected.

2) Furthermore, the introduction section needs considerable effort (concise and brief). The problem being investigated should be described clearly.

The introduction, e.g., should lead the way throughout the paper. In addition, the benefits coming from this paper should be made clearer in the introduction and throughout the paper.

- The entire introduction section is written in two paragraphs, while the first paragraph has been ended with a semi-colon (full stop needed)
- Furthermore, briefly describe the major contributions in bullet form, just before the organization paragraph.
- The final paragraph of the introduction section should be the organization flow.

3) I suggest adding a separate section that illustrates the related work. Various sections in the paper should be moved to this section. Moreover, a summary of the related work should be sketched into a table with respect to their characteristics. The authors should put their proposal into this table for easy comparison.

This will make it clearer to readers and they will be able to see what was missing in the literature; and how this is addressed in this paper.

4) Moreover, It would be better to add all notations in a Table for easy understanding.

5) The organization of the paper should be improved. For example, Sec. 5 has the proposed technique as well as results. I suggest putting them into two separate sections.

6) Does the results shown in various tables and figures refer to multiple runs (average)? In the latter case, I will suggest adding standard deviation bars. The reason behind this is to ensure that whether the results overlap with the closest rivals or not.

7) How about the time and space complexity of the proposed algorithms?

8) The evaluation section is written well. However, it is not clear how the simulations were performed? Which optimization tool was used? The experimental section, in particular, the setup and parameters being used, needs further explanation. Furthermore, the evaluation metrics should be briefly described.

9) I suggest adding pseudo-code for the proposed algorithm. Various procedures in the algorithm needs explanation. I suggest step-by-step details of the algorithm.

10) English can be improved. Proofreading should ensure the appropriate use of grammar, tenses, and punctuations. Longer sentences should be converted into smaller ones. Many words are missing their articles (the, a, an). Many punctuation characters are missing, and some are redundant.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IMPROVED GENETIC ALGORITHM OPTIMIZED LSTM MODEL AND ITS APPLICATION IN SHORT-TERM TRAFFIC FLOW PREDICTION
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors have addressed my comments properly. Therefore, this paper can be accepted in its present form.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IMPROVED GENETIC ALGORITHM OPTIMIZED LSTM MODEL AND ITS APPLICATION IN SHORT-TERM TRAFFIC FLOW PREDICTION
Review round: 2
Reviewer: 2

Output:
Basic reporting: The revised version is much better than the original submission. However, structure of the manuscript should be improved.
Experimental design: The authors have improved their experimental discussion in the revision stage.
Validity of the findings: The manuscript is well written, and the conclusions are supported by the experiments.
Additional comments: Please address the following comments:

1) I suggest adding a paragraph that describes the organization of the paper in the introduction section. Usually, this should appear just after the contribution list.

2) Furthermore, some sections are not numbered, while others are labeled incorrectly. I suggest that authors be consistent all over the paper.

3) The paper desires proofreading and substantial struggles to correct grammar, linguistic, and punctuation errors. I cannot list them all, but the majority are punctuation mistakes. Some sentences do not make sense at all and should be rephrased.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IMPROVED GENETIC ALGORITHM OPTIMIZED LSTM MODEL AND ITS APPLICATION IN SHORT-TERM TRAFFIC FLOW PREDICTION
Review round: 3
Reviewer: 1

Output:
Basic reporting: Well improved.
Experimental design: Well written.
Validity of the findings: The revised version has potential merit for publication.
Additional comments: N/A


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: THE ACCURACY OF RANDOM FOREST PERFORMANCE CAN BE IMPROVED BY CONDUCTING A FEATURE SELECTION WITH A BALANCING STRATEGY
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: 1. According to the Abstract and Introduction sections, the purpose of this paper is to improve the accuracy and processing speed of random forest, but the actual research only reflects the accuracy and does not analyze the processing speed improvement.
2. The parameters or hyperparameters of machine learning algorithm also have a great impact on the accuracy of the model. This work mainly studies the influence and improvement methods of feature screening and unbalanced data on the model accuracy of random forest, but does not study the influence of algorithm parameter or hyperparameter optimization methods (such as Bayesian hyperparameter optimization, refer to sun et al. 2020, 2021, Doi: 10.1016/j.geomorph.2020.107201, Doi: 10.1016/j.enggeo.2020.105972), and the present work cannot cover the subject. It is suggested to add "feature selection" and "SMOTE" to modify the title.
3. Does the features selection methods proposed in this work have comparative advantages with the widely used recursive feature elimination (RFE) methods, such as Zhou et al. (2021) used in DOI: 10.1016 / j.gsf. 2021.101211?
4. For the Conceptual method and Epilepsy data sets, the model effect is not good with lower accuracy and F1 score, whether it is improved or not. The possible causes should be analyzed in the discussion section.
5. There are meaningless to use broken lines in figure 2-4. It is suggested to change broken lines to histograms.
6. The number of factors of each data set by using each feature selection method are recommended to be expressed in a table.
7. Compared with table 2, the expression of conclusion a and b is inaccurate and confusing, which is recommended to be modified.
8. Table cs-68916-7_CNAE in the Supplemental Files seems incomplete, missing feature X1.
9. It is strongly recommended to add the content of the Discussion section.
10. See the Attachment PDF File for other writing errors.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: THE ACCURACY OF RANDOM FOREST PERFORMANCE CAN BE IMPROVED BY CONDUCTING A FEATURE SELECTION WITH A BALANCING STRATEGY
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper proposes a method to improve accuracy performance in random forest. Some important comments:
1.	The main contribution of this paper is not explained, for example the difference with previous studies.
2.	Explanation of the flow of thought on how IG uses the standard deviation (SD) as a threshold, because the standard deviation is a measure of the spread of data, not the concentration of data. Is the purpose of using SD as a threshold described in the paper the same as the research of Sindhu and Radha, 2020?
3.	In line 66, it is explained that the IG threshold using the standard deviation is less successful in determining the threshold when considering data balance, then in this study using the median. How to explain the flow of thought?
4.	It is necessary to explain the flow of thought why IG needs to be transformed using FFT, then to determine the threshold only use the real part. What does the real and imaginary part after the FFT process represent?
Experimental design: 1.	In this study using several unbalanced datasets, for performance measurement why only use accuracy, it is better to add others.
2.	Why for data separation using k-fold cross validation, not using a certain proportion between training and testing?
3.	In Figure 1, a flow chart that explains the SMOTE process for those with more than two classes, needs to be described in more detail, preferably with an example.
Validity of the findings: 1.	In Figure 4, the term first-SMOTE appears, what does it mean, where is it explained?
2.	In the results and discussion, the results obtained need to be explained more comprehensively based on the results of previous studies. For example in Figure 4A, why does the dermatology-first smote accuracy decrease sharply using the median threshold.
Additional comments: 1.	Split data using K-Fold validation instead of K-Folt validation (typo)
2.	It should be added in the background, the difference between the current research and previous research conducted by the researcher (it can be described as a kind of research roadmap).
3.	In the conclusion section, write a comprehensive conclusion, not just in the form of points without the connection between points.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: THE ACCURACY OF RANDOM FOREST PERFORMANCE CAN BE IMPROVED BY CONDUCTING A FEATURE SELECTION WITH A BALANCING STRATEGY
Review round: 2
Reviewer: 1

Output:
Basic reporting: 1) I think the 3 references mentioned in Concern #2 and # 3 of the last review comments should be added to the references to elaborate the research background in the Introduction section.
Experimental design: None
Validity of the findings: 2) Due to feature selection and SMOTE are common model optimization method in machine learning, it suggested to further strengthen the major contribution of this work concern on the influence and improvement methods of feature selection and SMOTE on the model accuracy and processing speed of random forest in the Introduction and Discussion section.
Additional comments: None


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: THE ACCURACY OF RANDOM FOREST PERFORMANCE CAN BE IMPROVED BY CONDUCTING A FEATURE SELECTION WITH A BALANCING STRATEGY
Review round: 2
Reviewer: 2

Output:
Basic reporting: All previous concerns have been addressed.
Experimental design: The corrections have been done as per requirement.
Validity of the findings: All suggestions have been fulfilled
Additional comments: The authors must improve the grammar of English writing


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: THE ACCURACY OF RANDOM FOREST PERFORMANCE CAN BE IMPROVED BY CONDUCTING A FEATURE SELECTION WITH A BALANCING STRATEGY
Review round: 3
Reviewer: 1

Output:
Basic reporting: All previous concerns have been addressed.
Experimental design: None
Validity of the findings: All previous concerns have been addressed.
Additional comments: None


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FG-DROID: GROUPING BASED FEATURE SIZE REDUCTION FOR ANDROID MALWARE DETECTION
Review round: 1
Reviewer: 1

Output:
Basic reporting: The English language used in the paper is not up to publishable academic standards. The abstract is written in the past tense. Apart from numerous grammatical mistakes, there are many spelling mistakes in the paper. What makes it worst, the text written from lines 174 to 177 is in some other language (possibly Turkish).
The above-mentioned mistakes prove that the author did not bother to proofread the paper before submission. This is unacceptable.
The literature review skeleton is good. However, a lot of language mistakes make it difficult to follow. Moreover, a comparison table of the reviewed papers should be included.
Figure 5 (Pseudo Code): It appears that the Pseudocode is generated through some automated tool. Convert it to write it in standard algorithm notations. The current form is unacceptable,
Experimental design: Needs to be reconsidered
Validity of the findings: The author proposes a reduced feature set (using just 11 static features) that is enough for classification tasks for Android malware and Benign apps. Moreover, the author claims that there is no need for an extended feature set for the classification task. I have some concerns in this regard:
1) Most of the literature published in the Android malware detection domain use significantly big feature sets for malware detection and classification tasks. The proposed techniques (I can not mention a single here because there are loads of them) have claimed up to 99% detection accuracy by using larger feature sets. Moreover, a lot of them are using fairly big datasets as compared to this study. How would you compare your results with them?
2) Another concern is about adversarial evasion attacks. A small feature set is more vulnerable to evasion attacks than compared to a diverse feature set. Have you considered it?
3) I can see that this study uses around 6k malware and 900+ benign samples. How can you justify this distribution? Normally, balanced datasets are employed. However, in real-world situations, malware samples are always less than benign, therefore, an unbalanced dataset (more benign and less malware still makes sense (As used in Drebin)).
4) You have used Drebin and Genome datasets. Just to let you know that all the malware samples in Genome are already a part of the Drebin dataset. Therefore no need to use the Genome as a separate dataset.
5) Drebin dataset is now obsolete. The apps in Drebin were gathered from 2010-to 2012. That's a decade old now. Please consider new datasets. A good option would be to use KronoDroid (2021).
6) Figure 8 is an example of poor presentation of results.
Additional comments: Although the paper should be rejected, however, I am suggesting a major revision at this stage point. All my concerns should be addressed one by one in rebuttal for further considereation.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FG-DROID: GROUPING BASED FEATURE SIZE REDUCTION FOR ANDROID MALWARE DETECTION
Review round: 1
Reviewer: 2

Output:
Basic reporting: The English language should be improved to ensure that an international audience can clearly understand your text. Their are too many grammatical mistakes, the article need to go through proofreading
Introduction should be rewrite again
Missing sentences in Line 16
This has resulted in the undesired installation of Android apks that violate user privacy or malicious ???
Line 32 it -> It, Also, too many use of 'it' in paragraph
Line 45 ->When malicious applications Access user mobil devices - correct it
Line 51 ->suffiecient -> correct it sufficient
Line 55 -> Malware detection mechanism in different types have been proposed to address this (WHAT IS THIS?? IN THIS SENTENCE)need in the security mechanism. These (?) are signature-based approach ...
Line 79 wit -> with
Experimental design: Line 238 While adding to the csv file, malicious and benign applications were labeled. Please explain How the data is labeled in Figure 2
Line 241 The processing steps shown in Figure-2 were carried out by means of an automatic code, and it
is a very fast feature extraction process. How this process is fast? please explain in detail
Automatic Feature Extraction and Pre-processing section need to be explain properly. Things are overlapping and explained shortly.
Validity of the findings: Explain how FG-droid is efficient as mentioned in contribution?
Line 427 -> While some of these obtained features contribute positively to the classification performance, some may have no effect at all, and some may have a deteriorating effect. Explain How? What are their limitations? What have you achieved?
Additional comments: In this study, FG-Droid, a machine-learning based classifier, using the method of grouping the
features obtained by static analysis, was proposed. It was created because of experiments with Machine
learning (ML), DNN, RNN, LSTM and GRU based models using Drebin, Genome and Arslan datasets.

However, Clear, unambiguous, professional English language is missing throughout the paper. Overall the structure of the paper need to be modified again. Experimental evaluation with other studies are missing in terms of novelty.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FG-DROID: GROUPING BASED FEATURE SIZE REDUCTION FOR ANDROID MALWARE DETECTION
Review round: 2
Reviewer: 1

Output:
Basic reporting: I strongly recommend the authors to re-write the last sentence in the abstract. The word "thanks to" should be replaced. Apart from that, the paper satisfies the publication standards.
Experimental design: Good
Validity of the findings: Justified
Additional comments: The author has put a lot of effort to enhance the quality of the work. I am happy to accept the paper for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A RETROSPECTIVE STUDY OF 3D DEEP LEARNING APPROACH INCORPORATING COORDINATE INFORMATION TO IMPROVE THE SEGMENTATION OF PRE- AND POST-OPERATIVE ABDOMINAL AORTIC ANEURYSM
Review round: 1
Reviewer: 1

Output:
Basic reporting: a.	Clear and unambiguous, professional English used throughout.
The paper is generally well-written and easy to follow, despite some sentences are confusing and need revisions:
Line 252-253 (“The experiments were implemented using the PyTorch (v1.8.0) deep learning library with Tensorflow backend in Python”) is confusing as PyTorch and TensorFlow are two different DL libraries, and as far as I know, TensorFlow cannot be used as backend of PyTorch;
Line 273-274 (“… the non-contrast and contrast-enhanced datasets in consecutive arrays were combined in another experiment”) says there is “another experiment”, please make it clear which experiment is referred to here.
b.	Literature references, sufficient field background/context provided.
The paper introduces sufficient background on AAA segmentation. However, while coordinating information and transfer learning are the two main techniques of this paper, the related work section only briefly introduces them. As they are not originally proposed by the authors, it is important to provide more details on these two methods.
Also, in lines 156-157 (ResNet is a kind of popular network that has been proven effective in medical data), ResNet deserves a more detailed explanation, and reference should be provided on the claim that it is effective in medical data.
c.	Professional article structure, figures, tables. Raw data shared.
Figure 5 training curve is hard to read. Increasing the resolution of the font is encouraged.
d.	Self-contained with relevant results to hypotheses.
No comment.
e.	Formal results should include clear definitions of all terms and theorems and detailed proofs.
No comment.
Experimental design: a.	Original primary research within the Aims and Scope of the journal.
No comment.
b.	Research question well defined, relevant & meaningful. It is stated how research fills an identified knowledge gap.
The paper conducts extensive experiments on the proposed coordinate integration methods. As the use of positional encoding is a hot topic in computer vision, how does the proposed scheme compare to other positional encoding method, for example, the fixed positional encoding in [1] (use sinusoid to encode the position) or learnable positional encoding?
[1] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I., 2017. Attention is all you need. Advances in neural information processing systems, 30.
c.	Rigorous investigation performed to a high technical & ethical standard.
No comment.
d.	Methods described with sufficient detail & information to replicate.
Line 233 is suggested to change into “the input data for CNN will be two or four”.
Validity of the findings: a.	Impact and novelty not assessed. Meaningful replication encouraged where rationale & benefit to literature is clearly stated.
No comment
b.	All underlying data have been provided; they are robust, statistically sound, & controlled.
No comment.
c.	Conclusions are well stated, linked to original research question & limited to supporting results.
c.1: In table 2, we can see that the integration of coordinate information as an additional input only has marginal improvement over the baseline UNet. For example, in the contrast-enhanced dataset the DSC and JSC improvement is less than 0.2% in UNet. This challenges the main contribution of this paper.
On the other hand, the coordinate information significantly improves the performance of DenseVoxNet, and the authors explain in line 376-379 that coordinate information make the network converges easier. I would suggest investigating further on this direction.
c.2: The author states that the main limitation of CNN-based methods is the limited size of input data, due to the GPU memory constraints. The common practice in medical deep learning is to use patch-based inference, which means a 512x512x64 volume can be patched into multiple, e.g. 256x256x32 patches so as to maintain the original resolution. In this paper, the authors choose to scale down the image volume.
Additional comments: This paper proposes incorporating voxel position as an additional input for AAA segmentation. The paper needs more work on the related work section, including how the proposed coordinate encoding scheme compared to previous works, and why the scheme would be a good fit for the problem as well as transfer learning.
The authors conduct extensive experiments on various network architectures, nevertheless, the improvement is marginal over the baseline UNet. Also, the novelty of the proposed method is insufficient and most of the work is derivative. I suggest looking at more methods to incorporate positional information, e.g. in multiple layers, or a better encoding scheme.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A RETROSPECTIVE STUDY OF 3D DEEP LEARNING APPROACH INCORPORATING COORDINATE INFORMATION TO IMPROVE THE SEGMENTATION OF PRE- AND POST-OPERATIVE ABDOMINAL AORTIC ANEURYSM
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper presents a new 3D AAA segmentation approach that incorporates coordinate information to improve the segmentation results. The authors have tested the proposed method on various network architectures, including UNet, AG-DSV-UNet, VNet, ResNetMed, and DenseVoxNet, and transfer learning from a network pre-trained on the pre-operative dataset to post-operative EVAR.

1.	The authors didn't explain how to obtain the coordinate information for test images, which was critical to applying the proposed method in real-world applications. We usually use prediction methods for unseen images in practice, and obtaining coordinate information for these images could be difficult.
** Please provide the related explanation in the paper.

2.	Some recently published related works are not referred to and compared in the paper. For example
[1] Wang, Yan, Florent Seguro, Evan Kao, Yue Zhang, Farshid Faraji, Chengcheng Zhu, Henrik Haraldsson, Michael Hope, David Saloner, and Jing Liu. "Segmentation of lumen and outer wall of abdominal aortic aneurysms from 3D black-blood MRI with a registration based geodesic active contour model." Medical image analysis 40 (2017): 1-10.
[2] Salvi, Anish, Ender Finol, and Prahlad G. Menon. "Convolutional Neural Network-based Segmentation of Abdominal Aortic Aneurysms." In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 2629-2632. IEEE, 2021.
[3] Dziubich, Tomasz, Paweł Białas, Łukasz Znaniecki, Joanna Halman, and Jakub Brzeziński. "Abdominal aortic aneurysm segmentation from contrast-enhanced computed tomography angiography using deep convolutional networks." In ADBIS, TPDL and EDA 2020 Common Workshops and Doctoral Consortium, pp. 158-168. Springer, Cham, 2020.
[4] Wang, Yan, Florent Seguro, Evan Kao, Yue Zhang, Farshid Faraji, Chengcheng Zhu, Henrik Haraldsson, Michael Hope, David Saloner, and Jing Liu. "Segmentation of lumen and outer wall of abdominal aortic aneurysms from 3D black-blood MRI with a registration based geodesic active contour model." Medical image analysis 40 (2017): 1-10.
** Please read and cite these papers accordingly in your paper.
Experimental design: 1. No validation set is created and used in transfer learning so it is unclear to me how do the authors tune the model hyper-parameters in this case. Please describe it in detail in the paper.

2. Please provide a detailed definition of dice score, including how to compute it.
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTIVE ATTRIBUTED NETWORK EMBEDDING WITH INFORMATION BEHAVIOR EXTRACTION
Review round: 1
Reviewer: 1

Output:
Basic reporting: Throughout the manuscript, this paper uses clear and unambiguous professional English. Although adequate literature references and field background/context are provided, the literature discussion is missing a few related works. The proposed method is self-contained, with results correlating to hypotheses. The formal results should contain precise definitions of all terms and theorems, as well as detailed proofs.
Experimental design: Numerous evaluation results corroborate the proposed method. The research question is clearly defined, pertinent, and meaningful. It is stated how research fills a knowledge gap that has been identified. Extensive investigation conducted to the highest technical and ethical standards. Methods described in sufficient detail and detail to permit replication.
Validity of the findings: The underlying experimental results have been provided in its entirety; they are robust, statistically sound, and well-controlled. Conclusions are succinct, relevant to the original research question, and limited to supporting data.
Additional comments: This manuscript is written in clear and unambiguous professional English throughout. While adequate references to the literature and background/context of the field are provided, the literature discussion is missing a few related works [1,2,3,4,5]. The proposed method is self-contained and produces results that are consistent with hypotheses. All terms and theorems should have precise definitions in the formal results, as well as detailed proofs.

[1] WebFormer: The Web-page Transformer for Structure Information Extraction---WWW 2022
[2] A vector-based representation to enhance head pose estimation---WACV 2021
[3] Sg-net: Spatial granularity network for one-stage video instance segmentation---CVPR 2021
[4] DenserNet: Weakly supervised visual localization using multi-scale feature aggregation---AAAI 2021
[5] Video object detection for autonomous driving: Motion-aid feature calibration---Neurocomputing 2021


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTIVE ATTRIBUTED NETWORK EMBEDDING WITH INFORMATION BEHAVIOR EXTRACTION
Review round: 1
Reviewer: 2

Output:
Basic reporting: This manuscript captures the structural and attribute features of the network using existing network embedding algorithms, and proposes the TNE model to extract additional information behavior features. Finally, the link prediction performance of the network is improved.

1.The number of references in the past three years (2019-2022) is only two, and it is recommended to add more latest references.
Experimental design: 1. In EXPERIMENTS, the latest comparison algorithm is from 2017, and it is recommended to add newer algorithms for comparison.
2.From the descriptions of Section 3 and Section 4, it can be seen that the extraction of node information behavior features requires additional topic information. From the experiment, the topic information seems to be the category information of the node. That is, in Table 2, the algorithm of the manuscript is supervised, while the algorithm used for comparison is unsupervised, that seems unfair. It is recommended to add some supervised/semi-supervised graph neural network, such as GAT, GIN, for comparison (the amount of information (category information) used should be consistent).
3.The overall performance of the algorithm is improved, but the increase in time complexity caused by adding information behavior features is worrying, such as the calculation of parameters \alpha. It is recommended to add time complexity analysis, including elaboration of the average number of cycles (convergence rate) of the ISR algorithm.
4.The number of topics in Zhihu described in Table 1 is empty, what is the basis for setting the number of topics as 13 in the experiment?
5.It is recommended to include a graph of the Importance Score distribution of nodes in the experiment to eliminate the concern that the importance may all be clustered into a very small number of nodes.
6.The manuscript adds Parameter sensitivity analysis, that is good. In the analysis of the topic number j, it is best to add a description of the implementation, for example, for nodes with unknown topics, how are they handled in the ISR algorithm?
Validity of the findings: no comment
Additional comments: 1.The manuscript does not mention the algorithms incorporating information behavior in INTRODUCTION or RELATED WORK, which seems to imply that this manuscript is the first network embedding work that incorporates information behavior. Is this true? If so, please state clearly in ABSTRACT and INTROCUCTION. If not, please add the description of the corresponding algorithms, and add the comparative experiment of some corresponding algorithms in the experimental section.
2.Are the information behavior features and node categories the same? If so, a large number of related papers have proposed, that adding information behavior features cannot be a bright spot. If not, does the dataset in the experiment reflect the difference between them?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTIVE ATTRIBUTED NETWORK EMBEDDING WITH INFORMATION BEHAVIOR EXTRACTION
Review round: 1
Reviewer: 3

Output:
Basic reporting: This paper presents an embedding method of attribute network through ISR algorithm and topic sensitive embedding matrix. Then, the above embedding is spliced into the embedding obtained by the existing characterization methods through appropriate methods. In this paper, experiments are conducted to verify the effectiveness of the method. However, this paper also has some issues.

1.	The main purpose of this method is to obtain the embedding of attribute network, but it is not compared with the latest attribute network embedding methods. It is suggested to add relevant baseline algorithms, such as GCN[1], GAT[2] and AM-GCN[3].
2.	There are lots of advanced attributed network embedding methods that should be discussed in the related work, such as MAGNN[4], FAME[5].
3.	The datasets used in this paper are relatively small, and efficiency verification should be carried out on large-scale datasets.
4.	Ablation experiments should be conducted to verify the effectiveness of model components.

[1] Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.
[2] Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR.
[3] Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, and Jian Pei. 2020. Am-gcn: Adaptive multi-channel graph convolutional networks. In KDD. 1243–1253.
[4] Xinyu Fu, Jiani Zhang, Ziqiao Meng, and Irwin King. 2020. Magnn: Metapath aggregated graph neural network for heterogeneous graph embedding. In WWW. 2331–2341.
[5] Zhijun Liu, Chao Huang, Yanwei Yu, Baode Fan, and Junyu Dong. 2020. Fast Attributed Multiplex Heterogeneous Network Embedding. In CIKM. 995–1004.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: no comment