Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CONCURRENT DESIGN OF CONTROL SOFTWARE AND CONFIGURATION OF HARDWARE FOR ROBOT SWARMS UNDER ECONOMIC CONSTRAINTS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper has a good presentation quality with sufficient background.
Experimental design: The paper's relevance lies in the novel automatic "configuration" of hardware. The use of extensive simulation suffice.
Validity of the findings: Results are well presented. Particularly, Figure 5 is very useful and provides an interesting overview of the automatically obtained controller.
Additional comments: The work extends AutoMoDe to include the automatic "configuration" of hardware alongside the existing automatic design of control software. The proposed hardware configuration involves the selection of appropriate range-&-bearing receivers and transmitters as well as the selection of the number of robots in the swarm subject to battery and economic constraints. Although the work is carried under simulation and using hypothetical range-&-bearing modules, it represents a valuable insight into the automatic selection of hardware modules for swarm robotics and it is worthy of publication.

I provide below a few minor suggestions(+) and strongly encouraged changes (*), which I hope will improve the paper:

+ I am not sure that hardware “configuration” is the best term, as it seems that the automatic design select appropriate hardware modules/components.

+ In the introduction it could be beneficial to include an explanation of what are constraints and what are objective functions in an optimisation context. Then later, when you talk about economic and battery constraints mention that the objective function depends on the task that will be solved.

(*) Page 5 lines 176-177: The authors state that “In a probabilistic finite state machine, states are instances of behaviours and edges are instances of conditions” and this is not true for all probabilistic FSM. See the difference between Mealy and Moore machines, and check the following work for another perspective (citation to the work not required):

Lopes et al. Probabilistic supervisory control theory (psct) applied to swarm robotics. AAMAS 2017.

Thus, the authors may want to state that in their case states are associated with behaviours and transitions are associated with conditions, but not state that this is always the case.

+ Page 5 lines 203-204 (Table 3) states 6 receivers and 2 transmitters, but notice that $T^1$ has 3 range options and $T^2$  has 2 range options. When looking in Figure 4 this gets non-intuitive as the x-axis relates to the 6 receivers plus no receiver and the y-axis states the range. Maybe present it as 6 transmitters would be more clear.

+ Figure 1 is a quite unconventional way to represent a finite state machine (automaton). Usually, double-line circles represent marked states and an initial state should be indicated by an unlabelled arrow. The transitions could be represented by labelled arrows without the lozenge. I would also suggest describing the symbols in the figure's caption.

(*) Table 2. Please provide a detailed description of the components of the aggregate position (V).

(*) I also would like to see 2 or 3 scenarios (shown in a figure) and the resulting aggregate position (V), as defined in Equation 1, this would help to understand the meaning of this reading.

+ Page 4, line 182 it would be nice to include the version of the e-puck under consideration. Note that since 2018 e-puck2 is available, but it seems that version 1 is used.

+ Regarding the objective functions in Equations 2 and 3, I am curious about what would be the implication of divide the summation by the number of robots, so we have the ration of robots instead of the absolute quantity. How is the impact on the number of robots & hardware selected? I risk speculating that if the control software is poor it would tend to select a lower (and perhaps odd) number of robots, as it tends to obtain an average 50%-50% distribution between the black areas (7-8)/15 = 1/15 which would be bigger than (14-15)/29 = 1/29. However, a controller able to aggregate most of the robots in a particular black area as long as it has a “good” (and therefore more expensive) hardware may also select a low number of robots to attempt obtaining 15/15 = 1.0. This would be different from the current results that tend to select many robots. In principle simulations under this new objective function would be ideal, however, this could be discussed and presented as future work.

+ In table 4 you may consider using “Unconstrained” instead of “unlimited”

+ Page 9, line 309: add “on” --> “...swarm depends not only on the formation ...”


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC REVIEW ON HAND GESTURE RECOGNITION TECHNIQUES, CHALLENGES AND APPLICATIONS
Review round: 1
Reviewer: 1

Output:
Basic reporting: This is a review paper, which aims at providing an overview of current tools and techniques within the field of hand gesture recognition. The paper is generally well referenced, but the text is somewhat hard to read, and I would recommend additional editing with a focus on making the text more readable. The authors provide a nice overview figure of their method, but it might be helpful if some figures could be added that also visualise the outcomes of the work. My main concerns, content-wise, with this work lies in the initial sections - the introduction and the methodology (see experimental design).

Introduction
I find it unclear what the authors really mean with hand gesture. Using gestures on screen has been a standard technology for a long time, an in essence already the mouse is an indirect hand gesture interface. The intro, on the other hand, is written as if hand gesture recognition is something that lies in the future.

I also question the general statement that users need to be experts; yes, gesture UI:s are often designed that way, but this could be seen as a design feature – if the system would be truly intelligent (at some point in the future) it might respond appropriately to user gestures the way humans are able to. Thus, this point needs some explanation and discussion.

The intro is written with image recognition as the main technology for gesture recognition. I would suggest it more appropriate to use more generic wording – after all the literature study does not focus on image recognition alone.

I must admit that I question that focusing on 2016-2018 will give us “the valid base of the current situation and technologies of hand gesture recognition.”.  With this focus, standard technologies and well established tools will be missed. You also miss early innovative work that may have been discontinued due to technology immaturity or just trends in the technology field. As an example: in the related field of haptics, a lot of work was done in the 1990’s- early 2000’s, and then, since much of the technology development turned mobile, less work was done, leading persons who revisit the field to miss quite a lot of the existing knowledge if they only take the most recent work.

To conclude, I would like to see the introduction rewritten to:
1)	Better motivate the very limited selection
2)	Clarify what is meant by gesture
3)	Improve the explanation of gesture UI:s

A note: the authors may want to check out the ISO standard 9241-960 Framework and guidance for gesture interactions.
Experimental design: Survey methodology
As already explained under introduction, I question the assumption that limiting the search to 2016-2018 will provide answers to the questions that are listed as the research questions. Either provide a better motivation for why this is the case, or change the questions so that the study answers the questions you have.

I am also slightly skeptical to the search terms. It is always a challenge finding the terms people use, and one may easily miss chunks of research simply because slightly different phrases/terms/expressions are used in an area. I would have liked to see an early exploration of which terms that could be used for the search. Eg. unless some exploration is done, you won’t know if some authors might talk “gesture interaction”, “gesture interfaces” leaving out the “hand” even though the actual interaction/system they discuss is hand based.

I would like more details on the process. It is stated “excluded for having titles with no relevance to the review in general” – please explain a little more what this actually means. The same goes for “All 244 papers were reviewed and evaluated in reference to the research questions, this process excluded 100 papers” – what was really excluded at this stage? Please explain better, maybe provide examples? I find it unclear what was really meant by “49 unwell-organized documents were removed due to lack of validation and weakness of results justification” – please provide more clarity on this process.

To conclude, this section needs revision and clarification. Please:
1)	Improve the motivation for the research questions alternatively rewrite the questions themselves.
2)	Provide more detail on the search and selection process. As it is, the reader cannot really follow what you did, and is thus cannot judge how appropriate your results are.

Results and analysis
Not being an expert in gesture recognition (I have used it, and have implemented basic gesture recognition but would not consider myself an expert), there is much in this section that appears quite useful. At the same time, the section is hard to read, and I wonder if it wouldn’t be possible to organize the text more also in the subsections (maybe provide a short overview first in each subsection, and possibly some illustrative images)? As I stated initially, I don’t have problems with the things that are found – these appear sound – what I am worrying about is what it is that might be missing, given my stated methodological concerns.

To conclude
1)	Please re-work this section so that it becomes more readable.
Validity of the findings: To sum up, while this texts does provide a useful overview of tools and techniques in gesture recognition, my main concerns with this work is methodological. I question the limitation to 2016-2018, and I am not convinced the method actually answers the stated research questions.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC REVIEW ON HAND GESTURE RECOGNITION TECHNIQUES, CHALLENGES AND APPLICATIONS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper follows the appropriate structure for a review paper.

In the main, the English language is easy to read, however, it would benefit from proofreading particularly for grammatical errors.  For example, (1) there are multiple examples of run-on sentences (e.g. line 79 “In artificial intelligence, machine learning enables the computers to learn without being explicitly programmed (Margaret, 2016), there are two types of learning; the process when algorithms ….), (2) incomplete sentences (e.g. line 82: “Classification aims of building a model based on previous training hand gestures to classify new hand gestures.” and  (3) poor/incorrect wording, including use of the word “researches” where “research” would be more appropriate (e.g. in the abstract’s Conclusion “We shall also introduce the most recent researches..” and reference to “unwell-organized documents” in line 122 and elsewhere.

The raw data is supplied as a link to a .rar file, which I did not seem to be able to open without downloading and installing additional software.
Experimental design: This paper presents a review of prior work on hand-gesture recognition.  The topic is timely and interesting, however I feel the survey methodology has a number of important limitations:
1.	The search was limited to IEEE Explore, and hence would have missed a large body of related literature outside of IEEE, for example, papers in the ACM Digital Library.
2.	Two main “keywords” were used:  “hand gesture recognition” and “hand gesture techniques.” It’s unclear if these search terms would have found papers that use other related terminology (e.g. in-air gestures) or terms that describe some of the common technologies (e.g. Leap motion).  It is also important to clarify if touchscreen gestures are included in this review or not.
3.	The methodology needs further elaboration and examples of exclusion criteria, for the reader to be able to understand fully and have confidence in the method.  For example “after removing duplicates” – how were duplicates defined?  “papers were excluded for having titles with no relevance to the review” – what were the criteria for relevance, who applied them, are they unambiguous and repeatable by different reviewers?  This also calls into question the appropriateness of the search terms (see #2 above).  “unwell-organized documents were removed” – again, how was this defined and applied?

It would also be good to see some further explanation of how many researchers reviewed the final list of included papers and what were the steps involved in reviewing the papers.

I also wonder if “scoping review” or “state of the art review” would be a more accurate reflection of this work, since “systematic review” often suggests a review of studies that have a common aim of answering a (healthcare-related) question.
Validity of the findings: The authors present a substantial body of literature.  To a large degree, it reads as a list of who did what and it would be more valuable to the reader if there was greater synthesis of the literature, drawing out insights derived from having oversight of the field.

The authors have grouped the results/analysis into headings and subheadings, which is important for a review.  However, I felt the chosen structure of headings and subheadings did not work very well, particularly for section 1 and its subsections.  For example, the choice of features depends very much on the underlying data that has been captured – and so it does not seem to make much sense to list features used by computer-vision based systems alongside EMG features, without the context of what each system is aiming to achieve. One suggestion is to organise the literature in terms of the acquisition systems, and then present the features and classification methods that have been investigated for each acquisition system.  This would enable like-for-like comparisons to be made, and also enable comparisons across different acquisition systems.  Similarly, the technical challenges are highly-dependent on the type of system being used, and so again, it would make more sense to present the challenges by system, and then draw out common themes within and across types of system.

The section on applications presents a very limited view of where gestures are being used and explored currently, and I suspect is a result of the search being limited to IEEE Xplore.  It does not seem to reflect the current state of the art, and I think it likely that expanding the search would enable the authors to expand out the “Others” section quite substantially.

The section on “the future of hand gesture recognition” is quite speculative, and while I understand speculation is acceptable and welcomed in PeerJ, I feel it still needs a good grounding in the literature and this grounding is not evident in this section.

The conclusions don’t all seem to follow from the literature – for example, much of the literature presented in the paper appears to use camera sensors, whereas the paper concludes that surface EMG “were the most acquisition tools used”.
Additional comments: The topic of this paper is timely and would be of interest and value to the research community.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC REVIEW ON HAND GESTURE RECOGNITION TECHNIQUES, CHALLENGES AND APPLICATIONS
Review round: 2
Reviewer: 1

Output:
Basic reporting: I think the authors have generally done a good job content-wise, and am generally happy with the content of the text. There are still some language problems that needs to be fixed, but this is hopefully a minor revision.
Experimental design: The authors have generally answered to my concerns. A remaining comment: on row 123 it is stated that other keywords were used. Please list which they were.
Validity of the findings: The authors have generally answered to my concerns.
Additional comments: As I stated initially, I think the authors have done a good job on addressing my concerns content-wise. What remains is to go over the language again. I am not good at proof reading, so I will not provide a complete list, but I will point to places where I have noticed problems:
On row 43 "Where the movement ..." is an incomplete sentence. My guess is that this is actually meant to be part of the previous sentence. If not, it is still unclear which type of hand gestures you are considering (my understanding is in-air hand gestures, not on-device gestures like gestures on a touch screen).
The sentence on row 44 "Which gives an example...." is unclear - what does the "which" point to. I would suggest re-phrasing this sentence (as well as the one before) to make the intent clearer.
Row 63 "huge expand" - is this correct english?
Row 100 "intensity" of a study - what is this? I don't quite understand what you mean - please re-phrase
Row 126 the sentence about the duplicates is long - maybe split it to make it easier to read?
Row 136 is "unwell-organised" really good english? Maybe re-phrase (eg. poorly organised).

The text in 1.1 and 1.2 is quite technical, and not being an expert I find it harder to comment on.

In section 2 there is some phrasing which is dated - in the old days one would say deaf & dumb, because deaf persons might not speak (not seeing the point since they were unable to hear). I would suggest using persons with hearing or speech impairments (if this is what you mean). I also note that you say "sign language", but it appears what is actually meant is finger spelling? In a full sign language facial expressions play a part and you might use your hands on another part of your body - eg. touch your ear as to indicate earrings /show the shape of a breast on the chest to indicate "woman" (I am no expert on sign language, these are examples I have been told).

Section 3 again, is technical. The final two sections work for me, although the final sentence on row 762 could possibly be made less passive to make it flow better (something along the lines of  "a common problem found in many of the studies was overfitting in the datasets").

Since this list is not complete, I would encourage the authors to make a general final round of language "polish".


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 1
Reviewer: 1

Output:
Basic reporting: In this work, authors propose a centrality metrics for complex networks based on the idea of entropy. Given one node, the centrality of which we want to calculate, paths starting from it are generated, and the entropy associated with the probability of reaching other nodes is evaluated. With respect to previous works, here a complication is introduced: paths (or flows) can split. In other words, suppose that a flow is exiting a node A, the latter being connected to B and C. Usually two possibilities are considered: either the flow passes through B, or it passes through C. In this work, on the other hand, the flow can be arbitrarily divided between B and C (such that, for instance, the 30% of it goes to B, and the 70% to C).

I have two major, and really important, negative comments about the proposed idea.

First of all, it is not clear why such complication is really needed, and whether the same problem could be solved by “renormalising” all link weights. Let me explain this with a slightly more complex example. Suppose the flow is exiting node A, which is connected to B and C. Also suppose that we have two possible “output patterns”: either the flow only goes to B with a 50% probability, or it goes to the pair (B, C) with a probability of 50%; also, in the latter case, the flow is split, such that half of it goes to B, and half of it to C. While this seems complex, it is easy to see that it is equivalent to say: 75% of the flow goes to B, and 25% to C. Interpreting this as probabilities, by starting from A we have a 0.75 probability of ending up in B, and 0.25 of ending up in C. Therefore, by renormalising the probabilities, we end up with a graph that can be analysed by more standard approaches - in a much more computationally efficient way. Thus, why do we need the framework proposed by the authors?

Secondly, and even provided that simpler solutions would not work, I fail to see when this approach would be really important. There are not many real-world situations that fulfil the proposed scenario; and actually, it is curious to see that this “split” mechanism is not used in the three examples provided by the authors! To illustrate, let us consider the case of the airport network. The full idea here presented, when using the example I reported in the previous point, would imply something like: “Airport A is connected to B and C. Passengers departing from A have two options: either take a flight whose destination is B, or take a second flight which will bring half of the passengers to B, and the other half to C.” Clearly this is quite weird!
I think that the authors completely fail at explaining why this tool, and its underlying hypotheses, is relevant in a real-world scenario.

Additionally, I have a set of smaller comments, which are reported below.

Table 1. The reported centrality is difficult to be analysed, as it is not compared with other standard approaches. For instance, I am pretty sure PWM would rank very high when considering a Betweenness or an alpha centrality. Authors should present this comparison, also to explain what is the difference between these metrics and the proposed one.

I find Fig. 1 quite useless, as the same example is then reported in Fig. 2 and following ones. Actually, the latter version is much better, as the figure reports more information (as probabilities, self-loops, etc.). Same for the text in the introduction; essentially, the same scenario is repeated several times. I don’t think that a very detailed description of this example is needed in the introduction, as it will be explained in Section 2.

The style of the text can be improved, and especially can be made more formal. I’ve lost count of the number of times the expression “say” is used in the introduction!

The bibliographic section could be made more complete. For instance, I find it surprising that no citations about complex networks theory are included. Note that the average reader of PeerJ may not be familiar with complex networks.

Line 188: “We notice the similarity with (1), which shows that the split-and-transfer entropic centrality is equivalent to the transfer entropic centrality, assuming the suitable computation of edge probabilities.” I find this sentence not really useful. According to definition 1, and if we consider just a “similarity”, every measure based on the Shannon entropy is also equivalent to the transfer entropic centrality!

Lines 209-210: bad formatting. A full stop is in the wrong position.
Experimental design: See previous section
Validity of the findings: See previous section
Additional comments: See previous section


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 1
Reviewer: 2

Output:
Basic reporting: The manuscript entitled "A split-and-transfer flow based entropic centrality" presents a new centrality measure which is based on a previously proposed entropic centrality.

The manuscript is well written and there are just a few typos. However there are some corrections to be performed in the text. Some of them are listed below.

- line 98: missing the number of the section which are being referenced
- line 164: "... corresponds to a set of edges (u,v) for v a neighbor"
- line 210: punctuation is missing before "The circled"

The main background and the introduction of the theme are well presented, however there are no references concerning related works in literature besides the one from which the proposed method is derived. Therefore, the introduction must be improved for what concerns other centrality measures and related works. What has been done so far?

The figures would be improved by using a vector format, like pdf or eps.

All the tables must be improved concerning their formatting. Captions must be located above the table. Table 1 should be divided into two tables.

In Example 1, the authors should assign a number to the sample figure.
Experimental design: The purpose of the study described in the manuscript is well defined and the main topic is relevant to the scope of the journal. The methodology is also well described and the several examples in the text illustrate very well the proposed approach.

The manuscript is well structured, presents a logical sequence of sections and a good summary of the obtained results. However, the conclusion part is missing. I also missed some discussion to what extent the proposed methods can be compared with other methods in literature.
Validity of the findings: It is missing a clear description of the data concerning its availability. The datasets can be freely accessed?

The supplementary material should contain a document explaining how to use the data and the available code.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 1
Reviewer: 3

Output:
Basic reporting: The paper is designed to proposed a novel split-and-transfer flow for centrality entropy measure. Usually, a flow is considered to be indivisible instead to have an arbitrary split across choices of neighbors. Also, the authors show a scaled version of the proposal to allow probabilities on edges based on the application criteria.
Three case studies are presented to show the proposal.

The paper is very insteresting and show the proposal measure with very detailed images and accounts.

However, the work has several issues that should be modified prior to its publication.

1 - The writing mainly in the introduction section is sometimes very informal with many repetitions of words and to many uses of `say`. Try to substitute the word graph by network, or egde by connections just to improve the reading flow.

2 - The definitions of the terms are good mainly due to the draws in the paper.

3 - The number of sections at the end of the introduction are missing.

4- Wikipedia reference is not accurate for a scientific paper. Try to find another reference to enforce your sentence and the reference does not seems correct in the text.
Experimental design: The content of the paper is relevant and results prooves that the proposed model is applicable for real cases analysis.

Previous works focused on indivisible flow, while the study in question aims to compute centrality entropy in split-and-transfer flows.

Three case study are provided. the first in the main airport traffic, the second with shareholdings in tehran stock market and finally, the last case which analyses bitcoin transactions.

Results proves that the importance of nodes is the network proportional to the entropy centrality captured by the split-and-transfer flow from each node. Also, using a scaling function, the result is even better.

Since the definition of the flow and measures are explained with equation and examples, the problem seems to be reproductible.
Validity of the findings: The  papers does contribute with the area and the results are very good.

A conclusion section could be added to summarize the proposal and reinforce its contribution with the results found the case studies.
Additional comments: - The work is very good in general, with good contributions to the science and very detailed explanations of the formulas.

- The network figures should be analysed to fit it better. The wikipedia reference is not very scientific and should be changed. Also, major revisions must be performed in the grammar (words repetitions and unformal english), latex references (sections numbers do not appears) to improve the reading.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 2
Reviewer: 1

Output:
Basic reporting: While the paper has improved from the initial version, I still have problems with it, and specifically I fail to see the relevance of what proposed.

If I’m not wrong, the main assumption and novelty of this paper resides in the fact that flows are considered as divisible. From this we have the scenario of Fig. 3: in Fig. 3a one flow splits between v2 and v3, and in 3b another flow splits between v3 and v4.
Now, the authors state that: “We thus showed that the split-and-transfer entropic centrality is equivalent to a transfer entropic centrality, assuming the suitable computation of edge probabilities.” This is exactly my point: isn’t it always possible to reduce this complex split-and-transfer scenario to a more standard one, by changing the link probabilities?

Note that the authors previously answered:
“The high level observation made by the reviewer, that a renormalized link weight may capture the effects of arbitrary split of flows is correct. In fact, this is precisely what we demonstrate in this paper.”
But then, what is the point of the paper? Authors introduce a complex split-and-transfer centrality, but at the same time they demonstrate that this is equivalent to the standard transfer entropic centrality, under a link probability transformation. Then, what is the point of introducing a new metric?

Or, is there a scenario for which such transformation cannot be made? And thus where split-and-transfer centrality is the only viable solution? If is that so, authors should clearly describe it.

It seems to me that this paper is an overly complicated work to show something fairly simple…
But maybe I’m missing something… if so, please let me know!!
Experimental design: No comment
Validity of the findings: No comment
Additional comments: Please see my comments in the first section.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 2
Reviewer: 2

Output:
Basic reporting: no comments
Experimental design: no comments
Validity of the findings: no comments
Additional comments: All the main issues raised by the reviewers were carefully addressed by the authors. The main purpose of the manuscript, the literature review and the related works are now clearer. In addition, the figures and tables are now well formatted. The manuscript was improved regarding the clarity of the presented methodology and results, as well as the text correctedness.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 2
Reviewer: 3

Output:
Basic reporting: The paper proposes a new centrality measure that is designed to overcome the lack of metrics to compute centrality of not indivisible flows such as financial transactions.

In this new version, the authors added the comparative analysis with the results with other centralities measures.

Also, the introduction was improved with new background references (wikipedia reference was removed) and description of previous centralities with advantages and disadvantages according to the data analyzed.

Although one example was removed from the original manuscript, the authors have now related their work with other literature methods. Furthermore, the conclusion was improved.
Experimental design: The study is a novel measure for not indivisible flow, the authors took care to explain the proposal with detailed diagrams and showing the computation of the measure with examples.

 However, Section 2.2 could be broken into paragraphs to help the readers to take time to understand each examples (Figures 3 a, b  and c).
Validity of the findings: Two real cases were used to reinforce the application of the methodology. The data is explained in the supplementary data excluding the Tehran stock exchange dataset which is private.

The paper is suitable for publication in journal.
Additional comments: to be improved:

l.266 - The alpha and Katz centralities are giving, as expected, very similar results…


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 3
Reviewer: 1

Output:
Basic reporting: It seems we are at a stalemate here, as my main concern (and it has not changed) is about the relevance of the new metric - I still don't see why it is so important to use it. Nevertheless, I went back to the requirements of PeerJ, and referees are NOT asked to evaluate "potential impact; degree of advance; possible readership etc.". Therefore, I will not block further the publication of this manuscript.
Experimental design: No additional comment
Validity of the findings: No additional comment
Additional comments: No additional comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 3
Reviewer: 2

Output:
Basic reporting: The introduction has very long paragraphs. This is not common in scientific writing.

The paper has only 6 references from the last five years, authors could include more recent references.

Nowadays exist several centrality measures, authors cite only six measures: degree, closeness, betweenness, eigenvector, Katz and PageRank,
More measures could be cited and related to the proposed work since the focus is a new centrality. For example, see this recent work:

- Influencers identification in complex networks through reaction-diffusion dynamics
Flavio Iannelli, Manuel S. Mariani, and Igor M. Sokolov
Experimental design: Authors should provide the computational cost to compute the entropic centrality.

How the measure is influenced by the presence of degree-degree correlations (assortativity)?

How the measure is related to the diffusion process, like epidemics? (see reference above)

Employ more graph-datasets and provide statistical analysis of the results.
Validity of the findings: Some authors show there exists a high correlation among centrality measures, for example:
- Correlations among centrality indices and a class of uniquely ranked graphs
David Schocha, Thomas W. Valente, Ulrik Brandes

- A multi-centrality index for graph-based keyword extraction
Didier A.Vega-Oliveros, Pedro Spoljaric Gomes, Evangelos E. Milios, Lilian Berton

To analyze the correlation authors should compute also the Pearson coefficient and employ:
1) more measures especially using random walk
2) more datasets, two datasets are not enough
3) compute the correlation distribution among the datasets and measures.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SPLIT-AND-TRANSFER FLOW BASED ENTROPIC CENTRALITY
Review round: 4
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: Authors included two recent references, one more dataset, calculated the assortativity and mentioned the time complexity as suggested.
It is not clear if the time complexity is the same as DFS, authors could mention O(V+E) in this case.

Some suggestions to improve readability:
1) URL could be included as a footnote.
2) the addresses in Bitcoin network are very long char, authors could use a simple code to mention each one in the text.
3) Tables could be standardized (about the information and font size).
4) standardize the number of decimal digits, in some tables, it is four, five or six.
5) if the authors have lack of space they could remove fig. 5, it does not aggregate much information in the work.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RANDOMIZED ROUTING OF VIRTUAL MACHINES IN IAAS DATA CENTERS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Please see the general comments.
Experimental design: The experimental design is moderate.
Validity of the findings: Not convincing.
Additional comments: In this manuscript, the authors propose randomized method to assign the virtual machines to Physical Servers in IaaS data centers. The authors try to minimize the total power consumption under certain QoS constraints.

The reviewer has the following concerns:
1. The virtual machine allocation problem is not well defined. In the introduction, the authors should first introduce some background and then introduce the VM allocation problem.

2. The authors claim the the behaviour of a VM is unpredictable (which is the main motivation). It is better to explain what are the behaviours.

3. Towards the technical solution, the authors propose a randomized solution regardless of the current status of each Physical Server aiming at minimizing the total power consumption. The reviewer feels that the optimization problem is not well formulated, given the condition that all the VM will be randomly allocated.

4. There are many typos and grammar errors, such as ‘let to’ ‘be able of’.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RANDOMIZED ROUTING OF VIRTUAL MACHINES IN IAAS DATA CENTERS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper needs some good proofreading. Some examples here:
The cloud computing providers is running → are running
 in data center -- in the data center
This let us to choose -- this let us choose
has been developed for initial placement of VMs -- for the initial placement
that PMs is fully utilized -- PMs are fully utilized
New request are either admitted or rejected → new request is
We suggest a randomized router after admission control phase. -- after the admission
Extensive analysis of huge data center -- huge data centers

Some figures lack units. (Figure 3, x-axis. Figure 5, y-axis)
There is a gap in Figure 5.

There is no explanation for many figures in the paper.

No background introduction. It would be better to introduce those compared methods (RR, TH, MT, BF).
Experimental design: no comment
Validity of the findings: The biggest problem for me is the simulation results. The authors compared their methods with several previous methods and the round robin method. However, from the Figure 5b, it seems that the proposed method works exactly the same as round robin, and worse than all previous work. Unless the authors can demonstrate that the proposed method performs better than those previous methods at least in some perspectives, I can’t find any advantages of the proposed one.

The proposed method is based on some very simple assumptions. For example, most VMs have approximately the same utilization; the power consumption is linear related to the utilization; feedback system is impossible in data centers. Those assumptions are not valid in real data centers.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RANDOMIZED ROUTING OF VIRTUAL MACHINES IN IAAS DATA CENTERS
Review round: 1
Reviewer: 3

Output:
Basic reporting: The Abstract and Introduction are easy to read and follow.

However the rest of the sections are hard to follow.

One minor typo: Line 12: providers 'are' running

The references do not follow the standard template (e.g. line 180: Khani, H. (2016). Simulation code. Accessed: Nov 2016.)
Experimental design: 'no comment'
Validity of the findings: It is not clear what is the challenge the authors are trying to address in the paper. The motivation of the paper is not clear.

Figure 3 is not clear. As beta decreases, the number of PMs increase and power consumption increases. So what is the author's recommendation from this figure? Same observation with Figure 5a.

The authors mention they use Google traces, but provide no further information of those traces.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: RANDOMIZED ROUTING OF VIRTUAL MACHINES IN IAAS DATA CENTERS
Review round: 2
Reviewer: 1

Output:
Basic reporting: None
Experimental design: None
Validity of the findings: None
Additional comments: The authors have fully addressed my concerns.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SECURITY DURABILITY ASSESSMENT THROUGH FUZZY ANALYTIC HIERARCHY PROCESS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper has made several contributions toward the establishment of security durability during software development in industry. The manuscript introduces the security durability and assessment. The quality of the article is good, the content is novel and the structure is complete.
The work presented should be more precise and related to work done in the paper. Too many general stiffs discussed should be avoided. It is suggested to carefully check the format.
Recommended for inclusion.
Experimental design: Entrance exam software of BBA University has been taken as an experiment. But the attributes of design hasn't been provided. Although, it is made clear that only security attributes are taken. But I think more attributes of design and security can be merged together to form the hierarchy.
Validity of the findings: Fuzzy MCDM methods have taken to assess the security durability and use the values of alpha and beta as 0.5. The researcher is given the sensitivity analysis of the results with respect to value of alpha and beta. More methods of MCDM  can be used for better validation of the results.
Additional comments: Please add more security analysis on the proposed system.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SECURITY DURABILITY ASSESSMENT THROUGH FUZZY ANALYTIC HIERARCHY PROCESS
Review round: 1
Reviewer: 2

Output:
Basic reporting: Please revise the format of all tables.
Experimental design: This paper provides original results of tha application of a multicriteria methods already largely used. In terms of relevance it was not clear to if the merit of this paper. So, please better explain the relevance of its paper as it is a crucial aspect to approuve its publication.
Validity of the findings: In addition below I provide other comments to be addressed:
Abstract
Methods: description of methods in abstract is very generalist, please revise this part focused on methods and procedures used.
Results: description of results in abstract provide no usefull information, please revise it.
Material and methods
Line 206 to 258. This part is not material and methods, it is literature review, please move it to the correct section.
Line 259: AHP is one of the most largely used multicriteria methods and it presents several disadvantages over other methods like compensation and rank reversal. So, please justify why AHP was adopted instead of other methods that does not present its disadvantages.
Line 468 to 490. This part is not material and methods, it is literature review, please move it to the correct section.
Line 750 to 865. This part was very generalist and detached from the rest of this section, please condense the information provided here.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SECURITY DURABILITY ASSESSMENT THROUGH FUZZY ANALYTIC HIERARCHY PROCESS
Review round: 1
Reviewer: 3

Output:
Basic reporting: The text is well structured but the main objective of the article was not shown in the abstract clearly. It is better the introduction section rewrite.
You need to present the text structure at the end of the Introduction.
Experimental design: No comment
Validity of the findings: No comment
Additional comments: I’d like to suggest that the authors get editing help from someone with full professional proficiency in English.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DEEPMONEY: COUNTERFEIT MONEY DETECTION USING GENERATIVE ADVERSARIAL NETWORKS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Paper is well written, English is good. References are properly marked.  Fig 5 (picture) is bit crowded, could have been better structured so could be easier to follow.
Experimental design: Good, Question/problem well defined. Details are good enough.
Validity of the findings: Impact is not fully assessed, meaning the loss of 20%, under what conditions the accuracy of detection is failing is not clearly articulated.
Conclusions are well stated.
Additional comments: Overall good.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DEEPMONEY: COUNTERFEIT MONEY DETECTION USING GENERATIVE ADVERSARIAL NETWORKS
Review round: 1
Reviewer: 2

Output:
Basic reporting: Literature Review / Related Work is not adequate. In fact it is merged with the Introduction section. Although several techniques have been proposed to this problem for the last 25 years. Moreover following works have been presented on this area since last 2 years in various conferences, and journals, which authors did not refer:

Verifying Authenticity of Currency and Tracking Duplicates by M Alicherry - 2017

Database for detecting counterfeit items using digital fingerprint records by DJ Ross, BJ Elmenhurst, M Tocci, J Forbes-

Bank note processing system having a combined florescence and phosphorescence detection system by S Kayani

Image Processing Based Detection of Counterfeit Indian Bank Notes by M Singh, P Ozarde, K Abhiram

Currency Recognition System Using Image Processing by SS Rajebhosale, DS Gujarathi, SV Nikam, PP Gogte

Counterfeit prevention by S Micali, S Devadas

MINIATURIZED COUNTERFEIT DETECTOR by R Phillips

Identification of Fake Currency: A Case Study of Indian Scenario by S Snehlata, V Saxena

Only two references are used from last two years which is inadequate.

The related work should compare the use of GAN with other techniques used in solving this problem. Moreover the Discussions on Neural Network should be done in Introduction section so that Problem Statement can be elaborated accordingly in Section 3.

Apart from other minor mistakes in terms of spelling and grammar line 215 and 216 contains typo errors. Table on results associated with figure 10 is neither labeled nor cited anywhere.
Experimental design: In line number 73 authors claim that this is the first research on forged money using GANs. This statement needs to be proved in light of following work:
Generative adversarial nets by I Goodfellow, J Pouget-Abadie, M Mirza and available at https://papers.nips.cc/paper/5423-generative-adversarial-nets . This paper is published in 2014 and many advances have been made by now (requires evaluation). Moreover GANs has been extensively used in detecting fake object. Such works should also be reported in related work section

The authors should also report about the unique features in Pakistani Currency that makes it unique from other countries. Otherwise just because it is a refined area and since no one did it for Pakistani currency does not seem to be valid hypothesis.
Validity of the findings: Data Preparation and Augmentation section is extremely confusing. The whole section needs revision for better readability. The discussion on function calling and parameters (see line 202-204) is made in haphazard way. A pictorial representation of program flow may better be used to show its correctness.
Tabular results are not shown properly. It is not shown how Confusion Matrix is calculated.

Figure 13 - 15 are showing the results as graphs. However these graphs lack the visiblity issues. Instead the authors should show in results how (for example 80% classification accuracy) was achieved. The screenshots of Tensorflow Graph Visualization is confusing and better methodology should be adapted for reporting the results
Additional comments: Apart from my comments mentioned in above review, various coloring schemes are used to explain the paper. From visualization purpose it is good but in print it may be represented on gray scale and thus make it useless (for example confusion matrix). You may need to check with the Editors in this regard


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DEEPMONEY: COUNTERFEIT MONEY DETECTION USING GENERATIVE ADVERSARIAL NETWORKS
Review round: 1
Reviewer: 3

Output:
Basic reporting: 1.	It is recommended to bring the nearest and state-of-the-art works in a tabular form and place it in the “related work” section by identifying purpose, methods, limitations, results etc. of each study.
2.	Please extend the proposed scheme to bring more novelty and contributions.
3.	Tell me the connection of the proposed work with the state of art techniques for big-data. Besides, the literature is limited.
Experimental design: 4.	Pose research questions and try to find their answers with experiments.
5.	The experimentation and implementation section is weak, where authors should give a detailed experimental setup and find answers to their posed research questions.

6.	The pseudo code can be represented to depict  the methodology steps followed.
Validity of the findings: 8.	The results and discussion section should contain both quantitative and qualitative discussion. I want to see some innovative findings out of your implemented work.
Additional comments: Written English must be improved as this does affect the overall understanding of the manuscript, there are many mistakes, equations without numbering, and many long sentences/paragraphs without references
	The conclusion and abstract sections should be revised and synchronized accordingly.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DEEPMONEY: COUNTERFEIT MONEY DETECTION USING GENERATIVE ADVERSARIAL NETWORKS
Review round: 2
Reviewer: 1

Output:
Basic reporting: Paper is refined
Experimental design: Experiments are well presented now
Validity of the findings: findings are now of sufficient standard to be accepted
Additional comments: All required changes incorporated


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DEEPMONEY: COUNTERFEIT MONEY DETECTION USING GENERATIVE ADVERSARIAL NETWORKS
Review round: 3
Reviewer: 1

Output:
Basic reporting: N/A
Experimental design: N/A
Validity of the findings: N/A
Additional comments: The paper has been revised well.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: STEADY STATE PARTICLE SWARM
Review round: 1
Reviewer: 1

Output:
Basic reporting: This paper proposed a steady state update strategy for PSO algorithms. The proposed algorithm was tested on a set of unimodal, multimodal, noisy and rotated benchmark functions. Finally, a parallel SS-PSO has been implemented and compared to the standard PSO. But some weakness can be observed:
(1) The latest references should be cited. The review of the existing PSO algorithms  is very necessary.
Experimental design: (3) The results are not persuasive. Comparison with the standard PSO is not acceptable, the authors should present their results compare to related papers.  Is the proposed algorithm better than other swarm algorithms?
(4) The 'Mean' and 'Std' values should be provided in Table 2.
(5) The convergence curves of the proposed algorithm and the other algorithms should be added.
Validity of the findings: The steady state update strategy for PSO algorithm is not descrided clearly enough in the paper. So the paper is lack of innovation.
Additional comments: This paper proposed a steady state update strategy for PSO algorithms. The proposed algorithm was tested on a set of unimodal, multimodal, noisy and rotated benchmark functions. Finally, a parallel SS-PSO has been implemented and compared to the standard PSO. But some weakness can be observed:
(1) The steady state update strategy for PSO algorithm is not descrided clearly enough in the paper. So the paper is lack of innovation.
(2) The latest references should be cited. The review of the existing PSO algorithms  is very necessary.
(3) The results are not persuasive. Comparison with the standard PSO is not acceptable, the authors should present their results compare to related papers.  Is the proposed algorithm better than other swarm algorithms?
(4) The 'Mean' and 'Std' values should be provided in Table 2.
(5) The convergence curves of the proposed algorithm and the other algorithms should be added.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: STEADY STATE PARTICLE SWARM
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper address an interesting topic:  A Steady State OpenMp parallel version of PSO.

Authors take inspiration from SOC as applied by Bak-Sneppen to coevolving species in 1993.  Authors are not only interested in the model, but also in the possibilities for parallel implementations, and their benefits.

The topic address is of interest, and the paper is well-written.  However, it is not clear to me why the "parallel" part of the study has been included:  it is a small section of the paper, that would really require a large study and probably an additional paper.  Actually, when I read the title, given the inclusion of "OpenMP parallel" I though this paper would really deal with a parallel algorithm, which is not the case.  Right now it is confusing what the main goal of the paper is.

If we focus on parallel versions of PSO, authors describe the problem of unbalanced distribution of particles when synchronous models are used.  In other areas, load-balancing is a topic always addressed, and a wider discussion is pertinent here, given that methods and algorithms have been widely studied and are available in this area (76-80).  So, the literature review should specifically address this topic.

Authors state that quality of solutions and speed are the main goals, and the proposal relies in reducing the number of evaluations every time step.  This is not a new idea, and has been frequently used in the context of Eas reducing fitness cases evaluations, when every individual must check a number of cases.  (88-90)  If this is one of the main ideas here, a review of the literature would be pertinent so that readers can understand the connection with previously published papers on the topic.

On the other hand, steady state approaches are not new for population-based metaheuristics, and are also frequent in the EA literature.  Nevertheless, the traditional approach is that a single population is managed, where new individuals are born, and only they are evaluated every time step, and some pass over, so that the size of the population does not change.  A comparison among the approach described and the standard one should be included in the review, providing references to interested readers. (100).
Experimental design: (194) A description on how the algorithms has been tested should be included:  although that information is provided below (50 runs per experiment), this is the best place to include it.

Given that two particular values of neighbourhoods has been tested, (5 and 9), and given that each of them require different computing time, results should also include time to solution for a better assessment of results.  A wider study should include the range of neighbourhoods values so that we can understand why some values are better than others.  If this is one of the main components of the algorithm, better to test them in the context of the new proposal.

(283) some comments should be added about why in f1 and f3 the new proposal does not found better results.

(402) the first and only time when references to tables are correctly included.  Why this time are correct and previously never? (error in other cases).
Validity of the findings: Although results shown for the algorithm are clear, regarding the parallelization section there are some problems that must be addressed.

(416)  Parallelization

The topic is important enough to be addressed separately.  But the information provided is quite scarce to be able to understand the implications.  For instance, no information on how the parallelization has been applied is included.  Authors state that run-time has been compared when using 1-8 threads.  But what technology, programming library, etc,  for parallelising the code has been applied?  Are authors simply relying on compilers decisions?  Even if the code is shared, the paper should include the required information for an occasional reader to know the details without checking the code.

(434)  authors refers here to parallelisation at the evaluation level.  But the results in this case should provide fitness quality against time for a proper comparison.  Speedup is not enough.  Also comparison between time-to-solution should be included.

(441) Authors state:  “Unfortunately, large run time variations were observed, making it difficult to perform a direct comparison. This was to be expected, since a single unsuccessful run is enough to skew the results towards either algorithm. Repeating the test with different seeds often lead to opposing results.”

This shouldn't avoid a correct comparison:  maybe number of successful runs under time intervals could be used.  Also, time provided can be longer thus assuring all the runs finish; or difficulty of problems increased, so that no run finish along the time studied, and quality can be analyzed along the experiment.

Given the specific problems that arise when parallelisation is addressed, I suggest to remove this topic from the paper and try it as a specific topic in a future paper.
Additional comments: I don't understand why no correct links to figures and tables are included in the paper, and instead only “Error” messages are included.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: STEADY STATE PARTICLE SWARM
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: no comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: REVERSE ENGINEERING APPROACH FOR IMPROVING THE QUALITY OF MOBILE APPLICATIONS
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1) There are some typos and some sentences that are not clear as highlighted in red in the document
2) Some sentences need references and some context is not relevant as mentioned in the attached document
3) The paper lacks organization and structure and the figures and tables are not well formatted and need more description
4) not clear
5) results are not clear
Experimental design: 1) The paper includes case studies about Android Applications and how to improve the anti-design patterns in them so it may not be within the scope of the journal

2) Not clear

3) Not clear

4) Yes
Validity of the findings: 1) Novelty is limited and not highlighted properly

2) Somewhat

3) Not clear

4) __
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: REVERSE ENGINEERING APPROACH FOR IMPROVING THE QUALITY OF MOBILE APPLICATIONS
Review round: 1
Reviewer: 2

Output:
Basic reporting: Abstract Editorial Comments

Methods:
1)	Include the decision criteria used to select the 29 apps used in testing and source of downloads. Include what version of Android was used. What was the criteria?  Why did you select these apps?   The list of apps tested includes two browsers (Firefox and Chrome) which have significantly different characteristics than some of the other apps.
2)	While the method of reverse engineering and analyzing the Android apps does appear to be novel and potentially provide for improvement in app design and app revisions, it does not necessarily mean that it will “lead the way for perfect long-time apps and ensure that the applications are purely valid.”  Rather,  the authors might want to suggest that this methodology provides developers with a way to help ensure that anti-patterns in existing apps are not repeated in-app revisions and may improve both operational characteristics and user experience.
3)	The authors should state that they present and test a new method of generating OWL Ontology of mobile apps and analyze the relationships among object-oriented anti-patterns and offer methods to resolve the anti-patterns by detecting and treating 15 different design’s semantic and structure anti-patterns that occurred in analyzing the 29 apps.

Results: The authors may want to combine the first two sentences into something like this.  “The proposed method is a general detection method to identify and help resolve semantic and structural design anti-patterns in Android mobile apps.   The analyses of the various types and frequency of anti-patterns and the potential uses of the demonstrated methodology in integrating the mobile applications are shown in order to validate the approach. “

General Editorial Comments:
1)	At first, the paper appears to be well structured with the sections on Abstract, Related Work, Ontology and Software Engineering, Method (not proposed if it was actually accomplished).
2)	However, what is unclear is why the authors chose to include the “Integration of Mobile Apps.”  Most app integration is for the inclusion of new skill sets for the app such as IOT or monitoring applications or potentially voice-activation integration into an existing app.  Seldom would we anticipate integrating 2 separate apps?  While the diagrams in Figures 2 and 3 are interesting, this entire section seems out of place and fails to establish the logic what apps are being combined and why?  Is this the only way to test and validate the methodology to identify and resolve anti-patterns?
3)	The Implementation section is well written and the descriptions appear clear for the re-engineering of an app and the detection of the anti-patterns in the UML class diagram, with the eventual editing of the OWL ontology of a single app using the reasoner ontology the Protégé editor.  Again, this begs the question of why the discussion of the integration of apps (Line 161). Unless, the authors are really addressing current needs, not of integrating two different apps, but rather the integration of new modules of skill sets into the apps?  (Note: Further reading in Empirical Validations shows that the authors did integrate “Viber” and “What’sApp” as a case study.  But no rationale is provided for why this integration is needed to demonstrate the methodology for identifying and resolving anti-patterns.
Experimental design: 4)	The Empirical Validation Section begins well with the statement that the authors propose to “reporting on the results…for the detection of 15 anti-patterns on the 29 popular Android apps downloaded from APK Mirror.”
5)	The Case Study is a very confusing section.  The authors first state, “ To explain the proposed method we presented a snapshot in a different case study, “ A vast Android Mobile Security.”  This case study is not explained nor cited.  Where did it come from? Why is it being used now?
6)	Actually, with some explanation of what it is the authors are examining (which is entirely unclear) this section has some very useful information for developers as it shows the number and locations and types of anti-patterns that were found in whatever mysterious app source the authors are exploring.  If this is a common pattern in Android app design development, then the identification of the resulting anti-patterns is very helpful.  The problem is the reader has no idea what app is being analyzed.  This is a major flaw in the article.

Integration of Mobile Applications
1)	Once again, the reader is mystified by what exactly is being explained and why the authors chose to integrate Viber and WhatsApp. The paper would likely be much stronger without this section or this analysis unless the authors can provide a rationale for the integration and provide an analysis that tells the reader more about the potential use of the proposed methodology.

Discussion of review of Tables and Charts
The authors should review each of the tables for alignment and correctness of the information and identification of data.  For example,  in Table 3, the heading might read “The Method of Correction.”  Check for placement of % marks, as they should always follow the numbers rather than precede. In Table 4 the alignment of the second column is confusing.  Also, in Table 4 what is “The Group”?  Is it the 29 Apps Under Review?
Validity of the findings: RESULTS and DISCUSSION
The results and analyses of the 29 apps and the resulting 29 anti-patterns and their classifications are the key results of this section (Please see editorial notes on Tables and Figures). There is a typo in line 256 (it should read 29, not 39). There should be additional discussion regarding any thoughts about the strength in numbers of the various anti-patterns shown in Table 9.  Why, for example might we anticipate that “anti-patterns in class” at ~50% is the most commonly detected?  But “anti-patterns in association” is nearly as common at ~45%.  The other 3 anti-patterns are detected at much lower levels. What can developers and team architects learn from this analysis?
The correlation discussion should include both negative and positive correlations ( remember a correlation of -1) is a strong correlation and should discuss any correlation over +/- .3 .  In Figure 9 we learn that “Attributes in Class” and “Attributes in Operations,” the areas with the greatest correlation also occur less than ~1% of the time.  So, what is this telling us? How is this helpful?
The discussion of the ANOVA test among the three tools used sequentially in the proposed methodology and the detected anti-patterns is interesting.  What is not clear from the discussion is that the analyses were proposed to be done as a comparative, rather it appeared that the methodology being proposed was a step-by-step method.

Comparing the Mobile App Integration Methods
Here again, the logic of the paper fails and confusion sets in.  What 2 methods in 6.3?  Is this part of another paper? When did the single methodology proposed become 2 methods being compared-- Ontology Integration and UML?  What is the virtue or benefit of integrating Viber and WhatsApp?  What does it mean when the authors state that “semantic integration decreases the accuracy of the anti-patterns in apps??” It appears that using ontology separately does increase the ability to identify and potentially improve the coding and structure of the apps?  But how does integration help? This is very confusing

Discussion
There was no discussion section. Perhaps it would benefit the authors and potential future readers to identify the key findings and support them as building upon prior research and science. How do these results support prior work? How are they different?
What strengths and limitations exist in this work (after the paper is cleaned up…) Were there any unexpected outcomes or learning?

Conclusions
It is suggested that the authors closely consider the concerns and suggestions of this reviewer and others to re-construct this article.  The methodology proposed is indeed interesting and the potential for using ontology and the existing tools in the order proposed may indeed be helpful for app developers, especially those who are in a revision process. The authors should briefly restate their hypotheses and expectations. (This suggests a close review of the list from lines 65-77). The benefits and discussion of the 2 app integration should be closely considered for value.  What would perhaps be more beneficial would  be a comparison of code between an existing app and any package extensions being considered, for example for IOT or voice control integration.  As a stand-alone analysis it is sufficient to discuss the 29 apps and the presence and types of anti-patterns found. However, this requires a discussion and rationale for the selection of the apps—more than just that they are “popular.”
Conclusions should also synthesize what was really learned in the analysis and how can this help developers build and revise apps in a cleaner fashion by removing and correcting anti-patterns.  What are any supporting research results?  What are next steps going forward to further support or analyze this methodology.

Discussion of review of Tables and Charts
The authors should review each of the tables for alignment and correctness of the information and identification of data.  For example,  in Table 3, the heading might read “The Method of Correction.”  Check for placement of % marks, as they should always follow the numbers rather than precede. In Table 4 the alignment of the second column is confusing.  Also , in Table 4 what is “The Group”?  Is it the 29 Apps Under Review?
Additional comments: There is significant potential in this proposed methodology and the research demonstrates the value of the step-by-step method and the use of ontology in identifying and correcting anti-patterns.
However, the article as written is confusing and needs significant corrections and re-thinking of the overall structure.  Especially confusing is the integration of the 2 apps Viber and WhatsApp.  Why?  Rather the authors should consider the issues and problems of developers who are revising Android apps and integrating new packages of code skill sets.  A code review such as the methodology proposed could be very valuable in terms of not carrying forward existing anti-patterns and also not incorporating new code flawed with poor design.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: REVERSE ENGINEERING APPROACH FOR IMPROVING THE QUALITY OF MOBILE APPLICATIONS
Review round: 1
Reviewer: 3

Output:
Basic reporting: It would be useful to have improved consistency in Tables 2-6. E.g. Tables 2 and 3 name specific anti-patterns; whereas Tables 4 and 6 list groups of anti-patterns (without explicitly linking specific items to these groups); and Table 5 uses abbreviations.

Also it is difficult to follow Table 5 - if allowed by the journal style guide it might be easier to read with vertical column lines, or dashes ("-") rather than blanks for empty cells.

Figure 10 would be better presented as a Table. It was not clear what is being shown in Figures 7 and 8.

The language is generally clear, but could be improved throughout. Some specific cases where improvements are needed: e.g. in the abstract, "mind-bloggling programming frameworks" could be written more professionally. "... leading the way for perfect long-term apps and ensuring that these applications are purely valid" - it is perhaps too much to say that this research will lead to "perfect apps".

Line 273 refers to "Section 6.3" - which is perhaps from a different version of the manuscript?

Line 276: "semantic integration decreases the accuracy of anti-patterns in apps" - do the authors mean "decreases the occurrence  of anti-patterns"? (anti-patterns do not have accuracy).
Experimental design: Line 199 "We selected some popular apps" - was there a particular selection process? Why were these apps chosen?

Line 219 "we introduce... the correction method" - please clarify whether these are standard correction methods, or new methods that the authors are introducing in this manuscript?

Lines 243-247 - I was not clear on the rationale for integrating/combining two separate apps (e.g. WhatsApp and Viber) - perhaps the authors could clarify this?

Line 260-262 - the authors say they perform correlations, but it is not clear what is being correlated. In each case, is it the total number of occurrences? The rationale for these correlations was also not clear.
Validity of the findings: Lines 263-264 - "the greatest correlations were between attributes and operations groups" - however these had the smallest numbers - this should be acknowledged.

Lines 266-270 - the rationale for the ANOVA is not clearly stated - but I also don't understand the interpretation of these results. If there is no significant difference associated with any specific tool (p values are >> 0.05), does that mean we can use any *one* tool, rather than having to use them all?

Lines 278-280 - it is not clear how the % values are obtained. Also, these refer to "consistency checks" which are not described elsewhere.

Line 294 "there is a correlation between OLED and Protege" - Table 7 shows an r value of -0.060 which is negligible. Inferring a correlation from this result is unfounded.

Line 297: "guarantees consistency" - it is not clear from these results what is meant by consistency, and how this can be guaranteed.
Additional comments: As a general comment, I think this manuscript would benefit from a discussion on the relationship between the reverse engineering process and the anti-pattern detection process. Is there any consideration of:
i) does the reverse engineering process (in general, or for these specific apps) have risks associated with the quality of source code obtained?
ii) do your findings tell us anything specifically about this set of apps?
iii) would the process work as well if source code was obtained directly (e.g. from GitHub) rather than the first reverse engineering step?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: REVERSE ENGINEERING APPROACH FOR IMPROVING THE QUALITY OF MOBILE APPLICATIONS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors have sufficiently addressed my questions and concerns in their rebuttal and revision of the article.
Experimental design: no comment
Validity of the findings: No comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ON THE IMPACT OF SERVICE-ORIENTED PATTERNS ON SOFTWARE EVOLVABILITY: A CONTROLLED EXPERIMENT AND METRIC-BASED ANALYSIS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Within this submission, the authors present the results of a study they conducted. The study had the goal to show how service patterns influence the effectiveness and efficiency of software developers when working on service-oriented software.

While the results of this study may look disappointing at first sight, since most assumptions of the authors cannot be "proven" through the study, such results should also be published in order to give fellow researchers insights.

The paper is extremely well-written and easy to follow. I do about 50 reviews per year in different fields of distributed systems (from top journals to mediocre workshops), and this one is clearly within the top 5% of well-prepared submissions. I really wish that more authors would put a comparable effort into preparing their submissions.
Experimental design: As far as I can judge, the study is very well-conducted, following accepted best practices, and is conducted with the right amount of rigor. I would have expected that some aspects are taken into account in addition, e.g., if the sex or the age of the participants influence the outcomes of the experiments. It would be good if the authors clarify if such aspects are simply not mentioned because there was no significant impact of such parameters.

I especially like the self-critical approach of the authors and the discussion of the validity of the results and of the lessons learned from the study and experiments.

I would have expected that the authors do not only give (and discuss) average values in Tables 3ff, but also the standard deviations, since this gives very important information about the heterogeneity of the participants.
Validity of the findings: The most critical aspect of the conducted experiments is that the participants are very junior developers. In fact, I would even call Master students quite inexperienced developers, and even more so students which are mostly in their second or third Bachelor semester. Taking into account the rate of computer science students who stop their studies in Germany before getting their degree, it is very likely that a lot of the study participants have not the minimal level of knowledge needed to participate in such a study.

The goal mentioned in line 172ff (that it would be extremely meaningful if a positive effect could be shown even with inexperienced developers) is (with hindsight) therefore quite unrealistic. I completely understand that it is very difficult to get experienced developers to participate in such a study, but nevertheless, this significantly decreases the potential impact of this paper.

At the end of the day, it could be argued that the authors also "only" provide further insights for one particular group of developers, with an unclear outcome, i.e., similar to the studies mentioned in Lines 56ff. However, as can be seen from my overall recommendation, I still think the results should be accepted for publication in PeerJ CS.
Additional comments: * Line 50: I wouldn't necessary use "battle-tested" in order to describe that design patterns are well-accepted in software engineering. That's not necessarily scientific language.
* Please make use of vector graphics in order to improve readability and scalability of the figures.
* Section 4: I am not sure, if it's always the percentage that is meant here by the authors. In some cases, it might also be the percentage points instead. For instance, in line 248, is "(~13% more)" correct or should it rather be "~13 percentage points more"?
* The authors should carefully check their literature list, which is very heterogeneous with regard to layout and content. There is a lot of information missing, or double information. The authors should make sure that every single literature list entry provides the same layout and the same content.