Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SCALEUP: MIDDLEWARE FOR INTELLIGENT ENVIRONMENTS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors present a paper called “ScaleUp: Middleware for intelligent environments” that focuses on its usage for the Internet of Things. It is the opinion of this reviewer that it requires further elaboration before being considered for publication:

1. The authors mention “the developed mechanism serves as a viable alternative to the commonly used Universal Plug and Play (UPnP) and Web Services middleware systems”, which is quite a statement since WS and UPnP have been standards in industry and research for decades. At this stage of their middleware development, it is hard to make such a claim, so the authors should humble down this sentence.
2. Line 74 “For instance, if a device state variable, updated using one middleware system, was not reflected by all the alternative mechanisms used”. This looks like an incomplete sentence that should be finished.
3. The references provided in literature could be expanded with usage of middleware in other international research projects. A (non-exhaustive) list of references that could be added is : A Review of Middleware for Networked Robots, Middleware Architectures for the Smart Grid: A Survey on the State-of-the-Art, Middleware Technologies for Smart Wireless Sensor Networks towards Internet of Things: A Comparative Review, Taxonomy and Main Open Issues, An optimized, data distribution service-based solution for reliable data exchange among autonomous underwater vehicles, Maritime Data Transfer Protocol (MDTP): A Proposal for a Data Transmission Protocol in Resource-Constrained Underwater Environments Involving Cyber-Physical Systems or IoT Middleware: A Survey on Issues and Enabling Technologies. Of course, these are suggested just as a means to improve the paper, they are not mandatory to be included if the authors think otherwise and provide other similar ones.
4. The middleware architecture that has been created lacks a specific name, it is only referenced as part of the project (ScaleUp). That makes hard to understand the differentiating value of this architecture compared to the other.
5. It is not clear what the advantages of the middleware architecture are compared to the existing literature. An open issues section, where it is mentioned what open issues are being tackled by the proposal, must be added. Likewise, the contributions made by the proposal should be made clear in the manuscript in the first section.
6. There is very little information about fundamentals of middleware: a) is a Message-Oriented Middleware? A Middleware architecture? Or something else? b) How do the PDUs interchanged look like (if it applies)? c) What software components are included in the MW? d) What software layers are used within the middleware itself (Hardware Abstraction Layer, Service Application Points, etc.)? All these questions should be answered thoroughly.
Experimental design: It is detailed enough.
Validity of the findings: They are suitable for the topic of the journal
Additional comments: No comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SCALEUP: MIDDLEWARE FOR INTELLIGENT ENVIRONMENTS
Review round: 1
Reviewer: 2

Output:
Basic reporting: Paper is well presented, but I think it needs really hard work to Abstract
Inline 41 authors present a “new form”, in line 45, authors present a “new mechanism” and in line 49, authors present a “system”. It is necessary to concrete the product, or solution, presented in the paper. Title names a middleware, it is necessary to concrete the contribution of the paper.
Line 43: UPnP is commonly considered a protocol instead of a middleware, perhaps it is necessary
Lines 47 (end) and 48: "...types of the device...” change for “...types of devices...”
Line 49: authors relate a “clone devices” that implies that the system presented uses clonation, this aspect must be presented previously.
Line 50: the same of the previous comment, but about the concept of “master”.

Abstract uses specifically concepts that, probably, paper can clarify, I suggest rewriting the abstract.

Introduction
Line 57: I suggest to change “omnipresent” for “necessary” or “ubiquitous”, instead omnipresent is correct, it is more frequently to use the alternatives mentioned above.

The introduction clarifies the aims of the research. I miss a paragraph with the description of the paper structure (forget this comment if this paragraph is not mandatory by the journal guide)

Literature review
Line 109: instead of “bespoke” is correct, I suggest to use the more frequently expression “custom-made” or “proprietary” when the concept is used in commercial trades (Apple or Sony mentions).
Lines 124 and 125 present the interesting result that can be used in the paper (requirements), I suggest authors select from the 12 requirements which ones accomplish the system presented (for the sake of providing scientific validity to the work presented).

Line 183 (figure 1): If the objective of the figure is to demonstrate where the wrapper should be inserted, a figure is not necessary, it is enough to explain in the text the sequence between the device and the programming interface. However, I suggest extending the figure with the context of layers in which a middleware acts.

Line 184: the reference Doolet et al., 2011 probably is the reference Dooley et al, 2011

Line 189: I try to avoid referring to tools by means of URLs, I suggest looking in PeerJ or similar for any reference to a survey in which the use of Eclipse as a system is mentioned.

Line 190 (figure 2): I don’t know the significance of the home environment pictures related to the blocks diagram used in the figure.

Figures 1, 2, and 3 represents the role of the main blocks, or functionalities, offered between devices and interface programs. It could be interesting to unite them in a comparative figure where device and programming interfaces were common and the different ways of implementing middleware could be compared.

The Environment Development Kit (EDK)
Figure 4: a code snippet can be useful in an implementation section, but a code declaration does not seem convenient in this section. In the text, authors declare that figure 4 represents an architecture, but no code.

Line 258: an example of a middleware based on publish-subscribe can be found in this reference (I suggest to include in the paper):
“Simó-Ten, J. E., Munera, E., Poza-Lujan, J. L., Posadas-Yagüe, J. L., & Blanes, F. (2017, June). CKMultipeer: connecting devices without caring about the network. In International Symposium on Distributed Computing and Artificial Intelligence (pp. 189-196). Springer, Cham.”

In general, to understand the next lines about the different operations offered by the EDK, I really miss some figures (instead of the code fragments presented). These figures are:
A general figure, using block diagrams or similar methods, to represent the different elements of the middleware (devices, master, clones, etc.). This figure can be used to explain the different functionalities presented.
A figure (as a flow chart or UML sequence diagram)  to each process: Implementing master devices, Searching for devices, Searching for specific devices, etc.

Line 431: searching for devices, perhaps is a discovering mechanism.

Additionally, authors can present the operations in a section that describe the middleware and present a case of use as an example that describes the diagrams before mentioned.

Evaluation
I suggest renaming the section “Experiments” or “System validation”.
Experiments are well presented, I only miss the best quality in figures 19 and 20.

Conclusions and future work
Well presented and really concise and concrete.

Authors mentioned in lines 84 and 85 that the work is based on the project (Scale up) but this project is not mentioned in acknowledgements or Founding.
Experimental design: The experimental design is well done, is based on the comparison with other systems. Is not mandatory, but I suggest to compare other scenarios including different devices.
Validity of the findings: no comment
Additional comments: no comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SCALEUP: MIDDLEWARE FOR INTELLIGENT ENVIRONMENTS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The document has solved the issues that were mentioned before. I believe it is ready for publication. Usage of English is correct, background is provded correctly, the structure and notation are fine, it is relevant to the hypothesis that is put forward and terms are well-defined.
Experimental design: It was already good enough in the first review of the paper, so I believe it is still good.
Validity of the findings: Although limited, impact is real and significant. Scientific community can certainly benefit from innovative middleware ideas in the IoT. Required data have been provided. Conclusions are well defined.
Additional comments: The document has solved the issues that were mentioned before. I believe it is ready for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IOT-IIRS: INTERNET OF THINGS BASED INTELLIGENT-IRRIGATION RECOMMENDATION SYSTEM USING MACHINE LEARNING APPROACH FOR EFFICIENT WATER USAGE
Review round: 1
Reviewer: 1

Output:
Basic reporting: The literature review in the introduction is very poor.
Experimental design: The authors claim that "a smart irrigation system prototype for efficient usage of water and minimal human intervention has been proposed". However, I am unable to find this prototype in the paper.
Validity of the findings: The authors claim that "a smart irrigation system prototype for efficient usage of water and minimal human intervention has been proposed". However, I am unable to find this prototype in the paper.
Additional comments: This manuscript is about IoT-IIRS: Internet of Things based intelligent irrigation recommendation system using machine learning approach for efficient water usage. As a reviewer, I am unable to accept it due to the following comments:

1. The title is not in accordance with the work done, for example, I am unable to find the implementation of IoT in the manuscript.
2. The literature review in the introduction is very poor.
3. The quality of the figures is very poor.
4. The authors claim that "a smart irrigation system prototype for efficient usage of water and minimal human intervention has been proposed". However, I am unable to find this prototype in the paper.

Based on the above-mentioned reasons, the contribution of this paper is not clear.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IOT-IIRS: INTERNET OF THINGS BASED INTELLIGENT-IRRIGATION RECOMMENDATION SYSTEM USING MACHINE LEARNING APPROACH FOR EFFICIENT WATER USAGE
Review round: 1
Reviewer: 2

Output:
Basic reporting: The English writing should be improved.
Your introduction needs more detail. I suggest you improve lines 55-57 to show why you think the existing researches are inadequate.
The authors spend a lot of ink on related work (lines 75-129), but how exactly are these works related to the proposed method is not very clear.
Figure 2 should be improved.
Experimental design: The contribution of the paper is too long. I suggest the authors rewrite it to underline the original work they did.
The authors are also suggested to provide more detail on how to implement the proposed system, and analyze its advantages and limitations.
Validity of the findings: The results are not very convincing since the authors fail to analyze why IoT-IIRS is robust, and implement it using experiment.
Additional comments: Dear authors:
After reading your paper, I believe you have good knowledge of intelligent irrigation recommendation system and machine learning. However, the paper needs improvement before accepting. Here are some suggestions:
The writing of the paper is quite fluent except that there are a few grammar mistakes. For instance, the situation may lead to worse in the coming days.
It is difficult to understand the motivation of the paper.
Why some references in the article are in bracket while others are not.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IOT-IIRS: INTERNET OF THINGS BASED INTELLIGENT-IRRIGATION RECOMMENDATION SYSTEM USING MACHINE LEARNING APPROACH FOR EFFICIENT WATER USAGE
Review round: 3
Reviewer: 1

Output:
Basic reporting: The paper has been revised, but I didn't understand how authors have implemented IoT using GSM board ?? ! Moreover, there is a lot of "?" in the paper and I didn't understand why?
Experimental design: Why you have used Arduino UNO since there is a lot of embedded board which are didicated ti IoT application such as: Arduino nano IoT 33, ESP32, Raspberry ...
Validity of the findings: The paper has been revised, but I didn't understand how authors have implemented IoT using GSM board ??
Additional comments: The paper has been revised, but I didn't understand how authors have implemented IoT using GSM board ?? ! Moreover, there is a lot of "?" in the paper and I didn't understand why?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IOT-IIRS: INTERNET OF THINGS BASED INTELLIGENT-IRRIGATION RECOMMENDATION SYSTEM USING MACHINE LEARNING APPROACH FOR EFFICIENT WATER USAGE
Review round: 3
Reviewer: 2

Output:
Basic reporting: The manuscript has some typing error, in Cloud Level sub-section before de equation 1: you have has an extra letter in "literawture" word, please correct it.

Acronyms should be defined where they are first used (e.g. NIT, MSP430, API). In the abstract: NIT is not defined, in the introduction section: MSP 430 is not defined, in the results and discussion section not defined the acronym API, please add it.

Some references are not clear in the manuscript please check it, For instance, (res, 2020) (imd,2020) (rai, 2020), (PME, 2021).

Variables should be defined of the equation 1 (eg. E, Vo, Cf, and Wneed)

Figures 1 to 7 you can improve the legends (number on the bar), changing the size or number style.

In order to clarify, the author should be added a figure of the prototype experimental in real conditions work and figure of the application web.
Experimental design: Crop Field Level subsection, mentions that the microcontroller is connected to the motor. The question is: how is connected to the motor – microcontroller, you use a control circuit or integrated circuit? Can explain this.

How energy consumption of the Crop Field Level sensor, in sending messages, acquisition data, and sleep condition?
Validity of the findings: In my opinion, the strategy implemented in the work is not clear. For instance, Goap et. al. are reported IoT-based smart irrigation using Machine Learning, in my opinion, this work is very similar to your report. Please explain clearly the differences and your contribution.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: IOT-IIRS: INTERNET OF THINGS BASED INTELLIGENT-IRRIGATION RECOMMENDATION SYSTEM USING MACHINE LEARNING APPROACH FOR EFFICIENT WATER USAGE
Review round: 4
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors proposed an IoT-enabled machine learning-trained recommendation system for efficient water usage with the nominal intervention of farmers. The authors deployed IoT devices in the crop field to collect the ground and environmental details precisely. The gathered data are forwarded and stored in a cloud-based server, which applies machine learning approaches to analyze data and suggest irrigation to the farmer. The authors added an inbuilt feedback mechanism to this system To make the system robust and adaptive. Finally, the authors performance their IoT devices on the crop, and these results compared with their own collected dataset and NIT Raipur crop dataset is agreed. In my opinion, the authors have attended and corrected the observations and doubts previously issued, so that, the article can be publishable.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper presents a new approach to detect LP by stacking two CNN networks. It manages to obtain a state if the art results while using a small network in terms of the number of parameters.
Experimental design: The experimental Section is clear and well-designed.
Validity of the findings: The novelty of the approach is questionable, but the finding is interesting.
Additional comments: The paper presents another approach for LP recognition. The paper is clear, but there are several points that need further classification.
1. VGG seems to obtain the same performance, you should explain why you need another network
2. You need to explain why you have different results for the various datasets on average (table 11)
3. What is the point of including fashionMNIST
4. There is no Latin digits, the one you call Latin are Arabic and the one you call Arabic are Hindi


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 1
Reviewer: 2

Output:
Basic reporting: Some revisions are necessary on the presentation of the text itself. In my general comments to the authors, I’ve pointed out some grammar errors, typos and unclear passages.
This seems to warrant a full revision of the text to ensure the promising results of the paper are not obfuscated to readers.
Experimental design: The manuscript describes a classification architecture for license plates from multiple countries, which is also tested in common character and digit recognition datasets. Additionally it is also tested in FashionMNIST, a popular small grayscale image dataset. To my understanding, the strongest contributions of the paper are the proposed Full Depth CNN (FDCNN) model, and the proposal of a new license plate dataset called LPALIC. They seem to successfully address the knowledge gaps identified  by the authors.

Regarding the FDCNN model, promising results are reported for all the studied datasets, including state of the art results for MNIST when only stacked CNNs are considered, according to the authors. I believe that the methodology description and results must be entirely reproducible, especially since state of the art results in widely used benchmarks are involved.

Because of this, I believe the paper needs revisions to allow for easier reproduction and verification of results, and also needs more justifications on certain choices in the methodology, as explained in my comments further below.

In this current form of the manuscript, readers might not understand why many aspects of this FDCNN architecture were chosen, or how they were tested and validated. The lack of the use of a validation set, for instance, should be clarified, as directly optimizing a model on the test set could potentially introduce biases. This is true even if the justifications are of an empirical nature. I believe the authors should be able to revise the manuscript to make all of these points clearer and further highlight and solidify their already strong results, before publication.
Validity of the findings: All results are well described and data and codes are provided, but the points mentioned about experimental design and basic reporting must be addressed to ensure the validity of the results.
I believe once those matters are addressed and clarified the validity of the findings will be more easily asserted.
Additional comments: Line 36 - It would be interesting to additionally address or discuss these recent works, which also reportedly provide state of the art accuracies for the same task of classifying MNIST:

Hirata and Takahashi; Ensemble Learning In CNN Augmented with Fully Connected Subnetworks
Byerly et al.; A Branching and Merging Convolutional Network with Homogeneous Filter Capsules
Assiri; Stochastic Optimization of Plain Convolutional Neural Networks with Simple methods
Kowsari et al.; RMDL: Random Multimodel Deep Learning for Classification

Line 84  –  method was used

Line 100 – very little research was done

Line 100 – This paragraph starts with the idea that there are very few multi-language license plate datasets, but then cites works about license plate classification from what seems to be a considerable variety of countries and alphabet types. Please consider clarifying this paragraph by stating more explicitly how many datasets exist among the cited works, and all the different languages and character types used. This can create a clearer picture for the reader, and help to further highlight the new contributions of the paper.

Lines  101 to 104 - The grammar in this sentence is unclear. Please restructure.

Line 117 – Please revise the usage of “concerned” in this sentence.

Line 117 – Since the main contribution of the paper revolves around FDCNNs, it is important that the reader gets a clear understanding of other types of CNNs, what makes them different, and why the changes included in FDCNN are necessary. I feel that this information is currently lacking in the paper, and it’s not entirely clear how the authors justified pursuing this particular approach or why it’s necessary, or important, or better, to reduce featuremaps to 1x1 size before classification. Please include a more detailed discussion/justification.

Line 119 – Please restructure the sentence so FDCNN is not repeated twice in a row.

Line 158 – It would greatly help readers to have a table or something similar where all countries studied are listed, and what language/character types are used in license plates in that particular country.

Figures 1 and 2 – These figures show some interesting differences between datasets. Namely, Arabic alphabet LPs seem to vary much less in color. Could this possibly affect classification accuracies? There seems to be a marked imbalance in the number of Latin and Arabic images in the dataset. Would this warrant using additional metrics, besides accuracy, that are more sensitive to these imbalances? Please discuss this in the manuscript.

Line 187 – Please avoid qualitative/subjective qualifiers such as “modest”, used here. There are other examples of such adjectives in the manuscript. If necessary, compare directly to other values used in the literature, by choosing appropriate metrics for the comparison.

Line 205 – shrink

Line 250 – See comment about line 187. The same applies to the usage of “modest” in this line.

Line 251 – Please clarify the meaning of “needed iterations”. How is this defined?

Line 251 – Please specify batch size and momentum used. The supplemental files seem to show the minibatch used had a size of 120. They also mention a LearnRateDropFactor of 0.9, whereas the paper seems to mention one of 0.5 . It also seems that the standard momentum for Matlab’s ‘sgdm’ function is used, but this value is not stated explicitly in the paper. It would be interesting to add it.
Since one of the results of the paper is a demonstrable improvement upon the state of the art of stacked CNNs, it would be interesting, for reproducibility sake, to include these hyper-parameters in the methodology description.

Line 251- Additionally, it would be interesting to discuss how these parameters were chosen. If there were preliminary tests, or heuristics, how did other attempts affect the results, and by how much?
It would seem that no validation set was used, so how were metrics chosen? Was the model adjusted to get the best result directly on the test set, without using a separate validation set? Couldn’t this bias the models to adjust particularly well only to the test set?

Table 11 – The UAE test set is larger than the training set. I feel this decision should be justified in the paragraph above the table. I could not see a justification for it as is.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 2
Reviewer: 1

Output:
Basic reporting: A deeper error analysis is required.
Experimental design: The authors need to discuss the evaluation procedure.
Validity of the findings: The authors can discuss the results for other performance metrics.
Additional comments: The literature review needs to be updated with some of the recent works. Discussing the following works would make the manuscript richer for the readers:

DevNet: An Efficient CNN Architecture for Handwritten Devanagari Character Recognition. Int. J. Pattern Recognit. Artif. Intell. 34(12): 2052009:1-2052009:20 (2020)
Character recognition based on non-linear multi-projection profiles measure. Frontiers Comput. Sci. 9(5): 678-690 (2015)
Relative Positioning of Stroke-Based Clustering: a New Approach to Online Handwritten Devanagari Character Recognition. Int. J. Image Graph. 12(2) (2012)
Artistic Multi-character Script Identification Using Iterative Isotropic Dilation Algorithm. RTIP2R (3) 2018: 49-62
Character Recognition Based on DTW-Radon. ICDAR 2011: 264-268
Spatial Similarity Based Stroke Number and Order Free Clustering. ICFHR 2010: 652-657
Dtw-Radon-Based Shape Descriptor for Pattern Recognition. Int. J. Pattern Recognit. Artif. Intell. 27(3) (2013)


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 2
Reviewer: 2

Output:
Basic reporting: A new approach to LP identification by stacking two CNN networks is discussed in the paper. The core of the convolution block, which is the convolution Layer is followed by batch normalization and a non-Linear activation layer.
Experimental design: There is a well-designed experimental section. The manuscript's strongest achievements are the suggested Full Depth CNN (FDCNN) model and the recommendation for a new license plate data set called LPALIC. The strength of the paper is the selection of parameters, filters, and the process of training. The manuscript also gives an overview of FDCNN with respect to the use of memory.
The drawback of the manuscript is that the results obtained during the testing process on the test dataset are missing and no validation dataset has been used, so it is unclear that how fine-tuning of the model hyperparameters has been performed and the problem of overfitting has been addressed. I think the writers should be able to update the manuscript before publishing to make all these points clear and further illustrate and solidify their already good findings
Validity of the findings: All outcomes are well defined and codes are given, but to ensure the validity of the data, the points listed must be answered. I assume that the relevance of the results can be more readily asserted if the issues are discussed and explained.
Additional comments: The paper is clear, but there are few points that require Clarification
1. Explains the methodology used in order to correctly recognize Arabic zero numbers and letters written in continuous style.
 2.Discuss the criteria based on which a character is labeled as difficult /easy as specified in line number
 342.
3.You should clarify, why one should go for the proposed FDCNN model, when  Assiri,2019 stacked method has outperformed.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 3
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: The authors still only use a training set and a test set, with no validation set. The text doesn't guarantee in a sufficiently clear way that the results weren't optimized specifically for this test set, which could mean the data is biased to fit this specific test set.

The training hyperparameters should be optimized on a validation set, and the best parameters for the validation set should then be used to calculate the test metrics.
Validity of the findings: The previous comment on validation sets must be clearly addressed by the authors. It is not the same thing to use a validation set and then a test set, and to optimize directly for the test set performance (which seems to be the case here).

Furthermore, always prefer to state where the performance is calculated. If the accuracy was computed on the test set, mention it as "test accuracy". This must be abundantly clear to the reader.
Additional comments: no comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 3
Reviewer: 2

Output:
Basic reporting: Accept
Experimental design: Authors attended to the suggestions of the reviewers.
Validity of the findings: Authors attended to the suggestions of the reviewers.
Additional comments: Authors attended to the suggestions of the reviewers.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: FULL DEPTH CNN CLASSIFIER FOR HANDWRITTEN AND LICENSE PLATE CHARACTERS RECOGNITION
Review round: 3
Reviewer: 3

Output:
Basic reporting: The author has written in simple, unambiguous, and competent Language. The literature is well-referenced and applicable, with an introduction and history to show meaning. The paper discusses a new method for LP recognition that involves stacking two CNN networks. Batch normalization and a non-Linear activation layer follow the convolution layer, which is at the core of the convolution block.
Experimental design: All of the research questions have been presented and answered in a clear and concise manner, and they are all important and significant in the context.
Validity of the findings: All validation problems have been addressed; they are stable, statistically accurate, and well-controlled.
Additional comments: I congratulate the authors on their extensive data collection. Furthermore, the manuscript is written in plain, unambiguous language, and the comments have been satisfactorily addressed.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN AMALGAMATION OF YOLOV4 AND XGBOOST FOR NEXT-GEN SMART TRAFFIC MANAGEMENT SYSTEM
Review round: 1
Reviewer: 1

Output:
Basic reporting: The review of the state of the art is insufficiently addressed. Literature references should be improved. See for example:
F. Castaño, G. Beruvides, R. E. Haber, and A. Artuñedo, “Obstacle recognition based on machine learning for on-chip lidar sensors in a cyber-physical system,” Sensors (Switzerland), vol. 17, no. 9, 2017, doi: 10.3390/s17092109.
F. Castaño, G. Beruvides, A. Villalonga, and R. E. Haber, “Self-tuning method for increased obstacle detection reliability based on internet of things LiDAR sensor models,” Sensors (Switzerland), vol. 18, no. 5, 2018, doi: 10.3390/s18051508.
The quality of figures should be improved. Font size is too small in some cases.
Experimental design: Other machine learning methods should be explored. Why are not explored self-parameterization using gradient-free optimization methods? See for example:
R.-E. Precup and R.-C. David, Nature-Inspired Optimization Algorithms for Fuzzy Controlled Servo Systems, Butterworth-Heinemann, Elsevier, Oxford, UK, 2019.
R. Haber et al., "A Simple Multi-Objective Optimization Based on the Cross-Entropy Method," IEEE Access, vol. 5, pp. 22272-22281, 2017.
G. Wang and L. Guo, "A novel hybrid bat algorithm with harmony search for global numerical optimization," Journal of Applied Mathematics, vol. 2013, 2013.
R. H. Guerra et al., “Digital Twin-Based Optimization for Ultraprecision Motion Systems with Backlash and Friction,” IEEE Access, vol. 7, pp. 93462–93472, 2019, doi: 10.1109/ACCESS.2019.2928141.
Validity of the findings: Other performance indices and techniques should be included in the comparison to assess the real contribution of the proposed approach.
Conclusions are adequate according to the study.
Additional comments: The work addresses an interesting topic however the novelty and the progress beyond the state of the art is not sufficiently outlined.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN AMALGAMATION OF YOLOV4 AND XGBOOST FOR NEXT-GEN SMART TRAFFIC MANAGEMENT SYSTEM
Review round: 1
Reviewer: 2

Output:
Basic reporting: -This paper does not present a Professional English, since the authors must write the acronyms correctly, and some acronyms do not have their meaning. Correct this error. Also, the words "Figure", "Table", "Equation", "Algorithm", must be written with the first letter in capital letters. Review and correct it in the article.
-The authors have not placed some reference scientific literatures, so I suggest performing a more in-depth investigation and adding these articles:
----- Zambrano-Martinez, J. L., Calafate, C. T., Soler, D., Lemus-Zúñiga, L. G., Cano, J. C., Manzoni, P., & Gayraud, T. (2019). A Centralized Route-Management Solution for Autonomous Vehicles in Urban Areas. Electronics, 8 (7), 722.
--- Zhang, X .; Onieva, E .; Perallos, A .; Osaba, E .; Lee, V. Hierarchical fuzzy rule-based system optimized with genetic algorithms for short term traffic congestion prediction. Transp. Res. C Emerg. Technol. 2014, 43, 127–142
-The structure of the article is well detailed and will match the results with the hypothesis they present.
-The results are a bit weak on the part of the authors, because the authors must include some Figure with their respective explanation, when the scenario is not applied the prediction, and when the scenario is applied the prediction, that is noticed when it has exceeded the prediction.
Experimental design: -The authors do not present which scenario is being studied. Furthermore, they do not present the parameters for that study. Some Figure of the stage must be placed.
-Figure 2 can be transformed into an Algorithm, if it is done it would give more quality to the paper.
-Is there a figure that shows the prediction that the authors comment?
-With which simulator has the study been performed?
-More graphics are needed for the reader to understand more about the work done.
-How is the Smart Traffic Management System presented by the authors composed?
-What are the routes that the vehicles have taken to predict traffic?
Validity of the findings: -The authors do not present where the data comes from to perform the experiment. Which must be detailed, and referenced
-The conclusions must be improved according to the new implementations suggested by the reviewers that support the results.
Additional comments: -The authors must perform the changes suggested by the reviewers to improve the quality of the article.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN AMALGAMATION OF YOLOV4 AND XGBOOST FOR NEXT-GEN SMART TRAFFIC MANAGEMENT SYSTEM
Review round: 2
Reviewer: 1

Output:
Basic reporting: English should be carefully checked.
The review of the state of the art is not sufficiently addressed according to reviewers' suggestions.
Experimental design: Experimental design is fine.
Validity of the findings: Novelty is adequately addressed.
Additional comments: -


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN AMALGAMATION OF YOLOV4 AND XGBOOST FOR NEXT-GEN SMART TRAFFIC MANAGEMENT SYSTEM
Review round: 3
Reviewer: 1

Output:
Basic reporting: The paper can be accepted.
Experimental design: The paper can be accepted.
Validity of the findings: The paper can be accepted.
Additional comments: The paper can be accepted.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN AMALGAMATION OF YOLOV4 AND XGBOOST FOR NEXT-GEN SMART TRAFFIC MANAGEMENT SYSTEM
Review round: 3
Reviewer: 2

Output:
Basic reporting: -I suggest adding more references such as:
--Zambrano-Martinez, J. L., Calafate, C. T., Soler, D., Lemus-Zúñiga, L. G., Cano, J. C., Manzoni, P., & Gayraud, T. (2019). A centralized route-management solution for autonomous vehicles in urban areas. Electronics, 8(7), 722.
--Lv, Y.; Duan, Y.; Kang, W.; Li, Z.; Wang, F.Y. Traffic flow prediction with big data: A deep learning approach.
IEEE Trans. Intell. Transp. 2014, 16, 865–873
Experimental design: --There are no Figures of which are the polynomial curve for the predictive model.
Validity of the findings: Everything is fine.
Additional comments: Perform the suggested modifications.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: COMPI: A FRAMEWORK FOR PORTABLE AND REPRODUCIBLE PIPELINES
Review round: 1
Reviewer: 1

Output:
Basic reporting: Just a personal comment related to sentence beginning at line 216. There are many life science researchers who are circumstantial developers out there who are shy sharing their code, as they feel embarrassed about it. Therefore, most of those researchers are not used to version control systems yet.

First sentence in line 477 should be rewritten, so it is not misunderstood. It took me re-reading it at least twice, carefully, in order to get its meaning.
Experimental design: Line 485. I cannot agree with sentence "According to its documentation, Snakemake only allows the execution of external scripts written in Python, R, R Markdown, and Julia". If you have a look at https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html and at example at https://snakemake.readthedocs.io/en/stable/tutorial/basics.html#step-1-mapping-reads , you can see there that it supports "shell" command line run. Snakemake supports rules where some python code can be embedded through "run" keyword, similar to Nextflow functionality, where groovy code can be run using "exec" keyword https://www.nextflow.io/docs/latest/process.html#native-execution .
Validity of the findings: No comment
Additional comments: I have read the manuscript, and also the related documentation, scattered among the repository and the different web pages. I have both comments, issues, questions and suggestions:

Line 209 of the manuscript: I guess there is a typo, as "Compo Hub" should be "Compi Hub".

Related to Compi Hub and its links in the manuscript, I guess I have found a bug in Compi Hub behaviour (2021-02-22, Firefox browser). I tried both links at line 269 ( https://www.sing-group.org/compihub/explore/5d09fb2a1713f3002fde86e2 ), at line 357 ( https://www.sing-group.org/compihub/explore/5d807e5590f1ec002fc6dd83 ) and at line 390 ( https://www.sing-group.org/compihub/explore/5e2eaacce1138700316488c1 ), but they did not work. I went to Compi Hub in order to find the referred RNA-Seq, Metatax and GenomeFastScreen pipelines, and I found then at "compi-rnaseq-pipeline", "metatax" and "pss-genome-fs" with the very same links. Clicking there it worked, but copying and pasting the links in a sibling tab or window did not work. The same happens to https://www.sing-group.org/compihub/explore , which is the URL https://www.sing-group.org/compihub/ redirects to.

I have perceived that the different sites and documentation are a bit isolated among themselves.

* I have missed both in http://sing-group.org/compi/docs/ and https://github.com/sing-group/compi/blob/master/README.md a reference to the download area at http://sing-group.org/compi/#downloads , as well as installation instructions (at least, the easy ones).

* I have also missed in http://sing-group.org/compi/docs/ a reference to https://github.com/sing-group/compi/ .

* Also, I have missed in https://github.com/sing-group/compi/ an INSTALL.md or similar explaining the steps to build compi on different platforms or architectures (which could be needed by macOS users, for instance). This comment is related to sustainability of a workflow written in Compi .

* Releases at https://github.com/sing-group/compi/releases could be enriched with the installers you are already providing at http://sing-group.org/compi/#downloads .

I have been studying the custom runners behaviour, and they are a very interesting feature, albeit quite synchronous. For instance, Slurm runner example https://www.sing-group.org/compi/docs/custom_runners.html#generic-slurm-runner is using the synchronous executor srun. Scenarios which require implementing custom runners which involve asynchronous checks to some resource are harder to implement. Could you include in the online documentation an advanced example of "polling" in a custom runner?

As it is distilled from line 429 to line 438, and from line 444 to line 446 good (scientific) software practices should be encouraged. So, I propose you that "compi-dk new-project" command should create initial LICENSE.md, README.md and DEPENDENCIES.md files, as well as a "runners-example" directory with a couple of the more common custom runners, and "compi-dk build" should check their existence (and give a warning when they are not found).

I have also several questions and suggestions to the authors, related to future developments of compi and Compi Hub:

Have you thought on an expandable workflow metadata system for Compi workflows? Currently it is a key / string value system bound to tasks, so it is not possible to have metadata attached to the whole workflow, its inputs and outputs.

I have a question which might be related to previous suggestion. How the authors of a workflow can embed the details of its licence and attributions? Currently that information must be either it is derived from the git repository of the workflow or it should be embedded as a comment in the workflow, as metadata system does not support it.

Do you have sub-workflow support in Compi's roadmap? My question is more related to common sub-workflow patterns reuse than composing a meta-workflow.

Execution provenance. A way to report what happened in an execution would be generating a parse-able report with all the parameters, both explicit and implicitly set, along with additional information and metrics of the workflow execution and the custom runners.

Should workflow semantic versioning be only recommended or enforced? My question is two-fold, both philosophical and related to that the there is no restriction on the version string at XML Schema definition.

Although custom runners are a very powerful feature, they have greater potential. So, I recommend having in the roadmap stackable custom runners, so stacks of custom runners would allow easy combinations of, for instance, Slurm+singularity or SGE+conda environment activation.

An additional suggestion about custom runners is allowing them to use task context annotations or variables. In that way, custom runners would be more reusable. For instance, a runner needing a conda environment precondition with an specific package installed could learn which conda package should be installed from that received metadata. Or using different containers for different tasks, based on an annotation provided by the task.

Last suggestion involving custom runners is having in Compi Hub a way to upload custom runner templates, so researchers can reuse successful patterns.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: COMPI: A FRAMEWORK FOR PORTABLE AND REPRODUCIBLE PIPELINES
Review round: 1
Reviewer: 2

Output:
Basic reporting: This manuscript describes the features of Compi- an application framework to develop end-user, pipeline-based applications with emphasis on user interface generation, application packaging, with compi-dk, and application distribution provided through a public repository of Compi pipelines, named Compi Hub.
I find it quite interesting that the authors have developed an XML-based platform for creating workflows and I also had a look at the resources for Compi at (i) http://www.sing-group.org/compi/ and (ii) https://sing-group.org/compihub/explore and they seem to be pretty well-documented.
However I have some reservations, namely:
(i) The paper is written more like a user manual instead of an application paper - there are also some grammatical mistakes. e.g. line 117 has an incomplete sentence "Fourth, for security reasons, since Compi is intended to run pipelines defined by third party programmers." Similarly there are more.
(ii) The authors have not mentioned CWL - Common Workflow Language, at all. I would like to draw their attention to the fact that CWL (https://github.com/common-workflow-language) is a very mature workflow language, being used by many bioinformatics communities. They can check https://www.sevenbridges.com/cwl-seven-bridges-platforms/
(iii) Given that they have compared Compi with Nextflow, I would expect that they take a simple workflow as an example and demonstrate how it is defined in both in NextFlow and Compi so that readers can see the advantages of Compi compared to NextFlow.While there already exist NextFlow and CWL as two major workflow languages, readers must be convinced why they should use Compi.
Experimental design: No comment
Validity of the findings: No comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: COMPI: A FRAMEWORK FOR PORTABLE AND REPRODUCIBLE PIPELINES
Review round: 2
Reviewer: 1

Output:
Basic reporting: I have checked the bibliographic reference you have provided to CWL, it is a nice one (and one of the first citing CWL in some way, as I have learnt from https://www.zotero.org/groups/2294829/cwl/items/FZ7GRJWP/library). I recommend to keep this one, and also adding as reference what it is requested at https://www.commonwl.org/#Citation

Peter Amstutz, Michael R. Crusoe, Nebojša Tijanić (editors), Brad Chapman, John Chilton, Michael Heuer, Andrey Kartashov, Dan Leehr, Hervé Ménager, Maya Nedeljkovich, Matt Scales, Stian Soiland-Reyes, Luka Stojanovic (2016): Common Workflow Language, v1.0. Specification, Common Workflow Language working group. https://w3id.org/cwl/v1.0/ doi:10.6084/m9.figshare.3115156.v2

which is eligible as "Gray Literature" (https://peerj.com/about/author-instructions/#reference-section)
Experimental design: No comment
Validity of the findings: No comment
Additional comments: Meanwhile I was reading the detailed description of Compi Hub (around line 421) and checking the claims in the browser, I have missed in the overview tab of Compi Hub a legend explaining the meaning of each one of the possible symbols in the graph. For instance, I had to have a look at the XML of the workflow https://www.sing-group.org/compihub/explore/5d807e5590f1ec002fc6dd83#overview to realize that "join_pe" was a foreach, not an optional step.

Also, when I tried finding some keywords related to the command-line in the documentation at http://www.sing-group.org/compi/docs/ I have realized that there is some issue, as it did not work. I have tried both on Firefox and Chrome


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: COMPI: A FRAMEWORK FOR PORTABLE AND REPRODUCIBLE PIPELINES
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment
Experimental design: No Comment
Validity of the findings: No Comment
Additional comments: Based on the revised manuscript (in track-changes version) and the rebuttal letter, I am happy to note that the authors have really improved the manuscript and addressed my earlier comments.
They have discussed CWL as another well-known workflow framework, as well as provided an equivalent workflow in Compi for one in Nextlfow (in Supplementary 2)  and corrected the English grammatical mistakes.
Hence I now find the manuscript acceptable for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLUSTERING ASSESSMENT IN WEIGHTED NETWORKS
Review round: 1
Reviewer: 1

Output:
Basic reporting: "Clear and unambiguous, professional English used throughout."
Yes.

"Literature references, sufficient field background/context provided."
Some definitions, although they appear to be right, have not been referenced.

"Professional article structure, figures, tables. Raw data shared."
Yes.

"Self-contained with relevant results to hypotheses."
Yes.

"Formal results should include clear definitions of all terms and theorems, and detailed proofs."
In equation 1 explain the steps that were performed.
Experimental design: "Original primary research within Aims and Scope of the journal.
Research question well defined, relevant & meaningful. It is stated how research fills an identified knowledge gap."
Yes.

"Rigorous investigation performed to a high technical & ethical standard."
Yes.

"Methods described with sufficient detail & information to replicate."
The Louvain algorithm has parameters that can be modified, the parameters used must be presented.
The 1/2 used in equation 21 can be applied to graphs in addition to the directed ones, for example in weighted graphs (Ex: value -0.5/0.5).
Validity of the findings: "Impact and novelty not assessed. Negative/inconclusive results accepted. Meaningful replication encouraged where rationale & benefit to literature is clearly stated."
The article presents developments for the area.

"All underlying data have been provided; they are robust, statistically sound, & controlled."
The data in section 2.7 must be made available for download or the work must explain where they are.

"Conclusions are well stated, linked to original research question & limited to supporting results."
The article could feature a discussion session

"Speculation is welcome, but should be identified as such."
No comment.
Additional comments: The article brings significant contributions to the area, but some topics must be changed to better understand the reader.
The "scoring functions" of section 2.1 must be referenced.
Present equation 2 also as a summation.
In equation 18 add parentheses.
The computational complexity of the methods used must be explained. Many use the adjacency matrix, it has a high cost of n^2.
Say the scale of the graphics. The y-axis in figure 4 did not appear to me.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLUSTERING ASSESSMENT IN WEIGHTED NETWORKS
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors proposed a systematic approach to assess the clustering results in weighted networks. The theoretical presentations are sufficient and the experiments can support the evaluation metrics. However, some problems should be addressed as follows:
1.	For the abstract, the authors should give some specific performance analyses for the verification experiments.
2.	For the introduction, the background and the research significance should be re-expressed. Furthermore, the challenges and contributions should be explicitly summarized. There needs a paragraph at the end of this part to describe the organization of this paper.
3.	There exist a lot of acronyms without definitions for the first time, such as VAR, SBM, WSBM, and wSBM, etc., and some full names have been referred to repeatedly after the first acronyms, such as reduced mutual information. Please check the whole manuscript to revise them.
4.	In section 2.1, the definition and presentation of modularity should be given sub-title to highlight liking other metrics. Further, the variable Q has the same name with different definitions as in Section “Number of iterations”. Please revise the second variable name. In a complex network, Q is always considered modularity.
5.	For Figure 2, the “Forex club graph” should be corrected as “Forex graph” and “club” is “karate club graph”. For the second sub-figure, are the lines of variance and VI overlapped, please give the related explanation.
6.	Based on Figures 4 and 5, the trend analysis referred to the lamda>20 and lamda=15 may be inaccurate. Furthermore, why Figure 4 has not compared with the algorithm spin-class?
7.	In Section 3, some of the analysis referred to the Figures or Tables are inaccurate, for example, “the leading eigenvector algorithm the effect isn’t as pronounced” in line 451 on page 13, “internal density” in line 476 on page 13. Please check the related content in terms of the tables.
8.	Please improve the pixel for each figure.
9.	The manuscript has been presented in a disorganized way, the theoretical presentation and the experimental design should be given in different sections. Furthermore, the experiments and analysis should be reorganized.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLUSTERING ASSESSMENT IN WEIGHTED NETWORKS
Review round: 1
Reviewer: 3

Output:
Basic reporting: At a glance, in this paper, the authors present metrics to evaluate clustering algorithms, focused on weighted graphs.
Known the performance of community detection and clustering methods is important, especially when one does not know a priori the network topology or properties.
Despite the importance of evaluating the performance of such methods, the current work, in fact, is not able of telling readers how well a method performs. As the authors themselves say at the conclusion, the metrics help distinguish networks, which have a community structure from others that do not. It does not tell how well the metric can cluster or form groups, maintaining properties.
Experimental design: A second issue of the paper is related to its organization. Authors mix everything in a single section (materials and methods). The way they present it, it is not clear (i) what they are proposing. (ii) how they are planning to evaluate their proposal.
I suggest authors work on article organization, first by presenting the things you propose.
Second, in the following section, I suggest presenting the experimental design. In this section, you can present carefully all the material and methods.
You detail the way you generate random graphs. The way you generate perturbations.
Moreover, how do you design your evaluations (and comparisons)?
At this point, I think you have 3 steps to evaluate the metrics you are proposing:
The design of comparison with ground truth (the random graphs).
The design of the evaluations of synthetic graphs
In addition, the evaluation of real graphs.

In this same section, you may overview the clustering methods you will use in your evaluations.

Third, and finally, you present the evaluation section. In this section, you present your findings.
You must show how your metrics evidence that clustering methods are performing well or not.
In special, I suggest comparing to the state of the art. As your claim is “metrics to unweighted graphs are not appropriate… then you are proposing new metrics”. You must show this, in your result section. (that metrics for unweighted graphs are not appropriate AND… that your metrics outperform somehow existing state of the art).
Validity of the findings: In sum, the findings do no support the main goal of the paper. As you are presenting new metrics because the existing metrics do not perform well (or are not appropriate) to weighted graphs, you must show this clearly. Moreover, it is not clear that your metrics show how a clustering method performs. The way you conclude the article, you are just telling us that you can differentiate random graphs from graphs with clusters.
Additional comments: The abstract is confusing. I suggest focusing on the real paper contribution.
Developing a way to generate random graphs is not a contribution of the article. There are many methods to this purpose.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLUSTERING ASSESSMENT IN WEIGHTED NETWORKS
Review round: 2
Reviewer: 1

Output:
Basic reporting: All of my suggestions have been correctly modified.
In my opinion, the article is suitable for submission.
Experimental design: ...
Validity of the findings: ...
Additional comments: ...


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLUSTERING ASSESSMENT IN WEIGHTED NETWORKS
Review round: 2
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors have revised the manuscript according to the comments in detail. However, the figures should be re-generated with high quality, and resolution, font size, etc. Especially, the figure 1 and figure 2 are poor quality.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLUSTERING ASSESSMENT IN WEIGHTED NETWORKS
Review round: 2
Reviewer: 3

Output:
Basic reporting: My biggest criticism about the article remains about the lack of evaluation of how well a method performs. However, the authors clearly stated and answered me their main objectives.
I notice the improvements on the organization of the article and I have also followed the other reviewers questions and the authors questions.
I´m not an English language specialist. However, I think authors could use more formal of negative tenses in most part of the text, instead of abbreviation.
Minor suggestion: in line 52, page 6 – “that measure some topological characteristics.” -> can you specify the topological characteristics here to be more precise?
Figure 2 – labels are messy. One on top of other.
Experimental design: no comment
Validity of the findings: no  comment
Additional comments: no comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LOCAL NETWORK CONNECTIVITY OPTIMIZATION: AN EVALUATION OF HEURISTICS APPLIED TO COMPLEX SPATIAL NETWORKS, A TRANSPORTATION CASE STUDY, AND A SPATIAL SOCIAL NETWORK
Review round: 1
Reviewer: 1

Output:
Basic reporting: a)	The language of this article is professional, clear and technical.
b)	The introduction and background part have elaborated the gap and problem this work aiming to solve. Related literatures or prior works have been reviewed in introduction part.
c)	The structure of this article is identical to the requirement of acceptable format. Figures in this article also support the corresponding content. However, in my option, all figures seem to be blurred and authors should try to solve this problem.
d)	All the results in this article and supplementary materials can support the hypotheses or problem definition in this article.
e)	Symbol definitions are clear and relevant.
Experimental design: a)	This article is in accordance with the Aims and Scope of the journal.
b)	Question definition (optimizing network connectivity around a specific node with the introduction of new edges) and gap between prior works and this article (Optimizing the network connectivity of additional edges attempts to maximize the number of nodes within a given distance to a focal node to be connected and  minimizing the number and length of additional connections) are clear and the following content also discuss the pathway to fill this gap.
c)	The experimental design basically conforms the ethical standard. A study of urban transportation planning in experimental part shows the effectiveness of proposed solution path even though the other results are obtained by performing algorithm on random networks. I think this experimental design is acceptable.
d)	Method introduction or discussion is clear.
Validity of the findings: a)	The novelty of this article is acceptable. It really solves the problem mentioned in introduction and this problem is actually meaningful in the real world. Results data and discussion are sufficient.
b)	The conclusion part is well stated and summarize the whole content in the article.
Additional comments: Generally speaking, it is an interesting work. The problem it solves seems to be minor but actually improve the related researches of network connectivity optimization. By reviewing this paper, I think authors devote themselves to solve this problem step by step. However, I have just one question or suggestion for authors: I hope authors can perform their method on one more real-world network rather than random networks. The transportation network is good. Another study on the real-world networks such as wireless networks or vehicular social networks is encouraged.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LOCAL NETWORK CONNECTIVITY OPTIMIZATION: AN EVALUATION OF HEURISTICS APPLIED TO COMPLEX SPATIAL NETWORKS, A TRANSPORTATION CASE STUDY, AND A SPATIAL SOCIAL NETWORK
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: In "Network connectivity optimization: An evaluation of heuristics applied to complex networks and a transportation case study" authors provide new methods for a problem encountered in systems across many different fields -- namely, optimizing network connectivity to maximize the number of nodes within a given distance to a focal node and then minimizing the number and length of additional connections. This is important in several domains including transportation planning, telecommunications networks, and geospatial analysis.

I have very much enjoyed reading this paper. I find it comprehensive and clearly written, and introducing new, timely, and important results that will surely also inspire future research along these lines. The provided random spatial network formulation can be beneficial to complex systems research, network theory, and interdisciplinary applications. The introduction is also very comprehensive and informative, particularly for the PeerJ Computer Science audience. For these reasons, I am warmly in favor of publication subject only to minor revisions.

In terms of related works and reviews, I would recommend Saving human lives: What complexity science and information systems can contribute, J. Stat. Phys. 158, 735-781 (2015) where such research has been advocated for.

I would also encourage the authors to extend the abstract more with the key results. As it is, the abstract is a little thin and does not quite convey the interesting results that follow in the main paper.

Some references contain errors, missing or incorrect information, and inconsistent formatting. References should thus be corrected with the best care.

Apart from this, I am happy to congratulate the authors to an excellent contribution.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LOCAL NETWORK CONNECTIVITY OPTIMIZATION: AN EVALUATION OF HEURISTICS APPLIED TO COMPLEX SPATIAL NETWORKS, A TRANSPORTATION CASE STUDY, AND A SPATIAL SOCIAL NETWORK
Review round: 1
Reviewer: 3

Output:
Basic reporting: This paper studied the network connectivity optimization problem of minimizing the number of nodes within a given distance to a focal node by adding an edge. The authors mainly used a genetic algorithm to solve the problem. The ideas and results are good, but I think it is not suitable to be published in its current form due to the bad writing. A lot of necessary explanatory information was omitted, which makes readers unable to fully and correctly understand the thoughts the authors want to deliver. For instance, in Equations (2), (3), (4), (5), and (7), what is the meanings of C(i,j), B(i,j), O^t, gene(i, k)? And in the “Local search methodology”, the authors did not give a detail description of the crucial two-level selection process. Besides, other problems exit. For instance, sub-figures (B) and (C) of Figure 2 are vague and unreadable. And in Equation (1), it is confusing that the authors tried to maximize the cost of the newly added edge.
Experimental design: ok
Validity of the findings: ok
Additional comments: This paper studied the network connectivity optimization problem of minimizing the number of nodes within a given distance to a focal node by adding an edge. The authors mainly used a genetic algorithm to solve the problem. The ideas and results are good, but I think it is not suitable to be published in its current form due to the bad writing. A lot of necessary explanatory information was omitted, which makes readers unable to fully and correctly understand the thoughts the authors want to deliver. For instance, in Equations (2), (3), (4), (5), and (7), what is the meanings of C(i,j), B(i,j), O^t, gene(i, k)? And in the “Local search methodology”, the authors did not give a detail description of the crucial two-level selection process. Besides, other problems exit. For instance, sub-figures (B) and (C) of Figure 2 are vague and unreadable. And in Equation (1), it is confusing that the authors tried to maximize the cost of the newly added edge.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LOCAL NETWORK CONNECTIVITY OPTIMIZATION: AN EVALUATION OF HEURISTICS APPLIED TO COMPLEX SPATIAL NETWORKS, A TRANSPORTATION CASE STUDY, AND A SPATIAL SOCIAL NETWORK
Review round: 2
Reviewer: 1

Output:
Basic reporting: a)	The language of this article is professional, clear, and technical.
b)	The introduction and background part have elaborated the gap and problem this work aiming to solve. Related literature or prior works have been reviewed in the introduction part.
c)	The structure of this article is identical to the requirement of an acceptable format. Figures in this article also support the corresponding content.
d)	All the results in this article and supplementary materials can support the hypotheses or problem definition in this article.
e)	Symbol definitions are clear and relevant.
Experimental design: a)	This article is in accordance with the Aims and Scope of the journal.
b)	Question definition (optimizing network connectivity around a specific node with the introduction of new edges) and the gap between prior works and this article (Optimizing the network connectivity of additional edges attempts to maximize the number of nodes within a given distance to a focal node to be connected and minimizing the number and length of additional connections) is clear and the following content also discuss the pathway to fill this gap.
c)	The experimental design basically conforms to the ethical standard.
d)	Methods introduction or discussion is clear.
Validity of the findings: a)	The novelty of this article is acceptable. It really solves the problem mentioned in the introduction and this problem is actually meaningful in the real world. Results data and discussion are sufficient.
b)	The conclusion part is well stated and summarizes the whole content of the article.
Additional comments: Generally speaking, I am very satisfied with this version. The authors have already modified the papers based on the reviewers' comments. More specifically, I suggest the authors add a case study of the transportation networks and I have seen the improvement in this version. The authors give a detailed response to other reviewers' comments. The enhancement of this paper can be observed.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LOCAL NETWORK CONNECTIVITY OPTIMIZATION: AN EVALUATION OF HEURISTICS APPLIED TO COMPLEX SPATIAL NETWORKS, A TRANSPORTATION CASE STUDY, AND A SPATIAL SOCIAL NETWORK
Review round: 2
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: everything OK
Validity of the findings: everything valid
Additional comments: The authors have revised their manuscript comprehensively and with love to detail. I warmly recommend publication in present form.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LOCAL NETWORK CONNECTIVITY OPTIMIZATION: AN EVALUATION OF HEURISTICS APPLIED TO COMPLEX SPATIAL NETWORKS, A TRANSPORTATION CASE STUDY, AND A SPATIAL SOCIAL NETWORK
Review round: 2
Reviewer: 3

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors has addressed all of my previous concerns.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS GENERALISABLE HATE SPEECH DETECTION: A REVIEW ON OBSTACLES AND SOLUTIONS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper introduces a survey about the possibility of generalization of existing hate speech detection models and approaches.
The topic addressed broadly belongs to the sentiment analysis area and especially to the application of sentiment analysis techniques to the detection of hateful content and related phenomena, a field which is considered ad highly interdisciplinary and can be of some interest for the journal.
The paper is clear for what concerns the content and well written for what concerns English.
The organization of the paper is clear and allows the reader to easily find the section of possible interest.
The literature of the area has been carefully analyzed and reported including also very recent papers.
Experimental design: The article introduces a rigorous and precise survey of the hate speech detection task under a novel perspective which according my knowledge has never been systematically surveyed by other papers. Previous surveys and papers where the resources considered in this study are properly cited.

The methodology applied is carefully described and can be reproduced by other researchers.

The organization of the paper is that typical of surveys and help the reader in understanding the content of the paper. Moreover also novel proposals for addressing the weaknesses detected in the analyzed studies are provided by the authors showing their involvement in the area and paving the way for further discussing the topics addressed.
Validity of the findings: The paper analyzes a very interesting topic and provides enough support for the ideas expressed. The analysis is organized around a set of parameters that allows a discussion about all the crucial aspects.

What in my opinion reduces the interest of this paper is the fact that it doesn't take into account the multilingual perspective and, without an explicit citation of the fact that the study only focus on English language, it seems to assume that hate speech detection is only applied on English data. This assumption seems also supported by the citation of the survey about Arabic, as an exception, and by the fact that the possibility of extending the present study in a multilingual perspective isn't even listed in future work section. But this assumption is not correct: there are tasks about hate speech on languages other than English (e.g. for Italian within Evalita or for Spanish within SemEval and IberEval), and also some of the datasets cited and analyzed in the paper are multilingual (e.g. Basile includes data for English and Spanish both).
In my opinion, the generalization across languages is a crucial topic that is currently addressed in a growing set of papers about sentiment analysis and related tasks, and as a review, but also as a reader, it is a topic that I'd expected to find in a survey about generalization of hate speech detection. I suggest therefore a revision of the paper according this direction, by considering language as a further parameter for the evaluation of generalization, citing in the tables the languages involved in the datasets and so on.
Additional comments: I really appreciate the survey and I think that it introduces a perspective of analysis of a very hot task which can be especially interesting for people working on hate speech detection, but also for other researchers because it introduces a methodology that can be usefully applied in other areas.

It should be useful to put table 2 before table 1, thus providing early the references extensively used in the section and necessary to better understand the content of the section.

My major suggestions about how to improve the paper are expressed in previous points.

For what concerns the references, about the exploitation of non standard language in social media there are very recent papers about this topic and among them https://arxiv.org/abs/2011.02063 which provides a multilingual perspective about this topic.

I've detected some typos in the bibliography, where sometimes the first letter in the name of a language or a proper name is not in uppercase as it must be (e.g. english > English, twitter > Twitter, rnns > RNNs)


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS GENERALISABLE HATE SPEECH DETECTION: A REVIEW ON OBSTACLES AND SOLUTIONS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper presents a survey of research works aimed to investigate the generalisability of hate speech detection models. The paper also provides extensive consideration of obstacles and future directions for the task of hate speech.

The article is well-written, clear, and correctly organized. I think the paper is within the scope of the journal and the topic is currently of main interest to the Natural Language Processing community.

The introduction should more explicitly state the audience to which it is referring to and, in order to be more broadly understood, introduce the concept of “generalisation” before using the term (lines 46-48 should be mentioned later in the introduction).

Several literature reviews have been published recently on the subject of hate speech (Poletto et al., 2020; Vidgen and Derczynski, 2020; Fortuna and Nunes, 2018) but none of them cover the issue of generalisation of hate speech detection models. Making the paper an interesting review of a point of view not already surveyed.
Experimental design: The paper presents an unbiased coverage of the subject in the Section named “Generalisation studies in hate speech detection”. While the review comprehends all the papers approaching the problem of generalisation, the research works proposed in this direction are still limited (8 papers). Tables 1 and 2 could be very useful for the community, but the comment associated with these tables is very short and provides a list-based explanation of each of the 8 papers involved. A survey should be able to provide different views and aggregations of the surveyed papers for highlighting commonalities and differences. This is not possible with such a low number of surveyed papers, which makes me doubtful of the need for a survey on this topic at the current stage.
The authors should go more into the details of the surveyed models.

Please present Table 2 before Table 1. The current order makes it not possible to understand that, for example, Basile and Waseem (lines 141 and 142) are actually benchmark datasets associated with papers and not just names. You should also consider using references instead of just the first author’s surname.

The reference (Fersini et al., 2019) is wrong, as it should be associated with “Overview of the EVALITa 2018 task on automatic misogyny identification (AMI)”
Validity of the findings: The discussion and future directions of the paper should be referred to obstacles and suggestions to address these obstacles for generalisable hate speech detection. However, I find that almost any of the obstacles or suggestions are focused on the generalisation problem but instead on generic hate speech detection. This is a crucial critic of the paper, as the focus on the generalisation problem is the one that makes it different from the existing literature review. The authors should explicitly report, for each obstacle and discussion point, why it is needed for improving generalisation. This is a crucial issue that invalidates the two sections of the paper that are also the longest.

An example of this issue is in the “Reducing overfitting” paragraph, where the authors are suggesting to generalise by training on more datasets as a solution to overfitting. This is problematic because (1) the authors are not mentioning that they are referring to generalisation (which is the topic of the paper) and (2) it is not clear how they are suggesting it while they are also stating that there are big issues when it comes to generalise.

The authors should also comment on the future issues reported in  (Fortuna and Nunes, 2018), for example, open-sourcing and multilingual research.
Additional comments: Minor comments:
- Lines 109-114: the paragraph is unclear. Please re-write and clarify what you mean with “section headings of the body of this paper”
- Line 512: Please do not use contractions.
- Lines 558-565: Please consider to cite also the work titled “Comparative Evaluation of Label-Agnostic Selection Bias in Multilingual Hate Speech Datasets” recently presented at EMNLP 2020
- Line 595: Please consider to cite also the work titled “Contextualizing Hate Speech Classifiers with Post-hoc Explanation” presented at ACL 2020


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS GENERALISABLE HATE SPEECH DETECTION: A REVIEW ON OBSTACLES AND SOLUTIONS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The paper is well written and clearly organized.
In the revised version of the paper the references have been updated wrt the previous version, including missing papers too, and better organizing the discussion of the cited literature.
The aspect of tables should be improved, first of all by applying to them all the same layout and the same character size.
Experimental design: According to my knowledge, the paper provides a careful investigation, not always systematic, but quite complete and correct.
Validity of the findings: The contribution that this paper provides to the existing literature is clear and the conclusions well stated.
Notwithstanding the variety of surveys about hate speech detection and related tasks, the paper provides a quite original perspective on this research area.
Additional comments: The paper has been carefully revised bu the authors and the suggestions provided by the reviewers used for improving its quality.

For what concerns in particular the discussion about multi-lingualism, I also appreciate if some words in the final version will be spent about the fact that the most participated task about hate speech detection (Hateval, often cited in other parts of the paper) is in principle a multilingual task since it is based on Spanish and English data collected and annotated according to the same methodology.

The authors are moreover strongly encouraged to apply a final careful spell check because I've detected some typos (listed below) and probably some other occurs that I've not detected in my reading.

line 21: content are > contents are OR content is
line 77: data and corpora resources > data and corpora
line 173: around 30 when > around 30 points when
line 193: a handful of recent studies suggest > a handful of recent studies suggests OR recent studies suggest
line 803: abuse posts > abusive posts


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 1
Reviewer: 1

Output:
Basic reporting: Thanks to the author for using a clear and understandable English in article. It's nice to mention dynamic analysis and static analysis in the article. however, there are a few more approaches in the literature. For example, these are signature-based approach and hybrid approach. The author should mention them as well. In my opinion, the author can add them to line 61.  The author should briefly explain these approaches in the article.

Between lines 63 and 71, a comparison of Dynamic and Static analysis has been tried. It would be more appropriate to include this in a comparison table with citations. The author can add the signature-based approach and hybrid approaches in this table.

As the author mentioned in line 119, He used “static analysis”, “malware”, and “Android” in the CA database for his literature review. however, the keywords of the article are; Android; Review; Static analysis; Machine learning; Features. Author should add “malware” here like as others.

Table 2 and 7, Figure 9 can be removed. The author should explain “Figure 5” in section 2.3. Feature more. A little more explanation for Figure 5 in there is better.

A comparison table with 4 machine learning (CL, RL, NN&DL and EM) should be created with citations. This table should show the machine learning metrics (such as KNN,SVM,K Means, Deep Q,MLP or RF)  used as well. This table created can be added to section 2.4.1.This section should explain the advantages and disadvantages of compared machine learning methods. The advantages and disadvantages of static analysis against these ML methods should also be mentioned.

Table 9 mentions “studies conducted to overcome obfuscation”. Open source advantages and disadvantages in section 3.1 can also be added to this table with citations.

Table 10 is very nice. It is listed by years. The 2nd listed criterion may be reference numbers. The reference numbers in the table can be listed in ascending order. So it can look better

The interpretation of table 10 should be given more in section 3.3. It is useful to add explanations such as which ML was used the most and why it was preferred. The frequency of using static and dynamic analysis in studies can also be mentioned. Is the data set used or not used in the studies? Similar interpretations should be made for table

Check whether the web links in references are accessible. The author can use ”webcitation” for such links.
Experimental design: no comment
Validity of the findings: no comment
Additional comments: All section numbers in the article should be rechecked. It seems to me that it will be “3.2.1. Research to counter obfuscation” instead of 4.2.1. and it should be 3.2.2. instead of 4.2.2 as well.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 1
Reviewer: 2

Output:
Basic reporting: In this study, a review has been made on malware detection systems in the Android operating system, which is one of the current topics in the literature. It can be important as current issues are covered in the study. However, serious deficiencies are observed in the writing style and organization of this study. In addition, the studies examined need to be detailed. The study seems superficial in this state. For this reason, I see benefit in reorganizing the study. Otherwise, the work will look too disorganized and difficult to understand.
Experimental design: My suggestions are as follows:
Which journals have articles listed most at the end of your research? In other words, which journals have been used more? Survey publications are especially important for new researchers who will work in this field. How many articles from the most frequently used 5 or 10? These journals should be given in a table. In this way, it will be revealed which journals will mostly benefit people who will conduct research in this field. These journals can be added by opening a new subsection in Section 1 or where appropriate.
When referencing tables and figures, there is no need to use the above or below patterns. It is already clear which table or figure it is. As a matter of fact, the tables or figures in the munuscript I read are in a different file. Instead of the above or below, a statement like in Table x, Figure y will be more fluent.
The data sets used in this field are given in Section 2.1. When I examine some of the references provided in this section, I only see the names of the relevant data sets in the reference list. For example, in [31, 32, 34, 35] references only the names of the relevant data sets are written. The official link of the data set should be added to these references. If there are blank references other than these, they should also contain the necessary information. This is important for the reader.
While "apktool" and "aapt" tools are expressed separately in Figure 4, these two tools are given as "Apktool @ aapt" in Table 5. Why is it included in Table 5? Is it given as “Apktool @ aapt” because the related works use both tools? I suggest it to be given separately in the works using the "apktool" and "aapt" tools. If it will be used together, the necessary explanation should be made. Otherwise, the reader will be confused.
Are there any studies using the Reinforcement learning technique under section 2.4.1? While the studies using other techniques are given in Table form, there is no table related to this technique. If there are no studies using this technique, an explanation should be made about it.
Some paragraphs consist of one or two sentences. Paragraphs should be expanded using meaningful sentences or combined with other paragraphs in a way that does not disturb the integrity of the part.
On line 562, a section titled "Open research issues" is written. What is explained with the chapter title is not compatible. When I read the title, I thought it would give obvious problems in this area, but I did not encounter such a thing. I was disappointed as a result. This section needs to be rewritten.
There is a sentence in the subsection on line 637. This subsection should be expanded in line with the relevant tables.
When Table 10 is examined, A and M are similar to each other. This needs to be parsed. Because "Manifest file" represented by M can be evaluated as a subset of "apk, dex and XML properties" represented by A. This will make it difficult for the reader to understand.
Validity of the findings: Reviewed articles in this study should be detailed.
Additional comments: In this study, a review has been made on malware detection systems in the Android operating system, which is one of the current topics in the literature. It can be important as current issues are covered in the study. However, serious deficiencies are observed in the writing style and organization of this study. In addition, the studies examined need to be detailed. The study seems superficial in this state. For this reason, I see benefit in reorganizing the study. Otherwise, the work will look too disorganized and difficult to understand.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 2
Reviewer: 1

Output:
Basic reporting: A comprehensive survey study has emerged. I think contribution has been made to the field of Android malware detection.
Experimental design: There is no problem.
Validity of the findings: There is no problem.
Additional comments: The authors made the suggested changes. The study is acceptable.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 2
Reviewer: 2

Output:
Basic reporting: Survey methodology and research methods are well explained. The Introduction section provides sufficient information about the general content of the study.
Experimental design: 1- Different reverse engineering tools are mentioned in Section 2.2. The studies they are used and the way they are used are expressed. However, the reverse engineering tools themselves are not cited.

2- In the description of Figure 8, the expression "(B) However, DL involves automatic feature selection and malware classification" is used. However, in the B part of the figure, "feature extraction + classification" expression is used. This situation causes the two different terms feature extraction and feature selection to be perceived as the same thing. Likewise, there is the same confusion in the paragraph starting at line 470.

3- At the end of the paragraph starting on line 470, publication 106 is cited. However, I could not see the direct relationship of the cited work with DL or automatic feature extraction. In Table 8, publication 106 is cited again and the year of publication is written as 2015. However, the year of publication 106 in the references is 2019. I think there is a confusion in the citations.

4- Although the last reference number is 138 in the references section, I see that there are higher numbered references in Table 11. I think there are missing references in the references table.
Validity of the findings: no comment
Additional comments: The research methods and organization of the study are very good. It clearly presents the boundaries of the researched area to the reader. However, the confusion, especially in references, needs to be corrected.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 3
Reviewer: 1

Output:
Basic reporting: the article meets the standards, with suggested improvements.
Experimental design: issues given for previous version are completed.
Validity of the findings: Conclusions are well presented, related to research question & limited to supporting results.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 3
Reviewer: 2

Output:
Basic reporting: Some references do not show the year, volume, journal name. Missing references need to be edited.
Experimental design: No comment
Validity of the findings: No comment
Additional comments: 1) In Table 3, the journal "Expert Systems with Applications" is written twice. You can delete one of them.
2) A dot was used before the reference in lines 265 and 266. This should be fixed. “Feature selection is important in order to increase the accuracy of the detection system. [18], [42], [43]. " -> "Feature selection is important in order to increase the accuracy of the detection system [18], [42], [43]."
3) The concepts of feature extraction and feature selection are mixed. It is not the feature selection described in the subsection 2.3, but the feature extraction. Obtaining attributes such as permissions and APIs from application files is feature extraction. Attribute selection is completely different. This must be edited.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: MALWARE DETECTION USING STATIC ANALYSIS IN ANDROID: A REVIEW OF FECO (FEATURES, CLASSIFICATION, AND OBFUSCATION)
Review round: 3
Reviewer: 3

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors made the suggested changes.. In my opinion, the manuscript is acceptable.