Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A STORAGE-EFFICIENT ENSEMBLE CLASSIFICATION USING FILTER SHARING ON BINARIZED CONVOLUTIONAL NEURAL NETWORKS
Review round: 2
Reviewer: 1

Output:
Basic reporting: Revision well done
Experimental design: Revision well done
Validity of the findings: Revision well done
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A STORAGE-EFFICIENT ENSEMBLE CLASSIFICATION USING FILTER SHARING ON BINARIZED CONVOLUTIONAL NEURAL NETWORKS
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: No comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A STORAGE-EFFICIENT ENSEMBLE CLASSIFICATION USING FILTER SHARING ON BINARIZED CONVOLUTIONAL NEURAL NETWORKS
Review round: 2
Reviewer: 3

Output:
Basic reporting: The authors addressed the comments in a very good way.
Experimental design: The authors addressed the comments in a very good way.
Validity of the findings: The authors addressed the comments in a very good way.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LORM: A NOVEL REINFORCEMENT LEARNING FRAMEWORK FOR BIPED GAIT CONTROL
Review round: 1
Reviewer: 1

Output:
Basic reporting: This paper proposes a biped control framework based on reinforcement learning. The expert trajectory of the traditional controller is introduced to accelerate the training. And the exploration of reinforcement learning ensures the final model outperforms the expert instead of simply imitating the expert. To improve the training efficiency and performance, some improvements are also introduced into the framework. The method is validated by various experiments, including two tasks (walking as fast as possible & tracking specific velocity) and several different environments (plain, up-hill, down-hill and uneven floor). The work is conducted well and is promising in different control tasks. Both the framework and the tricks are inspiring for other works.
Experimental design: Problems in the content and explanation of the paper:
1. The description of the PPO algorithm is not detailed enough. The meaning of lambda and gamma in Table1 and their usage are not illustrated.
2. The experiments are rich for the LORM models. However, the performance of reference motion should be illustrated to show the improvement or difference between the proposed algorithm and reference motion.
Validity of the findings: Other details should also be edited:
1. Some paragraphs have indentation while others do not. For example Line 295 and Line 297; Line 273 and Line 276.
2. Please avoid capital letters in sentences. For example, Line 15 “ Learn and Outperform the Reference Motion (LORM), an RL based framework ... “ in the abstract.
3. Equation 17: f(x) = ..., however, there is no variable x or function f(x). I think it should be Criterion = ... .
4. The names of curves in Figures can be polished, for example, Fig11. And the captions can be used in figures to make the meaning of curves more clear.
5. The language should be further polished. Especially in the subsection “Symmetrization of actions and states”, I wonder whether it can be more clear? Though the description is understandable, it takes time to read and understand it.
Additional comments: In addition, I have two questions to be answered by the authors:
1. In the subsection “Symmetrization of actions and states”, why only the angles of joints are symmetrized while other observations keep unchanged?
2. The input observation of the agent contains many items which can be obtained in simulation software (base position, position of centre of mass). However, is it possible to obtain them in the real world?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LORM: A NOVEL REINFORCEMENT LEARNING FRAMEWORK FOR BIPED GAIT CONTROL
Review round: 1
Reviewer: 2

Output:
Basic reporting: An RL based framework for gait controlling of biped robot is proposed in this paper to overcome the complications of dynamics design and calculation. The results validated the efficiency and the advantages of the proposed method. However, there are several suggestions for the authors
1.	As the proposed method is claimed to be a novel method, there should be more literature discussion in the introduction part to clarify the state-of-art of the field and thus the novelty of the paper.
2.	In the result and discussion part, it is better to compare and validate the result with published works to make it more convincing.
3.	There are some typo and grammar errors in the paper, please give it a proofreading for the language check.
The results are sufficient enough to validate the aim of the paper. However, more discussions are expected to emphasize the novelty and significance of the method.
Experimental design: In the result and discussion part, it is better to compare and validate the result with published works to make it more convincing.
Validity of the findings: As the proposed method is claimed to be a novel method, there should be more literature discussion in the introduction part to clarify the state-of-art of the field and thus the novelty of the paper.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LORM: A NOVEL REINFORCEMENT LEARNING FRAMEWORK FOR BIPED GAIT CONTROL
Review round: 2
Reviewer: 1

Output:
Basic reporting: The manuscript has been well-revised. Literature review is sufficient with good background provided. The results contains clear definition of all terms.
Experimental design: The experiment design is clear with sufficient details and justified. The manuscript has met the standards of the journal.
Validity of the findings: The findings have been rigorously validated with sufficient details. Conclusions are well stated.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LORM: A NOVEL REINFORCEMENT LEARNING FRAMEWORK FOR BIPED GAIT CONTROL
Review round: 2
Reviewer: 2

Output:
Basic reporting: all the issues I raised have been addressed
Experimental design: all the issues I raised have been addressed
Validity of the findings: all the issues I raised have been addressed
Additional comments: all the issues I raised have been addressed


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: VEGETATION INDICES’ SPATIAL PREDICTION BASED NOVEL ALGORITHM FOR DETERMINING TSUNAMI RISK AREAS AND RISK VALUES
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1. There is a need to check the grammar, spelling, sentence structures, etc. of the paper.
	In the title “A New Algorithm for …….on Spatial Predicition of 3 Vegetation Index”. The word “Index” should be in plural form since more than one index is used in the paper.
	In other parts of the paper, I found several errors on the spelling of words, verb agreement, correct usage of articles, etc. There is a need to go over the entire paper to check on these errors and reconstruct or rephrase some sentences which are redundant (e.g.Lines 239-242 needs to be reconstructed to avoid redundancy). I would suggest that the authors use an APP to correct these errors.
2. A reorganization of the Abstract is recommended. The results of the algorithm should be written in the last lines and the presentation of the algorithm and the indexes used should be more profound to emphasize the objective of the paper.
3. The tsunami risk levels (Levels 1, 2, and 3) are based on indicators resulting from the algorithm (Lines 358-360). First, the basis of these indicators  was not explicitly explained. Second, how would you classify the risk if TRUE is within the range  (1,2) or (2,3)? Is it always the case that the algorithm will result to TRUE=2 in your simulations?
4. In your regression model (Equation 10), the authors need to explain  how they obtained the constant coefficients (e.g. 0.95, 0.007, etc.)
5. Comments on Figures
	Are the location maps taken by the authors themselves? If not, then there is a need to indicate the source below the figure. Figure 3, although it’s a captured image, its’ source should be written below. Also, the figure is more likely to be identified as a table rather than a figure. The x-axis and y-axis must be given labels on Figures 5,6,7,8 and the titles whould be more clear and concise. In Figures 9,10,11, the legend uses a color coding to show the level of risk is not properly defined. Since you are identifying the risk levels with color codes, some colors are not properly identified. Also, it would be appropriate to give the range of values for the risk levels so that readers of the paper can properly distinguish the different risk levels in the graph.
6. Comments on Tables
	Since the purpose of the authors is to make a comparative analysis on the amount of error generated using different algorithms and that of their proposed algorithm, I would suggest either of the following improvements:
a. Tables 2,3,4,5 can be reconstructed as one table showing the amount of error (MSE. RMSE, MAE) of each algorithm based on each index, or
b. Three separate tables for each classified error (i.e. MSE. RMSE, MAE) comparing the algorithms and the amount of error based on the 3 indexes.
Since each table enumerates the amount of error of each index with prediction results using the varied machine learning algorithms in your study, it would be better to fuse all 4 tables as one where all the algorithms are listed together with the indexes and the classification of errors. In this way, you can make a clear comparison of the amount of error. Alternatively, you may create three tables, each for the identified error MSE, RMSE, MAE separately, and compare the results of the algorithms.
Experimental design: the experimental design is well explained and detailed
Validity of the findings: The findings are valid based on the provided data. The objectives of the study were achieved.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: VEGETATION INDICES’ SPATIAL PREDICTION BASED NOVEL ALGORITHM FOR DETERMINING TSUNAMI RISK AREAS AND RISK VALUES
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper studies a new algorithm to detect tsunami risk areas based on spatial modelling of vegetation index.  I think three questions must be answered in this paper.

1. I suggest to add some more explanation for the flowchart/algorithm to make it more clear and understandable.
2. In Eq. (8), the closest point search technique is performed by using the Euclidean distance formula. Why did you prefer Euclidean distance instead of Hamming distance or Hausdorff distance?
3. Is it possible that the closest point search technique is performed by using the similarity between data points instead of Euclidean distance?
Experimental design: No comments
Validity of the findings: No comments
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: VEGETATION INDICES’ SPATIAL PREDICTION BASED NOVEL ALGORITHM FOR DETERMINING TSUNAMI RISK AREAS AND RISK VALUES
Review round: 2
Reviewer: 1

Output:
Basic reporting: The English structure of the revised paper shows clarity and a more improved version than the initial paper submitted. Figures and Tables are given the appropriate titles and the data of their results and the raw data were shared. All terms and Acronyms used in this paper are defined according to the context of their paper
Experimental design: no comment
Everything is accounted for
Validity of the findings: Findings are valid and conclusions are well stated with future recommendations
Additional comments: Dear Peer J Team,
By comparing my suggested comments and revisions of my previous review of this paper and the newly submitted revised paper, all my comments and suggestions were addressed which improved a lot the quality and consistency of this paper.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: VEGETATION INDICES’ SPATIAL PREDICTION BASED NOVEL ALGORITHM FOR DETERMINING TSUNAMI RISK AREAS AND RISK VALUES
Review round: 2
Reviewer: 2

Output:
Basic reporting: The authors have addressed my concerns
Experimental design: ok
Validity of the findings: ok
Additional comments: n/a


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A DEEP CROWD DENSITY CLASSIFICATION MODEL FOR HAJJ PILGRIMAGE USING FULLY CONVOLUTIONAL NEURAL NETWORK
Review round: 1
Reviewer: 1

Output:
Basic reporting: Authors in their work proposed a new dataset for crowd analysis. In addition to this, proposed a fully convolutional neural network (FCNN)-based method to monitor the crowd. There is merit in the dataset annotation but the manuscript still need a lot of work.

1) Literature references are not sufficient. The related work section needs more work. There is a lot of work done in this field is reviewed in the following papers [1,2].
 [1] Sindagi, Vishwanath A., and Vishal M. Patel. "A survey of recent advances in CNN-based single image crowd counting and density estimation." Pattern Recognition Letters 107 (2018): 3-16.
 [2] Gao, Guangshuai, et al. "Cnn-based density estimation and crowd counting: A survey." arXiv preprint arXiv:2003.12783 (2020).

2) The English language should be improved to ensure clearly understand your text. Some examples where the language could be improved include lines 80, 93, 157 (what is L denoting here ?), Mathematical notations should be well defined. (For example in 228, What is m and n)  – the current phrasing makes comprehension difficult.
Experimental design: These are major issues with the experimental setup.

1. Experiment design and experimental results
Both the proposed method and the benchmark methods are stochastic and the results from multiple independent runs are expected. What is currently reported in the paper is the results from a single run, which is not enough to draw concrete conclusions. Furthermore, multiple runs will be needed to conduct a statistical significance test.

2. Performance metrics
The typical accuracy is inappropriate to be used when the dataset is imbalanced (Shanghai Tech and UCSD) and multiclass (proposed dataset, Shanghai Tech and UCSD). We know for sure that the benchmark datasets in this study fall under that category. Then why the typical accuracy is still used to assess the effectiveness of the experimented methods.

3. Benchmark methods and fairness of the comparisons.
3.1- Benchmark methods
  I do not think any of the methods in the experiments was specifically proposed for crowd density classification. There are a number of studies that have proposed similar techniques to the proposed method [1,2], e.g., utilising CNN or other machine learning methods to classify/estimate crowd density. Why none of these was included in the comparisons despite some of such methods being discussed in the related work section.

[1] Gao, Guangshuai, et al. "Cnn-based density estimation and crowd counting: A survey." arXiv preprint arXiv:2003.12783 (2020).

3.2 - Fairness
(a) The proposed method utilises pre-trained models (transfer learning) whereas the benchmark methods are trained from scratch. I do not think this is a fair comparison unless the study is about transfer learning vs conventional learning.
(b) Overall dataset annotation looks fair except there is a chance of having a human bias in 5 classes. As it is very difficult to see the difference in the low and medium (3rd image) and the same is the case with medium and high. In my personal opinion, having three classes (low, medium and high) will be more appropriate than five classes to reduce human bias or error.
Validity of the findings: Experiments 1 and 2 have fundamental issues that need to be resolved before making any valid conclusions.

1. The main issue with stochastic methods is that different results are produced depending on the starting point of the search. In neural networks, the random value generator, more specifically the starting point of the random values generator, initialises the weights; hence, causing the network to start the process from a different point in the search space. Therefore, we must rerun the method multiple times using different seed values “while” keeping everything else untouched/identical.

2. How the other benchmark datasets where category wise evaluation  (Mentioned in Table 1) is not available were modified into a classification problem.

3  what are the hyperparameters for all the benchmarks and our proposed model?

4. what are the train and test sizes for other benchmark datasets.

5. Why are recall, f-score etc are not reported?

6.  All benchmark models come with pre-trained weights.  Which means they are different from each other.
For example, I can use a method that was trained to do anomaly detection and fine-tune it (re-train it) for crowd classification and then compare it to a method that was trained to perform cancer image segmentation in MRIs after re-train it for crowd classification. Can you see the difference between the base models?
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A DEEP CROWD DENSITY CLASSIFICATION MODEL FOR HAJJ PILGRIMAGE USING FULLY CONVOLUTIONAL NEURAL NETWORK
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This article needs important modifications to be suitable for this journal. I suggest major revision for
this paper. The main comments are:

1) In this study, the authors would like to propose a model of deep crowd density classification
for Hajj pilgrimage by using fully convolutional neural network. It can be noticed that the CNN
used in this manuscript has been studied and applied in the previous literatures.
2) The novelty of this paper should be further justified and to establish the contributions to the
new body of knowledge.
3) Abstract section should be improved considering the proposed structure from the journal.
4) In Introduction section, the authors should improve the research background, the review of
significant works in the specific study area, the knowledge gap, the problem statement, and
the novelty of the research.
5) The presentation of the results and conclusions were not enough; it should be highlighted.
6) In the conclusions section, the findings should be explained clearly.
7) The authors should elaborate more on the practical implications of their study, as well as the
limitations of the study, and further research opportunities.
8) The English writing does not influence, in all the paper. There are a lot of grammatical errors
which should be revised by the authors. So, the paper needs a professional English revision.
The author’s guide should be considered by the authors in the writing style in all the paper.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A DEEP CROWD DENSITY CLASSIFICATION MODEL FOR HAJJ PILGRIMAGE USING FULLY CONVOLUTIONAL NEURAL NETWORK
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This paper needs to consider some suggestions to improve it like: (not taken into account in the first revision)

1- In the conclusions section, the findings should be explained clearly.
2- Conclusions have significantly improved. The authors should elaborate more on the practical implications of their study, as well as the limitations of the study, and further research opportunities.

After the modifications, I suggest to Accept this paper


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HYBRID FEATURE SELECTION ALGORITHM AND ITS APPLICATION IN BIOINFORMATICS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Wang et al. developed a hybrid MMPSO method combining feature ranking and heuristic searching which yield higher classification of accuracy. To demonstrate the accuracy of such method, the authors applied the method to eleven datasets and identified 18 tumor specific genes, nine of which were further confirmed in a HCC TCGA dataset. Together, the study presents a highly effective MMPSO algorithm with great potential to identify novel biomarkers and therapeutic targets for cancer research and treatment.
Experimental design: The study is original and novel in the sense that it develops the hybrid algorithm by combining mRMR and MIC method. The methodology is very concrete as the authors compared the MMPSO method to eight other methods and clearly showed their superiority. Furthermore, the MMPSO was applied to the HCC biological dataset and gave a 18 gene signature which successfully separate 531 samples into cancer and normal groups. Many of the targets in the signature were validated by published research and/or their own Kaplan Meier method. Therefore, the study is well targeted at the Aims of Scope of Peer J Computer Science.
Validity of the findings: The findings in this study are thoroughly validated. The conclusions are well stated.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HYBRID FEATURE SELECTION ALGORITHM AND ITS APPLICATION IN BIOINFORMATICS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The authors aimed to develop an improved feature selection algorithm. They applied the algorithm to the public gene expression dataset and found some candidate biomarker genes for classifying tumor versus normal samples. This is an incremental work based on previous methods and has potentially interesting applications on tumor diagnosis. However, the following concerns should be addressed to meet the basic requirement of a bioinformatics method paper.
Experimental design: Major concern #1: the authors used the same dataset (LIHC) for selecting the tumor/normal gene signatures and testing their performance. This does not prove the superiority of the algorithm since it can simply overfit this dataset. I suggest 1) selecting features based on a subset of samples and validating on the left-out samples; 2) using another cohort of normal/tumor gene expression datasets for testing the selected features.

Major concern #2: the authors did not compare the gene signatures selected by their method versus other methods. For instance, using the top differential genes of a simple differential gene expression analysis with DESeq2 or EdgeR, do they separate the tumor samples better than these 18 genes? Given that in many of the ten datasets in Figure 1 and Table 2, mRMR performs very similarly to MMPSO, what is the performance on the LIHC dataset if only using mRMR to select features?
Validity of the findings: Major concern #3: In line 210, the authors listed the 18 gene signatures and “nine genes were significantly upregulated in tumors compared to normal samples” but did not clarify which genes were differentially upregulated or downregulated and what were the criteria. The authors also did not provide method detail of how they determined the high and low expression levels for sample stratification in Figure 4. For instance, I checked the average values of some of the genes in the two raw data files (data_tumor.csv, data_normal.csv) provided by the authors:

ACTN1, ENSG00000072110	Tumor = 12.78586	Normal = 12.33047
CACHD1, ENSG00000158966	Tumor = 7.624089	Normal = 7.308306

In lines 292-296, the authors discussed the literature that a higher level of ACTN1 is associated with HCC size, and in Figure 4, CACHD1 was described as an ungulated gene associated with worse prognosis, however, I do not consider a difference of < 0.5 as significantly different in gene expression. At the very least, the authors should provide a box-whisker plot with proper statistics of the expression of these 18 genes across the normal/tumor samples they used to demonstrate their points.
Additional comments: Minor comments. Please read carefully for typos. Line 18: “a optimal subset”. Line 70: “Yuanyuan et al. 2021”. In line 17, MMPSO should not be abbreviated because this is its first appearance in the whole manuscript.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A HYBRID FEATURE SELECTION ALGORITHM AND ITS APPLICATION IN BIOINFORMATICS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors have properly addressed the questions I raised.
Experimental design: The authors have properly addressed the questions I raised.
Validity of the findings: The authors have properly addressed the questions I raised.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors please make any changes to improve the readability of the paper.
The reference is insufficient. The authors need to add popular related articles to support the proposed arguments.
The authors need to add more detailed contents to enrich the proofing process.
Experimental design: No comments.
Validity of the findings: The paper proposed a conceptual approach. If possible, the authors please verify the applicability of the proposed algorithm.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 1
Reviewer: 3

Output:
Basic reporting: In this article, the author proposes a blockchain-based multi-keyword verifiable symmetric searchable encryption scheme. Compared with previous work, it improves the search efficiency, ensures the fairness of the search, and can perform multi-key Word search result verification. Finally, the author uses experiments to prove that this method is safe and effective in practical applications. Here are some comments for the authors to improve the quality of this manuscript.
Experimental design: The experimental design should be more detailed, and the pictures will be more convincing
Validity of the findings: This is the same problem, adding your pictures will make your findings more convincing
Additional comments: 1.	I don’t know if it’s due to the system or other reasons. I didn’t see your picture in the article.
2.	In the introduction, I think you can write more compactly. For example, line 45, the shortcomings of SSE are introduced before, and dynamic updates are also very important afterwards.
3.	It is recommended to use subtitles to make your article clearer.
4.	Please use correct template of this journal.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 2
Reviewer: 1

Output:
Basic reporting: The revised draft made progress.
Experimental design: It is acceptable.
Validity of the findings: It is acceptable.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 2
Reviewer: 2

Output:
Basic reporting: The authors modify the manuscript according to the comments. It improves a lot compared to the previous  version. However, there are still some details to be corrected to make this manusript better. Would the authors use the latex template if possible since the typewritting is not so neat, especially for the equations.
Experimental design: The proof of the scheme is sound. The simulation results can be improved if possible.
Validity of the findings: The novelty is fine. Are there any data in the simulation? Please list them or clain the origin if possible.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 3
Reviewer: 1

Output:
Basic reporting: The revised draft made progress.
Experimental design: It is acceptable.
Validity of the findings: It is acceptable.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS EFFICIENT VERIFIABLE MULTI-KEYWORD SEARCH OVER ENCRYPTED DATA BASED ON BLOCKCHAIN
Review round: 3
Reviewer: 2

Output:
Basic reporting: I recommend to accept it in the current version.
Experimental design: The experiments in this paper are fine.
Validity of the findings: The findings are sound.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DMPNET: DENSELY CONNECTED MULTI-SCALE PYRAMID NETWORKS FOR CROWD COUNTING
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper, at a high level, presents a way to cope with the scale variation problem in Crowd Counting research. The authors clearly write about their motivation for coming up with the proposals and also compare the counting performance with state-of-the-art methods on 4 major datasets. The main contributions of the paper are the proposed Multi-scale Pyramid Networks (MPN), which are densely connected for better information retention throughout the deep networks. I shall comment on the basic components of the paper that could be improved as follows:

1. I believe that the very first paper that uses dense connections is “Dense Scale Network for Crowd Counting” (DSNet in Table 2) [1]. Thus, (Dai et. al., 2021) should be mentioned when introducing the dense connections.

2. The published year of DSNet is not consistent. (Dai et. al., 2019) or (Dai et. al., 2021)?
In crowd counting research, to “keep the input and output resolutions unchanged”, the encoder-decoder structure (See [3] and [5])  is usually utilized. The introduction to the structure is worth adding for better context.

3. In the RELATED WORK Section, to better support the difference between this work and others, there should be examples of previous papers that add “extra perspective maps or attention maps”, for instance [2],[3], and [4]. Specifically [3] also adopts multiple branches of convolution with different kernel sizes, namely the ASPP structure.

4. In Figure 4., please carefully check that #groups = 2 or 4? Based on Figure 4., the incoming feature channels are divided into 2 groups?

5. About describing the configurations (kernel_size, #output_channel, ...) of a convolution operation, It is preferable to use \times over the “*” notation. The  “*” notation may be reserved for the convolution operation itself.

6. In Figure 5, Is there any particular reason, the larger kernel_size is, the smaller output channels and the groups are? Is this just to balance computational resources between branches?

7. (Typo) Change “MFFE” to “MFFN”?

8. (Typo) What is G (with \theta parameterization) in the density level consistency loss? Should G be F?

9. Is C in the calculation of MAE loss, the summation over all pixels? The definition of C can be made more clear. Do I and D^{GT} notations have the same meaning? As I_{i} cannot appear twice, please revise the MAE loss formulation.

10. \alpha and \beta for weighting the losses are chosen based on what criterion? Experiences? Random? or validation count performance?

References
[1] Dai, Feng, et al. "Dense scale network for crowd counting." Proceedings of the 2021 International Conference on Multimedia Retrieval. 2021.
[2] Zhu, Liang, et al. "Dual path multi-scale fusion networks with attention for crowd counting." arXiv preprint arXiv:1902.01115 (2019).
[3] Thanasutives, Pongpisit, et al. "Encoder-Decoder Based Convolutional Neural Networks with Multi-Scale-Aware Modules for Crowd Counting." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.
[4] Shi, Miaojing, et al. "Revisiting perspective information for efficient crowd counting." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[5] Jiang, Xiaolong, et al. "Crowd counting and density estimation by trellis encoder-decoder networks." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[6] Chen, Liang-Chieh, et al. "Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs." IEEE transactions on pattern analysis and machine intelligence 40.4 (2017): 834-848.
Experimental design: 1. On defining the evaluation metrics, you can use D^{GT} instead of Z`. The notations need consistency.
There are no “bolded” numbers, indicating the best results in any Table.

2. Current SOTA methods are incomplete, [2], [3], [4], and [5] are missing.

3. In Table 3., it is not clear whether (w/ LPN, w/o GPN) and (w/ GPN, w/o LPN) are trained with MFFN or not? In the 3rd row, MPN should be MFFN? Another typo?

4. From Table 4., it is still unclear how many blocks of MPN blocks should be densely connected? In this paper, It is 3, but how about 1, 2, or even 4? This may be addressed in the ablation study section.

5. This paper recommends the use of the group convolution (over the normal one) but does not compare the number of trainable parameters of the proposed Multi-scale Pyramid Network, when G varies and the case that G = 1 for all the branches.

References
[1] Dai, Feng, et al. "Dense scale network for crowd counting." Proceedings of the 2021 International Conference on Multimedia Retrieval. 2021.
[2] Zhu, Liang, et al. "Dual path multi-scale fusion networks with attention for crowd counting." arXiv preprint arXiv:1902.01115 (2019).
[3] Thanasutives, Pongpisit, et al. "Encoder-Decoder Based Convolutional Neural Networks with Multi-Scale-Aware Modules for Crowd Counting." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.
[4] Shi, Miaojing, et al. "Revisiting perspective information for efficient crowd counting." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[5] Jiang, Xiaolong, et al. "Crowd counting and density estimation by trellis encoder-decoder networks." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[6] Chen, Liang-Chieh, et al. "Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs." IEEE transactions on pattern analysis and machine intelligence 40.4 (2017): 834-848.
Validity of the findings: 1. (Important) Since this paper would be perceived (if it gets published) as an incrementally improved version of DSNet, some audiences might get confused by the fact that the DMPNet only outperforms DSNet in terms of lowering MSE computed from the ShanghaiTech Part_B dataset. Hence, I advise that you can also compare the number of trainable parameters or inference speed for better justification of your proposed DMPNet. I guess that DMPNet might contain fewer parameters as you employed the group convolution operation.

2. As I mentioned above, it is hard to justify that DMPNet has “better in both mean absolute error (MAE) and mean squared error (MSE)” at the current state of the paper. So, there may be doubts whether “MPN can effectively extract multi-scale features” or not?

3. On lines 290-291, the claim that “Our DMPNet has well solved the problems of crowd occlusion, perspective distortion, and scale variations” is a bit exaggerated. It is unclear how exactly you deal with the perspective distortion? Is this solely based on your experimental results of the UCF_QNRF dataset?
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DMPNET: DENSELY CONNECTED MULTI-SCALE PYRAMID NETWORKS FOR CROWD COUNTING
Review round: 1
Reviewer: 2

Output:
Basic reporting: The authors proposed a Densely connected Multi-scale Pyramid Network (DMPNet) based on VGG and Multi-scale Pyramid Network (MPN). The latter consists of three modules that aim at extracting features at different scales. The authors compared performances obtained by DMPNet with other methods at state of the art.

1. Figures and Table should be placed before the bibliography
2 Related work section should include all methods used for comparison.
3. Introduce every acronym before using it (e.g., CNN, ASNet, etc.)
4. Viresh etal. (line 97-98) and CP-CNN method (Table 2) are not present in the bibliography
5. Figures 3 and 4 have the same caption.
6. Captions should be improved
7. In table 2, no results are highlighted in bold
8. Figure 1 presents "Table 1" in the caption
9. In the manuscript and Figure 5, the symbol "*" was used. It is necessary to replace it with "x" (in Latex, \times).
10. lines 35-36, 53-54, 103-106, 107-108, 195-196  need references
Experimental design: The method is described with sufficient details. Moreover, the authors provide the code.

1. The training methods section can be integrated into Experimental and Discussion
2. The measure MSE (Section "Evaluation Metrics") is the "Root Mean Squared Error" (RMSE)
3. lines 267-271, 280-282 can be improved
Validity of the findings: 1. The ablation experiment section can be improved by studying these effects on the other considered data sets.
Additional comments: 1. There are some typos (e.g., SOAT --> SOTA)


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DMPNET: DENSELY CONNECTED MULTI-SCALE PYRAMID NETWORKS FOR CROWD COUNTING
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: INTERPRETABLE DEEP LEARNING FOR THE PREDICTION OF ICU ADMISSION LIKELIHOOD AND MORTALITY OF COVID-19 PATIENTS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors should decrease the number of tables as possible. Some tables could be merged or moved to supplement files, such as hyperparameters tuning. They should keep the most important parts for readers.
Experimental design: The research question is "interpretable". The authors only pick the important features from the model but not to the point "interpretable". They can explain the definition of "interpretable".
Validity of the findings: Nil.
Additional comments: The analytic process could be concise and to the point.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: INTERPRETABLE DEEP LEARNING FOR THE PREDICTION OF ICU ADMISSION LIKELIHOOD AND MORTALITY OF COVID-19 PATIENTS
Review round: 1
Reviewer: 2

Output:
Basic reporting: * The report is too detailed and includes trivial information which renders the reading experience to be exhausting. In scientific papers many different methods are used these days and it is necessary to think about what information should be included, what should be cited and what should be left out (as trivial information). This is a process which the current report can greatly benefit from. More focus needs to be directed towards the main findings of the papers and the methods that ended up being used in the most optimal model that was eventually applied.

* The figures and the tables lack detailed descriptions. Furthermore they are not completely labelled ans for instance the x axis and y axis labels are missing in the feature importance plots (i.e. figure 2 and 4).

* information provided in many tables are redundant, as for instance in many tables such as table 6 the "Without symptoms" percentages are the complements of the "With symptoms". Better and more brief means of describing the 'Description of data sets' and the 'statistical analysis' must be used that listing all these information in a sequence of tables. If some of these information are not relevant or significant they can be referred to in supplement documents rather than in the main paper.

* The following sentences require rewriting :

- Line 145: "ADASYN (He et al., 2008) is a synthetic data generation algorithm that has the benefits of not copying
the same minority data and producing more data which are harder to learn."

- In equation 2: Xk is not defined . What is Xk ?

- Line 153: "… same proportion of observations, and also due to the dataset 154 being imbalanced. The fold used was k=5 which means …"

-Top of page 4: "These operations are done sequentially in the order as shown in Equations 4, and 5:
FC =W(x) (10) …"
Do the equations need to be repeated ?! Please clarify how do they relate to formula 4 and 5.

In tables 5 and 8: Both columns are labelled as "no-ICU" !

Line 273: "Overall, over 50% of the individuals acquired a symptom of a disease before they died."
This does not seem correct! How far over 50% ? and by "a symptom" do the authors mean "at least one symptom" ? This sentence suggests that a bit less than 50% of patients died without showing any symptoms ?!

Line 392: "The key performance metric in this analysis is the AUC score
and it can be plotted using the precision-recall curve which demonstrates a trade-off between the recall score (True Positive Rate), and the precision score (Positive Predictive Value)."
AUC is usually referred to the area under ROC curve. Do you mean the area under Precision Recall curve here ? If yes, you can use AUCPR
 to avoid confusion.
Experimental design: * I was not convinced and completely informed about how the authors avoided overfitting to the data throughout the whole study. They mention that they consider a 5 fold cross-validation using stratified K-fold :
"The fold used was k=5 which means that the dataset was divided into 5 folds with each fold being utilized once as a testing set, with the remaining k - 1 folds becoming the training set."

However, further down in the report (line 299) they mention : "The ENTIRE dataset is split into training and validation in a 90%-10% ratio respectively, to enable a good number of data points to be trained to get better results. "
If the entire data is divided to train and validation, how is the 5 fold cross-validation implemented ?

* The pvalues and FDRs of the correlations are missing throughout the report. Furthermore, the correlation (distance) method hat was used should be clarified; was it "Pearson" ?
Validity of the findings: * My biggest issue with the study is that as the authors have also mentioned the data is small for the kind of prediction model that the authors are aiming for. Furthermore the data is imbalanced, whilst computational methods have been used to overcome the effects that the imbalance of data may have on the training and testing of the model. All these, together with the fact that only one data was used for the each prediction model (one for ICU prediction of COVID-19 and another for the mortality) makes any findings related to the most effective prediction factors to be imprecise and unreliable if they are solely dependent on the computational methods used by the authors. At least these findings should be directly supported by other litterateurs and studies. As for instance is the direct association of Ferritin with ICU visits or mortality of COVID-19 patients (rather than the indirect association through COPD that is mentioned by the authors) been discovered by other studies as well ?
Additional comments: My criticism regarding the study aside the inclusion and possibly publication of all the codes used for analyzing the data and producing the figures by the authors is an excellent practice that should become more common in the scientific community.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: INTERPRETABLE DEEP LEARNING FOR THE PREDICTION OF ICU ADMISSION LIKELIHOOD AND MORTALITY OF COVID-19 PATIENTS
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors answered the questions


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: INTERPRETABLE DEEP LEARNING FOR THE PREDICTION OF ICU ADMISSION LIKELIHOOD AND MORTALITY OF COVID-19 PATIENTS
Review round: 2
Reviewer: 2

Output:
Basic reporting: * This is a suggestion (and it is OPTIONAL!): Figures have titles only and lack descriptions! legends are described in the PeerJ “Instruction for Authors” to be optional, however I think that it would improve the readability of a paper. Legends and table/figure descriptions allow the person who reads it to fully understand the plot/illustration without needing to read the text. As for instance in figure 2 you can mention what the heat map mean (e.g. shows whether a feature is selected at a given decision step in the model), what the x and y are; and what the brighter colours stand for and etc.
Experimental design: * It would be interesting if the authors can elaborate that how can factors such as COPD and Myalgia mortality be amongst 3 highest predictors of mortality but not amongst the highest factors for ICU admission. Are they seen in the later and more advanced stages of Covid?

* I understand that Neural Networks does not necessarily model in the same manner as linear regression and etc, however it would still be inteersting if the authors could elaborate how the Shortness of Breath (SOB) feature which has the highest correlation with admission to the ICU unit  is not amongst its highest 3 predictors !
Validity of the findings: * Something that is still not fully clear to me is that how has the stratified k-fold split of 5 been implemented in the training and testing of the models, using the most optimal model for ICU admission as for instance. The applied cross validation (K-fold split of 5) is still with 5 replications; therefore, I assume that 5 sets of training and testing of each prediction model been executed? An ROC curve can be plotted for each prediction on each test subset, so this means that actually for each prediction model 5 PRC/ROC curves are resulted and 5 AUPRC/AUCs are measured? The authors however describe only one PRC/ROC curve and AUPRC/AUC. Are these the PRC/ROC curves of the most optimal run (with the highest AUPRC/AUC)? If that is true, it should be written clearly and optionally the authors may want to mention the variance of  the AUPRC/AUC across the 5 cross validation runs, or report the mean AUPRC/AUC  as well.
Additional comments: * Correct m_l and m_s in the line beneath equation 1 , beneath line 153 (some lines don’t have numbers !).
* Line 167: “TabNet deep learning model 15 is”. Is “15” a typo ?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: INTERPRETABLE DEEP LEARNING FOR THE PREDICTION OF ICU ADMISSION LIKELIHOOD AND MORTALITY OF COVID-19 PATIENTS
Review round: 3
Reviewer: 1

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TEMPORAL DYNAMICS OF REQUIREMENTS ENGINEERING FROM MOBILE APP REVIEWS
Review round: 1
Reviewer: 1

Output:
Basic reporting: Line 73: reviews. First, we collect, pre-process and extract software requirements from large review datasets. The authors might need to the name of the apps and explain the process in details in section 3.

Line 74: Then, the software requirements associated with negative reviews are organized into groups according to their content similarity. The authors might need to explain how

Figure 1. : It is very high level. Lack of analytical details need to show the steps in more explicit way.

Line 201: The reviews are from 8 apps of different categories. What are the name of these apps and their categories?

Line 318-9: It would be better to move it up. After you introduce the model.

Line 321: Does it mean that you consider the negative review as the change-point?. This part is lack of a clear explanation. Moreover, it needs more supporting example (e.g., why did the author consider the negative review as the change-point)
Experimental design: Line 76 and 217: software requirement from negative reviews: The authors need to explain how they classify the negative reviews in more details (not only by considering the low rate as negative), as well as The authors, might need to read the reviews to check if it can be considered as negative or positive reviews, even if the rate is low/high (1/5). Example, the reviewers might rate the app 3 out of 5 and it might be considered as negative review?

Line 219: Thus, we use RE-BERT to extract …..…... It is not clear how this step is preformed? How did the authors classify them into complaints, bad usage experience, or malfunction

Line 193: Does the training data contain negative review? How would the model be able to predict them ? The authors need to explain these points.

Line 233: It is not clear how BERT would help your model? why do you need it at the first place?

Line 234: It is not clear how this objective is related to the proposed model.
Validity of the findings: In general: Lack of a comprehensive experiment. The model should be evaluated with some of the existing solutions to see how its performance.

Line 355: we used a dataset with 86,610 reviews of: Did they collect the datasets or is it publicly available? if it is publicly available where is the reference and why did they choose it? What is the dimensionality of the dataset? What is the name of these apps.

Line 358: clustering stage (with k = 300 clusters) why k=300
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TEMPORAL DYNAMICS OF REQUIREMENTS ENGINEERING FROM MOBILE APP REVIEWS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper does not follow the standard sections for the journal. Furthermore, Acknowledgments acknowledge funders as additional contradiction to the guidelines. The authors do not share any raw data on their evaluation. Neither the app review nor the results are shared. Therefore, none of their results can be judged.
Experimental design: no comment
Validity of the findings: The data used for the evaluation was not made available. Therefore the validity cannot be judged. Explaining the data set as well as sharing it online would improve the paper a lot. Furthermore, the authors could also publish the results on the data when having applied the MAPP method.
Additional comments: The paper covers an interesting and promising topic. It is written in a clear and well understandable language. The two mentioned problems can easily be addressed with a revision of the paper.
As minor addition I would also suggest to extend the motivation with references that support the goal of continuous evaluation of user feedback side by side to the development process like T Palomba et al. [1] and Hassan et al. [2] as well as Scherr [3] et al. do.
1.	Palomba, F., Linares-Vásquez, M., Bavota, G., Oliveto, R., Di Penta, , Poshyvanyk, D., De Lucia, A.: Crowdsourcing User Reviews to Support the Evolution of Mobile Apps. Journal of Systems and Software(March 2018), 143-162 DOI:10.1016/j.jss.2017.11.043 (137)
2.	Hassan, S., Tantithamthavorn, C., Bezemer, C.-P., Hassan, A.: Studying the dialogue between users and developers of free apps in the Google Play Store. Empirical Software Engineering 23(3), 1275–1312 https://doi.org/10.1007/s10664-017-9538-9 (2018)
3.	Scherr, S., Hupp, S., Elberzhager, F.: Establishing Continuous App Improvement by Considering Heterogenous Data Sources. International Journal of Interactive Mobile Technologies (iJIM) 15(10), 66-86 (2021)


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TEMPORAL DYNAMICS OF REQUIREMENTS ENGINEERING FROM MOBILE APP REVIEWS
Review round: 1
Reviewer: 3

Output:
Basic reporting: - Missing the threats to validity and/or limitation section.
- Missing the discussions and/or implications section.
- It still isn’t clear to me how this can be used concretely as this is not detecting emerging issues, but rather predicting the future frequency of a requirement in negative reviews. Could you elaborate more on why this prediction is needed as the requirements from negative reviews should have already impacted the app's rating? Additionally, as a developer looking at the results, when would I say that the problem/requirement is becoming major? (e.g., the slope of change/the change in the frequency is greater than some particular threshold?). These could be addressed in the discussions and-or implications section.
Experimental design: - Some methodological decisions need more details (e.g., I’m not quite sure how you decide the starting number of k? does selecting a different starting number k affect the result in any way? what's the rationale behind selecting six clusters in your experiment? Can a requirement belong to more than one cluster?)
- Were reviewed cleaned (e.g., based on length?).
- Your experiment should also include using app update dates as changepoints because prior research suggested that naturally, you will see a spike of user reviews shortly after app updates.
Validity of the findings: - For a journal paper, a further extensive evaluation is needed. I would argue that reviews from food ordering apps did not contain rich data regarding the problems/requirements with the app itself but contain a great number of information/complaints on the services.
- How applicable can this be used for apps in other categories? Your experimentation is done only on one type of apps and only popular ones (please discuss this in the threats to validity and/or limitation section).
- The experimental evaluation is quite lacking compared to other sections. You could revisit the research question you introduced in the introduction.
- Sensitivity analysis may be needed.
Additional comments: - Interesting research, relevant to the journal.
- It is well written, structured, and presented.
- Extensive literature review.
- The authors provided a code to examine.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TEMPORAL DYNAMICS OF REQUIREMENTS ENGINEERING FROM MOBILE APP REVIEWS
Review round: 2
Reviewer: 1

Output:
Basic reporting: All good
Experimental design: All good
Validity of the findings: All good
Additional comments: All good


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TEMPORAL DYNAMICS OF REQUIREMENTS ENGINEERING FROM MOBILE APP REVIEWS
Review round: 2
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: The findings are explained in a more solid way due to the introduction of a discussion section. Nevertheless half of this rather short section is spent on the limitations and not on discussion the findings. I would like to see a deeper discussions of the interesting findings.
Additional comments: The paper covers an interesting and promising topic. It is written in a clear and well understandable
language. The authors provided a significant improvement over the first version.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACO-KINEMATIC: A HYBRID FIRST OFF THE STARTING BLOCK
Review round: 1
Reviewer: 1

Output:
Basic reporting: 1. When several titles are cited in the text it is better to be in increasing order (lines 33,53, 103, 114)
2. Use same format, when several titles are cited : [a] [b] [c] or [a,b,c] (line 103)
3. line 134. Section 4 is too short. the section 3 can become "problem statement and objectives"
4. line 140. After mentioning Dorigo include citation of his work.
5. line 141. The first who proposed variant of ACO for continues optimisation is Patrick Siarry (Continuous interacting ant colony algorithm based on dense heterarchy,
J Dréo, P Siarry, Future Generation Computer Systems 20 (5), 841-856, 2004), cite him together with [32]
6. line 169. Remove section 6. Kinematic equations are discussed in section 9.
7. line 207. equation (13). This equation do not show the path length and minimisation of this function do not give the shorter path. The objective function must be the path length with end point the target. The collision avoiding can be constrain. Safety avoiding of obstacles can be solved defining some safety distance from obstacles.
Experimental design: The paper needs new experiments after rewriting the objectives and constraints.
Validity of the findings: The findings will be valid after recalculating with new objective function.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACO-KINEMATIC: A HYBRID FIRST OFF THE STARTING BLOCK
Review round: 1
Reviewer: 2

Output:
Basic reporting: No Comment
Experimental design: No Comment
Validity of the findings: No Comment
Additional comments: This paper presents a combined technique based on ant colony optimization (ACO) algorithm and kinematic equations, named ACO-Kinematic, to solve the robot navigation problem in static environments. In this method, ACO is used to find a collision-free route to the next step, while kinematic equations are used to control and move the robot to the new selected step. The paper can be considered for publication in PeerJ Computer Science, if the following minor and major comments would be carefully addressed:

1) The main limitation of this study is to utilize the ACO-Kinematic algorithm for path planning of the mobile robot in static environments, i.e., considering only static obstacles with known location. Although the authors correctly addressed it as a limitation of their work in Conclusion, there is still a major issue: What about the static obstacles with unknown locations? Is the robot does not aware of the location of obstacles (even static), it cannot run ACO prior to find the full path for the mobile robot.

2) In the original ACO, pheromone and heuristic information (as available) are used to calculate the probability of the different choices (i.e., next steps in your study) using the ACO selection rule. Please provide more details about how next steps are selected via ACO? Why you did not consider heuristic information in your model? For example, the distance to the obstacles and angle to the target can be used as very informative heuristic information not only to speed-up the algorithm, but also to improve the solution quality. There is a fuzzy heuristic based ACO (combining fuzzy heuristic information and pheromone): “FH-ACO: Fuzzy heuristic-based ant colony optimization for joint virtual network function placement and routing”, recently published in  Applied Soft Computing, 107, 107401. It utilized multi-criteria fuzzy heuristic as the heuristic information to guide ACO in finding better solutions, and utilizes a multi-criteria fuzzy heuristic model as well as pheromone to construct the full path. You should mention this paper in Introduction or Literature Review, and discuss why you did not consider heuristic information in ACO selection rule?

3) How uncertainties are handled in your model? You should have a plan to handle uncertainties of the environment, or even discuss about it as a limitation of your work in Conclusion. Even for static obstacles, what is your plan if there are uncertainties in the location of the obstacles?

4) Please provide a time complexity analysis for the different phases of the ACO-Kinematic.

5) The results of the ACO-Kinematic should be compared and justified with recently published heuristic- or metaheuristic-based path planning techniques.

6) Finally, the paper should be carefully double-checked to be free of errors.

This manuscript can be reconsidered for the publication in PeerJ Computer Science, if the above issues would be carefully addressed.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACO-KINEMATIC: A HYBRID FIRST OFF THE STARTING BLOCK
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors corrected the paper according reviewers comments
Experimental design: The authors corrected the paper according reviewers comments.
Validity of the findings: The authors corrected the paper according reviewers comments.
Additional comments: The paper is corrected according reviewers comments.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACO-KINEMATIC: A HYBRID FIRST OFF THE STARTING BLOCK
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment
Experimental design: No comment
Validity of the findings: No comment
Additional comments: The revised version has been efficiently improved, and all of my comments have been corrected. The current version of the manuscript can be accepted for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: Given the complexity involved, the author has produced a number of positive and welcome outcomes including the literature review which offers a useful overview of current research and policy and the resulting bibliography which provides a very useful resource for current practitioners.

Most relevant papers (only relevant and state-of-the-art) should be qualitatively and quantitatively analyzed with what gaps were left in these works and what this work is proposed to overcome those gaps/challenges.

Experimental should be rigorously analyzed both theoretically and visually with proper justification of obtained results as well as with potential comparative studies

can add future work also


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 1
Reviewer: 2

Output:
Basic reporting: This study provides the implementation of various machine learning models to measure
the polarity of the sentiment presented in user reviews on IMDB website. For this purpose, the reviews are first preprocessed to remove redundant information and noise and then various classification models like support vector machines (SVM), Naive Bayes classifier, random forest and gradient boosting classifier are used to predict the sentiment of these reviews. The objective is to find the optimal process and approach to attain the highest accuracy with best generalization. Various feature engineering approaches such as term frequency-inverse document frequency (TF-IDF), bag of words, global vectors for word
representations and Word2Vec are applied along with the hyperparameter tuning of the classification models to enhance the classification accuracy
Experimental design: Satisfactory
Validity of the findings: Satisfactory
Additional comments: 1. The authors have to summarize the findings/gaps from recent literature in the form of a table.
2. Some of the recent works on NLP and feature extraction such as the following can be discussed: "An ensemble machine learning approach through effective feature extraction to classify fake news, Analysis of dimensionality reduction techniques on big data".
3. In the proposed methodology section, the authors should clearly map the proposed work with the limitations of the existing works and discuss how the proposed work overcomes them.
4. Present a detailed analysis on the results obtained.
5. Discuss about the limitations of the current work.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 1
Reviewer: 3

Output:
Basic reporting: Literature review is very superficial and the last paper that is reviewed has been published in the year 2019.
Literature review is not guided, and authors have made very generic assumptions.
Experimental design: Authors are advised to try the latest LSTM models that are more suitable for this type of problem.
Validity of the findings: There isn’t any significant contribution in terms of methodology or results.
Authors need to justify the results, why a particular model should yield better results.
Additional comments: The paper is basically a comparative analysis of various machine learning algorithms on the published dataset and has no real scientific contribution.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 1
Reviewer: 4

Output:
Basic reporting: No comment.
Experimental design: 1.There is no novelty in this research. The algorithms used (for preprocessing and classification) are common
algorithms that widely used for sentiment analysis.
2. The third contribution mentions the use of deep learning in this work which is not correct since the results only show four traditional ML algorithms.
3. Authors should specify the type of decision tree used in this paper and also other parameters such as maximum depth of the tree or number of features to consider when looking for the best split.
4. Authors should also specify the hyperparameters used and the kernels used in this paper.
Validity of the findings: 1. The results in table 15 show comparable results in previous work.  Hence, the impact and novelty of this work are not there. The author should indicate the strength of this work.
2. The use of "optimized machine learning algorithms" term in the title does not reflect the real research outcome. The authors should at least describe the optimization process involved in this work.
3. In terms of the evaluation validity, how the authors ensure the similar evaluation method were used by all the papers in Table 15?
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors have addressed all my comments. The manuscript may be accepted for publication.
Experimental design: Satisfactory
Validity of the findings: No comments
Additional comments: The authors have addressed all my comments. The manuscript may be accepted for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 2
Reviewer: 2

Output:
Basic reporting: The paper presents a study to find the optimal process and approach to measure the polarity of the sentiments in user reviews on the IMDB website.

In section "Related Work" authors present a wide research in the context of the gap of knowledge that they want to fill. However, two of the works in Table 1 are not reviewed in the text above: Giatgoglou et al (2017) and Mathapati et al (2018).

I thank you for providing the code, however your code file need more comments to be useful to future readers.  For example, the introduction doesn't match with the code (Multi-layer Perceptron is not used, according to the paper). Besides, there aren't any comments about the kind of features engineering method used in each point.

The way of citing the bibliographic references is not correct the most of the times. The cites must be between parentheses if order to make the text more readable. For example, in line 48, instead of "...as a significant research are during the last few years Hearst (2003)" should be  "...as a significant research are during the last few years (Hearst, 2003)" The kind of cite, using the name of the author outside of the parentheses, should only be used if the name of author is part of the sentence. For example: Singh et al (2013) conduct experimental work about...

Some minor errors are:
a) Bag of Words is abbreviate as BoW, but sometimes it is written as Bow. Authors should revise the text to homogenize this acronym and others (RF)
b) Some blank spaces are needed in lines 196 (between algorithms and Oghina), 201 (between consistency and Lee)
c) Equations must be referenced with their number between parentheses instead of saying "the following". For example, in lines 219 and 220, TF-IDF is calculated using TF and IDF as (3)
d) The sample reviews of the Table 5, 6, 7 and 8 contain point at the end of the sentences, when these were removed in Table 4
e) In Table 6, the first sample review after case lowering yet contains a "P"
f) In line 325, it is indicated that Tables 11 and 13 show BoW and TF-IDF features, but the tables are 9 and 10.
Experimental design: In line 181, bibliography reference to the data set is missing

A bibliography reference to TextBlow is missing. It is a Python Library for processing textual data that provides a simple API for diving into common natural language processing (NLP) tasks such as sentiment analysis between others. It is not a lexicon-based technique.

TextBlob is used to annotate the dataset with sentiments (negative or positive) in order to tackle with "the contradiction in the user sentiments in the reviews and assigned labels" How do the authors know that classification of the reviews is contradictory and TextBlob result is better? Authors must explain this point.

In line 197, what does authenticity of a learning algorithm mean? or it is a mistake and it should be "accuracy".

In subsection "Feature Engineering Methods" only Bag of Words and TF-IDF are explained. Authors should also explain and reference the other two methods used in their study (Global Vector and Word2Vec)

Authors should revise the equation (1) and its explanation in line 217. In it TFij appears, but the explanation talk about term "t" and document "d"

In subsection "Supervised machine learning Models", author explain Naive Bayes, but in the Introduction and in the results, they used Gradient Boosting Classifier. Since gradient boosting classifiers are a group of machine learning algorithms that combine many weak learning models together to create a strong predictive model, which weak learning models does the gradient boosting classified used combine?

The title of the paper contains "optimized machine learning algorithms" and Table 3 shows the hyperparameters used for optimizing the performance of models. Author should explain how and why they have chosen these parameters to ensure the use of optimized machine learning algorithms.

The BoW features showed in Table 9 don't match with those in the preprocessed text of sample reviews in Table 8

TF-IDF values in Table 10 are miscalculated. If a word have a a frequency of 0 in a document, its TF-IDF can't be upper 0

In Figure 3, rows of the matrix represent actual labels and columns represent predicted labels. However, in lines 337 and 338 the opposite appears
Validity of the findings: The results are not very conclusive and the analysis of them it is merely a reading of the table with no discussion or a deep explanation.

There are not statics test to see if the differences are significant
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 3
Reviewer: 1

Output:
Basic reporting: The authors have addressed all my comments.
Experimental design: Comment 10: A bibliography reference to TextBlow is missing. It is a Python Library for processing textual data that provides a simple API for diving into common natural language processing (NLP) tasks such as sentiment analysis between others. It is not a lexicon-based technique.

The bibliography reference to TextBlow must indicate that it is a online document, the url and the last query date.

Author Response comment 11: Reviewer’s is concern is genuine. For clarification we added Table 3 in the revised manuscript
which contains the ’original’ label from the dataset and label from the Textblob. Original label is ’Negative’ while Textblob assigned label is ’positive’. Textblob assigned labels are more reliable as is confirmed by the higher classification accuracy from the models when used with Textblob labeled dataset.

It make no sense to rely on the classification results to indicate that the labels in the original dataset are contradictory. I'm sorry but this point is not clear for me. Why do we have to give more confidence to a machine than to a human? I'll try to set this clear. From what I read, I understand that you are using the predictions from another tool (Textblob) to train and evaluate your proposal. In this way, you can say that your proposal is similar to a model created by Textblob but it may be a poor model to the real data. In fact, in this way Textblob model could not be improved because that model is the goal. So if that model is supposed to be perfect, what's the point of the research (we already have the model from Textblob). So the main question is: Why do you say the labels from Textblob are better than the original data? I am sorry to say that Table 3 is not clarifying because the text seems to have been processed and can not be interpreted. How do we know if it is positive or negative?
Validity of the findings: In the new discussion of results, the paper says:
 "Performance of machine learning models(*1*) is improved with Textblob data annotation(*2*)"

In the response to comment 11:
 *2* "Textblob assigned labels(*2*) are more reliable as is confirmed by the higher classification accuracy(*1*)"

This is a case of false proof. If we want to prove *1*, we need to prove *2*. If we want to prove *2*, we need to prove *1*. We enter in a recursive non-sense proof.

Regarding this, the conclusions talk about contradictions found between sentiments and assigned labels but this is not explained in the paper. Is the data set wrongly labeled? You seem to mean that Textblob is detecting this mislabeled instances but how has this been checked? Has anybody actually read the reviews and confirmed that they was mislabeled? If not, you can not say so because it will just create confusion, in a similar way as fake news do.

Besides, more care should be taken when writing. If the data set is about movie reviews, why mentioning "tweets" in the discussion.

The statistical test applied is not clearly explained. What differences has been tested? T-test is usually applied to see if the means are different. What two means have been compared?
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CLASSIFICATION OF MOVIE REVIEWS USING TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY AND OPTIMIZED MACHINE LEARNING ALGORITHMS
Review round: 4
Reviewer: 1

Output:
Basic reporting: The authors have addressed  my comments.
Experimental design: The authors have addressed  my comments.
Validity of the findings: The authors have addressed all my comments.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: JOINT USER GROUPING AND POWER CONTROL USING WHALE OPTIMIZATION ALGORITHM FOR NOMA UPLINK SYSTEMS
Review round: 1
Reviewer: 1

Output:
Basic reporting: "The authors introduced Whale Optimization Algorithm (WOA) and compare it with Grey Wolf Optimizer (GWO) and Particle Swarm Optimization (PSO) algorithms in the context of NOMA-based wireless system. The manuscript is easy to follow. The authors can consider the below suggestions to improve the quality of the paper.

As for the writing:
- abstract: "A user grouping,... assume..." to passive voice.
- please avoid using ambiguous expressions such as "high", "significantly comparable", etc.
- line 144 two "at".
- line 151 unfinished sentence.
- avoid starting sentences with Greek symbols or "Which".
- Grey Wolf Optimizer (GWO): The gender of the wolve is emphasized. How does it contribute to the algorithm?
- "Figure 2: Grey wolf hierarchy ?"
- line 379 "is assume" to passive voice.
- line 412 "simulations results"
Experimental design: The authors proper explanation  NOMA-based wireless system performance and detailed analysis. Neither experimental verification nor comparison to other methods/models is presented. and more comparison to recently published papers that may prove scientific value is not provided
Validity of the findings: Please restructure the paper, as sections "Spectral efficiency maximization" and "Solution of proposed model" can be subsections instead of being equal to sections "Introduction" and "Discussion".

If possible, please elaborate the discussion on the Simulation results and add application scenarios and/or future works in the Conclusion.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: JOINT USER GROUPING AND POWER CONTROL USING WHALE OPTIMIZATION ALGORITHM FOR NOMA UPLINK SYSTEMS
Review round: 1
Reviewer: 2

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This paper studies the uplink NOMA system with the WOA algorithm, But the reviewer cannot accept this manuscript. Comments are as follows:
1. The authors just applied the WOA algorithm to the uplink NOMA systems and the reviewer could not find any interesting idea or intuition from this manuscript.
2. No performance gain of the WOA algorithm compare with PSO or GWO algorithm from fig.7

3. Future works should be added in the last section.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: JOINT USER GROUPING AND POWER CONTROL USING WHALE OPTIMIZATION ALGORITHM FOR NOMA UPLINK SYSTEMS
Review round: 1
Reviewer: 3

Output:
Basic reporting: Although the language of the article is understandable, it needs some enhancements, for example, when the authors wrote "In this work, a newly developed Whale Optimization Algorithm (WOA) is implemented", they did not mention the purpose of using this algorithm specifically. And the article also lacks clarity when talking briefly about results in the Abstract section as in the sentence " Also, WOA attains improved results in compliance with system complexity". Moreover there are some grammatical mistakes and types like: "is capable to chose", "would honour the","where as ", "response is assume to be", "are describe as"....
Experimental design: WOA algorithm has to be written and described by taking into account the NOMA Uplink system,
One of the drawbacks of the article is that the baselines contain more details than the proposed algorithms.
Validity of the findings: I think it would be good to compare your results against other algorithms such as Cuckoo Search Optimization, Ant Colony Algorithm, and other state-of-the-art algorithms...
Additional comments: The article is missing the future work


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: JOINT USER GROUPING AND POWER CONTROL USING WHALE OPTIMIZATION ALGORITHM FOR NOMA UPLINK SYSTEMS
Review round: 1
Reviewer: 4

Output:
Basic reporting: The problem under study is interesting and trendy, but the major issue is that there is no significant and convincing novelty. In the following, some concerns will be discussed which hopefully can help the authors.

1.	The main concern about this work is that the manuscript consists of contradictory statements about the contribution. It seems the authors investigated the usage of three algorithms on the application of NOMA, not proposing a new variant of WOA. The authors should specify with clarity their novelty and contribution. For instance, the reader encountered the following inconsistent sentences,
-	“In this work PSO, GWO, and WOA are employed”, “a newly developed WOA is implemented”, “To solve the issue of complexity, a WOA with low complexity is investigated for an efficient opt…”, “The proposed algorithm for user pairing/grouping”, “… reduce the system complexity, an innovative existence metaheuristic optimization technique named WOA is proposed in this work.”, “This section evaluates the proposed algorithms WOA, GWO, and PSO…”.
2.	The abstract should be revised to reflect the main novelty and contribution of the paper.
3.	The literature overview in Introduction section is shallow, the authors should add an in-depth literature review of recent optimization algorithms such as MTDE and I-GWO (with DLH search strategy) algorithms.
4.	 It is recommended to provide more informative literature on the usage of metaheuristic algorithms for solving the NOMA uplink system.
5.	The Section entitled “Optimal and Sub-optimal User Grouping using WOA” is vague and does not show the methodology of the usage of WOA for user grouping tasks. The authors should describe the proposed algorithm step-by-step and show the sequence by using a flowchart or pseudo-code. (Algorithm 1 is the pseudo-code of WOA not the usage of it for user grouping)
6.	The authors should clarify what is the “WOA NOMA”. There is no explanation for “WOA NOMA” in the manuscript until the Simulation results section.
Experimental design: 7.	The authors mentioned in Introduction that “The results obtained through the algorithms proposed in (Sedaghat and Mu¨ ller (2018)), WOA, GWO and the popular PSO are exclusively compared in this study.”, but there are not any results from (Sedaghat and Mu¨ ller (2018) in the Simulation results section.
8.	The authors should correct the legend of Figures 4-6 to remove “[19]” and add a proper reference.
9.	The selection of comparative algorithms is not satisfactory. It’s necessary to compare the proposed algorithm with recently proposed and enhanced variants of algorithms.
10.	I could not find the number of iterations, runs, and population used in experiments; are they missed?
Validity of the findings: 11.	The authors should statistically analyze the proposed and comparative algorithms using Wilcoxon and Friedman tests.
12.	The authors have claimed about the aim of using WOA and “obtaining high spectral-efficiency with lower computational complexity”, but this complexity is related to WOA and the authors have no contribution to achieve or even to reduce it.
13.	It is recommended to analyze the performance and effectiveness of the proposed and comparative algorithms using the performance index (PI) as shown in Subsection 5.3.5 in the QANA paper.
Additional comments: 14.	It is noted that Figures 4, 5, 6, and 7 are illustrated before their description.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: JOINT USER GROUPING AND POWER CONTROL USING WHALE OPTIMIZATION ALGORITHM FOR NOMA UPLINK SYSTEMS
Review round: 2
Reviewer: 1

Output:
Basic reporting: The article appears much better. The content is clear and easy to understand.
However, I recommend referring to some recent research papers that are considered as related to this research paper:
- Alawad, N. A., & Abed-alguni, B. H. (2021). Discrete Island-Based Cuckoo Search with Highly
Disruptive Polynomial Mutation and Opposition-Based Learning Strategy for Scheduling of
Workflow Applications in Cloud Environments. Arabian Journal for Science and Engineering, 46(4),
3213-3233.
- Panda, S. (2020). Joint user patterning and power control optimization of MIMO–NOMA systems. Wireless Personal Communications, 1-17.
- Zheng, G., Xu, C., & Tang, L. (2020, May). Joint User Association and Resource Allocation for NOMA-Based MEC: A Matching-Coalition Approach. In 2020 IEEE Wireless Communications and Networking Conference (WCNC) (pp. 1-6). IEEE.
- Abed-alguni, B. H., & Alawad, N. A. (2021). Distributed Grey Wolf Optimizer for scheduling of
workflow applications in cloud environments. Applied Soft Computing, 102, 107113
Experimental design: The methods and figures are described snd formulated well.
Validity of the findings: No comment.
Additional comments: No comment.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: JOINT USER GROUPING AND POWER CONTROL USING WHALE OPTIMIZATION ALGORITHM FOR NOMA UPLINK SYSTEMS
Review round: 3
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: no comment.
Everything has been addressed well.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SECURITY AWARENESS OF SINGLE SIGN-ON ACCOUNT IN THE ACADEMIC COMMUNITY: THE ROLES OF DEMOGRAPHICS, PRIVACY CONCERNS, AND BIG-FIVE PERSONALITY
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper is well written and is easy to follow. The results are reported with professional diagrams and tables.

Improvement suggestion:
1. The figures should be vector graphics. I suggest the authors regenerate them into pdf or ps graphics
2. Citations from top security conferences (e.g., NDSS, IEEE S&P) are largely missing. The authors are suggested to look through literatures starting with the following paper: AuthScan: Automatic Extraction of Web Authentication Protocols from Implementations
Experimental design: The paper manages to make clear the hypotheses, and design use studies to validate them. I am very happy to see the experiments well follow the ethics.

Improvement suggestion:
1. The privacy concerns (Table 3) in the questionnaire seem not complete. For example, the relying party (RP) may access user's data on identify provider (IDP). It would be good if the authors may consider to include these.
Validity of the findings: The conclusions are well stated, and backed by the user studies.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SECURITY AWARENESS OF SINGLE SIGN-ON ACCOUNT IN THE ACADEMIC COMMUNITY: THE ROLES OF DEMOGRAPHICS, PRIVACY CONCERNS, AND BIG-FIVE PERSONALITY
Review round: 1
Reviewer: 2

Output:
Basic reporting: Thank you so much for the editor for giving me an opportunity to review this article.

Very well written article in Professional English. The research is clear and unambiguous, and very relevant to current topic of information security.

SSO is an area where a lot of security attacks can happen in organizations. Educational sector often overlooks security and privacy of information. This article is timely to elaborate with appropriate data about the security mindset of participants in the educational institution.

Please to verify spacing and formatting issues before sending the final copy.
Experimental design: The design and reserach fits within the Information Security domain

The research is well defined and attached sumary and survey results fills in the idenfied knowledge gap of SSO.
Validity of the findings: Data has been well analysed and conclusions are linked to original research questions.
Additional comments: No additional comments