Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DETECTION OF MALICIOUS CONSUMER INTEREST PACKET WITH DYNAMIC THRESHOLD VALUES
Review round: 1
Reviewer: 2

Output:
Basic reporting: Authors' main contribution is to provide an additional security solution to face the content poisoning attack (CPA) which will prevent malicious consumers from flooding the network with unwanted reporting packet in the context of named data networking
Experimental design: Overall, the idea is clear and simple. Yet, the threshold adaptativity should be related to many oher metrics like the cache size and the history of the requested data (can adopt a trust establishment solution for instance).
Validity of the findings: My main concern regarding the authors findings is related to the validation environment. All NDN research community agreed on NDNSim over NS3 because it has all NDN modules already implemented. Why you did not use it?!
Additional comments: Figures quality should be improved. Please also include this work in the comparative table, so that we can see its position compared to what already exists.

I also recommend citing the following research works:
https://doi.org/10.1109/ICCCN.2019.8847146
https://doi.org/10.1016/j.future.2020.01.043


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: DETECTION OF MALICIOUS CONSUMER INTEREST PACKET WITH DYNAMIC THRESHOLD VALUES
Review round: 2
Reviewer: 1

Output:
Basic reporting: The abstract needs to be revised. what is the takeaway from the paper that should clearly be mentioned?
This article’s main contribution is the addition of a security mechanism in the CPA mitigation. This statement is not sufficient. Need to support with data.
check grammar and typo mistakes in the paper.
mention various types of attacks in tabular form
Experimental design: Table 5. Qualitative Comparison-- check the column Proposed Approach  better support with values instead of mentioning Very High , high
Validity of the findings: Table 5. Qualitative Comparison  -give reference numbers
This table should support with data and the comparisons should be in graphs i.e pictorial representation
Additional comments: check the paper against the grammatical and typo mistakes
visibility of the figures need to be enhanced
check the numbering of tables and figures


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SHARING DIVERSE INFORMATION GETS DRIVER AGENTS TO LEARN FASTER: AN APPLICATION IN EN ROUTE TRIP BUILDING
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper reports on very relevant and contemporary topics to both computer science and ITS (intelligent transportation systems). It is, therefore, a very timely and significant article. It studies the effect of providing driver agents with current congestion information through vehicle-2-infrastructure strategies, and adapt Q-Learning to support their approach of a decentralised traffic assignment method.

The article is written and organised well, quite clear and easy to follow. Some typos are found, nonetheless are very easily addressed with careful proofreading (e.g. "We could shown that...").

Some concepts could be better explained and interpreted, though. It is not very clear why authors do not consider their proposed method as a "traffic assignment" method, as discussed in line 49 and on. After all, the proposed method is an iterative offline method that seeks to yield user equilibrium. In which sense is it not a traffic assignment problem then?
Experimental design: The proposed method is described well, and the experimental setup is appropriate to support the evaluation of the authors' approach.

Some issues on the definition of the network might be further explained, though:

1) How is the impedance of intersections modelled? Are intersections traffic-light controlled or are they priority junctions?

2) How dense is the graph representing the communication network? Are all CommDev connected to each other (i.e. |E| ~ V^2)? What would be the impacts of different assumptions for this graph on the results?

Also, please provide some more details on how this work extends the one previously published by the same authors, quite recently: (Santos & Bazzan, 2020). Much material in this paper seems to come as-is from the previous publication, especially concerning the proposed method.
Validity of the findings: Results seem to be good and very promising. Indeed, gains in the learning speed of RL drivers are significant. However, some findings seem very expected and intuitive; also the fact that providing all drivers with information won't benefit the system performance and considering a lower rate of informed drivers produces better outcomes, even in real-time setups (for rather localised information, in space and in time). Are there other findings that may stimulate the potential use of the proposed approach?
Additional comments: The subjects addressed in this paper are indeed timely and significant. As expected, information can improve and speed up the learning process of RL drivers resorting to two-way communication technologies as preconised in ITS directives. It is not very clear, however, how is this method different from other user-equilibrium offline iterative methods, regarding the concept of traffic assignment as a tool for planning, rather than for operational purposes.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SHARING DIVERSE INFORMATION GETS DRIVER AGENTS TO LEARN FASTER: AN APPLICATION IN EN ROUTE TRIP BUILDING
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper is very well written. ALL the contexts are clearly expressed. The references are up-to-date.
Experimental design: The experimental should be strengthened by providing comparison between their method and some well-known methods to show the novelty.
Validity of the findings: The novelty of the findings should be strengthened.
Additional comments: This paper devises a Q-leaning scheme for en-route trip building problem. After reading this paper, I think it is well written, but needs further improvement to clarify the novelty of findings.

1. The model of TAP should be introduced.

2. The comparison between their method and some well-known RL schemes should be provided to show the novelty.

So, I recommend this paper to be accepted as major.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SHARING DIVERSE INFORMATION GETS DRIVER AGENTS TO LEARN FASTER: AN APPLICATION IN EN ROUTE TRIP BUILDING
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: I am satisfied with the revised version, since all my concerns have been addressed. So, I recommend this paper to be accepted for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LSTM-BASED SENTIMENT ANALYSIS FOR STOCK PRICE FORECAST
Review round: 1
Reviewer: 1

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: The author uses machine learning and deep learning technology to predict the stock price, uses the Bert model to analyze the collected information's emotion, and inputs the analysis results and historical stock data into the LSTM neural network to predict the next day's stock price. However, many issues need to be addressed in this paper：
1. There are no keywords in the abstract. Please add keywords.
2. The background color of the abstract is not consistent with the whole article.
3. Variable B in Figure 1 is not represented in the figure; please, redraw.
4.The author ignores the influence of the error of emotion tags extracted by dichotomy on the whole prediction. Please explain why.
5. The description of the BERT method is too short, please introduce it in detail, and the structure diagram of the BERT model is not given, please add it.
6. The layout of all the pictures and tables is unreasonable, far away from the text description, and the impression is not good.
7. This paper lacks the figure of the model architecture and experimental flow-chart.
8. The sample stocks selected in this paper are less, and the experimental results are not universal.
9. The experimental results are not graphically represented, which is not intuitive enough.
10. If this is an innovative article, the model used is too old and lacks innovation.
11. The author designs four models with different inputs: LSTMP、LSTMN、LSTMF, and LSTMNF to compare the accuracy of forecasting stocks. Whether the four models have the same architecture, please give the architecture diagram. Only the experimental results of LSTMNF are given, and the other three models' experimental results need to be given.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LSTM-BASED SENTIMENT ANALYSIS FOR STOCK PRICE FORECAST
Review round: 1
Reviewer: 2

Output:
Basic reporting: No comments
Experimental design: No comments
Validity of the findings: No Comments
Additional comments: In this paper, authors presented sentiment analysis approach based on LSTM Neural Networks for stock price forecast. However, I have some suggestion for paper improvement as follows.
1.	The title of this manuscript should be changed. I suggest “LSTM-based sentiment analysis for stock price forecast”.
2.	Authors should make the abstract more attractive.
3.	The main contribution is not clear in introduction. Authors should discuss their main contribution in the form of bullets.
4.	In related work Section, authors should include some sentences before starting subsection 2.1. In addition, authors should include existing research work about LSTM or Deep learning-based sentiment analysis (‘An intelligent healthcare monitoring framework using wearable sensors and social networking data’, Transportation sentiment analysis using word embedding and ontology-based topic modeling’, ‘ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis’, ‘BiERU: Bidirectional Emotional Recurrent Unit for Conversational Sentiment Analysis’).
5.	BERT should be discussed in detail.
6.	Authors must include something before starting subsection (see 3.3 and 3.3.1).
7.	I suggest to bring equations from section 2 (related work) to section 3.5 (LSTM models).
8.	The future work should be included in conclusion.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: LSTM-BASED SENTIMENT ANALYSIS FOR STOCK PRICE FORECAST
Review round: 2
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: Thank you for addressing my comments.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CONTEXTUAL DIMENSIONS FOR CACHE REPLACEMENT SCHEMES IN INFORMATION-CENTRIC NETWORKS: A SYSTEMATIC REVIEW
Review round: 1
Reviewer: 1

Output:
Basic reporting: no comment
Experimental design: no comment
Validity of the findings: no comment
Additional comments: This paper conducted a systematic review of information-centric networks (ICNs) through SLR process to reflect the relationship among current researches, also provided a classification based on context dimension. The ICN-enabled scenarios were then summarized. Besides, this paper also pinpointed the relevant research directions in future works. This paper is well organized and easy to follow; however, more in-depth and related techniques should be involved to solidify the paper contributions. I still have some major comments are listed as follows.

1. It is necessary to give the full name of an abbreviation, before it appears for the first time in the text, even if it has been mentioned in the abstract. For example, authors should provide the full name of “ICNs” appeared in the first paragraph of introduction section.
2. Sentences in this paper should be examined carefully. For example, the verb in sentence “The authors advocate that each device can choose…” should be “advocated”, since this work was made in 2016 and the grammar of verbs should be consistent.
3. Since this paper is conducted based on information-centric networks, therefore, there are some recently related works here. Are the following studies helpful to the author? “Machine learning based code dissemination by selection of reliability mobile vehicles in 5G networks”, vol. 152, pp. 109-118, 2020. “A Trustworthiness-based Vehicular Recruitment Scheme for Information Collections in Distributed Networked Systems”, vol. 545, pp. 65-81, 2021. “An Intelligent Collaboration Trust Interconnections System for Mobile Information Control in Ubiquitous 5G networks”, IEEE Transactions on Network Science and Engineering, DOI: 10.1109/TNSE.2020.3038454, 2020.
4. The first line of each paragraph should be indented by two characters. Authors should carefully examine the whole paper to avoid such problem.
5. Figure 7 is not clear enough.
6. Recently, drone as an emerging technology, has received increasing attentions in ICNs. Authors should read more latest schemes to enrich their review.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CONTEXTUAL DIMENSIONS FOR CACHE REPLACEMENT SCHEMES IN INFORMATION-CENTRIC NETWORKS: A SYSTEMATIC REVIEW
Review round: 1
Reviewer: 2

Output:
Basic reporting: The basic reporting norms are met. This paper presents a survey of various caching
strategies/policies used in Information Centric Networks (ICN).  There
have been other survey papers on ICNs and caching; this paper
specifically considers the awareness of context characteristics that
can affect the behaviour of such caching policies.
Experimental design: The contents on the impact of various context elements such as
Node-Location, Node-Cache Size etc. presents a listing of some of the
techniques used and key results from the various papers. However, what
is lacking is a description of the different techniques. A researcher
working in this field would be even more interested in the techniques
themselves, than merely policy names and results. Further, the results
in a paper will be influenced significantly by the underlying system
parameters and hence, presenting only the results may not be very
beneficial.
Validity of the findings: The survey's taxonomy and research questions (RQ1-RQ4) are valid,
scientifically relevant and seem to cover essential questions to be
considered.

The section on "RESEARCH DIRECTIONS" can be quite useful to
researchers who plan to work in this domain.
Additional comments: Introduction:

* It does not adequately describe what ICNs are; it does not present
  the state-of-the-art of ICNs in terms of their adaptation. A better
  motivation for studying ICNs and caching schemes is needed.

* There is no description of where caching occurs, and why it is
  needed in ICNs. There is a serious gap in this regard. Hence, the
  lead-in to lines 49-60 is missing context.

* A proper definition of "scenarios" and "contexts" is also needed in
  this section.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CONTEXTUAL DIMENSIONS FOR CACHE REPLACEMENT SCHEMES IN INFORMATION-CENTRIC NETWORKS: A SYSTEMATIC REVIEW
Review round: 2
Reviewer: 1

Output:
Basic reporting: The authors have addressed my concern and it can be accepted now.
Experimental design: The authors have addressed my concern and it can be accepted now.
Validity of the findings: The authors have addressed my concern and it can be accepted now.
Additional comments: The authors have addressed my concern and it can be accepted now.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: CONTEXTUAL DIMENSIONS FOR CACHE REPLACEMENT SCHEMES IN INFORMATION-CENTRIC NETWORKS: A SYSTEMATIC REVIEW
Review round: 2
Reviewer: 2

Output:
Basic reporting: The paper meets the basic reporting criteria satisfactorily. The revisions have strengthened the paper's contributions in this regard.
Experimental design: This paper presents a survey of different existing techniques for cache replacement in Information Centric Networks.  The criteria provided by the journal have been met satisfactorily.
Validity of the findings: As it is a survey papers, there are no specific findings in the paper. The important sections of the paper relate to classifications/taxonomy of existing schemes as summarized in several Tables, and also a discussion on Challenges.
Additional comments: The changes made to the paper, based on the first set of reviews, are satisfactory.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC METADATA HARVESTING WORKFLOW FOR ANALYSING SCIENTIFIC NETWORKS
Review round: 1
Reviewer: 1

Output:
Basic reporting: The basic reporting is good, the very fact that the authors provide Pthon scripts shall be commended. The English is more or less OK, but the use or misuse of the capital letters is surprising. I show only a few errors:
 "apply Network Analysis using open citation"
"Eigen centrality"?? You do not mean here Manfred Eigen,  don't you?
"python script"

Some awkward constructions
"use case"
" instead of a software tool, we propose to use python libraries" But Python libraries ARE software tools
" identification of highly cited clusters of scientific clusters"
Experimental design: The research goal is clear, the methods are desrcibed with sufficient details
Validity of the findings: Dubious.  Allthough the technique is valid, the validation protocol is problematic.
Additional comments: In this work B.H. Butt, M. Rafi, and M. Sabih demonstrate a software tool to download data from the Open Citation database and to process it. They demonstarte an example of such processing- measuring various kinds of centrality of  the authors of a certain journal- Scientometrics.
While the protocol of downloading the whole Open Citation database shall be commended, the scientific benefits that the authors draw from the processing of this  database are  not clear enough. Hence, I do not suggest to publish this paper in its present form. However, , if the authors report only their downloading protocol and do not report data processing, such abridged paper is publishable.

Detailed comments.
The opening sentence of the paper "Identifying prominent authors (Gurus) of any field is one of the primary focus for researchers in that particular field"  is blatantly wrong. A serious researcher working in some field shall know all prominent authors in his field without citation analysis of the corresponding databases. To identify prominent authors through citation databases- this is usually made by beginners or by the researchers in adjacent fields.

Some of the statements of the paper look like typos:
"Eigen centrality"- as if the author thinks that Eigen is a person.
"APS The American Phytopathological Society"- This is a very naive error.  The  abbreviation of the American Phytopathological Society is indeed APS, but in the field of informatics, APS means  the American Physical Society and this is what the Ref. Singh and Jolad deals with.

"Data Source, primarily, is WoS or Scopus". Two of the  authors come from the Department of Computer  Science, why they are not familiar with CiteSeer?

Using their downloaded database, the authors choose to analyze the authors of the Scientometrics Journal. Thie goal is to identify the most prominent authors. However, one can't make such analysis basing on one journal- one needs to analyze the whole scientific field. There areseveral journals in this field: Scientometrics, Journal of Informetrics, Quantitative Science Studies, Journal of the American Society for Information Science and Technology. Prominent scientists in the the field of information science publish in these journals but due to rivalry between the journals, there is a certain association between some scientists and some journals. Moreover, european scientists tend to publish in the european journals. Hence, by analyzing citation network of one journal, it is impossible to find all prominent figures in the field- one shall analyze all journals atogether.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC METADATA HARVESTING WORKFLOW FOR ANALYSING SCIENTIFIC NETWORKS
Review round: 1
Reviewer: 2

Output:
Basic reporting: The authors go in great detail to explain what they have done, yet the article is lacking on several points. The language is not always used appropriately, several figures are low-resolution, and most definitions are given without reference to the related papers nor formally (e.g., for centralities).
Experimental design: The research questions are well-defined, yet unfortunately they do not focus on novel contributions.

RQ1 explores whether network centrality measures can be used to detect popular authors, or "gurus", which is something that has been extensively explored in previous work.

RQ2 is, instead, more novel in that it attempt to use an open citation index, COCI, and compare its results with WoS. Yet, this study has also been recently performed (in much greater detail): https://arxiv.org/abs/2004.14329
Validity of the findings: I do not have much to add in terms of findings, as the main issue with the paper is in its lack of novelty. I would, nevertheless, suggest to the authors to share their code (per se, a great thing to do) using online persistent repositories such as Zenodo.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC METADATA HARVESTING WORKFLOW FOR ANALYSING SCIENTIFIC NETWORKS
Review round: 1
Reviewer: 3

Output:
Basic reporting: See my general comments.
Experimental design: See my general comments.
Validity of the findings: See my general comments.
Additional comments: The contribution of this paper is in providing a set of Python scripts for performing scientometric network analyses in a reproducible manner based on open data sources. The paper does not aim to make a substantive contribution by providing new scientometric insights. I value the work presented in the paper. However, the authors need to be more clear about the contribution and the scope of their paper. For instance, I believe the following sentence needs to be removed from the abstract: “We have shown that centrality analysis is a useful measure for identifying prominent authors.” The paper does not show this. Likewise, the authors claim to answer the following research question: “Is it possible to identify prominent authors (Gurus) of any field, by applying Centrality measures on Scientific Networks?” I don’t believe the paper answers this question (except by summarizing some earlier literature, but this is not an original contribution). It therefore seems to me that this research question needs to be removed from the paper. The discussion section at the end of the paper also needs to be revised accordingly.

“Does the coverage of CrossRef (for Open Access publishers), hamper the Network Analysis as compared to WoS, or it can be replicated?”: This research question needs to be rephrased. Crossref provides data not only for open access publishers but also for subscription-based publishers, and for both types of publishers most data is openly available. Also, since the empirical analysis presented in the paper focuses on a single journal (Scientometrics), the paper offers only a partial answer to the question whether WoS-based analyses can be replicated using Crossref data.

The comparative analysis section is hard to understand. According to the authors, Milojevic (2014) “have fetched the data at least 5 years earlier than us, therefore, the total citation count is different”. This is difficult to understand. It is not clear to me how exactly the authors collected their data, and in particular for which time period data was collected. I don’t understand why the authors didn’t organize their data collection in such a way that it is as similar as possible to the data collection performed by Milojevic (2014). Having data sets that are as similar as possible is essential for a meaningful comparative analysis. If there are basic differences in the time periods covered by two data sets, I don’t see the value of performing a comparative analysis.

Ego networks play a central role in the paper, but the paper doesn’t provide a proper explanation of what an ego network is. A one-sentence explanation is provided on p. 2, but this is not sufficient. The authors should provide a more extensive discussion of ego networks and their relevance in scientometric analyses.

Also, while most of the paper is about analyzing ego networks, these networks are not considered at all in the empirical part of the paper. It would be very helpful if the authors could add a section to their paper in which they give a practical example of an analysis of an ego network.

“the results of a case study based on WoS data is reproduced to confirm that accessing metadata of publishers, which submit metadata to CrossRef as Open Access (such as Springer) does not hamper analysis as compared to WoS. However, the same would not be true for Publishers whose data is not yet available with CrossRef (such as Elsevier).”: The information in these sentences is not entirely correct. Both Springer and Elsevier make basic metadata such as titles and author lists of publications openly available in Crossref. They also both make reference lists of publications available in Crossref. The only difference between the two publishers is that Springer makes reference lists openly available, while Elsevier keeps reference lists closed.

Most readers won’t understand the discussion on partial vs. full counting on p. 9. This requires some additional explanation.

Throughout the paper ‘CrossRef’ should be written as ‘Crossref’.

The authors may be interested in a special issue on bibliographic data sources published in the first issue of Quantitative Science Studies: https://www.mitpressjournals.org/toc/qss/1/1. This special issue offers a lot of information on data sources such as WoS, Crossref, and OpenCitations.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC METADATA HARVESTING WORKFLOW FOR ANALYSING SCIENTIFIC NETWORKS
Review round: 2
Reviewer: 1

Output:
Basic reporting: Although the authors reworked their submission according to suggestions of the reviewers, I  find too many inconsistencies to recommend its publication in its present form.

While the review of previous studies and description of the workflow are good, the formulation of the research goals and the case study are problematic.

Indeed,  the abstract formulates the research goal as developing a methodology to construct a multinetwork from the same dataset. Namely, citation network overlayed with authorship network and collaboration network. This is an ambitious program and the authors seem to be able to show the workflow, algorithm how they perform this task. Such algorithms are known and the authors shall be commended for their thorough  description of  how these algorithms work together, in tandem or pipeline.

The authors present the example of such workflow, the case study. They downloaded the authors writing to Scientometrics, calculated different centralities associated with them, and compared their measurements to previous study  (Milojevic, 2014). Table 1  show dramatic difference between their ranking of prominent authors  and those of Milojevic, In other words, the authors shoot themselves in the foot, since discrepancies in Table 1 invalidate  their algorithm.

My feeling is that either description of some details of their algorithm is missing, or they do not explain in which aspect their measurements are different from those of Milojevic.
Experimental design: good.
Validity of the findings: Problematic. Probably, some details of the algorithm are missing.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC METADATA HARVESTING WORKFLOW FOR ANALYSING SCIENTIFIC NETWORKS
Review round: 2
Reviewer: 2

Output:
Basic reporting: See below.
Experimental design: See below.
Validity of the findings: See below.
Additional comments: Dear authors,

thank you for a thoughtful and extensive revision of your article. I believe its goals and scope are now clear, as well as its contribution. The article is also now well-embedded into previous literature, and the publication of the code on GitHub is crucial.

While I remain skeptical about the actual scientific contribution of this work, which I consider somewhat narrow, I believe that the authors have substantially improved on their previous submission and, if the editor considers their work of interest to PeerJ readers, I now support acceptance.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: A SYSTEMATIC METADATA HARVESTING WORKFLOW FOR ANALYSING SCIENTIFIC NETWORKS
Review round: 2
Reviewer: 3

Output:
Basic reporting: See my general comments.
Experimental design: See my general comments.
Validity of the findings: See my general comments.
Additional comments: I would like to thank the authors for the improvements they have made to their paper. Before I can recommend this paper for publication, there are some further improvements that I consider to be necessary.

The introduction of the paper, in particular the first paragraph of the introduction, needs to provide a better explanation of what the paper is about. The introduction is largely focused on discussing the problem of identifying ‘gurus’. This gives the incorrect impression that the paper may provide in-depth analyses of different approaches to identifying gurus. The introduction does not make sufficiently clear that the paper is almost entirely about providing a standardized workflow for identifying gurus. This needs to be made much more clear in the introduction.

The subsection ‘Research question’ in the section ‘Preliminaries’ can best be removed. If the authors want to specify an explicit research question, it is best to do so in the introduction.

The distinction between the section ‘Discussion’ and the section ‘Conclusion and future work’ is not very clear. My recommendation is to merge the two sections in a single section.

“Any study on co-authorship may not necessarily have all the citation links.”: I don’t understand this sentence. To study co-authorship, there is no need to have information about citation links. A study of co-authorship requires data on co-authorship. It doesn’t require citation data.

“the same would not be true for publishers whose metadata are not yet public although available with Crossref (such as Elsevier).”: As I pointed out in my previous review report, this is incorrect. Elsevier does make metadata openly available in Crossref. The only exception is citation data, which Elsevier currently doesn’t make openly available in Crossref. (By the way, this will change soon. See https://www.elsevier.com/connect/advancing-responsible-research-assessment.)


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: GASTRO-CADX: A THREE STAGES FRAMEWORK FOR DIAGNOSING GASTROINTESTINAL DISEASES
Review round: 1
Reviewer: 1

Output:
Basic reporting: In general, this paper is well written, although part of it should be improved to ensure that readers can clearly understand the text. Some examples where the language could be improved are listed as following.
Line 74, How to get the results after "Therefore"? The statements before "Therefore" are not reasons for "the early diagnosis of GI is essential …".
Line 402, What does "… as as …" mean?
Lines 423-425, FP and FN are all the images' sum that is not well identified for each class of GI diseases. What is the difference between FP and FN?
Line 655, "stage two" should be written as "the second stage".

Literature references and sufficient field background have been provided. Moreover, this article is well structured. Figures are relevant to the content of the article. However, the resolution of Figures 5-14 is not sufficient. The raw data are available.
Experimental design: The research question is well defined, relevant, and meaningful. Moreover, the rigorous investigation has been performed to a high technical & ethical standard. Methods have been described with sufficient information.

However, the simulation environment is not mentioned. Furthermore, there are some inconsistent statements for the methods, and some information is not clear. Some examples are listed below.

Line 287, in the first stage, four CNN are used as feature extractors.
Lines 312-320, in the first stage, valuable DL features are mined from the CNNs.
Lines 573-575, in the first stage, four DL features were extracted from images and used to train SVM classifiers.
The problem is which statement describes the real first stage.

Line 336, What are CD1 and CA1?
Validity of the findings: Although the results and compelling, the data analysis does not include the experimental settings' meaning corresponding to the best results.

The conclusions are well stated and linked to the research question.
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: GASTRO-CADX: A THREE STAGES FRAMEWORK FOR DIAGNOSING GASTROINTESTINAL DISEASES
Review round: 1
Reviewer: 2

Output:
Basic reporting: Comments are attached
Experimental design: Comments are attached
Validity of the findings: Comments are attached
Additional comments: No additional comments


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN EFFICIENT APPROACH FOR LOW LATENCY PROCESSING IN STREAM DATA
Review round: 1
Reviewer: 1

Output:
Basic reporting: The authors need to cite the following and other related newly research papers in the introduction and literature section 10.1109/MIS.2019.2942836 ; 10.1016/j.future.2018.08.031 ; 10.1016/j.ress.2020.106815 ; 10.1016/j.comnet.2020.107327 ; 10.1016/j.amc.2014.11.064 ; 10.1016/j.nahs.2015.07.005 ; 10.1155/2019/7875305 ; 10.1016/j.ins.2020.02.051 ; 10.1002/rnc.3980 ; 10.1016/j.isatra.2016.11.002 ; 10.1002/oca.2326 ; 10.1016/j.autcon.2019.02.014 ; 10.1016/j.aei.2019.100960 ; 10.1016/j.autcon.2019.102859 ; 10.1109/TIP.2018.2881828 ; 10.1016/j.amc.2015.06.036 ; 10.1016/j.autcon.2010.09.011 ; 10.1016/j.future.2020.08.021 ; 10.1109/TII.2019.2952565 ; 10.1016/j.swevo.2020.100697 ; 10.1007/s40857-020-00175-5 ;  10.1007/s11390-020-0350-4 ; 10.1109/TSP.2020.3007313 ; 10.1007/s11265-020-01610-6 ; 10.1093/imaman/dpaa009 ; 10.1109/TNNLS.2019.2955287 ; 10.3390/app10217924 ; 10.23919/JCC.2020.03.011 ; 10.1049/iet-map.2020.0090 ; 10.1631/FITEE.2000229 ; 10.1109/LED.2019.2903430 ; 10.1109/UCMMT47867.2019.9008340 ; 10.1109/LWC.2020.2982637 ; 10.1109/TCYB.2020.2970736 ; 10.1109/TITS.2020.3013928 ; 10.1109/TETC.2020.2974183 ; 10.3966/160792642020072104022 ; 10.1002/acs.3098 ; 10.1016/j.asoc.2020.106372 ; 10.1007/s12555-019-0972-x ; 10.1016/j.isatra.2020.08.022 ; 10.1504/IJDMB.2013.056078
Experimental design: no comment
Validity of the findings: no comment
Additional comments: The authors are advised to improve the research GAP. Finally, the authors are suggested to suggest such type of suggestion that are practically helpful.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: AN EFFICIENT APPROACH FOR LOW LATENCY PROCESSING IN STREAM DATA
Review round: 1
Reviewer: 2

Output:
Basic reporting: This paper presents quite a good work on stream data processing. I believe  this work is interesting and in the scope of PeerJ. I am aware that latency is the biggest issue in stream processing. The statistical approach used  by the author to handle latency is a good contribution. In fact, exploring different types of possible latency and defining an efficient approach to handle it is really interesting.  This paper proposed a model for end-to-end low latency stream processing. The overall framework makes sense and the logical structure of the paper is very clear.
Figures show appropriate structure and are well described.  The proposed approach is accepted.
Experimental design: Experimental design satisfies the relevance of the proposed work but the dataset used should be explained more and if possible in some suitable format. Author has put an effort to design a suitable dataset to explore the dependence and its effect on latency. Statistical experiments are interesting and relevant to the work.
Validity of the findings: Well organized literature findings.  The objectives of manuscript is well defined and covered.
Additional comments: The paper includes good theoretical justification. The Paper is well written and strongly accepted, however some English statements can be reformulated. I also feel if future direction of this work is defined in brief to continue this research further. Overall a good piece of work.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS SUITABLE DESCRIPTION OF REFERENCE ARCHITECTURES
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper is an SLR on approaches for the description of reference architectures. The topic is interesting both for the academic and industrial world. The work is well motivated and well presented. I really enjoyed reading the paper.
Experimental design: The study design is well described.

It may be improved considering the following observations.

The authors consider 4 reference architecture examples which are considered successful. How have they been chosen? For example, the authors cite Volpato et al, 2017 and mention that from the 20 analyzed reference architectures, 12 present no evidence, but it seams that other 8 do present evidence. Did you choose the 4 successful examples from these 8 reference architectures? Based on which criteria? Which are the criteria to consider them successful?

The authors mention the AUTOSAR example with 220 files and 22271 pages of documentation. This implies a significant financial, organizational, besides the development effort. Please comment this aspect related to its success in Section 4.4. The same observation holds for all the 4 examples.  In addition, reading, understanding and using such an amount of documentation is not simple. This observation is corelated with the second research perspective in Section 5.

Please insert the exact query you used for each of the search database because they may differ from one to another. For example, you searched the indicated keywords in the whole document or in the title and abstract or in the metadata, etc. Having the exact query allows the replication of the experiment.

Table 2: The most recent approach mentioned in this table is from 2015. Even if the study was conducted in 2018, there are no approaches between 2015 and 2018. Can you please comment this aspect? In addition, I noticed also a gap between 1998 and 2003 with no available approaches.

Line 533: what do you mean by “good examples”? Successful? Well documented?
Validity of the findings: The results are well described and the findings clearly outlined.
Additional comments: Section 5: Improvement in the collaboration between academia and industry: this aspect should be discussed also from the point of with of the cultural and financial aspects worldwide. There are countries in which this collaboration is very strong, in the same there are countries where such collaborations are not supported by cultural and financial rules. Also, the authors of this paper belong to the academic world. Is industry not interested in such an SRL?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS SUITABLE DESCRIPTION OF REFERENCE ARCHITECTURES
Review round: 1
Reviewer: 2

Output:
Basic reporting: The literature review is from the first half of 2018. I would advise to check for more recent papers. Given the structured approach of the paper, any additions should be easy to include.
The quality of English is very acceptable. A check by a native speaker might remove any remaining issues, such as in line 16/17 or 241.
Experimental design: The paper proposes to do a literature search and compare the results with four reference architectures from practice.
For comparison, some sort of reference is required. Such a reference can be developed from the literature, the practice viewed or both. Or it can be found externally. In this case the authors selected an ISO/IEC standard. I would expect an explanation for this choice. Also, it is a bit surprising to use a nine year old standard as the basis for analysis in a (as the authors themselves state) rapidly developing but still immature area.
The selection of reference frameworks from practice appears to have been based on success. This can be made more explicit, and also the definition od ‘success’ is missing (but can be inferred).
Validity of the findings: no comments
Additional comments: Nice work!


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS SUITABLE DESCRIPTION OF REFERENCE ARCHITECTURES
Review round: 2
Reviewer: 1

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: The authors have addressed all the reviewer comments and they have improved the paper based on these comments. I really appreciated the new version of the paper, as well as the motivations given by the authors. Therefore, I suggest the acceptance of the paper for publication.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: TOWARDS SUITABLE DESCRIPTION OF REFERENCE ARCHITECTURES
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment
This is a re-review. I am satisfied by the way the comments of the reviewers have been addressed.
Experimental design: No comment
Validity of the findings: No comment
Additional comments: No comment


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTS OF NETWORK TOPOLOGY ON THE PERFORMANCE OF CONSENSUS AND DISTRIBUTED LEARNING OF SVMS USING ADMM
Review round: 1
Reviewer: 1

Output:
Basic reporting: .
Experimental design: .
Validity of the findings: .
Additional comments: This paper investigated the key features of networks for distributed ADMM in the applications of SVMs. Extensive experiments have been conducted for this purpose. However, there are still several problems need to be addressed.
1.	As the ADMM is the key method used in this paper, it should appear in the title.
2.	The section ADMM is confusing. The problems (12) and (13) are not formal. The process of the used consensus-based ADMM algorithm should be provided. It will be better if the convergence analysis could be given rather than only presenting the experiment stduies.
3.	The transformation from SVM to ADMM is absent.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTS OF NETWORK TOPOLOGY ON THE PERFORMANCE OF CONSENSUS AND DISTRIBUTED LEARNING OF SVMS USING ADMM
Review round: 1
Reviewer: 2

Output:
Basic reporting: The wording of this article is more accurate, in line with the requirements of politeness and norms.This paper introduces the research background and current situation of the problem.The graphs are also clear.
Experimental design: This article should further explain how the Alternating Direction Method Of Multipliers (ADMM) algorithm can be applied to Support Vector Machine (SVM) training.Instead of just describing the Alternating Direction Method Of Multipliers algorithm and Support Vector Machine training model respectively.The article needs to add more details about the transformation between models.
Validity of the findings: (1)The simulation results in this paper are more comprehensive, and the conclusion of this paper is validated effectively.
(2)The computational part is persuasive.The parallel computations in this article are performed on Hebbe at Chalmers Centre for Computational 485 Science and Engineering (C3SE) provided by the Swedish National Infrastructure for Computing (SNIC).
Additional comments: (1)The paper should organize the references more strictly and unify the quotation format of multiple references.
(2)The language needs to pay more attention to details, such as the use of parentheses when referring to graphs.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTS OF NETWORK TOPOLOGY ON THE PERFORMANCE OF CONSENSUS AND DISTRIBUTED LEARNING OF SVMS USING ADMM
Review round: 2
Reviewer: 1

Output:
Basic reporting: N/A
Experimental design: N/A
Validity of the findings: N/A
Additional comments: The author has addressed most of my comments. I have no other concerns.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: EFFECTS OF NETWORK TOPOLOGY ON THE PERFORMANCE OF CONSENSUS AND DISTRIBUTED LEARNING OF SVMS USING ADMM
Review round: 2
Reviewer: 2

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: This paper investigates the impact of network topology on the performance of an ADMM-based learning of Support Vector Machine (SVM) , and mean-degree graphs, and some common modern network topologies. Further, the paper investigates to which degree the expansion property of the network influences convergence for iterations, training and communication time. The experimental part is very sufficient. I have no doubts now.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SUPERVISED ENSEMBLE LEARNING METHODS TOWARDS AUTOMATICALLY FILTERING URDU FAKE NEWS WITHIN SOCIAL MEDIA
Review round: 1
Reviewer: 1

Output:
Basic reporting: This paper utilizes ensemble methods to classify fake and legitimate news articles of the Urdu language. Authors built an annotated news corpus composed of Urdu news articles using Google Translate. Moreover, they perform experiments with traditional machine learning classifiers and five ensemble models to detect fake news and comparatively analyze the performance of their models. The language used in this paper is clear and unambiguous. Their work, innovations, contributions, and results are clearly presented in the introduction section, and the whole structure meets the requirements. However, the resolution of the figures needs to be improved, especially Figure1&2. In addition, all of the captions for the figures and tables are written as 'Figure 1' and 'Table 1'.
Experimental design: This paper presents a meaningful research topic and fills the gap in the current knowledge gap with high relevance for real-life situations.  This paper innovatively builds a corpus using translated labelled English corpus, which addresses the problem that the scarcity of Urdu language corpus. However, there exist two problems: first, the accuracy of the English-to-Urdu translation is not verified. As mentioned in lines 461-462, the idea that all models perform better on UFN than BET is not a sufficient reason to illustrate the accuracy of Google translate API. Second, line 351 mentions that they randomly selected news articles. Why they were randomly selected instead of using all the data?
Validity of the findings: All underlying data have been provided and the conclusions are well presented. The final experiments are able to demonstrate that they design a classifier that performed well on their dataset.
Additional comments: This paper presents a meaningful research topic and fills the gap in the current knowledge gap. Authors built an annotated news corpus composed of Urdu news articles using Google Translate, perform experiments with traditional machine learning classifiers and ensemble models to detect fake news. However, the section about corpus building needs to be supplemented to illustrate the accuracy of the methodology.  In addition, there are some typos:
line 38: is a fake news: a piece of fake news.
line 441: accruacy: accuracy
line 499: shown: shown in.
Moreover, the resolution of the figures needs to be improved and all captions of the tables and figures need to be revised.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SUPERVISED ENSEMBLE LEARNING METHODS TOWARDS AUTOMATICALLY FILTERING URDU FAKE NEWS WITHIN SOCIAL MEDIA
Review round: 1
Reviewer: 2

Output:
Basic reporting: In general, this paper is easy-to-follow. However, one concern is on the use of "Feature Selection"; I think the authors meant extracting the features (feature extraction) rather than feature selection (for dimensionality reduction).

It has a comparatively comprehensive reference list, while more advanced relevant work with the topic of fake news detection could be considered, e.g., neural-network-based studies.

Most figures and tables are clear. It would be more professional if (1) accurately numbering figures and (2) just like other figures, note "%" in Figure 3.

I didn't find access to the raw data.
Experimental design: This work is within the Aims and Scope of the journal. It contains a detailed experimental setup and some of the necessary experiments. However, the experiments are insufficient. If the authors aim to validate the effectiveness of ensemble learning methods compared to the single classifiers. Multiple groups of features are requested for validation. For example, separating the combined n-grams or trying other common feature sets, like LIWC.
Validity of the findings: The work's limitations have not been discussed yet. For example, the Urdu news articles were obtained not directly but by translating the English news articles. The translation quality is hard to be perfect; the data might need to be further improved from this perspective.
Additional comments: In general, this work contains most of the necessary information, components, and analyses. However, my major concern is about its innovation since rather than proposing news methods, the authors mainly use some existing methods to detect fake news. I suggest the authors explicitly claim their innovation in the paper.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SUPERVISED ENSEMBLE LEARNING METHODS TOWARDS AUTOMATICALLY FILTERING URDU FAKE NEWS WITHIN SOCIAL MEDIA
Review round: 2
Reviewer: 1

Output:
Basic reporting: In general, this version of the paper has been improved both in terms of content and writing. Several parts have been rewritten and now the quality of the paper is higher than before.
Experimental design: The authors address my comments satisfactorily, both the accuracy of the English-to-Urdu translation and the news articles selection.
Validity of the findings: The authors have added the limitations of this study to the manuscript, which makes it more reasonable.
Additional comments: The authors did some progress with the new changes, and I believe that it would be accepted.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SUPERVISED ENSEMBLE LEARNING METHODS TOWARDS AUTOMATICALLY FILTERING URDU FAKE NEWS WITHIN SOCIAL MEDIA
Review round: 2
Reviewer: 2

Output:
Basic reporting: Please see my general comments for the author.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: I appreciate the authors have addressed most of my previous concerns. Nevertheless,

1. the writing can be further improved before being accepted;

2. the review for low-resource language datasets is insufficient, where several newly released ones for fake news research are suggested to include, such as CHECKED and MM-COVID. Meanwhile, I hope the following surveys could help the authors with the review:
- Zhou, Xinyi, and Reza Zafarani. "A survey of fake news: Fundamental theories, detection methods, and opportunities." ACM Computing Surveys (CSUR) 53.5 (2020): 1-40.
- Sharma, Karishma, et al. "Combating fake news: A survey on identification and mitigation techniques." ACM Transactions on Intelligent Systems and Technology (TIST) 10.3 (2019): 1-42.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SEMANTIC MICRO-CONTRIBUTIONS WITH DECENTRALIZED NANOPUBLICATION SERVICES
Review round: 1
Reviewer: 1

Output:
Basic reporting: The paper presents exensions of the nanopublication architecture / server network previously published by some of the authors (Kuhn et al, 2016), with the overall aim to provide non/expert end users with an effective and easy to use solution to publish authenticated nanopublications - so-called "semantic microblogging". More in details, newly contributed extensions w.r.t. (Kuhn et al, 2016) consist in the support for:
* digitally signing nanopublications (section 3.1);
* superseding/retracting nanopublications (section 3.1);
* querying nanopublications via LDF (via new quad pattern fragments extension - section 3.2) and grlc APIs (sections 3.3);
* accessing these APIs and manipulating signed nanopublications via the new Nanobench web application via a template mechanism (section 3.4);
The new features are assessed in terms of performance and usability. Evaluation code and data is made available in supplemental material.

The manuscript report on an incremental work w.r.t. (Kuhn et al, 2016). The novel contributions appear appropriate - in terms of significance and scope - for a PeerJ publication according to the Journal reviewing guidelines.

Background and references are overall adequate, and the paper is mostly self-contained with two minor exceptions:
* the paper should provide an explicit discussion of how this work relates to (Kuhn et al, 2016) including which additional problem(s) / research question(s) it addresses and which novel contributions it provides, as currently I had to go back and read the other paper to figure out these aspects;
* the paper (section 3.2) should provide more details about how triple pattern fragments have been extended to quad pattern fragments, as interested readers currently have to figure out this information in the linked specification.

The paper is overall well written and structured, the quality of English is good, and listings, figures and tables are appropriate. Some suggestions:
* [section 1] I suggest providing more details on the contributions of the paper, possibly reusing content from [section 3, lines 106-114]
* [section 1, line 33] "Such files are not regularly indexed ... and which therefore often lack discoverability" -> typo: remove "which"
* [section 3.1, line 159] "that can work on any URL scheme" -> suggest writing "any URL scheme and identification mechanism providing dereferenceable user identifiers"
Experimental design: A research question is not explicitly stated in the paper (if strictly required for the journal), although the paper provides new useful features / services extending the nanopublication architecture of (Kuhn et al, 2016) and therefore mostly addresses the same research question of that work:

[[
Can we create a decentralized, reliable, and trustworthy system for publishing, retrieving, and archiving Linked Data in the form of sets of nanopublications based on existing web standards and infrastructure
]]

with a specific focus on efficacy and usability by non-expert end users. In any case, I would suggest authors to improve the introduction and be more specific about the particular problem addressed here and how this work fits in / extends the one presented in (Kuhn et al, 2016).

I have some concerns with the approach design in terms of trust assumptions, scalability and fault tolerance, which I report in the general comments section. Ignoring these concerns and leaving out scalability and fault tolerance from evaluation dimensions, then I believe the experimental design is appropriate.

Approach code and evaluation code and data are provided (via supplemental material / external links), this way ensuring adequate reproducibility.
Validity of the findings: The findings drawn from the two evaluations (section 6) are overall justified by the evaluation results (sections 4.2, 5.2).

Concerning usability, I have no remarks except suggesting - if possible, i.e., if Nanobench logic is fully implemented browser-side - to deploy Nanobench on the server network so that users don't have to install it locally, this way further enhancing usability. If that is possible and there were user complaints about the tool installation, then authors may argument / roughly quantify in the paper how usability results may improve if installation steps (and possibly associated complaints) are removed.

Concerning performance, I'm a bit puzzled by the success/error percentages and analysis of section 4.2 / figure 3. Put that way, it would seem that there is no guaranteed way to successfully query the two APIs (i.e., it's expected to encounter errors!), although multiple servers are available and a client (like Nanobench) is in principle able to use different servers and thus cope with a specific server error/timeout (as reported for grlc). I would suggest authors to extend the analysis by considering also a different notion of success / error as perceived by the end user: a query is successful for a certain client location if there is at least a server able to answer it without errors (so a client contacting different servers from that location would eventually succeed). Put this way, I think grlc queries will likely achieve 100% success rate, and LDF queries would also improve. Concerning the latter, it would be interesting to consider also (client-side) timeouts larger than 60s.
Additional comments: Major comments:

* System design and assumption of trusting the server network -- the proposed solution featuring the two grlc and LDF APIs assume implicit trust in the servers offering these APIs. For instance, we must assume that they will return correct and complete (pending timeouts/errors) answers to queries, that the data they provide match the nanopublication data uploded by users, and that data in the two administration graphs (if publicly accessible, as it seems) is correct and complete. Note that these trust assumptions, which are relevant in this grlc+LDF service layer and likely in any other service layer, are less relevant for the core nanopublication distributed architecture, since there the client can only lookup a trusty URI and then it can verify the integrity of the associated content itself. Now I'm not much concerned with these trust issues (although it would be nice to discuss them in the paper). My point, instead, is that since a client already has to trust the servers, then why not delegate all integrity checks to the servers, instead of asking clients to perform them themselves through complex queries such as Query 16 (section 4.1, supplemental material) that negatively affect usability and performance? For instance, the server may offer a separate grlc/LDF endpoint offering only the latest versions of nanopublications whose signature and integrity have been successfully checked, this way greatly easing performance and usability (i.e., Query 15 will suffice). If trust in the server network cannot be assumed, then a possibly interesting solution would be to offer an API that, given a query method (grlc, LDF, or other), returns only / also the raw content of the nanopublications from which query results can be extracted from, so that the client can verify the authenticity / integrity of such data and compute / verify the query results itself. If I got this aspect wrong, or if there are precise motivations for designing the system this way, I suggest authors to comment or argument in the paper.

* Scalability and fault tolerance -- these are relevant aspect of a distributed system like the one being proposed, especially given sentences like the one concluding section 6 (semantic microblogging possibly being "the starting point of bringing Linked Data publishing to the masses"), but also in case the scope of the proposed solution is not web-wide but restricted to specific use cases / deployments as argued in (Kuhn et al, 2016). Concerning scalability, while I trust that the key-value stores used by the core nanopublication architecture are scalable, I have some concern about the scalability of the triplestore backing the new APIs since the system design asks for storing all queryable nanopublication data in it. Given the breadth of the paper contributions, I'm fine if scalability is left out, but I would suggest explicitly stating it and possibly pointing at more efficient designs (as future work) in case the authors already have ideas to that respect. Concerning fault-tolerance, the experimental setting using Nanobench already permits to roughly assess it (see suggested analysis with different error/success notion), but also here I think authors may present the solution as an initial step and explicitly leave out fault tolerance from the paper.

Minor comments:

* [section 3.1, lines 151-152] Concerning the requirement that updates and retractions must use the same private/public key pair: what if such key pair is invalidated and a new key pair is associated to the user? I'm in no way a security expert but I believe a similar issue occurs with legally-recognized digital signatures (where a personal key pair may also expire), so probably similar solutions may be adopted in this case, also to guarantee that previously posted nanopublications are still recognized as legit in case the user later updates his/her key pair.

* [section 3.4, lines 258-260] Editing the ORCID profile to link it to the nanopublication providing the public key appears crucial to me (vs. only optional), as without this step no third party can fully trust the nanopublications issued by a user.

* [section 3.4, lines 291-293] I think what needs to be sorted in the form are placeholders, rather than statements. How does "statementOrder" work precisely, to that respect? Suggest specifying better in the paper.

* [section 3.4] Suggestion: while I acknowledge the usefulness of templates, I think the Nanobench application may benefit from a general RDF file upload functionality allowing users to post arbitrary content not supported by available templates (size limits and data validation may be enforced prior to publication, in this case).

Overall, I believe the work described in the paper is interesting but the paper can be further clarified / improved, so I recommend a major revision and another review round.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SEMANTIC MICRO-CONTRIBUTIONS WITH DECENTRALIZED NANOPUBLICATION SERVICES
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper tackles the very relevant issue of authoring and publishing Linked Data. It describes a decentralized approach, based on nanopublications and inter-connected services, that allows users to publish small sets of statements. The system incluses a template-based interface, called Nanobench, that helps authors to compose these statements. The article is completed with an empirical evaluation of both the performance of the architecture in answering queries and the usability of the interface.

I found the work interesting and solid. On the other hand, I would suggest a major revision mainly to improve the presentation.
More comments in the 'Experimental design' review.
Experimental design: I see two main issues. First of all, the paper does not make a clear distinction between the specific contribution of this work and the overall approach based on nanopublications proposed by the same authors. The background section points to some previous works but the paper should better state which research questions are answered by this specific work and which is the advancement to the state-of-the-art.

Related to this, the paper combines two very different perspectives and the connection between these looks rather weak. On one side, there are architectural aspects (identifying and distributing statements, building services, querying, etc.); on the other, usability ones (template-based authoring of LD). The paper, and in particular the introduction, should identify the exact focus and novelty. Is it on the interface or on the services? Or both? How is different from other contributions?
This is also reflected on the evaluation part. The two parts are quite disconnected and the usability one looks weaker.
If the focus is on the semantic blogging approach, the evaluation should involve more users and go deeper in the features of the interface.

The paper would also benefit a stronger separation between the literature review, to identify the limitations of the existing approaches, and the background section, to better understand the global approach proposed by the authors.
Validity of the findings: No comment.
Notes about impact and novelty in the 'Experimental Design' review.
Additional comments: Some more detailed comments (including some minors):

- L.17: the sentence “It also imposes a large overhead… …or original content” is not clear in the abstract, please rephrase
- L.18: use “do not” instead of the contract form “don’t”
- L.29: the reference about the remarkable adoption of Linked Data goes back to 2011, should be changed with a more recent one
- L.37: the introduction should also mention other research efforts on decentralized services for publishing data, as described in Section 2
- L.42: the introduction mentions semantic microblogging but this is lost in the rest of the paper; it should be given more relevance; a more detailed comparison between the use of ‘semantic microblogging’ in this work and in (Passant et al. 2008) would also be helpful
- L.53: the relation of ‘semantic publishing’ is not clear here; the introduction does not mention it and it is focused on Linked Data in general; later in the section it becomes clear though
- L.93-103: the discussion about decentralized approaches is too short. The analysis of the reference (Third and Domingue, 2017), for instance, is only a few lines even if this work is very related. A more accurate discussion of the literature would strngthen the overall approach and would help to highlight the novelty of the paper.
- L.106: a picture summarizing the overall approach and components would be very helpful for the readers
- L.121: the proposed approached seems to be weak, since the nanopublication can be signed by anybody claiming any ORCID. This is clarified later in the paper (L.153-) but should be anticipated here
- L.161: QPF is explicity mentioned here for the first time; it should be given more relevance in the first part
- L.192-211: a table showing the queries instead of a long paragraph would be much more readable
- L.212-214: the discussion is very short even if these problems are complex; this part should be extended
- L.254: a list of predefined templates is loaded; is it possible to add new templates? How? Is it complex? Did authors evaluate this part?
- L.327: add some details about the process for selecting these 14 queries
- L.419: add details about the test: how many nanopublications were published? tow complex are they? Even if there is the supplementary material (which is very good) some more hints would help
- L.425: if possible, the authors should add some more information about the background of the testers. How did they publish Linked Data in the past? When they say that ‘Nanobench was easier to use’ which other approaches they compared?
- L.435: did testers also provide suggestions to improve the interface?


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SEMANTIC MICRO-CONTRIBUTIONS WITH DECENTRALIZED NANOPUBLICATION SERVICES
Review round: 1
Reviewer: 3

Output:
Basic reporting: The paper discusses a novel decentralized strategy for supporting the publishing of semantic microblogs (as defined within the text) into the Linked Open Data (LOD) cloud in a more easy and efficient way.
Beyond the theoretical aspects, the authors presents a user interface through which the users can manually provide new knowledge.
The paper is sound with the scope of the journal, well presented and motivated, and the authors provided the supplementary material for better checking the evaluation.

On the one hand the paper is interesting since the problem of managing all the knowledge of the LOD cloud is known to the community and, at the time of this review, it seems that no specific efforts are planned to address this problem.
On the other hand it seems anyway that the problem is more technical-wise than research-wise. Hence this consideration affects part of the motivation behind your work.
How can the authors comment about this?

Another point is related to two novelty aspects.
First, interfaces and tools for simplifying the gathering of knowledge from users and domain experts have been widely studied in the last decade and several state of the art tools are currently available (MoKi, WebVOWL, ecc.). Is Nanobench of the same family of these tools or it has different aims? Obviously, the fact that Nanobench works with microbloggings while the other tools have been thought for ontology design does not count.
Second, the decentralization of information, and also of semantic knowledge, is something that already started to be investigated. As example consider this paper:
- Harm Delva, Julián Andrés Rojas Meléndez, Pieter Colpaert, Ruben Verborgh: Decentralized Publication and Consumption of Transfer Footpaths. SEM4TRA-AMAR@SEMANTICS 2019
Please argue if your work belongs to this research direction or not and why.

Finally, my opinion the evaluation is incomplete.
How were the queries chosen? Were they validated in some way by some experts or not?
How were the Linked Data Fragment built?
Here the problem is quite thorny since there are many way for creating the fragments and also the hardware equipment significantly affect the performance.
Please argue on that.
Experimental design: The experiments have been conducted properly even if further details and discussions should be provided as highlighted in the report above.
Validity of the findings: No comment.
Additional comments: Overall, the work presented is good and I do not find big reasons for rejection.
However, I warmly invite the authors to address the comments in order to improve the quality of the paper and to make it more of impact for the community.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SEMANTIC MICRO-CONTRIBUTIONS WITH DECENTRALIZED NANOPUBLICATION SERVICES
Review round: 2
Reviewer: 1

Output:
Basic reporting: The manuscript is a revised version of the original submission I previously reviewed, so I will focus my review on authors' changes. I would also like to thank the authors for their response and for having addressed the comments raised in my previous review.

The paper research question, novel contributions, and relation to previous authors' work (esp. Kuhn et al, 2016) are now explicitly stated, positively addressing a concern in my original review.

The paper presentation is good and has further improved in this revision, also due to the above clarifications. I also appreciate the added approach diagram (Figure 1). Spotted typos for authors' convenience:
* [section 3.2, line 202] "allows allow" -> "allow"
* [section 3.2, line 230] "can be join" -> "can be joined"
Experimental design: A research question is now clearly indicated in the introduction, and appears appropriate. I previously found the experimental setup appropriate and appreciated the availability of code and data to ensure reproducibility.
Validity of the findings: As previously reported, I believe the findings summarized in the paper (section 6) are justified by the conducted evaluation and its results (sections 4, 5).

I appreciate the analysis of which / how many queries can be successfully evaluated by trying multiple servers (section 4.2), which addresses a concern I previously raised and demonstrates the benefit of replication.

I thank the authors for their clarification (in the authors' answer) about the difficulties of deploying Nanobench on the server network. While I see technical solutions to the central point of failure issue (e.g., by replicating Nanobench on multiple servers and using techniques for directing users to the closest online server, similar to the ones adopted in content distribution networks), I didn't consider the need for Nanobench to access the user's private key and I find this much more tricky to deal with. I agree with the authors' arguments, and I think it may be worth explicitly mentioning this private key issue when discussing private keys in section 6 of the paper.
Additional comments: The revised manuscript provides indications of how the proposed solution would accommodate future work extension to address concerns related to trust in the infrastructure, scalability, and fault tolerance. I think this is appropriate and sufficient for this paper, whose main contribution is to lay the foundations for enabling end users to easily publish nanopublications in an authenticated and decentralized way.

Related to trust, I previously suggested to further rely on the underlying assumption of trust in the infrastructure (which can be "lessened" as hinted by authors, but not completely eliminated) in order to simplify end user queries, exemplifying this with Query 16, which I found very complex from an end user point of view. I agree with the authors' response and the fact that it must be possible to formulate complex queries such as Query 16 that can access all data, including retracted or non-authenticated one. I just wondered how frequent such queries will be, with respect to "simpler" queries interested in up-to-date and authenticated data only, and therefore advocated for having the trusted servers to provide further data / access mechanisms to simplify those "simpler" queries. I think the solution suggested by authors of adding additional triples in the administrative graph (e.g., to materialize checks precomputed by servers) can be appropriate to that respect, and it may be worth to mention that in that paper when commenting on the complexity of Query 16.

I thank the authors for all their clarifications to the minor comments I made in my previous review. I'm fine with their answers, esp. regarding leaving key invalidation as future work and the statement-based organization of forms justifying the way 'statementOrder' is used (I imagine that even if a placeholder is used in multiple statements, the end user will be asked to fill it only once in the form).

Overall, I believe the paper has improved and all of my concerns have been addressed, in the paper or the response and in some cases (which I agree with) leaving them as future work. The few additions to the paper I'm suggesting in this review are very minor and the manuscript is already fine as it is now, so I recommend acceptance.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SEMANTIC MICRO-CONTRIBUTIONS WITH DECENTRALIZED NANOPUBLICATION SERVICES
Review round: 2
Reviewer: 2

Output:
Basic reporting: The presentation in the revised version was improved, with enough references, contextual information, figures and structured data.

More details in the general comments.
Experimental design: The experiments are clearly illustrated in the revised version, as well as the research and technical questions.

More details in the general comments.
Validity of the findings: Data and code are freely available and the supplementary material is rich.

More details in the general comments.
Additional comments: The revised version of the article looks very clear to me, and the rebuttal was satisfying. The introduction, in particular, explains very well how the article is positioned in the literature and how its different parts are connected each other (which were my main concerns on the first submission). I also like the term micro-contributions. On the same line, this new version clearly explains the relation between this work and the previous ones by the same authors.

In the current form, the article looks ready for publication in my opinion. Yes, it is focused on technical and implementation aspects but I think it is a nice piece of work in the overall framework of nanopublications.

Just some (very) minor points:
   - lines 52-54: the paragraph looks a bit convoluted, please check
   - lines 183-185: some references to research works on retraction would further complete the article
   - lines: 248-251: add a very brief description on what can be done with each service; the text mentions how the services are implemented and the list of possible queries is right after but some anticipation would help the reader
   - lines 505-508: if I understood correctly from the rebuttal, the authors do not have information about the background (role, position, background knowledge, mastered tools, etc.) of the testers. Is that true? If you have any details, it would be helpful to include them in the paper
   - lines 508-: I would include the suggestions of the authors about the Nanobench interface, even if they were given only by a few testers, just to complete the picture


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: SEMANTIC MICRO-CONTRIBUTIONS WITH DECENTRALIZED NANOPUBLICATION SERVICES
Review round: 2
Reviewer: 3

Output:
Basic reporting: No comment.
Experimental design: No comment.
Validity of the findings: No comment.
Additional comments: I want to thank the authors for the answers provided.
The changes carried out on the paper satisfy the issues highlighted on the first version of the manuscript.
The only doubt that I still have is the research contribution of this work.
Actually, I do not have anything against papers having a stronger technical contribution rather than a research one.
I only want to highlight this aspect to the journal's Editors in order to judge the paper properly with respect to the journal's policies.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACCELERATED IMPLEMENTATION FOR TESTING IID ASSUMPTION OF NIST SP 800-90B USING GPU
Review round: 1
Reviewer: 1

Output:
Basic reporting: Paper is reasonably well structured, but the text is sometimes hard to read. I would suggest to seek the help of a native English speaker.  If not possible, the general recommendation is to use shorter sentences which are easier to understand.
Experimental design: Authors propose a GPU implementation of subset of tests specified in NIST SP 800-90B. As I understand, the main performance bottleneck (in the straight-forward implementation) is utilization of GPU multi-processors. Only 8 CUDA blocks are scheduled if "Parallelization Method 1" is used. The authors propose to solve the problem by parallelizing across statistical tests - "Parallelization Method 2". They further suggests to combine several tests and execute them jointly to reduce load on GPU global memory. Authors suggest that 2x speed-up is achieved with these optimizations. I believe authors are right looking for performance in this 2 particular areas. But I think this is only first steps of the work and significantly higher performance gains are possible if authors take performance analysis and optimization in a more systematic way.

Even using the proposed optimization, many recent GPUs will be significantly under-utilized. With 72 blocks, "Device A" will be running at about 30% occupancy (64 warps at each of 30 SMs at maximum occupancy). Furthermore, according to table 5 "collision test statistic" takes order of magnitude more time than all other tests combined. Consequently, even more hardware resources will be idling while the 8 slowest blocks are computing collision statistics.
1. First of all, this aspect is not discussed in the paper. It should.
2. Then, there is a rather easy approach to increase occupancy if GPU memory allows it: just detect amount of available global memory and increase "N" (number of iterations) accordingly. Authors should discuss this option and explain why they decided against it.
3. It might be possible to further increase occupancy using a "parallel reduction" approach. Multiple threads may be assigned to compute a single test. It could be considerable work to optimize all tests, but it seems that majority of time is taken by "collision test" and it seems reasonable to invest time optimizing at least this one.
Validity of the findings: The performance-evaluation over-focus on work-to-thread mapping and register usage. This is only one, while important, aspect affecting the performance. From the presented analysis, it is even unclear if register usage play any considerable role. Even if they are, the included discussion is not rigorous enough. As I understand authors just present number of registers reported by NVIDIA compiler. This information can't be interpreted straightforward. CUDA optimizer generates code based on multiple constrains. Compiler often tries to reduce the number of used registers, but to do it the compiler might increase amount of computations (by re-computing some variables on the fly instead of assigning registers to them) or some variables might be spilled in the local memory. And this is only the simplest effect. There are more uses for registers. For instance, the registers are also used for streamlining access to the global memory. Even if amount of used registers have not increased, the compiler might have sacrificed something to allow higher occupancy. I don't think that it is happens in this particular case, but argumentation is not rigorous. Being not rigorous it does not contribute to the value of paper and should be rather omitted than presented in such form.
Additional comments: - The benchmarking procedure is not explained. Particularly, it would be interesting to learn if authors had considered effects of GPU boost while measuring performance. The clock rates of NVIDIA GPUs vary significantly depending on load and chip temperature and might introduce significant fluctuations in the measurements if not handled with care.

- The execution time in Table 5 sums up to 10.22. However, Table 4 gives only 9.38 for "Device A". Yes, I understand that fluctuations happen, but this is almost 10%. The same speed-up is claimed in the section "Coalesced memory access". How authors can be sure that it is a real effect of optimization and not another fluctuation in measurements?

- While comparing with CPU-version, it is interesting to see how optimal is the CPU version. Can we tune a CPU version instead of including GPU and get the same result? I.e. is it multi-threaded or single-threaded? Are performance optimizations considered in the code or not, e.g. are SIMD instructions used?

- The summary is a bit overstates the achieved results: "When applied to an IID noise source, the proposed program was 10 times faster than the NIST program written in C++. Moreover, for a non-IID noise source, our proposal improved the performance up to 23 times." According to table 7, 10 times was only achieved for sample sizes of 1- and 4-bits. The performance was only twice faster for the 8-bit use-case which was mostly discussed in the paper. Actually, it would be interesting to learn why the GPU performance favors 1- and 4-bit sample sizes. Considering GPU architecture, I would rather expect the opposite outcome.

- In the introduction, major motivation for work is that IID test needs to be executed repeatedly and, for this reason, performance is crucial (p.2 l. 73). However, as I understand once source is determined to be non-IID no further tests are needed. In the conclusion, the authors focus on speed-up for non-IID use case. But considering the Introduction this speed-up is relatively unimportant. The practical use-case is for IID tests. I think this should be made clear in the conclusion.

- Authors does not explain how bzip2 test was handled when the performance was compared between CPU and GPU implementations. There is no GPU version for this test. Have the GPU implementation executed 1-test less than the CPU? Or was the CPU-based bzip2 test executed in GPU-implementation?  Or was bzip2 test excluded from both CPU and GPU implementations?

- Furthermore, it would be interesting how much time is required for bzip2 test compared to all other tests as presented in Table 5.

- On Fig.2 (and some others), letter "T" is used to denote 1) number of threads in block 2) result of statistics calculation. This is a bit misleading. I suggest to use different letters.

- In captions of Table 4 and 5, the authors state "number of CUDA blocks = 8". I guess this is a typo. Earlier authors say that 8 blocks are used for "Parallelization Method 1", but 72 for the optimized version (and 144 for method 2).

- p.4 l.141: typo "10,0000" => "10,000"


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACCELERATED IMPLEMENTATION FOR TESTING IID ASSUMPTION OF NIST SP 800-90B USING GPU
Review round: 1
Reviewer: 2

Output:
Basic reporting: The paper proposes a GPU-based parallel implementation of the NIST SP 800-90B IID assumption testing, and carefully designs the implementation logic and finely control the resources. The proposed implementation shows several times speedup than the NIST package.
1) Overall, the paper is well written and easy to understand;
2) The introduction and background are concise, and give a clear mention of SP800-90B;
3) The presentation of algorithm and optimizations are good and ordered in a comprehensible fashion,
4) Figures and tables are comprehensive.

Here are some points that should be improved:
1) SP 800-90B relevant background is sufficient, but the preliminary of GPU and CUDA programming are absent that may impact on its readability for readers that are unfamiliar with GPU;
2) There are plenty of prior GPU-based cryptography relevant literatures, which should be appropriately referenced and reviewed. Some of the prior literatures should be updated to the state-of-the-art ones, e.g., Manavski, 2007; Szerwinski and Gu¨neysu, 2008;
3) The authors should explain the motivation further. The authors say that “the NIST program requires approximately 1 h to estimate the entropy of one noise source. Therefore, at least 20 h are required to analyze (20 noise sources)” (1st paragraph, Page 3). Why must the 20 tests execute serially? How about running the 20 tests in a 10-core hyperthreading CPU? Besides, as far as I know, SP 800-90B is a professional test mainly used by CMVP and its accredited labs, and only applied several times for each type of entropy. And it is an off-line test and rarely used in run-time of an RNG. All the above reasons make performance not a vital concern for SP 800-90B.
4) The basic situation of the NIST package should be clarified. “Written in C++/Python” is just not enough. Extra information should be introduced to reflect implementation level of NIST package, e.g., AVX acceleration, multi-thread feature etc. 23 times speedup sounds fine, but it is based on how good the reference is.
Experimental design: The basic idea of the experimental design is straight and clear, but it can be improved in two aspects:
1) the paper presents several optimization techniques, e.g., statistical tests merging and Memory coalescing. But there is no incremental analysis for the proposed solution to demonstrate how each optimization impacts the overall performance of the proposed solution.
2) the paper only compares the results with the CPU-based NIST package, but lacks of comparisons with relevant GPU-based applications. I cannot tell whether it is a high-level GPU-accelerated application.
Validity of the findings: It is a common sense that GPU-accelerated applications have great performance advantage over the typical CPU implementations. However, the performance advantage of this paper is not clear.
1) The authors claim “the experimental results demonstrate that our method is ‘at least’ 23 times faster than that of the NIST package”. However, Table 7 reports “up to” 22.5 times speedup than the NIST package.
2) And the paper does not report the experiment setups for the NIST programs, e.g., single-thread, or multi-thread, etc. According to Table 7, performance advantage is 2 to 23 times, in most of cases, 10 times. However, if the such advantage is achieved by comparing with single-thread NIST package, the result is disappointing.
Additional comments: I thank you for providing the source code, but I wonder whether the if-else semantics in the source code can run as expected under the SIMT architecture, i.e., the different statistical tests can really run concurrently as shown in Figure 7. I think the multi-stream technique may be more appropriate.


Instruction: You are one of the reviewers, your task is to write a review for the article. You will be given the title of the article, the number of the round in which the article is located, and your order among the reviewers.

Input:
Title: ACCELERATED IMPLEMENTATION FOR TESTING IID ASSUMPTION OF NIST SP 800-90B USING GPU
Review round: 1
Reviewer: 3

Output:
Basic reporting: The paper mostly explains the general executions of the prior NIST algorithm on GPGPU platform. It is obvious that the parallel platform like GPGPU will enhance the execution speed of NIST considerably. However, the contribution of this research work to improve the NIST algorithm is very significant. At the same time, Coalesced memory access technique is a very small contribution in this work. Furthermore, the structuring of this paper is very confusing. Hence, the paper needs to be improved with further contributions and it requires complete restructuring.
Experimental design: The shown experimental design in this paper is not very innovative.
Validity of the findings: The findings are good but does not meet the expections.
Additional comments: Please try to improve the existing algorithm for claiming your contribution.