{'fraction_non_alphanumeric': 0.01821725131333672, 'fraction_numerical': 0.005931198102016607, 'mean_word_length': 4.889720558882235, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Sentiment analysis system for movie review in Bahasa Indonesia using naive bayes classifier method

There are many ways of implementing the use of sentiments often found in documents; one of which is the sentiments found on the product or service reviews. It is so important to be able to process and extract textual data from the documents. Therefore, we propose a system that is able to classify sentiments from review documents into two classes: positive sentiment and negative sentiment. We use Naive Bayes Classifier method in this document classification system that we build. We choose Movienthusiast, a movie reviews in Bahasa Indonesia website as the source of our review documents. From there, we were able to collect 1201 movie reviews: 783 positive reviews and 418 negative reviews that we use as the dataset for this machine learning classifier. The classifying accuracy yields an average of 88.37% from five times of accuracy measuring attempts using aforementioned dataset.


Introduction
Information presented in cyberspace is now more diverse and the media used means in the process of information diffusion are growing. One of the main media used in the process diffusing information in cyber media is text or document media. The ability in order to extract information from documents is absolutely necessary. The method of extracting information from data in the form of documents is known as text mining. Over 80% of information is currently stored in the form of text, so that text mining is believed to have high commercial value potential [1]. describe the steps taken in text mining: tokenizing, filtering, stemming, tagging, and analyzing [2]. Sentiment analysis is one of the new branches in the domain of text mining or data extraction in the form of text, consisting of processing and extracting textual data automatically in order to obtain information [3,4,5,6,7]. Sentiment analysis can be utilized as a tool in seeing the public response of a particular event, either positive or negative response, so that the next strategic steps can be undergone immediately. An example of sentiment data in the form of document is movie review from various sites on the internet. Reviews obtained from movie reviews sites can be used a reference for movie fans to know recommended and also a medium for movie producers to know the public responses towards the movie released. Movie review can be divided into a number of categories based on the sentiments contained in the document.
There are some difficulties in developing sentimental analysis study on movie review using Bahasa Indonesia partly due to the lack of studies on movie review documents using Bahasa Indonesia and the lack of corpus or textual literature for movie review using Bahasa Indonesia as the original language of the document.
Based on the description above, the researchers are interested in creating a system which is capable of categorizing movie review in Bahasa Indonesia into two categories of positive and negative sentiments using the method of Naïve Bayes Classifier [8,9]. Naïve Bayes Classifier method is one of the supervised learning models based on statistics and probability with high level of accuracy [10].

Result and Discussion
The steps in this study include the step of literature review, data collection, and system development. The method employed in system development is the method of Naive Bayes Classifier [11]. The flow diagram of the study steps is illustrated in Figure 1.

Data Collection
The data collection is performed in order to obtain the information needed to determine the system requirements. The data used in the study is divided into two categories based on the data source: primary data and secondary data. The primary data required in the development of the Sentiment Analysis System for Movie Review is a dataset of movie review. The movie review dataset is made by the researcher using a movie review obtained from the movie review web page of Movienthusiast. Secondary data required in this study is the root words in the Great Dictionary of the Indonesia Language (KBBI) and stop words dictionary or set of words which are considered less important and is able to be removed from the movie review text [12,13,14]. The dictionary of stop words used comes from the study conducted by Tala.

The Naive Bayes Classifier Method Implementation
The Sentiment Analysis System for Movie Review which is developed implements the Naive Bayes Classifier method in classifying the movie review documents into two classes: positive sentiment class and negative sentiment class. The flowchart regarding the overview of the Sentiment Analysis System for Movie Review is illustrated in Figure 2 The explanation on Figure 2. above is: firstly, the derived from the movie review is submitted by the user one by one. The submitted review may be categorized into training data or test data based on the user's wishes. The system, then, executes the document extraction process in order to generate a collection of terms from each review text and so-called bag of words. The next step is to train the system. At this step, the system constructs a model of the training data by calculating the occurrences of each term in the training data and likelihood value [15]. The result of the system training process is a table of feature sets containing the number of occurrences and likelihood values of each term in the training data. The likelihood value is, then, incorporated into the next step which is the calculation of the system accuracy [16].
The system performs the classification process using the Naive Bayes Classifier method in order to obtain the best class of sentiment of each review in the category of data train. The results of the accuracy calculation are the sentiments of the classification of each test data review and the confusion matrix table obtained from the original sentiment comparison of the test data review and the sentiment of the analysis result. System accuracy can be concluded from the confusion matrix. The planning steps which are implemented based on the waterfall method [17].

Requirement Analysis
System requirement is the ability of the system to meet the condition desired by the users. System requirement analysis is performed by grouping the needs into functional requirements and nonfunctional requirements.

Functional Requirements
The functional requirements of the Sentiment Analysis System for Moview Review are as follows: 1. The system can manage (add, read, update, and delete) movie review datasets, 2. The system can manage (add, read, update, and delete) the dictionary of root words in Bahasa Indonesia, 3. The system can manage (add, read, update, and delete) the dictionary of stop words, 4. The system can perform the classifier training process and display the model in the form of feature sets of the term data from the training data, 5. The system can display the test data result and display confusion matrix generated from the classifier testing, 6. The system can display a set of movie review dataset terms derived from tokenizing, filtering, and stemming processes, and 7. The system can display sentiment analysis result derived from reviews submitted by users.

Non-Functional Requirements
Meanwhile, the non-functional requirements of the Sentiment Analysis System for Movie Review are as follows: 1. The system uses an authentication through login process in order to differentiate user level. 2. The system can run in various web browsers which support the system environment, 3. The system gives a fast response, and 4. The system has a user-friendly interface design.

System Design
The design of the Sentiment Analysis System for Movie Review uses Unified-Modeling Language (UML) as the modeling language and object-oriented programming concept. The documentation of the created system includes:

Business Process
Business process diagram describes a series of inputs, needs, and outputs processed by the system in order to produce the desired destination by the user. Description of business process system can be seen in Figure 3.

Use case Diagram
Use case diagram describes the system's functional needs in the form of diagram. Use case is needed to describe typical interactions between users and system. Use case diagram of the Sentiment Analysis System for Movie Review is illustrated in Figure 4.

Deployment Diagram
Deployment diagram is a diagram illustrating the physical layout of the system. This diagram is included in order to describe the hardware and software required for the system to run according to the system needs. Deployment diagram of the Sentiment Analysis System for Movie Review can be seen in Figure 5. Sentiment Analysis System for Movie Review is developed as a web-based application. The webpage of Sentiment Analysis System for Movie Review is divided into two levels based on the user level: private page for the administrators and public page for the visitors. The private page contains the management dataset menu of the movie review, the stop words management menu, the training system menu, the accuracy calculation menu, and menu containing the set of terms from each movie review on the dataset. The public page contains an interface for visitors wanting to obtain sentiment analysis result from the movie reviews within the system. The main features displayed in the discussion are the dashboard page, system training page, system accuracy calculation page for the visitors.

System Interface Implementation`
The dashboard page is the first accessed page once the administrator has successfully logged in to the system. The dashboard contains brief information on datasets, such as the number of review for each category, the ratio of training and test data as well as the ratio of positive and negative training data. The information is displayed in the form of numbers and graphs of donut charts. System training page contains a table on the occurrences and likelihood values of each term in all train data. The process of system training in order to form feature sets is performed by the administrators through the interface page by selecting the button of "Latih".
The accuracy calculation page contains a list of reviews which become test data within the dataset, probability of the positive sentiment and negative sentiment for each test data, the result of the class analysis with the highest probability, and the accuracy value of the analysis result towards the original sentiment of the reviews. In addition to the review table of the test data, there are accuracy values of the system's last accuracy test result which have been converted into percentage format and confusion matrix table of the last accuracy test result. Accuracy calculation process is also initiated by the administrators through the interface page by selecting the "Hitung (calculate)" button.
The visitor page is the page which is for the visitors. The visitors can access this page in order to enter the review data they want to know the sentiment of. After selecting the "Lihat Sentiment (view sentiment)" button, the system classifies using the Naive Bayes method in order to obtain positive and negative sentiment probabilities as well as the sentiment result obtained based on the probability ratio of the two sentiment classes.

Accuracy Calculation Result
The testing process is conducted five times. The first test was conducted by selecting 600 reviews as the training data and 200 reviews as the random test data. Further tests were performed by adding 100 movie reviews as the test data and test data of 20 movie reviews into the dataset utilized for the previous test. Details of the composition of each training and test data for each test process can be seen in Table 1.