{'fraction_non_alphanumeric': 0.014578569095066971, 'fraction_numerical': 0.0026543613198301208, 'mean_word_length': 5.287291399229781, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
A Study on the Quality Assessment Improvement Scheme of Realistic Representation Effects based on User Quality Assessment

Realistic media, which allow people to feel presence and reality by stimulating the five senses, are new forms of media differentiated from the existing media. While ultra-high definition content enables people to feel presence and reality by maximizing the visual effect that people can feel, realistic media are the contents that offer a high level of presence and reality by providing auditory, olfactory and touching effects as well as a visual effect. The realistic representation effect represents the phenomenon occurring physically in the real world using the realistic representation effect device. The realistic representation effects that frequently occur in realistic media contents include light, water, wind, vibration, movement, etc. The process of creating a realistic representation effect and inserting it into content is complying with MPEG-V standard. The realistic representation effect is complying with MPEG-V Part 3 and the physical effect additionally generated can be added to the future standard and used1,2. Realistic media content producers create metadata in the form of XML in accordance with MPEG-V standard to create a realistic representation effect. In providing users who feel the realistic representation effect with reality and presence, the synchronization with the visual and auditory effects is important. However, a realistic representation effect is produced with the subjective judgment of producers. The realistic representation effect used to provide users with immersion and presence sometimes interferes the immersion of realistic media contents. The previous work measured the QoE of a Abstract


Introduction
Realistic media, which allow people to feel presence and reality by stimulating the five senses, are new forms of media differentiated from the existing media. While ultra-high definition content enables people to feel presence and reality by maximizing the visual effect that people can feel, realistic media are the contents that offer a high level of presence and reality by providing auditory, olfactory and touching effects as well as a visual effect. The realistic representation effect represents the phenomenon occurring physically in the real world using the realistic representation effect device. The realistic representation effects that frequently occur in realistic media contents include light, water, wind, vibration, movement, etc. The process of creating a realistic representation effect and inserting it into content is complying with MPEG-V standard.
The realistic representation effect is complying with MPEG-V Part 3 and the physical effect additionally generated can be added to the future standard and used 1,2 . Realistic media content producers create metadata in the form of XML in accordance with MPEG-V standard to create a realistic representation effect. In providing users who feel the realistic representation effect with reality and presence, the synchronization with the visual and auditory effects is important. However, a realistic representation effect is produced with the subjective judgment of producers. The realistic representation effect used to provide users with immersion and presence sometimes interferes the immersion of realistic media contents. The previous work measured the QoE of a realistic representation effect with a limited environment and specific content by selecting a certain population to improve the quality of the realistic representation effect. Thus, the existing research technique can maintain a high level of objectivity by having users practice the assessment method. As its drawbacks, however, it has less diverse contents, realistic representation effects and evaluators. In order to overcome these limitations, therefore, a quality assessment that can reflect the assessment contents of diverse users and objectively assess the realistic representation effect of the existing realistic media contents is needed. In addition, a quality improvement scheme from the quality assessment is needed as well.
In this paper, an assessment was made on the realistic representation effect of realistic media contents with unspecified users being evaluators, and the quality improvement scheme, where the realistic representation effect in the form required by users was available by modifying the attribute value of the realistic representation effect metadata that were already created, was investigated. This paper is organized as follows: Section 1 describes the overview of the research, Section 2 presents various forms of quality improvement assessment schemes, and Section 3 discusses the quality improvement scheme of the realistic representation effect proposed by this paper. Section 4 draws a conclusion on the quality improvement scheme of the realistic representation effect and describes the direction of future works.

Relate Work
The QoE of a realistic representation effect is the quality of services subjectively determined by users who are provided with realistic media content. The QoE is affected differently by the experience and environment of users. The realistic representation effect can be negatively applied or reflected as a strongly positive factor when it is provided to users who have not experienced the effect before. In addition, since it stimulates the 5 senses of human being, it is not possible to provide the same realistic representation effect to users with a variety of characters. The providers of realistic media contents use a variety of realistic representation effects to maximize presence and reality during the process of creating contents. Since the realistic representation effect provides a physical effect occurring in the real world with the realistic representation effect device, users experience a different level of the QoE depending their experience. Thus, in order to maximize the realistic representation effect, the characteristics of user groups, including gender, age and the number of experience of users, need to be identified. A research needs to be conducted on the technique that can provide realistic media contents by identifying the realistic representation effect preferred by each user group.
The existing schemes that assess the quality of realistic representation effects are based on ITU-T P.910 standard 3 . ITU-T P.910 was derived from the standard created to assess the QoE of multimedia and to assess the subjective quality of videos. The numeralization of the result of user's subjective assessment of the QoE is referred to as MOS (Mean Opinion Score) and its representative assessment method is ACR (Absolute Category Rate). ACR is an assessment method, where the target content of assessment is designated and the scenes of less than 10 seconds are shown repeatedly. The scale of assessment is in the 5 steps of very bad, bad, average, good and very good.
Waltl et al. presented a web-based system to assess the QoE of a realistic representation effect [4][5][6] . They employed such genres as action, cartoon and documentary and sport and used the realistic media content with shorter than 30 seconds realistic media contents to assess the QoE. As assessment methods, they used DCR (Degradation Category Rating) and ACR-HR (Absolute Category Rate with Hidden Reference) 7 .
In the DCR assessment method, an assessment is made with a repeated process where the realistic media that don't include realistic representation effects are shown, followed by 5 seconds of holding, and then the realistic media that include realistic representation effects are shown. In the ACR-HR assessment method, an assessment is performed every 15 seconds by consecutively showing the realistic media including realistic representation effects. Since the realistic representation effect assessed in the previous work used only the effect of light, it had a limitation in complexly assessing the QoE with various realistic representation effects. A total of 20 subjects (11 men and 9 women) aged 20-30 participated in the quality assessment. The result of quality assessment showed that the assessment on the realistic representation effect varied depending on the genre. Although there were a lot of positive responses to the realistic representation effect in the genres of action and sports, negative factors were found to the realistic representation effect in the genres of comedy and documentary. In particular, it was found that the content with a more dynamic movement, such as the genre of sports, showed a positive response to the realistic representation effect. In the genre of documentary with more static scenes, on the contrary, there were many evaluators who showed negative or no responses to the realistic representation effect. As for various genres, thus, a research is needed to categorize the positive and negative realistic representation effects by performing quality assessment because the realistic representation effect varies depending on the genre of contents. In addition, a quality assessment needs to be made depending on the various forms of realistic representations effects. Moreover, a correlation analysis for each realistic representation effect is needed because more than 2 realistic representation effects are complexly provided in a single scene in realistic media contents.
As a work on the quality assessment of realistic representation effects, Murray et al. investigated the impact of the synchronization of the olfactory effect, among the realistic representation effects, and realistic media contents on the perception of users 8,9 . Consistent with the previous research, they assessed the quality of realistic representation effects using ACR and DCR. Since, however, the research was on the effect of the synchronization of realistic representation effects and realistic media, an experiment was conducted in the interval of 5 seconds by discordantly setting the synchronization time of each scene and olfactory effects from -30 seconds to 30 seconds. In addition, age, gender and culture were considered in order to analyze the impact of the olfactory effect in realistic representation effects on the perception of users. The questions for assessment in the previous research were as to whether the realistic representation effect was perceived either positively or negatively. Unlike the work by Waltl, the research performed an assessment by variously categorizing the questions on quality assessment. The first question was on the synchronization of the olfactory effect and scenes with 5 criteria to ask whether it was represented too quickly or appropriately. The second question was on whether the olfactory effect acted positively or negatively after it was represented in a certain scene. Finally, a question was asked as to whether the intensity of the olfactory effect was appropriate. The findings of the experiment showed that the time of perceiving the olfactory effect varied by age. It was found that it took longer for those who were older to perceive while those who were younger responded immediately. In addition, the time of response to the effect varied depending on ethnicity. It was also found from the experiment with gender that the response of men in perceiving the effect was slow, while that of women was sensitive. The experiment technique presented by the previous research has changed the existing recognition that scenes and realistic representation effects need to be performed in the same time. The result of the research suggested that age, gender and culture should be considered when creating realistic representation effects because the olfactory effect has the time of physical representation and its perception time varies individually.
This paper aims at investigating a technique to improve the QoE by performing a quality assessment based on the multiple evaluators and various contents, which were pointed out to be the limitation of the previous research.

Quality Improvement Scheme
The realistic representation effect used in realistic media is defined in MPEG-V Part3 and can express various physical effects. The process of generating SEM (Sensory Effect Metadata), the realistic representation effect metadata to maximize the presence of contents, is created by the subjective judgment of producers based on the intention to provide content users. The realistic representation effect reflecting the intention of content producers sometimes acts as a factor to maximize the presence to contents users, but it also acts as a factor to interfere the immersion in the contents. Thus, a system that can improve the attributes, such as intensity, interval and synchronization of realistic representation effects, is needed in the form desired by content users. In addition, a research is required on the quality improvement scheme to enhance the realistic representation effect metadata based on user assessment.
In order to assess the quality of the realistic representation effect used in realistic media contents, a quality assessment for the realistic representation effect is performed on users who have experienced realistic media contents. The quality assessment scheme in the previous research used the designated assessment method by selecting an assessment group in advance for quality assessment. Thus, it has an advantage that the objectivity of user quality assessment can be ensured, but it has a drawback that user assessment groups cannot be diversely secured. In order to improve the user QoE of realistic representation effects, a variety of QoE assessments need to be performed on users with experience in realistic media contents. Unlike the existing assessment method, an assessment is thus made on users who have experience in realistic media contents. The users who make assessments evaluate the realistic representation effect on the content that they have experienced. In addition, in case a direct assessment is not available, an indirect assessment scheme is used, so that the quality of realistic representation effects can be improved using the result of quality assessment by various users.
The workflow of the quality improvement system of SEM, the realistic representation effect metadata, is illustrated in Figure 1. The realistic representation effect metadata of realistic media contents are changed by the system in the form requested by users, depending on the result of user assessment. Evaluators refer to the users who have experienced realistic representation effects and perform an assessment on all the realistic representation effects expressed in the contents by each effect. As for the assessment items, an quantitative assessment is made as to whether the realistic representation effect felt while using realistic media contents was positive or negative, and the attribute value of the realistic representation effect is changed depending on the result of the assessment. In case a number of evaluators assigned (-) scores to a certain effect, the attribute value of the effect is applied to a lesser degree. In contrast, the number on the existing attribute value is adjusted upward in the realistic representation effect assessed with (+) scores. Since the scores of evaluator quality assessment change whenever the number of evaluators increases, the attribute value of realistic representation effects changes only when it is out of the designated range. However, the range of the attribute value that can be changed is differently set by each content because the characteristics of realistic media contents are diverse.
Evaluators quantitatively assess the realistic representation effects reflected on realistic media contents by individual effect. The results of the assessment by the evaluators are stored in User Assessment Database. User Assessment Database stores the assessment scores by the category of the evaluator, content and realistic representation effect. The reason for the categorization by evaluator is to classify and analyze the degree of evaluators' experience on the realistic representation effect by gender, age and region.
For the realistic representation effect of realistic media contents, its synchronization with scenes is important. Thus, using SEM Authoring Tool, in which a realistic representation effect can be produced while watching scenes, can create a realistic representation effect with a more accurate synchronization. As presented in Table 1, information on evaluators is stored in User Assessment Database when assessing the realistic representation effect. Evaluators perform assessments by accessing the assessment system using mobile devices to perform the assessments. The realistic representation effect is assessed by entering information on the evaluators, which is set to analyze the characteristics of evaluators, and selecting the contents that the evaluators have experienced. The evaluators assess all the realistic representation effects and perform assessments as to whether the realistic representation effects are positive or negative by quantitatively selecting the degree of evaluators' experience. It is possible to classify the realistic representation effects that are not suitable for a certain genre by analyzing the realistic representation effects provided by realistic media contents based on their attributes. It is also possible to produce a realistic representation effect by designating a specific user and to reflect a realistic representation effect with a high experience, thus making it possible to shorten the time to produce a realistic representation effect in the future.
User Assessment Database and SEM DB store realistic representation effects separately, as shown in Table 2. Intensity-Range is an attribute to set the range of intensity available for the realistic representation effect device. The Intensity-Range of a realistic representation effect device is set differently because there are various devices that can represent the same effect. The Intensity-Range is used as an attribute value to set the intensity of a realistic representation effect in a set range. If a range is set between '0 100, ' the intensity of the realistic representation effect device can be determined by setting an integer value in the range. Activate is an attribute that can temporarily deactivate the implemented realistic representation effect at the request of a user. It also can deactivate the effect by changing it to False in an environment where a realistic representation effect is not available. Fade is an attribute to set the realistic representation effect to be activated in a designated time. Given the characteristics of a realistic representation effect device, there exists a device that cannot stop perfectly at the designated time. Thus, Fade is an attribute that can maintain the synchronization with scenes by gradually reducing the effect at the designated time. In addition, the number of realistic representation effects can change by content and the attribute of realistic representation effect devices can be differently set depending on the characteristics of the devices. The workflow of a user assessment database is shown in Figure 3. The database receives information on evaluators and the results of quality assessments for realistic representation effects from evaluators and stores them in the table by categorizing them by attribute value. In addition, when the number of evaluators exceeds 20, the assessment scores are transmitted to SEM Analyzer. Upon completing the transmission, the scores are on standby until the number of evaluator reaches 20, and when it exceeds 20, they are added to the evaluation results of the existing evaluators and transmitted. The process is repeated until evaluators stop assessments. The already created realistic representation effect is stored in SEM DB, maintaining the same form of table with the user assessment database to store data. SEM DB has both file and table forms of XML and transmits the realistic representation effect data in the form of table at the request of SEM Analyzer.  The workflow of SEM DB is illustrated in Figure 4. SEM DB compares the already stored data and the data modified and transmitted by SEM Analyzer, and stores them in the table. After storing is completed, the attribute values of the XML file are changed into the changed table attribute values and stored. After all the processes, the changed realistic representation effect can be checked using SEM Authoring Tool. In case the effect is much far from the intention of a producer, the producer can change it again. The attribute values changed by the producer can be set not to be modified, and when it is not designated, it can be re-modified according to the assessment results of evaluators.  The workflow of SEM Analyzer is described in Figure  5. SEM Analyzer compares the data in the user assessment database and those transmitted from SEM DB. When the result of the analysis is (-) value, it is transmitted to SEM DB by changing the existing data into a changed attribute value. However, when the result is (+) value, it is returned after inserting a record that the data were not changed. SEM Analyzer can identify the fact that the already produced realistic representation effect maintains high quality by many evaluators and can be used as a technique to draw a content genre suitable for the realistic representation effect.

Conclusion
Realistic media are the media that provide users with immersion and presence by adding a realistic representation effect to an ultra definition video. Realistic representation effects are used to provide immersion and presence to users, but the QoE of users varies. Since a realistic representation effect is created by the subjective judgment of content producers, it can acts as a negative factor in using contents due to the provision of a realistic representation effect that is not desired by users. Although it is not possible to produce a realistic representation effect by taking all the tastes of countless users into account, it is feasible to improve the QoE in such a way that it is generally accepted.
The previous work performed a QoE assessment as a way to improve the QoE. It assessed quality in a limited way by selecting a population from specific contents. The existing quality assessment technique has an advantage that the assessment results of evaluators can have objectivity and consistency because it performs a limited quality assessment. However, it could not reflect the opinions of various users because the number of evaluators is small. Therefore, a research is needed on the technique that can perform a quality assessment of realistic representation effects by providing an assessment opportunity to all users who have experience in using realistic media contents, not the quality assessment technique for specific evaluators.
In this paper, we investigated a technique to improve the quality of the already produced realistic representation effect metadata by performing a quality assessment of realistic representation effects. The assessment was performed on all the users with experience in realistic media contents, using mobile devices for a better accessibility. The results of the quality assessment by evaluators are reflected on the existing realistic representation effect and modified through the quality improvement system. It is now possible to analyze which realistic representation effect has a higher haptic effect in each genre of contents using the assessment results from various evaluators. Thus, it is possible to produce realistic media contents according to the characteristics of users because the realistic media contents are produced based on the analysis results.
As future research agenda, we intend to perform a quality assessment for realistic media contents using the postings on social networks of users using realistic representation affects and work on the technique and system design to improve the quality of realistic representation effects. In addition, a research is needed on the technique to check whether the results of transmitting the modified realistic representation effect metadata through the realistic media transmission system are accurately reflected and to improve the QoE by considering the characteristics of the realistic representation effect device.