{'fraction_non_alphanumeric': 0.02642503592527543, 'fraction_numerical': 0.013172600989940922, 'mean_word_length': 5.3395748987854255, 'pattern_counts': {'":': 0, '<': 2, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improving creative problem solving performance of mathematics students by digital multimedia in graph theory course

Integrating learning through digital multimedia can be a disruptive learning innovation in high education. One subject in Mathematics that can has integration with the digital multimedia is graph theory. The reason is that some graph theory topics has application in the real world, hence students implementing these topics to solve problems can be analysed using creative problem solving (CPS) performance. Topics in Graph theory to be analysed by CPS are minimum spanning tree, traveling salesman problems and shortest path. Main objective of this study was to investigate the effect of the integration of digital multimedia to improve creative problem solving performance of mathematics students. Sequential mixed method was used in the study. Students are divided into 2 groups: experimental group that has integration of digital multimedia in the teaching-learning, and control group that has conventional learning. Phase of this study are 1) involving quasi-experimentation to determine the effect of the integration of digital multimedia to improve creative problem solving performance, and 2) exploration of creative problem solving performance through qualitative interviews. Instrument to measure creative problem solving are direct assessment using CPS indicators: understanding the challenge, generating ideas, preparing for action, and planning approach. Finding from this research is a significant increase creative problem solving performance of mathematics students by integrating graph theory course with digital multimedia. Student’s ability in generating ideas and preparing for action mostly is good. Student’s perception about the implementation of digital multimedia is that 67.7% of students very agree that learning activities improving creative problem solving. This digital multimedia implementation can be used as an alternative to disruptive learning innovation in mathematics learning.


Introduction
In order to respond to the rapid development of information technology and the industrial era 4.0, it is a challenge in the world of education, especially in higher education, to develop learning innovations. The development of educational devices must follow the latest development to respond to the need for the emergence of the 4.0 industrial revolution (based on the use of cyber-physical). The use of cyberphysical integrates Information and Communication Technologies (ICT) and humans to get solutions, solve problems and find new innovations. Digital technology integration can be seen in previous researchers [1], integration of android applications in education 4.0 [2], integration of digital learning in Industry 4.0 [3], industry challenges 4.0 [4,5].
Integrating ICT by developing digital multimedia in learning is needed to keep pace with industrial revolution 4.0. Learning will be more effective and efficient by integrating ICT. The integration of ICT in learning is of concern to world researchers, for example digitalization of interactive learning can be seen in previous researchers [6,7,8,9]. The use of ICT in learning mathematics can be seen in previous researchers [10,11,12], and the use of mobile learning in learning mathematics mentioned in [13].
Other studies linking creativity and creative digital multimedia are creative approaches to educational problems [14], the development of creative digital multimedia in higher education to increase creativity [15], and creative digital in learning process in schools [16]. Creativity is also associated with the development of ICT for example creative through ICT [17,18]. Meanwhile the development of creative learning with e-learning can be seen in previous researches [19,20].
The trends and issues of multimedia learning can be seen in [21], the usefulness of multimedia digital video objects in [22], and use of interactive multimedia [23]. Other researchers are interested in efficient multimedia [24], while the effects of multimedia can be seen in [25,26,27]. Other researchers explore the use of digital multimedia [28], digital multimedia in mathematics learning [29] and [1], effects of multimedia on creativity [30], and improvement of mathematical skills with digitalization [31].
The integration of ICT in learning innovation is currently being developed at Malang State University. The new paradigm of learning development at Malang State University is being sought to shift, among others, to capability development models, life-based / real world-based learning, curricular services that are multi-disciplinary based, with a new orientation to the growth of science and technology disciplines and 21 st century skills. One of the courses in the UM mathematics department that can be developed into life-based / real world-based learning is Graph Theory courses. The reason of choosing Graph Theory is due to its practicality in real life. Topics in graph theory that have a wide field of application in real life problems is minimum spanning tree problems, shortest paths, and traveling salesman problems. Learning graph theory with digital multimedia can be also associated with increased problem solving abilities such as the influence of web learning on problem solving strategies [32], and Creative Problem Solving (CPS). CPS on blended learning can be seen in previous researchers [33,34].
Digital multimedia implementation in graph theory course has not been done previously by other researchers. Solving real problems by students varies in ways, processes and results. It is necessary to study more deeply the integration of digital multimedia in graph theory course and their effects on student creative problem solving performance. How to improve the learning outcomes of graph theory for mathematics students by implementing digital multimedia? How was the CPS indicators description of mathematics students' criteria by integrating digital multimedia in graph theory course? What was the perception of students with the implementation of digital multimedia in lectures? These results will be examined in this article.

Research Methods
Sequential mixed methods were used in the study. The first phase of the study was quasi experimental with control and experimental groups, as well as pre-and post-tests. The first phase of the study was to determine the effect of the integration of digital multimedia to improve creative problem solving performance. The second phase of the study was exploration of creative problem solving performance through qualitative interviews. An experimental group (N = 59, 20 male and 39 female) used digital multimedia in the teaching-learning based on Powtoon application. In the control group (N = 61, 15 male and 46 female) conventional learning was performed. Quantitative data were collected in the form of pretest and posttest using Kahoot application. The instrument test was validated by an expert. Qualitative data were collected from problem solving results from assignments about real problems in minimum spanning tree, shortest path, travelling salesman problem, with short videos based on Powtoon applications. Qualitative data were taken from task-based interviews using a semi-structured format. The instrument of the research is used to portrait problem solving skills by using creative  [35]. Instrument to measure creative problem solving used indicators: understanding the challenge, generating ideas, preparing for action, and planning approach. Quantitative data was analyzed by Minitab. Qualitative data were analyzed in some points: categorization, reduction, exploration, interpretation, and drawing conclusion.

Analysis of improvement in Creative Problem Solving mathematics students
Analysis of the data for the first research question is about how to improve the learning outcomes of graph theory mathematics students by implementing digital multimedia. The experimental group use pretest and posttest based on Kahoot application, and for control group use paper-based test. Homogeneity test of pretest for the control class and the experimental class can be seen in Table 1.
Analysis of the homogeneity data shows that there is no average difference between pretest for the control class and the experimental class. Individual value plot of pretest for the control group and the experimental group can be seen in Figure 1. Average difference between posttest of the control class and the experimental class can be seen in Table 2. Table 3 depicts effect of the experimental group treatment by implementing digital multimedia in learning process. Analysis of the test results that average difference is p<0.05. This shows a significant increase in student learning outcomes by implementing digital multimedia.   T-Test of mean difference = 0 (vs < 0): T-Value = -32.30, P-Value = 0.000. This means that with a 95% confidence level it can be stated that the average pretest value is smaller than the average posttest value. So that the treatment given to the experimental class has a significant effect on increasing the value from pretest to posttest

The description of the Creative Problem Solving indicator
The task of graph theory case studies of minimum spanning tree, shortest path and traveling salesman problem is used to analyze the second research question. The CPS criterion of mathematics students by integrating digital multimedia based on Powtoon application in graph theory course, shown in Figure 2. It shows that the average is good for all CPS components. Figure 3 and 4 are samples of two components: appraising tasks component and the data exploring component. Both diagram show that 51% students marked as very good.

The Student's Perception about implementing digital multimedia
Data for the third research question was collected by given a questionnaire to students about the implementation of digital multimedia in problem solving. About 67.7% of students have perceptions that they are very agree about learning activities can improve creative problem solving. The results of the description are shown in Figure 5.   Table 3 with an average pretest of 61,627 and post test average of 83,627 this shows a significant increase with a confidence level of more than 95%. Finding from this research is a significant improvement of students' Creative Problem-Solving by implementing digital multimedia in learning graph theory. These findings confirm the previous studies [32,30,33].
Refer to the previous research about Engineering Education using Web-Based Interactive Multimedia Applications [36], interactive learning using digital assistance [37], the effects of webbased professional improving teachers' Problem Solving Strategies [32], effective integration of interactivity into future online courses [9], and the use of digital multimedia in mathematics learning [29], these results support the idea of implementing digital multimedia in learning graph theory.
Description of all CPS components with good results (understanding the challenge [constructing opportunities, exploring data, framing problems], generating ideas, preparing for action [developing solutions, building acceptance], planning approach [appraising tasks, designing process] is shown in Figure 2. Figure 3 and 4 are example of CPS components, about exploring data and appraising tasks. Both are very good, over 50%.
Analysis of questionnaire is shown in Figure 5. It is about the perception of students with the implementation of digital multimedia in lectures. It can be described that 67.7% of students very agree on point that learning activities can improve creative problem solving. Significantly, the implementation of digital multimedia based on kahoot and powtoon applications can be used as an alternative to disruptive learning innovation in mathematics teaching-learning.

Conclusion
This was important for the implementation of digital multimedia on graph theory course because it could improve the creative problem solving performance of students. This paper has argued that the Kahoot application as digital multimedia is creative as a fun evaluation tool for learning graph theory. In this investigation case study of graph theory that was the minimum spanning tree, shortest path and traveling salesman problem. We assess the provision of digital multimedia-based tasks such as the 67.7 29.7 Learning activities can improve creative problem solving Very agree (%) Agree (%) Less agree (%) Not agree (%)