{'fraction_non_alphanumeric': 0.021524768026993223, 'fraction_numerical': 0.008900782454405306, 'mean_word_length': 4.686404234204432, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Pisa Question and Reasoning Skill

The objective of the study is to determine the level of the reasoning skills of the secondary school students. This research has been conducted during the academic year of 2015-2016 with the participation of 51 students in total, from a province in the Black Sea region of Turkey by using random sampling method. Case study method has been used in this study, since it explains an existing situation. In this study, content analysis from the qualitative research methods was carried out. In order to ensure the validity of the scope, agreement percentage formula was used and expert opinions were sought.The problem named Holiday from the Chapter 1 of the normal units in Problem Solving Questions from PISA (Program for International Student Assessments) [35] are used as the data collection tool for the study. The problem named Holiday consists of two questions. Applied problems were evaluated according to the mathematical reasoning stages of TIMSS (2003). The findings suggest that the students use proportional reasoning while solving the problems and use the geometric shapes to facilitate the solution of the problem. When they come across problems related to each other, it is observed that they create connections between the problems based on the results of the previous problem. In conclusion, the students perform crosscheck to ensure that their solutions to the problems are accurate.


INTRODUCTION
The ability to reason is the most important feature that distinguishes individuals from other living things. Because, with the ability to think, it is possible to make sense out of events and rearrange conditions according to yourself [41]. [40] refers to reasoning as deduction, to try to find a solution, to solve the problem. When the literature for reasoning is examined, [34] interpreted it as a process involving many forms of thinking, whereas [41] described it as a process emerging in the later stages of thinking, and [27] defines it as producing an idea and defending this idea. Generally speaking, in the light of this information, reasoning can be expressed as analyzing a problem well to solve it, presenting an idea and making suggestions to support this idea, testing the suggestions it is available, and deciding the correctness of the result. In this process it can be expressed as using mathematical reasoning skills.
Recent research in mathematics education reveals the need to resolve real life problems associated with everyday life, either from abstract applications or from imaginary questions [16]. This process, referred as mathematical literacy, has become an educational reform in many countries. The Program for International Student Assessments (PISA) administered by the Organization for Economic Co-operation and Development (OECD) and spreading to a wide geographical area is the result of this (Kabael and Barak, 2016). Almost most of OECD countries participate in this test. This exam requires that the levels of students are determined. OECD describes mathematical literacy as formulating, using and interpreting. Students are expected to use these three situations to solve their problems in order to be problem solvers [30]. In other words, students are expected to use skills such as expressing problem situations mathematically and reasoning, etc. to solve existing problems. In this respect, these skills are considered to be important skills as indicators of mathematical literacy. For this purpose, PISA reports the educational conditions of the countries. In the literature, there are studies that reveal the situation of students, teachers, and prospective teachers against the PISA questions [1,4,5,8,25,26,37,43]. Opinions were taken in these studies and compared to the existing problems in existing books, the level of practitioners or the problem solving levels of the students were examined.
[37] conducted a study aimed at revealing the difficulties that 140 mathematics teacher candidates had in solving the PISA questions. As a result of the study, it was seen that the prospective teachers were having difficulty to express mathematical arguments, focused on the solution, and struggling to obtain a result without using mathematical reasoning. [43] stated that by asking prospective teachers two PISA questions, it will be important in terms of teaching how they will explain this subject. Teacher candidates cannot fully use mathematical expressions and experiences in the process, but if they are provided with training, they will be a fully equipped, qualified teacher. In situations where prospective teachers are having difficulty approaching problems, determining the approach style of the students to problems will help future teachers to have information about the potential of the students. In this way, teachers will have the opportunity to know what is missing in their students and to have the opportunity to make up their existing deficiencies. Starting from this, it is thought that it will contribute to the field in determining how students use their reasoning skills in PISA questions. It is believed that revealing the reasoning skill levels of students and having an opinion about it will be instructive in the future PISA examinations. From the answers given by the students in the process, it is aimed to examine in detail how they approach the problem and how they reached the solution. It is the purpose of this work to determine how students will make a reasoning for the solution of the problem by leaving them alone with the problem they may encounter in real life. In the process, the mathematical reasoning processes of each student and their approaches to problems are analyzed.

METHOD
Case study, which is one of the qualitative research designs, was preferred for the research. Case study is a type of study that examines in depth the solution of the problems that can be encountered in everyday life [11]. The limitation of the existing situation in such studies is to determine the facts in the study, to determine the datasets, to reveal the findings, to interpret and conclude according to the result of this [10].

Study Group
The study was carried out with a total of 51 students selected by a random sampling method in the Black Sea region of Turkey in the fall semester of the 2015-2016 academic year. [15] and [32] have stated that all units have equal chances for the random sampling method. Here, all the units to be applied are sorted and one of them is randomly selected. All the schools in the Black Sea region of Turkey were randomly sorted and a school was randomly selected for applications to be made with permission from the Directorate of National Education.

Data Collection Tool
In the study, the problem called "Holiday" in section 1 of the normal units was used from "PISA (Program for International Student Assessments) Sample Problem Solving Questions (2015)". Holiday problem consists of two questions. These questions serve as the data collection tool of the research. The data were collected by interviewing each student individually. In the study, a total of 51 students were interviewed in order to try to find out how the students approached the question and how they used metacognitive considerations based on the data given in the PISA question. Interviews were planned in a quiet environment so that the student will feel comfortable. The students were asked 2 questions, respectively. They are asked to voice what they think when solving the questions and to solve the problem in a loud voice. The interviews were both noted and recorded by the researcher and the process was taken under the record. Each student was interviewed for 15-30 minutes. The data were collected for 2 months. In the process, quotes of students were used to try to show the stages of reasoning and how they approach the problem.

Analysis of data
In order to achieve the purpose of the study, students used the problem called "Holiday" from part 1 of the normal units from the Sample Problem Solving Questions (2015) of PISA (Program for International Student Assessments). Holiday problem consists of two questions. The applied problems were evaluated according to the mathematical reasoning stages of [39] and the points to consider are given below.
According to TIMMS (2003) Mathematical reasoning is examined in 5 dimensions. First, students need to analyze, generalize, make connections, make decisions and solve non-routine problems [36]. When these stages are examined; The data obtained were resolved by two researchers according to Creswell's research approach, according to data analysis and presentation technique [11]. This technique primarily starts with data organization. In this process, files are created and edited for the data. Then the text is read and notes are taken on the edges, which at the same time form the pre-codes of the work. Then the structures that constitute its context and the situation are determined. Interpretations are used, with quotations from participants. Natural generalization of what is "learned" is made. Presented using tables and figures that describe the situation in depth. Analyzes were made with careful attention to these steps 3. FINDING

Results and Comments belonging to the solution of the first problem
The findings of the mathematical reasoning skills of the first problem are given below in order.

3.1.1.Analyzing
The first step in mathematical reasoning is the step of analyzing. The expectations from the student in analyzing; determining the relationship between the concepts available in the problems given to students and identifying the purpose and objective. Able to use geometric shapes according to the given problems, if there is a proportional reasoning between the situations asked in the question, it should identify these and if a three dimensional visual is found in the given problem, it should be able to determine the formation and expansion of this visual, use the available data in the problems in different ways and obtain valid results with the available information.
For the first question, student quotations for the analysis step are listed below. S1: "first of all, I need to interpret the   teacher. I want to return it... " When the quotations of the students were examined, it was determined that a large number of the students analyzed the data correctly. Too many students tried to solve by randomly adding the numbers on the table. Some of them did not consider the shortest distance but calculated the numbers between two places. From the quotations of the students, it is seen that only 7 students were able to interpret well and place the given data on the visuals. This makes us think that students can analyze the existing information. The rest of the students did not use the information given in the table correctly even if they understood the given. It is understood that 5 students did not answers the question from their papers and quotations. In addition, the students stated that they could not interpret and understand the table because they found it different, and they did not feel the need for an answer because they thought it was a different style problem.

Generalizing
According to mathematical reasoning steps of TIMSS (2003), the second step is the step of generalizing. In the generalization step; Students should be able to achieve certain results by reasoning in the case of problems given to them and by using the steps of problem solving and to generalize their results in other situations. When we look at student quotations, it is seen that Polya have given place to problem solving steps. Student quotations for the analysis step are listed below.
S28: "…if we look at what is given to us in the problem, there are a lot of things. The distance between the towns is given in the It is noteworthy that when student quotations are examined in terms of generalizing, the students have mentioned Polya's problem solving steps. First of all, they realize the step of understanding the problem by determining what is given and what is desired in the problem. Then, it is seen that the students were processing what is given on the figure and went on to application of the problem. Even if they have not indicated how they choose a strategy for solving the problem, it is seen that they ended up using the given data. They mentioned that other problems could be solved like this as well. We see that they use the information they learned in their other lessons in the problem. They reported that air flight distance can be useful in measuring distance. Also, indicating what to do makes us think that there is also the ability to make decisions.

Making Connections
According to mathematical reasoning steps of TIMSS (2003), the third step is making connections. At this stage, the expectation from the students is being able to use the solved problem in other problems in their lives and transfer the knowledge. The students have made the expected solution for the first question at this stage, but when they solved the other problem they used the information they got for the first question. They did not make any knowledge transfer for the first question. Therefore, student quotes were not included.

Making Decisions
According to mathematical reasoning steps of TIMSS (2003), the fourth step is making decisions. At this stage, students are expected to decide whether the problem they solved has beneficial results. They need to test the result of the problem. Student quotations for this step are given below.   When the student quotes were examined and when the expressions of the other students were analyzed, many of the students backtracked their processes and tested their answers. While some students seemed to be able to control over the table, some students only made checks on the map. While some of them reached the right conclusion and made the right decisions, some students believe that their solution is correct even if they solved it incorrectly. They tried to mathematically prove their mistakes.

Solving non-routine problems
The expectations from the students in the solving non-routine problems step, which is the final evaluation step of the problem; students should be able to solve the problems encountered in daily life instead of solving pure mathematical problems at all times by using available mathematical knowledge. Here, students are analyzed on the basis of solving the problem given to them by reaching the right result.    : S41coded student solutions When the student papers are examined, students coded S1, S13, S18, S22, S28, S48, S51 have solved the first question of the problem by using the correct logic. It is seen that they have identified what is given and required in the problem, reading and interpreting the table, have the ability to do operations from the table. Some students who answered correctly were seen to correlate the problem situation with situations that exist in everyday life. It is seen that students coded S7, S9, S23, S32, S41 cannot understand what they are given in case of problems, cannot analyze required and given data, and cannot read and interpret the table correctly. Therefore, these students did not make any judgments about the solution of the problem. The remaining students did not reach the correct result because of reasons such as not being able to understand what is given in the problem, unable to read and interpret the table, not being able to use the data properly, and not being able to choose the path between the distance to be traveled correctly and appropriately. It is understood that very few students have solved the problem correctly.

Results and Comments belonging to the solution of the second problem
The analysis steps applied in the first problem are also valid for the second question. According to these steps, the findings of student papers and quotations are given below.

Analysis
The analysis step was briefly analyzed with careful attention to detail, such as distinguishing the given and required items in the problem, making sense of the problem, and giving clues as to which reasoning to use. Student quotations are listed below.
S24, S27, S38: "…Canan, who lives in Akgaz, wants to go to Kadı and Lale. She walks maximum 300 km in one day. She's taking a break when she is tired. She needs to spend two days in every town... " S19: "…first of all, the camp locations need to be identified. Since she travels 300 km, she needs to move 300 km and go around the village... " Figure 20: S19 coded student solutions S2, S34, S42,: "…camp locations need to be identified. Kadı  When the student quotes are examined, it is seen that most of the students clearly read the problem, and almost most of them expressed the given and required. The students are aware that Kadı and Laleli needs to be visited. They stated that a maximum of 300 km should be traveled per day. It is also seen that express the rule for staying for 2 nights in every town. Two students stated that they did not want to solve the problem by expressing that they did not understand the question, just like in the first question. Since our study was based on voluntariness, the students were not forced and their blank papers were taken back. It was stated that if they want to solve later, they will be interviewed again. But both of the students did not want to solve the question. The other 3 students who left blank in the first question tried to solve the problem. But as a solution, they read the question and then wrote the names of the places on the table in order by using the statement "if villages are to be visited, first location is selected and all village is visited in turn". It is seen here that students cannot analyze the problem situation in general.

2. Generalizing
According to mathematical reasoning steps of TIMSS (2003), the second step is the step of generalizing. The things to pay attention in this step for the second question; Students being able to analyze by reasoning what they are doing in problem situation, to obtain certain results using the problem-solving steps and to analyze the existing information for other situations. Examples of student quotations are given below. S11: "…we will calculate in km and we need to calculate for the place to go. We looked at the given. What is required of us is to determine where we will camp. We go by choosing from the map. We have to start from Akgaz and return to Akgaz. And then we have to go everywhere twice so I'll take it one forward and bring it back and do what is required... " Figure 21: S11 coded student solutions S13: "…we can solve the problem by using the data from first problem. The things that are given to us and required from us are obvious we are going to go from Akgaz to camp sites and visit the entire village. I think we have to go starting from Akgaz as already done in the problem. If we do Akgaz Laleli, Akgaz Mengen, Akgaz Nurdan, Akgaz Miras, Akgaz Kadı, we can return to where we started. In this way we will be visiting the whole village. In the end, this question can be solved by reasoning... " Figure 22: S13 coded student solutions When we look at the statement of the students, it is obvious that they do not understand clearly and fail to distinguish according to the problem solving steps. This is because even if the students have indicated what are given and required in the problems, they could not use the information given in the implementation of the solution properly and could not focus on the result with a wrong strategy because they do not understand the desired information clearly. When it is identified that there are two students who understood the question clearly, it is determined that the students could not analyze the process well. Because of that, they have done wrong applications with wrong strategy. hould be set to single line spacing.

Making Connections
According to mathematical reasoning steps of TIMSS (2003), the third step is making connections. At this stage, the expectation from the students is being able to use the solved problem in other problems in their lives and transfer the knowledge. Since there are two problem situations here and the problems are related to each other, the process should be analyzed as follows. Student quotations for the analysis step are listed below.
S35: "…The question is asking us to determine the places to visit. She can go 300 km. In the first question, we have already calculated the distance between the two villages and the other villages. The distance between some places were already written in the  When student papers are examined, the students are aware that in the process, they should use the information in the first question. They did not recalculate the distances when solving the problem and tried to solve the problem through the table or the previous ones. It is seen here that students made connections with the other problem. But, because they do not understand correctly what is required in the problem, they have realized the solution by choosing the wrong way to solve. It has been found that the students were careless, switched to solution without understanding the question and tend to complete as soon as possible.

Making Decisions
According to mathematical reasoning steps of TIMSS (2003), the fourth step is making decisions. At this stage, students are expected to decide whether the problem they solved has beneficial results. The students need to test the result of the problem. Student quotations for this step are given below.  Figure 27: S26 coded student solutions Figure 28: Incorrect student solutions (respectively, S16, S3, S48) When student answers are examined, it is determined that the students tested the results of their problems. Even if they do not answer the questions correctly, they have used this step in their problems, confident that their answers are correct. Some students did not show any test on the correctness or incorrectness of their problem. It has been determined that they did not use this stage. The reason for this is that they approach the question with a tendency to solve the problem as soon as possible. Students are in an effort to achieve a result rather than the correctness of the problem. Therefore, it is thought that they skipped the testing of the problem stage. It seems that they think that they have solved the problem by writing the places to go in order.

Solving non-routine problems
The expectations from the students in the solving non-routine problems step, which is the final evaluation step of the problem; students should be able to solve the problems encountered in daily life instead of solving pure mathematical problems at all times by using available mathematical knowledge. Here the students are analyzed on the basis of the solution situations that will lead to the right result.

S24: "…Canan, who lives in Akgaz, wants to go to Kadı and Laleli.
She walks maximum 300 km in one day. She's taking a break when she is tired. She needs to spend two days in every town. If camped between Akgaz and Kadı, first day. The second day goes 300 km from here. That is, she continues traveling between Akgaz and Kadı. She camps between Kadı Laleli when she arrives at Kadı. She will spend another night in Kadı and make another camp. Then goes to between Laleli Akgaz and camps there. Then she camps between Laleli Akgaz again because she will stay for two days. Then goes to Akgaz and the journey. In total she may have made a total of 2100 km with 300 km a day for 7 days ... "   When student quotes and papers are examined, it is seen that students solve problems by using information they have available in the course of solution of non-routine problems. It has been determined that some students analyze well the situations given to them and try to use all the available information on the problem. Some students missed details while using available information. Not paying attention to these details have caused them to solve the problem incorrectly. Almost all of the students in the process made the mistake of thinking the entire village needs to be visited. For this reason, some have forgotten to look at the days, and some at the kilometer information. Solutions were always wrong in the process due to not being analyzed well.

DISCUSSION AND CONCLUSIONS
According to the findings, since students cannot use the problem solving steps clearly, it is generally determined that they perform the arithmetic operations in order as they do them in the class. [19,20] investigated the problem-solving steps of teachers in the classroom and as a result they reached the conclusion that students solved the steps of each problem differently and did not include all the steps required in the problem-solving process. This suggests that the students have not sufficiently understood the problemsolving steps and have experienced difficulties in implementing these steps. If the teacher is always following the same problem-solving steps, it is very likely that the students will solve the problems using those steps. The study made contains similar results. Students tend to try solving a problem by following the same steps as with the arithmetic order they solve the problems in their classrooms. [31] interpreted in his study researching the failures in PISA questions as not reading the problem correctly. For success, he stated that students should be left face to face with different problems which they will use problem solving strategies. In this study, it was determined that students solved the problem incompletely due to not completely reading the problem or not paying attention to all the information given. [14] stated that they focus on a certain numbers and words in the problem-solving process, do not think much about the results rather than process, and tend to obtain a concrete result as a result of arithmetic operations. It has been seen that the students who are in the effort to achieve the result have solved the questions without paying attention to the given and required.
The students read the question from beginning to end and done this by emphasizing given and required were obtained as a result of interviews conducted. However, it has been determined that students have difficulty in implementing the remainder of problem solving process. The students tried to solve the problem without paying attention to most of the steps in the process. [29] and [12] point out that it is important for the students to find out the answer to the problem by short way rather than the problem-solving process when the Turkish Education System has an evaluation process consisting of multiple-choice items. This suggests that students have a tendency to get results in the shortest way rather than analyzing and thinking the problem. It turned out that for both problem situations, students tend to add the numbers or do what is required last in the question root rather than do a good analysis. [21] points out that students focus on the results since the grades students will be getting from the exams are more important than learning the problem-solving steps. This indicates that the difficulties students encounter in solving problems may be partly due to the steps teachers are taking to solve the problem. In other words, when compared with the result here, it is possible that the students cannot perform the required steps clearly because of the teachers, not themselves. [37] and [3] have concluded in their studies that participants have failed to read and interpret the graphics and tables in the PISA questions. Similarly, it is understood that students cannot read and interpret the table correctly. Especially for the second question, many students tried to solve the question without paying attention to the information given to them in the table while reading the first question correctly and filling the second question according to the points that should be noted in the table.
While [2] and [26] report that they correctly read and express the concepts and mathematical expressions in problem situations, in the study of [37], it was identified that they were not able to use mathematical skills and reasoning skills clearly in their work. Similar results were obtained for all three studies. The reason for this is that it is understood from the student quotations that they express the concepts given in case of problems but do not use mathematical skills clearly in using them in practice and do not use reasoning skills. In a study by [37], participants noted that they were trying to solve the problem with daily life information instead of mathematical reasoning. Similarly, it has been determined that the students approach the questions by giving examples from their own lives and try to solve them. [24] reports that conducting activities oriented to test technique in Turkish Education System causes not to think about the reasoning of the solution. They also stated that students generally prefer the approach of "reading the roots of the question first, then read the problem". This process in turn causes the students to fail in the solving phase of the problem. The study conducted also supports this result. Because many of the students have tried to solve the problem without paying attention to what was given earlier and paying attention to the root of the last question. [29] stated that teachers usually do not use the step of verifying and checking the problem because of the time constraint. They have stated that the students who were trained this way tend not to use this step. It has been observed in this study that students generally use this stage and tend to confirm their results on the map or on the table. It is thought that the use of this stage may have been due to the fact that they were in an effort to confirm that they were doing it correctly during the interview. Because students stated that they felt the need to get a confirmation after every operation in the process of solving the problem. This was felt in all of the students even though no student was confirmed in the process.
[9] stated that with the teacher's anxiety to make the deadline with the curriculum, they prefer teaching problem solutions in only one way. When the answers of the students were examined, it was determined that almost all of them tended to solve the problem in the same way.
[22] and [28] have obtained the result that students were indecisive in being confident in proving solutions to non-routine problems. Once students solved the problem, they were not very successful with both problems in the course of checking the validity. In the study by [42], some students stated that they used excellent strategies to solve the problems but they could not reach the right result by making simple mistakes. [13] stated that instead of focusing on the result, they should focus on the process [trns. 42] Similarly, even if students have identified the right strategy for solving the problem, the simple mistakes they make and the percentage of achieving the right result remain at a very low level. [17] have concluded, according to the results of the study they made for determination of table and graphic reading levels of teacher candidates, that the higher the performance of interpretation of tables and graphics, the higher the ability of reasoning to judge. However, in our study, the result was that the students did not have good reasoning skills even though they read the table correctly. Especially for the second question, most students cannot reason about the problem because they do not understand the question clearly. [18] Mathematical reasoning is used in the decision-making process to determine what meaning to obtain from reading. He stated that it is a high-level thinking ability. At this stage, the result can be obtained that the group in this study does not clearly understand what they are reading. In general, Güler (2013) stated that the students' ability to reason in the face of PISA questions is weak. This result is supported by the work done. [7] stated in their study that the PISA questions were inadequate in terms of improving the ability to think effectively, communicate, perceive and problem solving in the Turkish education system, and thus the achievement is lower compared to other countries. Altun, et al., (2012) suggested that teaching programs should be supported by teaching situations that address vital problems and that students should be able to develop and defend proposals for specific purposes in order to be successful in solving PISA questions. The work done may have a similar suggestion because it is a fact that students have a low success in real life problems. Altun, et al. (2012) stated in the study they made students should be supported with teaching conditions addressing vital skills in teaching programs and support that students to develop suggestions in the face of such problems and defend their suggestions. They stated that this way failure will be avoided. [38] stated that PISA questions are questions that require high-level thinking skills, and even students who succeeded in the LGS exams may fail in the process. The reason for this is that LGS questions were abstract and based on memorization. Students were also not successful due to not structuring the problem well. Even the most successful students in the class could not correctly answered both problems. Only one student from 51 students answered both problems correctly.
[23] investigated the inadequacies in solving real life problems. He stated that it is necessary to provide environments that will allow students to become problem solvers. When the literature is examined, it is revealed that the education given to the students to be good problem solvers contributes to the development of the students [6,14,33,44,45]. [43] stated participants should be left alone with PISA questions which they will make solutions by using mathematical reasoning. He said that when faced with such problems, there is an opportunity to use the reasoning skill. As a result, it is seen that the students have low level of reasoning ability and they have failed in the process due to the difficulties they have experienced in interpreting and reading the tables.