{'fraction_non_alphanumeric': 0.02611418432313955, 'fraction_numerical': 0.009852833733430749, 'mean_word_length': 5.098790579248886, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Improving the Effectiveness of Physics Homework: A Minds-on Simulation-Based Approach

Students spend much time in doing their physics homework. Whether this effort results in deep learning depends on the quality of the mere homework. Therefore, we designed a minds-on simulation-based approach to physics homework and conducted a pretest-posttest quasi-experiment to compare its effectiveness to the effectiveness of traditional homework. Our student sample consisted of 39 first year high-school students from Bosnia and Herzegovina. In two school hours, all students received the same lectures about gas laws. Next, the experimental group students solved simulation-based homework in which their planning of actions, execution of actions and self-reflection was supported by a carefully prepared worksheet and survey. The traditional group’s homework consisted of three textbook problems and covered the same content, which is gas laws. Through analysis of covariance it was shown that the minds-on simulation-based homework was significantly more effective in developing students’ understanding of gas laws than traditional homework. The experimental group students perceived the simulation-based homework as interesting, challenging and useful.


INTRODUCTION
Learning does not happen only in school, but also in the home environment. Most physics learning in the home environment occurs through physics homework. Homework may be defined as tasks assigned by the teacher which are to be completed outside of school hours (Cooper, 1989). If carefully planned, homework may facilitate development of subject-specific competencies, as well as development of generic competencies and student-parent relationship (Cooper et al., 2006). However, in some occasions, homework may also have some negative effects such as decreasing motivation for learning, inducing cheating behavior and inducing stress in the students and their parents (Trautwein et al., 2006). Therefore, it is very important to carefully plan the homework and take into account the key factors that are known to influence effectiveness of physics homework.

Factors that Influence Effectiveness of Physics Homework
For careful planning of homework, it is important to have a theoretical model that illustrates what variables account for effectiveness of homework. One such model has been developed by Trautwein et al. (2006). Their model hypothesizes that homework affects student achievement through the interplay of the following variables: learning environment (e.g., quality and quantity of homework tasks, feedback), student characteristics (e.g., conscientiousness, cognitive ability), parental behavior (e.g., quality and quantity of help), homework motivation (e.g., expectancy and value components) and homework behavior (e.g., invested effort, invested time, cognitive and metacognitive strategies).
The teacher may primarily influence the effectiveness of homework through improving the quality of the learning environment. To that end it is important to maximize the quality and optimize the quantity of homework tasks, as well as to provide quality feedback.

Quality and Quantity of Homework Tasks
Traditionally, many teachers randomly assign a few physics textbook problems for homework. Then typically, only for a small part of that homework feedback is provided which results in poor learning from homework. Therefore, we can say that there is much place for improvement of traditional physics homework (Bao et al., 2008).
First of all, we should avoid the practice of randomly selecting homework problems. The homework tasks should be carefully selected to optimally complement learning that happened during the classroom hours. It is recommended that the teacher takes into account the key learning goals, while selecting homework tasks (van Voorhis, 2001). If the key learning goals are diverse, also the nature of homework tasks should be diverse. It is generally important to take into account students' level of knowledge, as well as their needs and interests for purposes of increasing homework motivation. In order to allow for significant learning progress, the homework should be perceived by the students as challenging and as of high quality (Dettmers et al., 2010).
How much students learn from the homework is mainly related to their effort invested in doing homework. Relevant effort is a way better predictor of effective learning than time invested in doing homework, because if students are not focused on the homework, they may spend much time on doing it, although the amount of relevant effort is low (Trautwein et al., 2006). Relevant effort may be increased by providing the students with feedback about their homework. Earlier research shows that it highly matters whether or not homework is controlled and/or graded (Cheng et al., 2004). Even more important is the quality of the feedback; it should be focused on discussing key fallacies in students' reasoning and it should be planned in a manner to allow the students to engage in correcting their mistakes (Tas et al., 2016;Xu, 2016). In addition, it should be provided at the right moment which is when the students are cognitively ready and motivated to utilize it for further development of understanding. It has been shown that regular feedback facilitates a student's ability to self-monitor her/his learning (Tas et al., 2016).

Homework and Self-Regulated Learning
Generally, learning in the homework environment often shares many features of self-regulated learning (SRL). In self-regulated learning it is expected from the student to autonomously identify her/his learning needs, to set learning goals, to identify relevant learning resources, as well as to select appropriate learning strategies and continuously evaluate her/his own learning (So et al., 2019). Most models of SRL differentiate three stages of SRL learning: forethought (e.g., task analysis, setting goals), performance (e.g., self-observation, time management, self-instruction) and self-reflection (e.g., retrospective evaluation of own learning) (Wong et al., 2021;Zimmermann & Moylan, 2009).
Evidently the main challenge of SRL is that students are expected to intensively engage in metacognitive processes because in the homework environment there is no teacher to control and (re)direct their learning. Therefore, many authors suggest that students' self-regulatory learning in the homework environment should be supported through provision of different scaffolds (Wong et al., 2021). A scaffold may be defined as any element which is included within the learning environment for the purpose of supporting the learning processes (Moore et al., 2016). Scaffolding may be provided by the teacher, peer or technology with the purpose to support the cognitive, metacognitive and/or affective aspect of learning (Kim & Hannafin, 2011). It may help to lower students' cognitive load, to direct their attention at key aspects of the learning material and to raise students' interest for learning. Scaffolds are particularly needed for supporting novice learners and may be provided through: prompts (e.g., guiding questions, instructions), tools that facilitate processing and organizing data, tools that facilitate communication etc. They may be used to facilitate planning of the learning process, execution of target cognitive processes, as well as self-reflection, at the end of the learning process. For the scaffolds to be effective, it is important that they are understood by the students and that the level of support is compatible with students' needs, abilities and interests (Roll et al., 2018). As the students master the content, the level of support should be lowered which is known as fading.
One particularly popular way to support students' learning is to structure their learning processes through the use of worksheets (Adams et al., 2015). For example, in learning by inquiry worksheets may help in: inducing students' interest, clarifying the nature of the task, decreasing the degrees of freedom, prompting for key research processes, directing attention at key content.

Simulation-Based Physics Homework
Not all types of homework are equally effective for fulfilling certain learning goals. Researchers in the field of education have been calling for a new type of homework which could better motivate the students and make the student-parents homework interaction more productive (van Voorhis, 2001). One such type of homework is Teacher Involve Parents in Schoolwork (TIPS). In TIPS students are typically given few weeks to work on experimental problems and to share the results of their work with their parents. It has been found that assigning experimental problems for homework may largely facilitate classroom discussion about physics phenomena.
Another type of interactive physics homework is the simulation-based homework (Adams et al., 2015). A simulation may be defined as an "imitation of a real-world process or system over time" (Banks et al., 2010, p. 3). Learning with simulations is very convenient for the homework context because students are more motivated to learn with them compared to traditional learning materials . Also, simulations may be very effective for inducing relevant cognitive efforts. Concretely, with physics simulations it is relatively easy to interactively explore cause-and-effect relationships which makes them potentially effective for development of mental models about physics phenomena (Dervić et al., 2018). Thus, learning with simulations is similar to learning by inquiry. On the other hand, it is well known that most often novice learners need strong guidance when beginning to learn by inquiry. Otherwise, many novices diverge from the set learning goals and no systematic learning happens. Therefore, although simulations are characterized by in-built scaffolds (Moore et al., 2013), in learning with simulations it is recommended to additionally lower the degrees of freedom for novice learners by providing them with carefully designed worksheets. Such worksheets should include prompts for purposes of guiding the students through the forethought, performance and self-reflection stages of learning.

Learning About Gas Laws
The students have many misconceptions about gases. For example, some students believe that gases have no mass and that they do not occupy space, at all (Pathare & Pradhan, 2010). Others believe that gases exclusively result from boiling of water (de Berg, 1992).
When it comes to the concrete topic of gas laws, a significant obstacle for students' learning is related to the fact that students are expected to reason about ratios that are constant over time and reasoning with ratios is generally known to be very difficult for many students (Arons, 1997). Furthermore, there is also the "third" variable problem related to the fact that students tend to disregard one of the three thermodynamic parameters, even in situations when all of these parameters are free to vary (Vidak et al., 2019). This could be related to the more general issue of forgetting to control potentially confounding variables when investigating the relationship between two variables (Lin & Lehmann, 1999).
Another source of students' difficulties with learning about gas laws may be related to the high abstractedness of gas phenomena and processes. Achieving deep understanding of gas laws requires development of visually rich mental models that effectively reflect the mechanisms of how pressure, temperature and volume of gases are established and changed (Knight, 2013). In fact, for meaningful learning to happen it is necessary to establish a connection between observable, macro-world phenomena (e.g., gas expansion in a cylinder with movable piston) and corresponding micro-world processes (e.g., movement of gas particles and their collision with the inner walls of the cylinder). Interactive simulations may provide an effective tool for establishing this micro-macro connection.

RESEARCH PURPOSE
There are still many open questions when it comes to developing effective approaches for learning with simulations (Falloon, 2020). It is particularly challenging to find the right approach to utilizing the didactic potential of simulations within the context of physics homework, where students are typically expected to learn without direct contact with the teacher. Therefore, it is not surprising that in some of earlier research simulation-based learning was not significantly more effective compared to learning with traditional materials (Chang et al., 2008). In other words, if the simulation environment is not sufficiently utilized for inducing relevant cognitive effort, simulation-based homework may be equally or even less effective than traditional physics homework. It is of uttermost importance that the simulation-based homework provokes many relevant cognitive processes. In a minds-on simulationbased approach we would expect that the students are encouraged to think in all stages of selfregulatory learning. Also, a minds-on approach would be expected to induce the key cognitive processes earlier physics education research (PER) has shown to be important for developing understanding in the given area of physics. This all could be accomplished through worksheets and surveys that include PER-based prompts related to the forethought, performance and self-reflection stages of learning.
The purpose of this research was to design such a minds-on simulation-based approach to physics homework and to compare its effectiveness to the traditional physics homework. Concretely, we wanted to answer the following research questions: 1) Is the minds-on simulation-based approach to physics homework more effective when it comes to developing conceptual understanding about gas laws than the traditional approach to physics homework?
2) What are the experimental group students' opinions towards the minds-on simulation-based approach to homework?
Here traditional physics homework has the meaning of paper-and-pencil homework which includes physics tasks from typical introductory physics textbooks (e.g., solving quantitative problems and interpreting graphs).
This research is highly significant because it provides directions on how to make the physics homework more effective. This is particularly important in these times of COVID-19 pandemic in which much of the physics learning happens in the home environment.

Research Design and Procedures
For purposes of answering our research questions we decided to conduct a pretest-posttest quasiexperimental study with one experimental group and one control group (Ary et al., 2009). One week before the homework intervention both groups were administered the pretest which measured understanding of particulate nature of matter and thermal phenomena. Then both groups received the same online-lecture about gas laws (one teaching hour lasting 30 minutes). Next, both groups had one more online, video session in which they briefly revisited the concepts from the previous lesson, after which the teacher showed them how to apply the gas laws, mostly through interpreting graphs and solving of traditional, quantitative problems. Following this lecture, the control group has been assigned with traditional homework and experimental group with minds-on simulation-based physics homework. Finally, the students were administered a posttest which measured students' understanding of gas laws. Students from the experimental groups were also administered a survey in which they were asked about their opinions towards the minds-on simulation-based approach to homework. For completion of the pretest and posttest students were given 30 minutes each, and for completion of the opinion survey they were given 10 minutes.
It is important to note that due to COVID-19 pandemic all physics classes were implemented by the same physics teacher in an online-environment (Microsoft Teams) and were the same for both groups. For purposes of pretesting and posttesting, an online-survey was used with the students obliged to have their cameras and microphones on.
Taking into account that the groups had the same online-classes and differed only with respect to characteristics of their homework, a between-group difference in posttest scores may be related to the homework intervention. Each correct answer at pretest and posttest was worth one point, and pretest/posttest scores were obtained by summing correct answers. For purposes of checking the statistical significance of between-group differences, we decided to run an analysis of covariance (ANCOVA). Posttest score has been chosen to be the dependent variable and pretest score was chosen to be the covariate.
All but one of the items from the opinion survey were of Likert type. For each such item, the frequency of choosing each of the Likert options has been reported. In the one constructed response item in which students were asked about their general impression about the minds-on simulation-based approach to homework, students' answers were analyzed and, based on that, broader themes that occur in students' answers were identified by the first author of this paper. Next, the first and second author analyzed together the answer of each individual student and negotiated to reach a consensus on the broad theme to which the given answer should be assigned. Thus, each student's impression was assigned to one of the identified broader themes and frequency of themes' occurrences could be calculated.

Population and Sample
In Federation of Bosnia and Herzegovina, each of the 10 cantons has its own educational system. Taking into account that this study has been conducted in Canton Sarajevo, the population for this study consists of all first year high-school students (mainly 16-year-olds) from Canton Sarajevo. The student sample has been obtained by convenience sampling and it included altogether 39 students. 20 students were assigned to the experimental group and 19 students were assigned to the control group. 53.8% of students were male and the remaining ones were female. If we take the students' results at physics competitions as a criterion, we could say that our student sample performs above-average in physics, when compared to first year students from other high-schools in Canton Sarajevo.

Characteristics of the Curriculum
Students from the sampled high-school are expected to cover all introductory physics topics in the first two years (i.e., tenth and eleventh grade) of high-school. Thereby, the curriculum includes physics topics similar to the topics that are typically covered in US high-schools. Physics teaching is primarily characterized by lecturing and solving quantitative physics problems.
Typically, one teaching hour is devoted to covering gas laws with focus on isobaric, isochoric and isothermal processes. In this lesson also the ideal gas equation is introduced. This is followed with an application lesson in which most often the teacher demonstrates how to apply gas laws for purposes of solving quantitative physics problems.

Characteristics of the Homework
The two sampled groups of students were assigned different homework about gas laws. At the end of the second gas laws online-lesson, students from the traditional group were told that for homework they are expected to solve the following three problems: 1. With one inhalation, 1 liter of 15°C air flows into the lungs. What is the volume of that air at temperature 37°C if pressure is considered constant (comment: note that this is a highly idealized assumption)?
2. Please fill out the empty cells, assuming that the table is describing an isothermal process.
3. Illustrate the isochoric process with a p-T, V-T and V-p graph Please note that the first homework task was adopted from the Physics Problems and Experiments book by Čolić and Mehurić (2000). The remaining two tasks were also taken from the same book, but in a slightly adapted form. Outside of the school hours, the students solved these problems on a paper and uploaded the scanned version of the homework onto their Google Classroom. The teacher checked whether the homework was solved and tried to identify students' common difficulties with the homework. At the beginning of the next online-lesson the teacher provided brief, general feedback about the quality of students' work.
When it comes to students from the experimental group, at the end of the second gas laws online-lesson, students were told that a gas laws worksheet in MSWord has been posted in their Google Classroom and that they are expected to fill it out and upload it back to the Classroom. Also, they were told that immediately after solving the homework, they are expected to reflect on their homework by taking the online-survey for which a link has been also posted in their Google Classroom. Outside of the schoolhours, the students from the experimental group downloaded the gas laws worksheet which contained a link to the Bosnian language version of the "Gas properties" Phet simulation and instructions for completing the homework. The completed, English version of the worksheet is provided at the following address: http://fizika.pmf.unsa.ba/wp-content/uploads/2021/09/WS_G_L.pdf The worksheet was designed to support forethought and performance stages of learning. Forethought/planning was facilitated by providing the students immediately with a print-screen of the simulation window with printed comments on elements of the simulation which they were supposed to use in next steps and with a call to briefly try them out. Then an advance organizer was provided which reminded the students on what they had been doing in classes, and what they will be expected to do for homework. The students were asked to use the simulation for experimentally investigating: 1) isobaric processes 2) isochoric processes 3) isothermal processes In order to fulfill the segmenting principle of multimedia learning (Mayer, 2009), the worksheet included three separate experiments. However, each of the three experiments had a similar structure. Firstly, the aim of the experiment was explicitly stated, and the student has been guided on how to prepare the "experimental setup", so that everything is ready for doing the necessary measurements. Clearly, this part of the worksheet was mainly related to the forethought/planning stage of learning.
Next, in each of the three experiments the students have been asked to change one of the thermodynamic parameters and to write down the corresponding values of the other parameter; each time, one of the three parameters was fixed. Temperature and pressure were directly measured with a given thermometer and barometer, respectively. Expression for gas volume has been determined from the volume of the cuboid gas container (a*b*c), where students have been instructed to measure the length "a" with the given ruler.
After writing down, for multiple gas states, the values of the two parameters which are free to change, the students were asked to verbally describe what they have learned about the relationship between the two thermodynamic parameters and to try to explain the observed relationship based on the "behavior" of gas particles. Next, the students have been provided with instructions on how to represent the explored relationship with an Excel-scatterplot and were asked to copy the obtained plot into the worksheet. Thereafter, students were firstly asked to take any two measurements of the first thermodynamic parameter and to find their ratio, as well as to repeat this procedure for corresponding measurements of the second thermodynamic parameter. Finally, the experimental group students were expected to analyze the obtained ratios and to draw a final conclusion about the relationship between the two parameters which are free to change in the given activity. Besides the described activities which were common for all three experiments, the exploration of isochoric process contained two additional questions/activities related to explaining how the microscopic model accounts for gas pressure and to finding ways to change gas pressure.
When it comes to the performance phase of learning, we can conclude that students' reasoning processes have been facilitated through the use of prompts which particularly focused on developing micro-macro connections and using multiple representations of gas laws. Besides facilitating forethought/planning and performance processes, we aimed to also facilitate self-reflection about the homework activity. To that end the students were required to take a survey which contained four open-ended questions and six questions with 7-point rating scales. A description of questions from the self-reflection survey is provided in Table 1.
After the deadline for submission of homework passed, the teacher checked whether the homework was solved and tried to identify students' common difficulties with the homework. Also, a completed version of the worksheet has been uploaded and at the beginning of the next online-lesson the teacher provided brief, general feedback about the quality of students' work.

Assessment Instruments
For purposes of answering our research questions, we had to develop instruments that provide us with an insight into students' understanding of gas laws, as well as with experimental group students' opinions towards the minds-on simulation-based homework. Taking into account that in earlier grades students did not learn about gas laws, we decided that our pretest should measure understanding of a familiar topic which is particulate nature of matter and thermal phenomena.
The pretest and posttest items were designed with the purpose to engage the students in cognitive activities that are an indicator of understanding, such as interpreting, comparing, identifying, predicting and explaining. Most items were adapted from existing literature which may contribute to the validity of the pretest and posttest. Each item had between 3 and 5 choices, out of which only one was correct. The Cronbach's alpha for the posttest-scores amounted to 0.52 which is relatively low but acceptable (Bowling, 2005, p. 397). A brief description of pretest and posttest items is provided in Tables 2 and 3, respectively.

Item 1
Identifying the picture that best illustrates the trajectory of a gas molecule in a sealed container

Item 2
Comparing the number of atoms in 1 mol of helium and 1 mol of neon

Item 3
How decreasing of temperature to 0°C affects motion of water molecules?

Item 4
Comparing the picture of water molecules and their spatial arrangement in ice and liquid water

Item 5
Comparing the picture of gas molecules' spatial arrangement at 300K and 100K

Item 6
Identifying the picture which correctly shows the direction of heat conduction

Item 7
Predicting what will happen with temperature of boiling water if we increase heating

Item 8
Explaining why one can handle being in sauna at air temperature close to 90°C, but not in water of similar temperature

Item 9
Explaining how an express pot works

Item 10
Comparing the temperature of objects made of different materials if left for a long time outdoors

Item 11
Explaining how a sweater makes you feel warm

Item 12
Identifying the most probable temperature of water when it just starts boiling

Item 13
Explaining how a boiled egg cools down in cold water

Item 14
Identifying the physical quantity which does not change as a result of heating an object

Item 15
Explaining why a metallic ruler feels colder than a wooden ruler

Item 16
Explaining the energy transfer for the process of freezing small amounts of water

Item 17
Predicting whether ice wrapped in paper will melt faster than ice that is not wrapped in paper

Item 18
Predicting whether mass of water changes as a result of freezing

Item 19
Predicting whether freezing of water affects its mass and volume

Item 20
Comparing internal thermal energy for water of different mass, but same temperature Table 3. A brief description of posttest items

Item 1
Predicting how cooling down a balloon affects its volume

Item 2
Identifying the gas state characterized by highest temperature, based on a p-V graph

Item 3
Predicting how heating the gas affects gas pressure, when holding the gas volume constant.

Item 4
Identifying the gas process for which the final temperature is highest, based on comparison of corresponding p-V graphs

Item 5
Predicting the ratio of final and initial gas temperature based on the given p-V graph

Item 6
Predicting how pressure changes in isothermal compression

Item 7
Identifying the procedures that may result with an increased gas pressure

Item 8
Identify the procedure which simultaneously results in an increased mass and pressure within a basketball

Item 9
Identifying the type of gas process from a verbal description of the experiment

Item 10
Identifying the graph which does not represent an isochoric process

Item 11
Comparing gas pressure for cylinders with different cross sections and masses of pistons

Item 12
Comparing gas temperature for two cylinders with same gas pressure and number of moles, based on a picture of cylinders

Item 13
Explaining why a piece of marshmallow in a syringe with a sealed tip increases in its size if we pull out the piston of the syringe

Item 14
Explaining how breathing-in works

Item 15
Predicting how isothermal expansion affects the processes at the micro-level The opinion survey included six statements for which students from the experimental group were expected to specify their level of agreement on a four-point Likert scale (Table 4).
Besides specifying level of agreement with these six statements on a Likert scale, the experimental group students were also expected to answer the following open-ended question: "What is your general impression about the gas laws homework?".

Students' Understanding of Gas Laws
A first insight into the effects of the two different approaches to gas laws homework may be obtained by analyzing the two groups' means on the pretest and posttest ( Table 5).
On a percentage scale, the between-group difference on the pretest amounted to 5.8 % in favor of the experimental group. The between-group difference was larger on the posttest and amounted to 16.8 %.
Next, we wanted to check whether the between-group difference is statistically significant when the pretest difference is taken into account. For that purpose, we decided to conduct an analysis of covariance (ANCOVA). However, before conducting ANCOVA it is necessary to check whether the ANCOVA assumptions are met. We could prove that the following assumptions were met: independence of covariate and independent variable, normality of residuals, independence of errors, equality of error variances and homogeneity of regression slopes. That the independence of covariate and independent variable assumption is met, was evident from the fact that the between-group differences on the pretest proved to be non-significant, t(37)=-1.71, p=0.10. The normality of residuals and independence of errors assumptions were checked by running ANCOVA as a multiple linear regression, with the pretest score (covariate) entered into the regression model in the first block and independent variable in the second block (Field, 2009, p. 396). The standardized residuals were saved and the Saphiro-Wilk test for these residuals proved to be non-significant (p=0.14). The Durbin-Watson statistic amounted to 1.6 which is inside the interval from 1 to 3 and indicates independence of errors.
When it comes to equality of error variances, Levene's test proved to be non-significant (p=0.59) indicating that this assumption was also met. Finally, the homogeneity of regression slopes assumption was checked by including the covariate-independent variable interaction into the ANCOVA model. This interaction proved to be non-significant (p=0.08).
After proving that all above mentioned assumptions were met, we ran the ANCOVA procedure. The results of ANCOVA showed that there is a significant effect of homework approach on students'

Item 1
The gas laws homework made me interested for learning thermodynamics.

Item 2
The gas laws homework helped me to considerably improve my knowledge about thermodynamics.

Item 3
Although I invested efforts into the gas laws homework, it was difficult for me to complete it.

Item 4
While doing the gas laws homework, I have been learning physics with understanding.

Item 5
Learning thermodynamics boils down to memorizing facts and equations, and mathematical solving of these equations.

Item 6
We should more often have simulationbased physics homework. understanding of gas laws after controlling for pretest differences, F(1,36)=10.44, p=0.003. The betweengroup difference in posttest means, adjusted for pretest differences, amounted to 2.25 points which corresponds to a difference of 15 % on a percentage scale. This is a large effect size which can be also concluded from the fact that partial eta squared value for our ANCOVA amounted to 0.22 (Pallant, 2010).

Students' Self-Reflection and Opinion on the Simulation-Based Homework
As has been emphasized earlier, experimental group students were asked to provide some feedback about their experience with the simulation-based homework in two occasions. First, they had to reflect on their homework immediately after they submitted the worksheets to the teacher, and after the posttest (seven days later) they were again asked to report on their experience with the simulationbased homework by filling out an opinion survey. Four questions from the homework self-reflection survey were open-ended and the remaining six questions were given with a seven-point rating scale. Students' answers on the open-ended questions were firstly carefully analyzed by the first author of this paper. Based on that analysis, for each open-ended question, a few broader themes that occur in students' answers were identified. Next, the first and second author of this manuscript analyzed together each individual answer and negotiated to reach a consensus on the broad theme to which the given answer should be assigned. In certain cases, one answer could be assigned to multiple themes. Table 6 provides a brief overview on how students answered the open-ended questions from the homework self-reflection survey.
Students answers to the rating scale questions from the self-reflection survey are shown in Figure 1. Table 6. Brief overview of students' answers on the self-reflection survey; 18 out of 20 students filled out the survey. Sum of numbers is sometimes higher than 18 because one answer could be assigned to multiple themes Question Theme N Q1: What was your physics homework? What content, phenomena and processes was it about?
Brief answer: "gas laws" 12 More detailed answer (relationships are specified) 6 Q2: What difficulties did you encounter? How did you overcome them?
Overcoming difficulties: "asked others for help" 2 Overcoming difficulties: "worked longer" 2 No difficulties, at all 5 Graphs: "plotting graphs" 4 Calculations: e.g., "setting expression for volume" 5 Technical difficulties: e.g., "using the simulation" 2 Q3: What do you know better after the solved homework than before the homework? What did homework help you to understand better?
Generally everything: e.g., "whole lecture", "gas laws" 7 Learning with understanding 2 Negative experience: "more confusing than helping" 1 Some parts: e.g., "some parts of the lecture", "Charles' law" 6 Micro-macro connections 2 Q4: How much time (in minutes) did you need for completing the homework?
Less than 30 minutes 1 Between 30 and 60 minutes 5 Between 60 and 120 minutes 7 Between 120 and 180 minutes 4 More than 180 minutes 1 Finally, we also present the results of the delayed post-intervention survey on experimental group students' opinion about the gas laws homework (Figure 2).
As earlier emphasized, the experimental group students also were expected to answer the following open-ended question: "What is your general impression about the gas laws homework?". We carefully analyzed students' answers to this question and identified four themes in students' answers. Each of the students' answers was assigned to one of these themes, i.e., types of answers. An overview of students' answers is provided in Figure 3.

DISCUSSION
Students from the control and experimental group scored similarly on the pretest and received the same online-lessons on gas laws. Their differences in learning about gas laws were primarily related to the two different approaches to gas laws homework. Taking into account that the posttest measured understanding of gas laws and that between-group differences on the posttest were statistically significant, it is reasonable to conclude that simulation-based homework was more effective than traditional homework when it comes to developing students' understanding about gas laws. On a percentage scale, the experimental group students on average scored 67 % on the posttest and their adjusted posttest score was by 15% percentage points larger than the adjusted posttest score of control group students. This may be regarded as a large effect because it corresponds to a difference of at least one grade on a typical grading scale in Bosnia and Herzegovina schools. Miller et al. (2021) consider already a 5% to 10% score difference that results from a homework intervention as an indicator of substantial improvement over the traditional physics homework.
For purposes of a better understanding of the cognitive achievement differences, it is useful to firstly interpret the results of the self-reflection survey and delayed opinion survey. When it comes to openended questions from the self-reflection survey, one first observation was that students generally demonstrated conscientiousness. In fact, 18 out of 20 experimental group students filled out the selfreflection survey. However, most of them decided not to provide detailed answers. This may be related to the fact that they spent already a lot of time on solving the homework and consequently were not motivated to engage in deep self-reflection. In addition, through earlier schooling they were not trained to perform self-reflection. However, we still can gain important insight from their answers. For example, we can see that experimental group students mostly faced difficulties with plotting graphs and setting up mathematical expressions and that they were pretty autonomous in doing their homework. In addition, most students stated that homework helped them to improve their knowledge, but provided pretty generalized answers (e.g., "helped me to learn better the whole lecture"). However, some students were more concrete and specified that the simulation helped them to better understand the micro-macro Figure 3. Experimental group students' general impression on simulation-based homework connection which has been already considered very important by Knight (2013). For the majority of students, the homework took relatively much time, between 30 and 120 minutes. From their answers, one could conclude that this mostly may be related to the fact that this kind of homework was new for them and that they had some difficulties in creating the plots in MS Excel. That is why many students reported technical difficulties in doing their homework. However, a large majority of the students still considered the homework as interesting and believed that it helped them to better understand the gas laws. It also seems that they perceived the homework to be of average difficulty. Generally, Flunger el al. (2017) found that in students who were observed to invest large efforts into their homework large amounts of homework time was associated with high motivation and conscientiousness. On the other hand, in students who exerted low homework effort, large amounts of homework time were associated with low motivation and conscientiousness.
Students' answers from the post-treatment opinion survey further corroborated their claims from the self-reflection survey which speaks for the stability of their impressions about the simulation-based homework. Thereby, socially desirable responding is not a good explanation for their answers, because the large majority of students (above 80%) agreed with the statement that they should have more often simulation-based physics homework. The fact that many students stated that the homework was interesting and long, probably indicates that the homework succeeded in motivating them but induced much mental effort. That students generally enjoy in learning physics from simulations has been already observed in the study by Mešić et al. (2015) in which learning with Physlets was compared with learning from traditional static images and Physlet-based sequences of static images.
Next, we will use Trautwein et al.'s homework model (2006) and results of the student surveys to explain how the simulation-based homework helped the experimental group students to develop better understanding of gas laws. When it comes to homework characteristics, already earlier research showed that the quality of homework as perceived by the students affects homework effectiveness (Dettmers et al., 2010;Rosario et al., 2018). In our study, most students perceived the simulation-based homework as interesting, challenging and useful, which certainly contributed positively to students' achievement. This is at least partly related to the fact that students are motivated to learn with new technologies Mešić et al., 2015). In addition, the quality of our homework was achieved by using interactive curricular material and a carefully prepared worksheet. Most experimental group students approached the homework conscientiously and autonomously. Besides implicit guidance provided by the simulation itself (Moore et al., 2013;Roll et al., 2018), they were also provided with explicit guidance through the worksheet. The worksheet provided guidance for forethought and performance stages of learning, whereas the self-reflection survey helped the students to reflect on their homework. In other words, the experimental approach was designed to facilitate meta-cognitive reasoning which is important for learning with simulations (Moser et al., 2018). The worksheet activities also encouraged the students to establish micro-macro connections which are considered important for developing deep understanding of gas laws (Knight, 2013). Altogether the activities induced in students a high mental effort which is evident from the time spent on the homework. Some of the mental effort was irrelevant (e.g., handling technical aspects of the simulation or MS Excel), but there was also a large amount of relevant mental effort (e.g., establishing micro-macro connections). Irrelevant effort resulted in complaints about technical difficulties and longer homework time, and relevant effort resulted in a deeper understanding of gas laws.
Although in earlier research it is sometimes recommended to use only light guidance when learning with Phet simulations in the classroom setting (Chamberlain et al., 2014), we decided to use stronger guidance because the students from our study had no earlier experiences with simulation-based physics homework. In the study by Adams et al. (2015) which was also conducted in a homework setting, a moderate guidance approach proved to be most effective. Probably, the inconclusiveness of earlier research when it comes to guidance in learning with simulations is related to the fact that the choice of learning methods should depend on a large number of factors (e.g., learning setting, students' physics foreknowledge and knowledge of computers, earlier experience of learning with simulations, students' conscientiousness, quality of the simulation) and there is no golden rule about the "right" method which would hold in all contexts.
The main limitation of this study is related to the fact that the simulation-based homework was probably somewhat more extensive than the traditional homework. Thus, an alternative explanation for the observed between-group difference on the posttest would be the different quantity of homework. However, earlier studies have already shown that solving even a thousand of traditional physics problems is often not effective in developing conceptual understanding of physics (Kim & Pak, 2002). Thus, a more reasonable explanation is that not the quantity but the quality of the activities primarily accounted for the observed effect. In fact, simulation-based homework allows for a different type of activities (e.g., interactive, visually-rich) to be assigned which makes it easier to provoke cognitive processes (e.g., micro-macro connections) which physics education research community considers important for developing understanding about gas laws.
Another possible limitation is related to the fact that the reliability of posttest scores is barely acceptable. The relatively low reliability may be partly explained by the fact that our instrument consisted of only 15 items. One of the main consequences of low reliability is an increased probability of type II error, i.e., false negatives (Matheson, 2019) which was not an issue in our research.
Finally, the trustworthiness of the qualitative data analyses is potentially compromised due to the fact that no intercoder-agreement studies have been conducted. However, we hope that by coding students' answers through a negotiation process which included the first and second author of the manuscript, we achieved a sufficient level of credibility of our qualitative data analyses.

CONCLUSION
Learning is not only limited to the classroom environment, but much of it may also happen within the home environment. In fact, home is a natural environment for self-regulated learning. However, the effectiveness of this learning highly depends on the homework design.
In this study we explored the effectiveness of a minds-on simulation-based physics homework. We came to the conclusion that simulation-based physics homework may be significantly more effective than typical traditional physics homework, when it comes to developing conceptual understanding of physics. However, for the simulation-based physics homework to be effective one has to carefully plan how to support students' learning in the homework environment and to select simulations that allow for interactive learning. In our study, we could conclude that one way to support the students is through worksheets that facilitate the forethought and performance stages of learning, as well as through encouraging the students to engage in self-reflection. Thereby, it is of highest importance that the worksheets include prompts which activate cognitive processes that are known from earlier disciplinary education research to be indicators of deep understanding of the given topic. In this study, we came to the conclusion that for the gas laws topic, deep understanding may be developed by encouraging students to make micro-macro connections.
Another finding from this study is that minds-on simulation-based physics homework is promising when it comes to developing students' interest for physics and their perception of quality of physics homework. This is a very important effect, because interest for physics is a prerequisite for life-long learning of physics and students who perceive homework positively are more often ready to invest higher mental efforts into solving the homework and to persevere in completing it, even if the homework is long.
In our future research, we plan to explore the effectiveness of minds-on simulation-based physics homework for different areas of physics and to compare it with other types of interactive physics homework. Kim