{'fraction_non_alphanumeric': 0.0328515111695138, 'fraction_numerical': 0.012614980289093298, 'mean_word_length': 5.249589490968801, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Assessing Collaborative Problem Solving in the Context of a Game-based Learning Environment

Game-based learning environments provide students with the opportunity to collaborate with peers and solve ill-defined problems that align to a problem-based learning (PBL) inquiry cycle. However, game-based assessments must not only be psychometrically valid, but also fun. Drawing on problem-based learning (Hmelo-Silver, 2004) and a collaborative problem-solving framework (Liu et al, 2016), we present a conceptual piece detailing the design of collaborative work products and PBL assessments in the context of our game-based learning environment. Problem-Based Learning (PBL) refers to an instructional method that presents an ill-defined problem that students solve collaboratively with the aid of scaffolds (Hmelo-Silver, 2004). Although PBL has often been used in the classrooms, game-based learning environments can provide a rich context to situate problems. Shifting to a gamebased learning environment means that generating a validated assessment can be challenging since it requires accounting for assessment design, psychometrics and the element of fun. To address this challenge, we utilize Liu et al.’s (2016) evidence-centered design (ECD) framework of collaborative problem solving (CPS). We aim to explore the relationship between scaffolds and the impact it has on group collaboration and learning outcomes. In this conceptual paper, we highlight how CPS framework aligns with steps in the PBL process and the design of collaborative work products in the context of a game-based learning environment. It is hoped that by making these aspects visible in our work, we can facilitate discussions about how to design game-based PBL assessment in collaborative settings. Evidence-Centered Design (ECD) approach to game-based assessments Evidence-centered design (ECD) is an approach towards designing assessments by attending to evidentiary arguments, wherein claims about what students know can be reasoned from what they do (Mislevy et al., 2014). There are multiple layers that need to be addressed in the design of assessments; domain analysis, domain modeling, conceptual assessment framework (CAF), assessment implementation and delivery (Mislevy et al., 2014). The ECD framework has been widely utilized in game-based assessments, allowing researchers to examine constructs such as emotional engagement (Kim & Shute, 2015). Given the centrality of collaboration in PBL, we utilize the collaborative problem solving (CPS) model designed by Liu et al. (2016) who outline four categories of skills (table 1), that were crucial to successful collaborative problem solving. In their work, the authors found that individuals who were able to share ideas with others performed significantly better at understanding seismic events (i.e., their outcome of interest) than those working on their own. In table 1, we present an alignment between CPS, the PBL cycle and activities in CRYSTAL ISLAND: ECOJOURNEYS. In ECOJOURNEYS, students work in groups of four and have to solve an aquatic problem. As students engage in the game, they use a whiteboard tool, hypothesis board and chat to share their ideas and solve the problem. Table 1: Mapping of the CPS model, PBL cycle and activities in the game CPS skills Problem-solving in PBL Activities in the game 1.CPS Sharing ideas 2.CPS Negotiating ideas 1.PBL Identifying facts 2.PBL Generating hypothesis 3.PBL Understanding gaps in knowledge 4.PBL Presenting solutions to the problem Whiteboard & Hypothesis board: A shared environment where students identify facts and share information after meeting assigned stakeholders and gathering data. Students evaluate evidence to generate hypothesis and solutions to solve the CSCL 2019 Proceedings 893 © ISLS