{'fraction_non_alphanumeric': 0.024713353832006665, 'fraction_numerical': 0.007212850943992643, 'mean_word_length': 4.998965695569729, 'pattern_counts': {'":': 0, '<': 0, '<?xml version=': 0, '>': 0, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Using Problem Based Learning to Develop Class Projects in Upper Level Social Science Courses: A Case Study with Recommendations

Problem Based Learning is often used as the pedagogy for an entire course. However, instructors wanting to try PBL for the first time may find this intimidating. An alternative is to use this pedagogy for a class project and not the entire class. Students in an upper level psychology course used Problem Based Learning to create a transitional facility for ex-offenders in a rural county where currently none exists. Students gained insight into community services, the needs of the target population, and how to match clients’ need with services in the community. This project can be used as a model for instructors in the fields of psychology, sociology and social work.


Introduction
Pairing field-based and service-learning experiences with traditional campus-based courses is becoming more typical in professional preparation programs (Academy of Criminal Justice Sciences, 2005;Davidson, Petersen, Hankins, & Winslow, 2010;Harvey & Mitchell, 2006;Kenny, Simon, Kiley-Brabeck, & Lerner, 2002;Kezar & Rhoads, 2001). However, there are often a great many challenges to setting up service learning opportunities for an entire class in the short span of a semester (e.g., Kretchmar, 2001;Tapp & Macke, 2001) An alternative non-field-based instructional model for courses that intentionally combines campus-based pedagogy with authentic, real world-referenced work is Problem Based Learning (hereafter, PBL). PBL is a pedagogy model in which students work to solve real-life problems, engaging in critical thinking and deep learning through an instructor-supported but largely student-driven inquiry experience (e.g., Thomas, 2000;Thomas, Mergendoller, & Michaelson, 1999;Woolfolk, 2010).
Typically conducted in student groups, PBL offers relevant opportunities for course instructors to embed authentic learning experiences in their college coursework (e.g., Murray & Summerlee, 2007). In PBL, real-world problems are used to stimulate deeper engagement in course content, hopefully resulting in deeper understanding of concepts and practices used in the discipline (Blumenfeld et al., 1991;Marx, Blumenfeld, Krajcik, & Soloway, 1997). Core to PBL is the instructor's facilitation of student-driven constructive investigation through inquiry, knowledge gathering, and generation of solutions (Grant, 2002). Typically, problem investigation is pursued over a period of time, culminating in final products that address authentic problems and/or that mimic products such as those that would be generated in the field (Jones, Rasmussen, & Moffitt, 1997;Thomas, Mergendoller, & Michaelson, 1999).
Using this pedagogy, content material that is normally disseminated in lecture form during class instead is learned as the students go through the process of solving problem cases presented by the instructor. Class meeting time is devoted to working on the presented problem case, typically in small groups, with the instructor serving as a facilitator for the students' efforts rather than as the lecturer regarding course content. The problem case may take several class sessions to solve, or new problem cases may be presented at each class session. The activities that students experience in PBL are similar to those skills required of most people in the workplace: developing and using novel solutions to problems encountered, effectively working with others to solve the problems under consideration, and adjusting activities based on feedback from supervisors.
According to Barron et al. (1998), there is ongoing commitment among educators, standards-producing organizations, and accreditation bodies to "push the envelope" on connecting knowledge to application; PBL has been cited as a model that can, indeed, do that. Additionally, Barron et al. (1998) reported that students experiencing PBL instruction demonstrated better understanding of interrelated concepts. Beyond traditional knowledge and skills outcomes, PBL is associated with higher expectations for student ownership of learning, as well as meaningful opportunities for students to self-assess (Brown, Bransford, Ferrara, & Campione, 1983;Stiggins, 1995).
Much of the research on PBL describes using this pedagogy as the structure for an entire course, with all class sessions being comprised of problem solving work by students, and all content being acquired through the problem solving process (e.g. Murray & Summerlee, 2007;Searight & Searight, 2009;Thomas, 2000;Woolfolk, 2010). However, this type of complete and radical change in the way a course is organized can be off-putting for an instructor wanting to try PBL for the very first time. One way to incorporate the key aspects of (and the benefits associated with) PBL into a course initially is to use PBL for completion of a large class project rather than for the entire term. The project may be worked on throughout the entire course as one part of each class meeting, or during a set number of class sessions. All work on the project is done using PBL, but the instructor can still provide much of the content for the course through lectures.
Courses in the social sciences (e.g., social work, criminal justice, and psychology) lend themselves particularly well to exploring the use of PBL. First, the subject matter of these disciplines often focuses on problems within society where only limited success has been achieved in finding good solutions. Second, many students in these disciplines will become licensed practitioners with required competencies in specific skill sets needed to qualify for licensure. PBL helps students develop skills that are in line with the skills and knowledge required by these disciplines for board certification. Finally, students entering these professions will most often be required to assess the needs of clients and help ensure those clients receive services. This component of assessing a current problem state and finding an effective solution is at the heart of PBL.
The project described here was one component of an upper-level forensic psychology course on incarceration and community reentry. The project gave the students the opportunity to work on the real-world problem of matching service providers in the community with needs of ex-offenders transitioning from prison back into the community. Other than the work on this class project, class sessions were comprised of lectures by the instructor and guest speakers and did not use the PBL pedagogy. The specifics of our project are only used to illustrate the principles of designing such a class project and can serve as a template for an instructor wanting to include PBL as one component of a course. Our example can be adapted for any upper level social science course where the instructor desires the students to work on a real world problem in the discipline.

Designing the PBL Project
When using PBL to develop a class project, there are several things to consider: construction of the problem case, determining how group assignments will be made, and determining how the final project will be assessed. The instructor should spend time completely developing each of these elements prior to assigning the project to the students. In this section, we will discuss the development of each of these elements and provide templates from our own experience.

Problem Case Construction
Following the definition of PBL developed by Thomas (2000), the problem case used for the project should focus students on central concepts within the discipline, and should be based on a real problem in the area of inquiry. The instructor should pick one or more of the learning objectives for the course and develop a good problem case that is not more than one paragraph in length (For an overview of steps to creating good PBL problem cases please see Searight & Searight, 2009). Our project was designed to give students a way to identify the full scope of services needed by ex-offenders upon their release from incarceration, and to evaluate the accessibility of these services in a rural setting. The full problem case statement for our project was: "Can you create a facility that can provide transitional services for criminal offenders being released into our county? You will need to find a potential building space to rent, decide which services to provide inside the facility (and which to access elsewhere), and organize your facility to effectively provide services and information between the hours of 8 am and 6 pm. You may provide any service at the facility you desire, but for all services, there must be a likely source of funding (e.g., grants, allocation of current funding streams). Although your facility is entirely imaginary, your facility must be viable in the real world; running a facility and providing services costs money, thus you are required to find a potential source of funding for every service provided." The example problem case statement above includes several tenets of good problem case construction (Searight & Searight, 2009). First, the problem case should have a pre-determined scope of enquiry. If a problem case statement is too vague, students' inquiries may expand in ways that make it difficult for them to actually come up with a good solution to the problem. If the problem case statement is too strict, students may not discover enough of the content material that makes PBL a good alternative to lecture-based instruction. In our example, the scope was limited to creating a non-residential facility and requiring a potential course of funding for each service provided. By requiring students to find funding for all services provided it was less likely students would dream beyond a truly feasible real-world solution. Second, the problem case statement should include specific "clues" to areas and places the students should explore to find solutions to the problem. In our example, it was indicated that some services could be provided outside the facility, hinting that students should explore service providers in the surrounding community. In addition, by specifically requiring funding sources for services proposed, students should be alerted to explore sources of funding in the government and the community as part of their project. Finally, the problem case statement should leave plenty of room for novel solutions to the problem. For projects in the social sciences, part of the interest for the students will lie in coming up with a solution to a current real-world problem that has not already been solved. In our example, there is no such transitional facility in the county the college is located so there is no concrete model to follow nearby. Making the facility non-residential and requiring potential funding put real world limits on the scope of their information search, but the design, location, and choice of services provided at their facility was entirely up to the students in the course. The balance between limiting the scope too much (which can dampen creativity) and having too few constraints (which can end with the students overwhelmed by choices well beyond a feasible end product) is a delicate one. Keeping with the three tenets of problem case construction listed above should help the instructor to strike the right balance.

Group Assignments
There are always challenges when determining working groups for large class projects and these can be even more difficult when attempting to follow a PBL approach. The first challenge is determining who will choose the categories or topics each working group will focus on. The instructor may predetermine group topics based on desired learning outcomes, or the instructor can allow students to determine the exact groupings from a larger list of choices. The latter is more in line with the PBL approach, but there is risk of not having the right working groups to achieve the desired learning outcomes. Thus, even if the instructor allows the students to self-select group topics, the instructor should provide guidance to help ensure each group has a separate part of the whole project, with no one grouping having too large a workload.
A second challenge is deciding how individual students will be assigned into each working group. Students may be randomly assigned to working groups, the instructor may assign students to working groups based on a predetermined criterion, or students may be allowed to self-select their working group. Again, the latter is most in line with the PBL approach. In addition, research indicates that when students self-select their group for assignments they are more invested in the projects and achieve better learning outcomes (Myers, 2012;Papadopoulos, Lagkas, & Demetriadis, 2012). It is best to have each group have the same number of participants, if possible, and if the working groups are to coordinate with each other to produce a single final outcome, it is best to limit the number of individual working groups.
In determining groups for our class project, we used somewhat of a hybrid of the approaches listed above. For ease of working on such a large project, as the course had 30 students enrolled, the instructor pre-determined that the class would be divided into five working groups of six students each. The students were told that each working group should focus on providing one service area to the ex-offender. The instructor led a discussion of the various service areas that might be provided, writing them on the board in a list as students suggested them. However, students were not told which of the areas on the list to include in their facility; instead, they were to determine this as a class through discussion. Students were told to first determine the five areas of service provision for the facility, and then assign themselves to the group that most interested them personally. It was stressed that there were to be exactly six students in each group, and that compromises might need to be made so that every student could find a group that was satisfactory to them. If there was insufficient interest in one service area, perhaps another service area would be a better choice for the facility. The instructor left the room, and the students were given 30 minutes to decide their groupings.

Project Assessment
The last major component of designing a class project using PBL is to determine how the learning objectives will be assessed. There are many ways of assessing student performance and learning efficacy. The most common way is having all assessment of the project done solely by the instructor. The instructor can assess progress at various stages of project completion, or can do one summative evaluation of the final completed project. If the instructor desires a more complete picture of the problem solving process inside the working groups, we also recommend including peer reviews and self-assessment. A growing body of research indicates that including these other types of assessment are useful for improving student learning outcomes, especially in group work (e.g., Crack, 2007;Saito & Fujita, 2009;Scott, Van der Merwe, & Smith, 2005;Topping, 1998Topping, , 2009 For our project, students presented their solution to the problem case in one large oral presentation with each working group presenting individually as one section of the whole. Instead of presenting solely to the instructor, the oral presentation was given to a group of approximately 60 stakeholders from the rural community surrounding the college (e.g., social workers, parole officers, correctional officers, court appointees, and law enforcement). Each working group was to present their groups' contribution to the whole solution. We included the three types of assessment mentioned above: summative assessment by the instructor, peer review by working group members and self-assessment. We will cover each of these assessment types in turn.

Assessment by the Instructor
When the instructor is assessing the project, using rubrics allows for more uniformity in grading and, if given to the students at the project's start, serves to solidify the instructor's expectations for the final product (for an overview of creating and using rubrics for assessing learning, please see Burke, 2011;Quinlan, 2011;Stevens & Levi, 2004). Research indicates that students generally favor the use of rubrics in grading (e.g. Holmes & Smith, 2003;Reddy & Andrade, 2010). Students perceive that rubrics increase their understanding of instructors' expectations and rationale for point deductions (Gezie, Khaja, Chan, Adamek, & Johnsen, 2012), however, rubrics are regarded most useful when given to the students at the time the assignment is made (e.g., Petkov & Petkova, 2006;Reitmeier, Svendsen, & Vrchota, 2004). In addition, simply handing out the rubric is not enough; many students need instruction on how to use the rubric as a tool toward increased performance (Green & Bowser, 2006;Reddy & Andrade, 2010).
For our project, the rubric used to grade the oral presentation was distributed at the same time as the assignment was given. A copy of the rubric used can be found in Appendix B. Students were shown that there were two parts to the rubric: one part evaluated the cohesiveness and content of their working group's presentation, and the second part evaluated their personal performance during their group's presentation. The majority of the criterion for individual student performance was focused on their oral presentation skills. Individual students were required to orally present at least two minutes of their working group's entire presentation. This ensured that each student contributed to the actual oral presentation, not just to research and problem solving strategy for the final project. The majority of criterion for grading the performance of each working group was focused on content. Each working group was required to turn in copies of all materials used during their presentation (e.g., handouts, copies of Powerpoint slides) to assist in accurate grading. Our rubric used a 4-point Likert-type scale with point values corresponding to Exceptional 4), Adequate 3), Marginal 2) and Unacceptable 1). As mentioned earlier, there are many resources to assist instructors with making assignment-specific rubrics. (Several well respected online sites that offer assistance with creating rubrics for specific assignments and disciplines are: rubistar.4teachers.org, www.teach-nology.com, and rubrics4teachers.com).

Peer Review
A common complaint students express regarding group projects is the worry that someone in their group will not perform well and this will affect the final grade for the others. A typical desired learning outcome for group projects is for individual students to successfully work together as one unit. When per-sonality and strategy conflicts arise between students, the PBL pedagogy encourages the instructor to direct the student(s) to go back to the group and work it out without interference. Resolving individual conflicts within the group is, indeed, part of the project. However, simply encouraging the students to work out individual conflicts themselves within the structure of the working group (much as they would have to in the work environment), often does little to ease frustration when conflict is already evident. One way to diminish this type of frustration is for instructors to be very clear about their expectations for the group work process beforehand (Burdett, 2007). The use of grading rubrics can help all students in the group be equally aware of the performance expectations and should help to diminish conflict regarding what the students should do in order to receive high marks. An additional way to ease the frustration of group conflict is to give the students in each working group a chance to assess their fellow group members using peer review. Research by Yining and Hao (2004) showed that students' favor peer review as a way to reduce group conflict and social loafing on the part of individual group members. The same study also indicated that to be most effective, students should be told how the peer evaluations will be used. Students primarily take the assessments seriously when their feedback is incorporated into their peers' final grades. If the peer reviews are merely a way to inform the professor of social loafing or group conflict, and are not incorporated into the final grading scheme for each student, the motivation to take the review seriously is greatly decreased (Yining & Hao, 2004). In order to be most beneficial, students should be trained on how to complete fair and consistent peer reviews (e.g., Khabiri, Sabbaghan, & Sabbaghan, 2011).
For the peer review for our project, we designed a rubric for the students to use that addressed three areas of group work: quality of work, team membership and communications (A copy of this grading rubric used can be found Appendix C). For each area addressed, the rubric included a letter grade and a detailed description of the level of performance that was required to earn that grade. Students were specifically told that the rubric provided was to increase assessment consistencies between working groups as well as within each working group. In our case, the peer review rubric was the same rubric used for self-assessment, which we will turn to next.

Student Self-Assessment
Including student self-assessment is useful for gaining further understanding into the group process and dynamics during completion of the project (e.g., Falchikov, 2004;Li, 2001). This form of assessment affords the student a chance to reflect on the entirety of the group process and their personal contribution to the final product. It is also a good window for the instructor to view the contrasts between group-mates' perceptions of each other and a student's perception of their own efforts and contributions. The scoring grid used in our project can be found in Appendix A. It was our choice to use the same rubric for self-assessment as for peer review. The advantage of this approach is that students will be grading themselves on the same criterion as they graded their peers. The disadvantage of this approach is that it is more limiting in the types of self-reflection the instructor is able to ask the student to complete. Although we did not choose to do this, it may be preferable to add several questions to the self-assessment that are solely focused on self-reflection. (Our project chose instead to have all students rate their experience with the PBL process).

Consideration of Target Audience
An interesting opportunity presents itself when asking students to work on a real-world problem. If the problem case used in the project is relevant to stakeholders in the community surrounding the campus, the instructor should consider inviting interested stakeholders to the final presentation. Inviting community members to attend the students' presentation of their solution to the problem case adds a level of realism to the students' efforts often rare in class assignments. The instructor will want to monitor the progress of the project very carefully to ensure that the presentation will warrant the time investment of the community members.
In the case of our project, the stakeholders included people in law enforcement, probation/parole, social services, and local volunteer and faith-based organizations involved with reentry. Many of the stakeholders were already known to the instructor through their work in their primary area of research. However, contact information for people in these agencies and organizations is readily available online. Approximately five weeks prior to the presentation date (which was during final exam week at the college), the instructor informed the students in the course that the results of their group effort on this project might be of interest to stakeholders in the community and that stakeholders in the area of reentry would be invited to their presentation. The instructor shared a list of all the agencies and organizations that an invitation would be sent and asked the students to indicate any omissions. Once the invitation list was finalized, the instructor sent out a brief synopsis of the class project along with an invitation to attend the presentation. In addition, the letter indicated that the addressee should feel free to invite any other interested parties in their organization. The invitations were sent on letterhead through the postal service as well as through email. The instructor invited approximately 100 stakeholders and the night of the presentation more than 60 were in attendance.

Managing Progress of the PBL Project
Once the class project is assigned, the instructor's role during class sessions should move into one of support and away from one of direct leadership. The instructor should refrain from structuring the work time for groups or leading the groups' pursuit of background information, solutions, or design. The primary assistance provided during the work sessions should be to remind students of the pre-set scope of the project. In our example, students were reminded that they needed to have a viable source of funding for each element of their programming or service offered in their transitional facility.
It is important to note that during in-class work sessions, students often will ask questions such as, "Can we do X?" or "How should we accomplish Y?" In order to make PBL effective, the instructor should turn these questions back over to the students. In order for students to learn how to solve problems in their own manner, they are the ones who need to find answers to questions they encounter along the way (Thomas, 2000). Even if the instructor knows the answer to the question asked, the instructor should reply along the lines of, "I don't know, can you?" or "How long is typical for appointments with clients to last?" For example, when our students asked which social service agencies they should contact for information, the in-D. V. MCMAY ET AL.
Copyright © 2013 SciRes. 66 structor guided students to explore the county's online phone book and determine the agency that best fit their needs, but did not answer the question directly.

Example Class Project Outcome
At the end of the semester, more than 60 stakeholders attended a presentation of the students' class project. Each working group did an oral presentation that lasted approximately 15 minutes, with five minutes for questions. Each student in a group was required to contribute at least two minutes to the oral presentation of their group. Elements of the facility presented by each group included a list of services that could be provided by that group, the sources of funding that could be used to provide these services, and a mock-up schedule of how their group could put their part of the building space to best use. Each group presented their service component in isolation, culminating in a final summation in which one person from each group presented as a member of a core group of organizers. This last part of the presentation was completely developed by the students themselves, and had not been a part of the project requirements.
These five core students had served as liaisons among all the individual groups, determining how all the services could best work together to help a person in transition or family members of those still incarcerated. During the presentation, this group discussed a sample case of one person coming to the facility with various needs, demonstrating how the facility would coordinate as a whole to help this individual.
It is important to note that in the subsequent semesters this class project has been assigned, there are always unique elements developed by the students. When students are required to solve a problem, they rise to this challenge in surprising ways. Often the topics chosen for each working group are not the same categories as previous semesters. Given the changing availability of funding, the changing nature of community service provision, and the specific interests of students from semester to semester, the same problem case statement can result in dramatically different solutions. This variance contributes to the excitement for both students and instructors.

Student Evaluations of Experience
In addition to their peer reviews and self-assessment, as part of the general end-of-course evaluation, the students were asked to rate the PBL experience. The students overwhelmingly liked using PBL, and they reported that they benefited from it. When asked if they would take another course that incorporated PBL, 100% of the students chose ratings from "maybe" to "definitely", with 44% saying that they "definitely" would. Students were also asked to assess their learning experience by responding to seven statements specific to the nature of the project. A 5-point scale was used with anchor points ranging between "strongly agree" and "strongly disagree". Table 1 summarizes results of this questionnaire. The first three statements asked the students to rate how well work on the project increased their ability to find and use resources from a variety of sources. These statements assessed students' impressions of their increase in general problem-solving skills after completing the project. When asked if work on the project had increased their ability to gather and use information to solve problems, 96% of the students responded that they "agreed" or "strongly agreed" with those statements. With regard to increasing the general ability to find and analyze information, 92% of students answered "strongly agree" or "agree".
The last four statements on the evaluation asked about discipline-specific skills that students should have developed. For example, one question asked the student to indicate the extent to which he/she understood resources and services currently available in the county; this is information that a person working in the field of social work and/or mental health services would be expected to know. Ninety-six percent of the students indicated they "strongly agreed" or "agreed" with that statement.

Discussion and Recommendations
Completing a project of this type can provide students a way to learn about the challenges similar to those that they will face in their future careers. If the project is well designed, students will work at high levels of critical thinking by evaluating and synthesizing design factors and their potential impacts. The "real world" aspect of the problem in their own community created special interest for many students who lived on campus and rarely left its grounds. One challenge to implementing PBL is the reluctance of students to engage in a new way of learning when their focus is often entirely on final grades. One way to counter this anxiety is to be clear about the problem statement, as well as both the parameters within which students are to work, and how their work will be evaluated. In this study, requiring students to find a potential source of funding for all services provided proved helpful in curbing students' desire to "fix all the problems in the world". In addition, the instructor's anticipation of barriers that students would likely face helped to diminish frustration. For example, the instructor chose to limit the facility created by the students to a non-residential facility operating between the hours of 8 am-6 pm. This pre-determination eliminated the major hurdle of zoning restrictions for residential facilities (particularly those for ex-offenders) that students would face when seeking an acceptable space to rent.
Using PBL in the classroom poses challenges for the instructor, as well. Students' expressions of difficulties should not always be taken as expectations for the instructor to solve the problems-or answer questions-for them. Fighting the urge to help and letting students struggle to find good solutions is truly at the heart of this pedagogy. The better the initial problem statement is designed by the instructor, the more opportunities there are to turn the students' questions back to them. When a student asks, "Can we ···?" It is entirely acceptable to reply, "I don't know. Can you?" This gives "permission" for students to make the decision.
Setting aside the lecture format, and thus a great deal of control over the course content, is one of the most frequently cited difficulties for instructors using this pedagogy (e.g., Pepper, 2008). Designing a specific structure for class meetings can help reduce this anxiety. For example, setting benchmark dates for students to make decisions about aspects of their project helps give students a focus for each block of class time, while reassuring the instructor that progress was made during that class session.
Another implementation element is the re-allocation of time and the change in the instructor's role in a PBL model. In our example, we used a "standard" course design for the first two-thirds of the course. For the last month of the course, the instructor role shifted to that of facilitator. Reminding students of potential resources in the community and asking questions that prompted students' own critical thinking became the new instructor roles. Gradually incorporating PBL into the course aided both instructor and students to adapt to the differing roles required of traditional vs. PBL pedagogy. An initial positive experience with using PBL in one segment of the course can give an instructor the confidence to use this approach in the future as the primary method of instruction for the entire course.
There are several areas that future instructors might consider implementing to improve the experience for instructor and student. First, projects can be enhanced when an authentic audience assists in actually evaluating project outcomes. Being given advanced notice and a student assessment form would allow the stakeholders to give feedback rather than just observe; the students can benefit from this type of feedback from people working in their future career fields. In addition, feedback from stakeholders can help the instructor determine if their grading of the students' efforts is too harsh or too lenient in comparison to industry standards. Finally, allowing the students to self-evaluate their whole group final product (rather than just the efforts of their working group) affords them the opportunity to assess what went well, and identify areas they might choose to do differently. Incorporating any or all of these suggestions should be beneficial to both students and instructors attempting this pedagogy for the first time.