Project Evaluation (Activity)

From Foss2Serve
Revision as of 20:15, 23 February 2017 by Darci.burdge (Talk | contribs)
Jump to: navigation, search


Project Evaluation



  • Identify HFOSS projects that are good candidates for new contributors
Process Skills

Assessment, Critical Thinking




Walk through of an evaluation of the OpenMRS project






Activity - To support student participation a project should be reasonably active. Number of commits can be used as an indicator of activity. Little to no activity over a year, for example, may indicate that the project is dead.

  • Click the "Graphs" tab then select "Commits"
  • Assuming "Active" means that approximately two-thirds of the weeks in a given quarter have commits, determine whether each quarter was active over the last year by recording yes or no in the rubric. Note: you can use approximate delineations for each quarter.

Number of contributors - A suitable project must have an active user community. A common fossism states that "It's all about community." The community members are great resources for both faculty and students and they begin to learn about a new project, its culture, and norms.



  2. Click “More tools”
  3. Click “Extensions”
  4. Scroll down and click “Get more extensions”
  5. Search for “GitHub Repository Size”

You should now see the repository size next to license type

Determine the size of the OpenMRS project and enter your findings in the rubric.

    • Follow the instructions for finding the number of open and closed issues and record your findings in the rubric.
    • Follow the instructions for assessing a project's willingness to guide new contributors and enter your findings in the rubric.

    1. Based upon the results from OpenHub (gathered in the FOSS Field Trip activity) and the information from the OpenMRS Technical Overview page, think about the size of the code base and how many different technologies and layers are involved in the application. What would you score this project for size/scale/complexity on a scale of one to three where one is "low" and three is "high".
    1. Based upon the number of commits (gathered in the FOSS Field Trip activity) would you consider this project active? Why or why not?
  2. Community - A suitable project has an active user community. While it is difficult to quantitatively evaluate the activity of a user community, some indicators include a regular history of project downloads and documentation updates over time, current activity on user mailing lists, and testimonials on the project web site.
    1. Examine download activity
      1. Go to and enter OpenMRS into the search box.
      2. Choose OpenMRS from the search results.
      3. Click on the number of downloads that is listed on the project page.
      4. Change the date range to give a graph of downloads over the last year.
    2. OpenMRS has begun migrating legacy mailing list activity to OpenMRS Talk. Examine discussion activity
    3. Examine the IRC logs
    4. Based upon the download history, discussion activity, and IRC activity, do you feel this project has a good community? Why or why not?
Mission Critical Criteria - Approachability
Here you are evaluating a project's on-ramp to contribution, scoring as follows:
1 - Insufficient - Few or no pointers on how to become involved.
2 - Sufficient - Suggestions about how to get involved other than contributing money with accompanying high-level instructions.
3 - Ideal - Obvious link to get started, list of suggestions for things to do and detailed instructions.
  1. Examine project on-ramp.
    1. Link to getting started - The website has a Get Involved page with links to ways you can contribute and share your ideas.
    2. Each of the links (Develop, Test, Document, Translate) contain more detailed information about what and how you can contribute.
    3. The Getting Started as a Developer page contains a detailed list of how to get started including a list of introductory issues.
    4. Detailed instructions - The Developer Guide contains instructions and information in many areas including process, architecture, tools, and developer documentation.
    5. Based upon the resources you looked at, how would you rate the approachability of the OpenMRS project?

Mission Critical Criteria-Suitability
  1. Appropriate Artifacts - Since evaluation is dependent on class objectives, in this example we'll assume the objective is to learn the process of working in an authentic development environment by contributing bug fixes to OpenMRS.
    1. Opportunities to contribute bug fixes - Examine the issues found at the bottom of the getting started as a developer page. How are the introductory issues categorized? How many issues are listed?
    2. Documentation on how to contribute bug fixes - On the Tickets page there is information on how to create and work on an issue, including links to coding standards and the code submission process. Review this information.
    3. Based upon the number of bugs suitable for students to tackle and information on the process of how to submit bug fixes, do you think this would be an appropriate project for your students? Why or why not?
  2. Contributor Support - Does the project have a high volume of guidance to help students as they learn?
    1. Communication Tools - Communication tools are directly available from any of the Wiki Spaces (Documentation, Projects, Resources). The Resources page contains links to OpenMRS Talk and IRC Chat, as well as links to group meetings (under Events), and training opportunities.
    2. Web Presence - Examine the IRC logs. Has there been activity during the last week?
    3. Operating Processes - Links to information about coding standards, the code submission process, and commit privileges can be found on the How-To Submit Code page. The process for making feature requests is available on the Tickets page. Are these processes well documented?
    4. Response to Questions - Review a few of the posts on the OpenMRS discussion platform. Do posts to this forum receive timely and supportive responses?
    5. How would you rate the support that newcomers to OpenMRS receive?

On your wiki page, write a summary of why you think the OpenMRS project would be suitable for your course. Be sure to include information about the course and reasons why OpenMRS would be a good/poor match.

Overall evaluation for Mission Critical criteria - If the mission-critical criteria seemed reasonable, then you may want to evaluate the following secondary criteria. If the mission-critical was not reasonable then the project would not be considered suitable for student participation.

Secondary Criteria - Viability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment
  1. Domain
    1. Does this project require domain knowledge that may be difficult for students to learn? OpenMRS is a medical records system. Students should be able to grasp it well enough to contribute a bug fix, which is the learning objective assumed in this example.
    2. How would you rate the understandability of OpenMRS?
  2. Maturity
    1. To have the organization support student learning, the project should have at least one stable production release. The Platform Release Notes page lists releases.
    2. Does OpenMRS have enough of a stable base to support student learning? Why or why not?
  3. User Support
    1. The project should have clear instructions for downloading, installing, and using the project. As noted previously, the Getting Started as a Developer page provides detailed information about setting up and using the required tools, in addition there are detailed instructions related to installation, configuration, system requirements, and troubleshooting, including videos.
    2. Does the documentation seem sufficient for getting students started?
  4. Roadmap
    1. Student learning is best supported by projects that have a roadmap that includes new feature development, a method for users to submit new feature requests and a process for identifying how new features are prioritized. Feature requests are made through JIRA, the OpenMRS issue tracker. Road map planning and the process for prioritizing feature requests is available on the Technical Roadmap Planning page. Here you will find information about the planning process and how to participate in the planning process. The Technical Road Map page identifies features, their current status, and a point of contact, in addition to expected dates of completion.
    2. Does the roadmap provide you with enough information to make a decision about using it in your course?

Secondary Criteria - Approachability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment
  1. Contribution Types
    1. Does the project contain opportunities for multiple types of contribution and of the type that fits the class? There are multiple projects for testers, tech writers, and developers. These can be seen on the Get Involved page. Are the number of and type of bugs available suitable for students given your course and the class size? Are there other ways that students could contribute?
  2. Openness to Contributions
    1. Acceptance of a student contribution to a project provides valuable affirmation to student learning. Determine whether the project accepts student patches. The process for contribution is documented on the Tickets page. Does this project seem likely to accept student contributions?
  3. Student Friendliness
    1. Do community members moderate the tone of communication? Review the discussion platform and IRC to gauge tone. Review the discussion platform and IRC logs. What was the tone of the communication?

Secondary Criteria - Suitability - Secondary criteria sections are OPTIONAL for the POSSE workshop assignment
  1. Project Description
    1. Students must be able to understand the purpose of the project. Does the project clearly describe the product? Can students understand the intended uses of the product? - The About page provides an overview of who, where, and what OpenMRS is, including a downloadable PDF file and a video.
  2. Platform
    1. What software and hardware platform does the FOSS project run on? Development environment can be built on Windows, Linux or Mac OS X completely with FOSS software. (Project development information found here)
    2. Are there resources to support these platforms?
    3. Are students familiar with the platforms?
  3. Development Features - Is the class dependent on specific development features? (Project development information found here)
    1. Programming language - What is the primary language?
    2. Development environment - What environments are supported?
    3. Supporting technologies - What technologies are suggested/required?


POSSE: On your user wiki page, a section describing your evaluation of OpenMRS as a suitable project for your course.

Notes for Instructors



  • How will the activity be graded?
  • How will learning will be measured?
  • Include sample assessment questions/rubrics.
The purpose of the project
Why the project is open source


  • What should the instructor know before using this activity?

Variants and Adaptations


Additional Information

Area & Unit(s)
Estimated Time
to Complete


Environment /

Michele Purcell



CC license.png

Suggestions for Open Source Community


Personal tools
Learning Resources
HFOSS Projects