OpenStack Technical Committee Update: Project Reform Progress

Over the last few months, the Technical Committee has been discussing plans to dissolve the binary concept of the integrated release and adapt our projects structure to the future needs of collaborative development in the OpenStack community. A specification was written to describe the rationale for the reform and its goals. In the past weeks, the OpenStack Technical Committee approved a first set of changes, affecting how OpenStack upstream teams and projects developed by our community are organized.

Project teams

The first of those changes is the transition from “programs” to “project teams”. We previously organized the work in OpenStack around thematic programs, like “Compute”, “Infrastructure”, or “Deployment”. This created several issues, including terminology collision with “trademark programs”, or naming confusion (“Compute” was used both as the official name for Nova and the name of the development team that was producing it). But the most important issue was that it was exclusive: there could only be one “deployment” program. No alternative solution could grow and prove itself without being somehow adopted by the incumbent team. That was stifling innovation and keeping valuable projects, arguably developed within the OpenStack community, forever out of “OpenStack”.

By calling them project teams instead, and naming them after the code name of their main product (“the Nova team”, “the TripleO team”), we avoid that exclusion. We also get real and call teams the way everyone calls them upstream. That doesn’t mean we’ll have alternative project teams to every existing project team: obviously competition doesn’t make sense for teams like the Infrastructure team, where there can only be one used anyway.  But allowing some competition to happen lets us foster innovation where it bring value to our users.

New requirements for project teams

The second change is in the requirements for new work groups to be officially accepted as an OpenStack project team. New programs used to have complex and very subjective requirements, including avoiding overlap in scope with existing programs. For programs working on a software deliverable, we would assess the maturity of the deliverables and judge their potential fit with the rest of the “integrated release”. That created a number of issues (detailed in the spec for those interested), ultimately resulting in our complete inability to accept new groups during the Juno cycle.

The new requirements are more social than technical. The goal is to determine if a project team is “one of us”: do they help fulfill the OpenStack mission ? Do they follow the OpenStack Way (the four opens) ? Do they run their daily development as the other OpenStack projects ? Do they follow our development cycles ? Do they use the openstack-dev mailing-list and participate to our Design Summits ? Then they are obviously a part of the OpenStack community and the software they produce should be recognized as “an OpenStack project”.

We will accept new project teams applications starting now, although the Technical Committee expects to process them serially and slowly at first, since we are still fine-tuning the requirements and the application process. We still expect a number of new project teams to be added before the Liberty Design Summit in Vancouver.

Some people are worried about the sudden influx of new project teams disrupting the Design Summit. My answer to this is that we actually already had more than just official programs at the Design Summit, we already gave time and space to projects that were obviously a part of our community. This is just making things clearer. Together with proposed changes on the Design Summit format, we are confident that the event can survive this change.

No more incubation ladder to the integrated release

The third change is that we started to deconstruct the “integrated release” concept into a set of more clearly-defined tags. As explained in the spec, the binary “integrated release” attribute ended up meaning different things to different people. It was originally just meaning “released together on the same date”. But since that was the only attribute around, people inferred stability, maturity or being a necessary block where there was just adoption of common development processes. That created a lot of confusion and also made it a very desirable badge rather than a technical attribute, resulting in an always-larger mix of projects, projects that were not consumed in the same way by our users. Time had come for that single and binary concept to die and be replaced by a series of more precisely-defined attributes that will ultimately provide a richer description of the projects in our ecosystem.

We created the framework for defining such tags. As a first step and to ensure a seamless transition, we created an “integrated-release” tag to describe the list of projects that will be released together at the end of the Kilo cycle. And in the coming months, we’ll start defining tags for the various facets of the “integrated release” concept and apply them to the projects that match their requirements. Those tags will help users make more informed decisions.

Since that will ultimately make the integrated release an obsolete concept, we also got rid of the “incubation” period. Incubation was the period during which projects were learning processes and lining up a series of requirements before we would feel confident they would be able to release at the same date as other “integrated projects” without jeopardizing the existing ones. Since we are getting rid of “integrated release” as a single objective, we are also getting rid of the incubation ladder that was slowly leading to it. The work to define rigorous standards of quality goes on, however, through the definition of specific QA-related tags and more direct mentoring of project teams.

What does that mean for existing incubated projects ? Well, they remain full-fledged OpenStack projects, produced by official OpenStack project teams. They will get tags applied to them (or not) as we define and introduce them.

What about core ?

OpenStack “core” is a set of rules, defined by the OpenStack Foundation board of directors (more precisely, its Defcore subcommittee), associated with specific trademark programs (think: what you need to implement in order to brand your solution asOpenStack Powered“). Defcore used to build their “core” rules out of the “Integrated release” list of projects, so how is it affected by those changes ?

First, Defcore already moved away from a simple list of core projects to an approach based on capabilities and required sections of code, which may then apply to multiple different trademark programs. The recently-adopted new bylaws wording still requires Defcore to pick required sections of code from a Technical Committee approved list of projects, so the Technical Committee still expects to have to provide lists of projects for Defcore to pick from. We’ll likely express our answers as specific tags, so that it’s disconnected from other attributes and can have specific deprecation rules that are aligned with the trademark programs specific needs.

What about focus ?

With the likely increase in the number of OpenStack project teams and delivered software, there is a natural fear that this will dilute attention away from the essential projects. We don’t expect that to be the case. First, this is not a zero-sum game: recognizing more project teams as being a part of our community will bring new blood inside our community, rather than keeping them artificially outside. Second, critical sets of projects will emerge as being the base sets the other projects all have to rely on. Those base sets will be much smaller than the current integrated release, so it will be easier for us to focus our efforts. The overloaded and ever-increasing “integrated release” forced us to increasingly dilute our focus, and this evolution will enable us to put greater focus on the foundational pieces.

What’s next ?

The next step in the reform is to start defining the tags that will replace the “integrated-release” single concept. For example, I plan to introduce a series of tags to describe the release model of each project. Are they coordinated by the release management team to ensure they release on time ? Are they just adopting the common release schedule and doing their best to hit the date ? Are they following their own feature-based schedule, trying to do a release to match the Kilo release date ? Or are they just releasing whenever they please ? Those tags will provide a much richer information to our downstream users than what we currently provide.

But that is just an example. The goal is for our community to step up and define the tags that will be useful to them, with documented criteria. The Technical Committee shouldn’t have the exclusivity in tag definition. I expect our operators, our distros, our end users to discuss the attributes that would be useful to them, and propose the corresponding tags. The Technical Committee shouldn’t have exclusivity in associating tags to projects either: tags should be applied by those who know best (and who applies them is actually part of each tag definition). The Technical Committee is just overseeing the tag framework — it’s up to us all to make it as useful as possible to navigate our ever-growing ecosystem.


One Response to “OpenStack Technical Committee Update: Project Reform Progress”

Leave a Reply
