commons-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Bateman <>
Subject Consolidating Subprojects
Date Sat, 21 Nov 2009 12:41:39 GMT
Howdy all,

Tonight while browsing through the latest commons projects, it
occurred to me the once modest commons project has grown substantially
over time and has a lot of great stuff.  There are now lots of sub
projects.  Actually, it's 37 active subprojects to be exact.  Plus the
archived projects.  And the release numbering, JRE requirements, and
dependency graph can get a bit confusing.  (Took me several hours to
map it out.)

So I started thinking about what really makes it important to separate projects:
+ Different large external dependencies you wish to keep separate.
+ Projects that aren't related.
+ Projects that have very different user communities (e.g.
commons-math versus commons-dbcp).

And on the flip side, the benefits that come from having one artifact
and one release number to track, instead of a whole heap of release
numbers and dependencies to manage.

Then it hit me... most of the dependencies are with other commons
projects, and on Java itself.  And quite a few of the projects run on
the same version of Java.  Could this network of interrelated projects
be simplified by consolidating a handful of projects together as a
single project?

If so, what are the tradeoffs and what might it look like.

So that's the question... would it potentially be beneficial to
consolidate some of the projects?  What do we lose?  What do we gain?

To help get discussion and thoughts flowing, I've attached some notes
to the bottom of this email (and in attached txt files).  Please don't
interpret these considerations as an actual final proposal... I'm
including them simply as a basis to initiate a possible discussion.


Attachment 1: A possible consolidation plan.
Attachment 2: A starter list of pros and cons.
Attachment 3: A map of project dependencies and minimum Java versions

Caveat: Please forgive me for any spelling errors or missing words
that went unnoticed while pulling all this together.  Spelling isn't
my strong suit.

*Attachment 1:  A possible consolidation plan...*

Commons-Lang (JDK 1.1/1.2 Compatible)
- lang
- collections
- logging
- primatives
- discovery

Commons-Utils (JDK 1.5... plus create one-time releases for JDK 1.3
and 1.4, using the appropriate past releases of the items below)
- attributes client api
- beanutils
- command line utils (CLI)
- codec
- compress/zip
- dbutils
- exec
- io
- jxpath
- pool
- proxy (unclear... this might not belong as it has some *optional*
- validator

Commons-Config (Could be in rolled into Commons-Utils)
- configuration
- digester

Commons-Net (JDK 1.4+.  Keep old release of original commons-net
available for those needing jdk 1.2 support.)
- net
- email
- vfs

Commons-EE (JavaEE 1.4+.)
- dbcp
- el
- fileupload
- transaction
- chain (Note: The non-JavaEE stuff could go into utils group above if desired.)

Keep as Independent Subprojects:
- sanselan (hasn't reached 1.0 yet)
- betwixt (hasn't reached 1.0 yet)
- scxml (hasn't reached 1.0 yet)
- daemon (contains native code)
- modeler (requires jmx jars unless jdk 1.6+)
- math (maintain focus on math/sci community)
- launcher (ant dependency)
- jelly
- attributes' ant tasks

*Attachment 2: Pros and Cons*

Benefits of the separate projects in the status quo:
+ *JRE Versions*: Some projects run on Java releases as old as Java
1.1.  Other projects have generics and need Java 5.0.
+ *Fewer Transitive Dependencies*: By keeping project A and B
separate, a project can use A without cluttering its transitive
dependency graph with all of B's dependencies.
+ *Smaller Jar Files*: Rather than having one single 1 MB jar file, I
can pick out just the handful of 150kb jar files I really need.
+ *Separate Release Cycles*: I can release project A without releasing
project B.
+ *Separate Committer Lists*: Project A and Project B can have
separate lists of committers, and access to the SVN directories can be
controlled accordingly.
+ *Legacy*: We've always done it this way, and it works.  Why confuse
people with change.

Important Observations:
+ *Similar Committer Lists*: When you look at the projects, there is a
core group of individuals who are listed as committers on the majority
of projects.  The committer lists between projects aren't all that
different.  Particularly since the subprojects are all so
interdependent on each other, one project is likely to find and fix
another projects bugs.
+ *Very Few Transitive Dependencies*: Most of the commons subprojects
simply depend on each other.  There are actually very few required
dependencies on external artifacts that could be undesirably included,
+ *Small Jars*: Most of the jars are only about 150kb.  If they
combined the resulting jar would still only be about 1MB, which is
hardly problematic for 90% of the apps of there.
+ *Stable Projects*: Subprojects that support old versions of Java
tend to be the more mature and stable projects.  Generally speaking,
new versions incorporate new features rather than bug fixes for very
very old features (there are exceptions, of course).
+ *Java 1.4 is 7 years old*: Java 1.4 came out in 2002.  People
running Java 1.1, 1.2, and 1.3 are most likely more interested in the
existing commons releases than they are desiring of the newest stuff.
(Certainly those who run old Java but want the newest stuff fall in
the <10% category.  As Craig once said... Apache designs primarily for
the more common uses (80%) than the exceptional uncommon cases (<20%).
 Craig... did I get the quote correct this time around. ;-)
+ *Java 5.0 is 5 years old*: Java 5.0 came out in 2004.  Sure, lots of
vendors were slow to fully support 5.0 (ahemIBMahem), but it's out and
established now.  Even dbutils has elected to support Java 1.5 for
it's newest releases.  If you need a Java 1.4 version of dbutils, one
can simply pickup the older releases.

Benefits of consolidating
+ *Simpler Dependency Management*: Rather than tracking down the
required versions of various dependencies and transitive dependencies,
it really can be much easier to just grap a single JAR with
everything, so long as you don't accidentally introduce new transitive
dependencies you didn't want.
+ *Easier Refactoring*: It's easier to manage clean design within a
single project (for example, there might be some sharable code between
commons-net and commons-vfs).

Other minor benefits of consolidating:
+ *Easier Corporate Approval*: So this might not seem such a big deal,
but organizations which have to strong-arm the corporate lawyers to
get approval for various open-source libraries don't do so well
handing the lawyers a list of 37 separate projects to approve.
+ *Less Bewildering*: A new person looking at the website is hit by a
list of 37 projects, and tends to instinctively become overwhelmed
before getting started.

*Attachment 3: Commons Projects and their dependencies*

Disclaimer: I put this list together based on a variety of sources,
ranging from truck/pom.xml to to project webpages.
Please don't flame me if I didn't get everything exactly right.  I put
this list together quickly to get a sense of what the issues are in
consolidating projects and dependency lists.

  - JDK: 1.4
  - Required: qdox
  - Optional/Provided: ant, maven-xdoc

  - JDK: 1.3
  - Required: commons-logging
  - Optional/Provided: commons-collections

  - Note: Not yet release 1.0
  - JDK: 1.3 + xml
  - Required: beanutils, collections, digester, logging
  - Optional/Provided:

  - JDK: 1.3
  - JavaEE: 1.3
  - Required: commons-digester
  - Optional/Provided: javax.servlet 2.3, javax.portlet 1.0, javax.faces 1.1

  - JDK: 1.4
  - Required: None
  - Optional/Provided: None

  - JDK: 1.4
  - Required:
  - Optional/Provided:

  - JDK: 1.2 with partial 1.1 support, pom.xml in trunk targets jdk 1.5
  - Required: None
  - Optional/Provided: None

  - JDK: 1.4
  - Required: None
  - Optional/Provided: None

  - JDK: 1.3 + xml, 1.4, or 1.5
  - Required: commons-collections, commons-lang, commons-logging,
              commons-digester, commons-beanutils
  - Optional/Provided: commons-jxpath, commons-codec, commons-jexl
              commons-vfs, javax.servlet, ant

  - Note: Includes native C code
  - JDK: 1.3
  - Required: None
  - Optional/Provided: None

  - JDK: 1.4 (1.3 version comes with JDBC 3.0 support)
  - Required: commons-pool
  - Optional/Provided: JavaEE, concurrent

  - JDK: 1.5 (generics)
  - Required: None
  - Optional/Provided: None

  - JDK: 1.5
  - Required: commons-loggging, commons-beanutils
  - Optional/Provided: None

  - JDK: 1.1
  - Required: commons-logging
  - Optional/Provided: None

  - JDK: 1.4
  - JavaEE: 1.4
  - Required: commons-logging
  - Optional/Provided: javax.servlet 2.4 (for JSP 2.0 support)

  - JDK: 1.4
  - Required: javax.activation,
  - Optional/Provided:

  - JDK: 1.3
  - Required: None
  - Optional/Provided: None

  - JDK: 1.3
  - JavaEE: 2.4
  - Required: commons-io
  - Optional/Provided: javax.portlet, javax.servlet 2.4

  - JDK: 1.5
  - Required: None
  - Optional/Provided: None

  - JDK: 1.4
  - Required: None
  - Optional/Provided: A compiler

  - JDK: 1.3 + xml
  - Required: commons-beanutils, commons-cli, commons-collections
              commons-jexl, commons-lang, commons-logging
              dom4j, forehead, jaxen
  - Optional/Provided: javax.servlet, jstl

  - JDK: 1.5
  - Required: commons-logging
  - Optional/Provided: None

  - JDK: 1.3 + xml
  - Required: commongs-logging
  - Optional/Provided: javax.servlet, jdom, commons-beanutil

  - JDK: 1.2
  - Required: None
  - Optional/Provided: None

  - JDK: 1.3 + xml
  - Required: ant
  - Optional/Provided:

  - JDK: 1.1
  - Required: None
  - Optional/Provided: log4j, logkit, avalon, javax.servlet

  - JDK: 1.5
  - Required: None
  - Optional/Provided: None

  - JDK: 1.3 + xml
  - Required: commons-digester, commons-logging, jmx
  - Optional/Provided: ant

  - JDK: 1.2
  - Required: oro
  - Optional/Provided: None

  - JDK: 1.3
  - Required: None
  - Optional/Provided: None

  - JDK: 1.1
  - Required: commons-collections
  - Optional/Provided:

  - JDK: 1.4
  - Required: None
  - Optional/Provided: Lots, but they're all optional

  - Note: Not yet release 1.0
  - JDK: 1.4
  - Required: None
  - Optional/Provided: None

  - JDK: 1.4
  - Required: commons-logging, commons-digester, commons-beanutils
  - Optional/Provided: javax.servlet, javax.faces, commons-el, commons-jexl

  - JDK: 1.2
  - JavaEE: 1.4
  - Required: commons-codec, commons-logging, log4j
  - Optional/Provided: JavaEE 1.4 (servlet, jta, etc, etc.)

  - JDK: 1.4
  - Required: commons-beanutils, commons-digester, commons-logging,
  - Optional/Provided: oro

  - JDK: 1.4
  - Required: None
  - Optional/Provided: None

View raw message