Part 1: The woes of the testbot




  • I no longer had gratuitious amounts of free time.




SimpleTest's journey to core


SimpleTest Transition to Core Commit Log


Following up SimpleTest with the testbot



Seeking sponsors


Code freeze and the request to stop


PIFR and PIFT commit log


Post-freeze undeployed features


  • Coder reviews in addition to testing
  • Beta support for contributed project testing with dependencies

Recent changes









Thanks for sharing your thoughts. The Drupal community needs to take testing seriously and it needs to be easier for the devoted people to get their patches into the code base.

I am curious for your new initiative ! Very good of you to bend this your way. Thank you for staying with us. Drupal needs people like you.

several hundred applications and it takes months to even receive a review

There are currently 250 in a "needs review" state, so depending on your definition of "several" that is true.
I don't think the claim that it takes "months" to receive a review is anywhere close to valid. In some cases it has taken weeks. But I think you'd be hard-pressed to find examples that sat for months with no review, and when you do they are usually complex modules that require external dependencies or specific knowledge to test.

Jthorson indicates on his blog it took months and I have seen plenty of people saying the same thing in IRC, but no doubt it is not always so.

To clarify, it is often months to get through the entire process ... up to a year in the case of some folks who didn't follow the 'migration to git' instructions posted on their applications. But this is different than the time to a 'first' review, which seemed to be averaging around 4 weeks when I applied; and the longest was 9 weeks the last time I performed a spot-check. Sorry if I was misleading. :)

@jthorson: Thanks for the clarification.

@others: Please keep in mind the primary point was things could be much better with automation and the only reason we don't have it is due to lack of resources. One of many, the next post will hit on this so more.

I suspect that you are always going to have a hard time finding maintainers for PIFR and simpletest, mainly because you are always going to find it hard to find someone willing to pay for testing, especially if you're not actually testing the finished product, but the framework. I.e. there's no way any of our clients would pay for us to help maintain PIFR so that their site built on Drupal was better. Maybe this is one case where the Drupal Association really does need to get involved financially, but I'd imagine that the DA has less money than everyone would like it too, and it might be too close to getting involved with development of core anyway.

Additionally, I wonder if PIFR is doomed to be unmaintainable, as most people doing testing won't want to write or use their own testing framework, they'll use something like Jenkins instead. If PIFR was a lightweight interface with some external tool, then you might find you'd get much more traction in the community, and many more people willing to help maintain the code you write. One of the main reasons that git was adopted by the Drupal community over other SCM tools was because the community seemed to go nuts whenever someone mentioned git. Maybe you need to find out how PIFR can become more useful for the Drupal community as a whole, and then re-work it light of that, and actually get somewhere other than d.o using it, and then you might find some maintainers.

Thank you for you hard work though, undoubtedly testing has made Drupal 7 a better release than ever before, and your efforts are highly valued by the community. I think we all want to see development of testing Drupal continue, and in a sustainable fashion.

That's a very good question.

Is anyone other than using PIFR for automated testing?

If not, why not? Should PIFR be modified so it is appealing to those organizations so they have a reason to maintain it? Should move to the system (Jenkins, or whatever) that those organizations use?

I hope all these questions and more will be answered in post #2 :)

Actually, there are a few companies a know of since they asked me to make specific changes. Examiner, as I mentioned in the post, hired me in part to work on PIFR since they are using it. The second post will provide more details on why Jenkins is not a good (or option at all) option for