Tuesday, March 18, 2008

Smaller Chunks

Estimating a task or project is difficult. It's easy to give a wild guess - "that'll take 3 months" and get within 30% or so. To get any more accurate, though, you really have to think about what you're doing, and that can be difficult for even well-known projects.

One of the QA engineers at work is learning estimation in general and this is the task he was working on:

Automatically put error messages in the test reports.

Some Background:
  • The test report already exists and is generated by an existing script.
  • There are differences in error message indicators (e.g., FAIL or ERROR or fatal) in each test suite.
  • All test suites and logs are located in a single folder (a subfolder per test suite)
The First Estimate:
"Gosh, I don't know. A couple of days?"
That officially falls into the category of guess.

Working Through It:
We don't know how long this task will take. So the simplest strategy I know applies here:

Make the task smaller until you understand it.

We'll break the task down into component pieces, and then break those pieces until we do understand it. Let's give it a shot. Here's what we have to do:
  1. Figure out how to grab the failure line
  2. Figure out how many lines around it we have to write in order to pick up the full failure
  3. Repeat for each test in the suite (note here that it should be the same for every test in the suite, so this is just a confirmation)
  4. Repeat for each suite (there are 30 of these)
Is that small enough? Nope. We still can't estimate that step 1. So let's break it down further. To complete step 1 we have to:
  1. Build the path to the test suite log directory
  2. Build the path to the individual failing test log
  3. Open the log file and read a line (any line) out of it
  4. Print the line into the test report
  5. Change the logic to find the actual failure line and print that instead
  6. Start printing some number of lines before and after the failure line
  7. make before or after or both a configurable parameter
  8. make the number of lines to print a configurable parameter
Now that we can estimate. Granted, many of these tasks are 5 minutes only, but going through this means that now we have an estimate we can be confident in.  Launder, rinse, repeat for steps 2 through n. Now we've got a good estimate.

So how do you get better at estimations? Keep breaking it down. Over time you'll be able to estimate larger pieces and it will go faster. But keep at it in the meantime and remember... this is hard stuff, but we can do it.

No comments:

Post a Comment