Wednesday, August 12, 2009

Functions As Designed

Many customers have different requirements or requirements that change backwards and forwards over time, especially in an early stage of the project. The developer's weapon against such ping-pong requirement changes are :

make it all configurable

Whatever the customer sets is what he gets, even if the settings don't make any sense in their combination.

It is awkward if you have changes to existing behavior without noticing the parties concerned.
Testers raise defects whereas the symptoms are more a result of new features implemented and activated by default rather than actual defects.

We have an unwritten law that guarantees customers don't need to adapt their configuration files just to ensure existing functionality doesn't get changed due to some (unwanted) new features or restrictions in the use of some old existing features.

It is not only unpleasant for the customer, our test scripts won't work either. Well, that's the good thing of test automation. We wouldn't have detected those anomalies at the same speed as if we had done this test manually. However, we still struggle with a straight reaction to the findings. There are still some roadblocks to master.

Monday, August 3, 2009

What does Swine Flu have in common with Testing?

The expected worst case scneario in Switzerland are 2 mio casesof Swine Flu infections (one third of the population) and about 5000 deaths. Is it really that alarming or are these messages alone already the disease?

What happens if the Swiss Health officials (BAG) warn us and nothing happens (as already seen with the bird-flu)?

What happens if they don't warn us and we're all left in a big disaster?

My work as a tester sometimes involved similar decisions (in a much smaller and less important world of course) when thinking about which priority / severity to set for a defect. If I continue to raise many defects with a high priority status, developers soon don't pay attention anymore, especially if a majority of the issues turns out to be far less important than initially rated. For the flu, if that disease is not going to come, people are going to pay even less attention to such warnings.

If I raise a defect and set it a lower priority, developers probably never look at it, because they have too many other important defects to work on.

I might have found an important security issue but not noticed its relevance. Neither may developers notice its relevance because I set it a low priority. It could be that someone else finds the issue and hacks the system. The consequences can be tough for the company. For the flu, if you don't warn people about a pandemia, the responsible officials get all the blame for the omission of informing.

As in most situations, what's left is a decision based on common sense. But I think it is also an egoistic one. Am I really out for raising all sorts of low rated defects? I will have to re-test all of them later at a moment when developers have finally worked off their stack of high priority issues and get over to fix the low priroity defects. That is happening at a time when I am in the middle of testing all the other high priority issues, and now I am the one who ignores all those low rated issues, may be for several months because there is plenty of other high priority fixes to re-test. You may get an email from an administrator who closed all the low rated issues, because none seems to take care about them and therefore he made the right decision. But what did you raise the defects then for?

Severity vs. Priority
When you as a tester think you found an important issue, it could be looked at differently by the business analyst or the developer or the customer or the project leader. Setting the priority is often a team decision as all team-members may be right or wrong.

A system crash might be a severe issue and rated a SEVERITY 1, because that feature is not usuable. But the customer may not use this feature in the next couple of weeks. That makes is far less important, hence becomes a lower PRIORITY, at least for a while.

On the other hand, you as a tester might find a bug of which you rate as not very important. The developer might notice it and understand that the issue will have severe (other) consequences if not fixed immediately.

Testers and developers may rate translation issues as not very important, but the Business Analyst may have a closer contact to the customer and know that users will not accept a system with tons of text pieces spread over dialogs that are not translated into the user's language.

Whatever you do, it is always a chain of thoughts before you make up your mind.