« Project Management and ITIL – A BoK Comparison
Approval and Sign-off »

Managing Testing Resources: Five Suggestions for the Project Manager - Developing a Project Plan for the Testing Sub-Project

April 17, 2008 | Author: PM Hut | Filed under: Project Plan Development, Quality Management

Managing Testing Resources: Five Suggestions for the Project Manager - Developing a Project Plan for the Testing Sub-Project (#4 in the series Managing Testing Resources: Five Suggestions for the Project Manager)
By Cem Kaner and Johanna Rothman

To determine whether the project has met the criteria (which is Johanna’s view of the testing task) or to prove that the product has not met the criteria (Cem’s view of the task), the test group has a lot of work to do, probably involving more than one person, over a period of weeks or months (or years). Somehow, they have to be able to tell what is supposed to be done, what they’ve actually gotten done so far, what’s left to do, and when they have achieved a goal or met a milestone.

Different test managers handle the planning problem differently. Forget about the incompetents and the turf-hoarders and the blame-it-all-on-you experts. Competent, cooperative test managers differ significantly in how they schedule and in how they communicate their schedule to the rest of the company. We’ve had success with variants of a method written up by Kaner and we recommend it to testing groups (”Negotiating Testing Resources: A Collaborative Approach.” Proceedings of the International Quality Week Conference, San Francisco, 1996. Available at http://www.kaner.com.). However, you can’t impose a method on a good test manager (you can try, but the next test manager might reject your method too, after the first one quits.) You can ask for clear communication.

What should be in the plan

Here are some of the things that we think it is fair to request (and expect) in the testing project plan:

First, a clear statement about the minimum level of testing that will be done for every area of the product (program plus documentation plus marketing materials plus associated hardware). Some areas will get more than this, but none will get less. What is that minimum? Do you think it is enough? Too much?

Second, a description of different levels of depth of testing that will be used in different areas of the program. For example, we often think in terms of four categories:

  • Mainstream or Normal Use testing works the product gently. The tester tries out the various options but is not intentionally using extreme values to break the product. In mass market software, this level includes a complete verification of the program against the user manual. (For a discussion of the need for documentation testing in mass-market products, see Kaner, “Liability for Defective Documentation”, Software QA Quarterly, volume 2, #3, p. 8., 1995. Available at www.badsoftware.com/baddocs.htm; Kaner & Pels, “User Documentation Testing: Ignore At Your Own Risk”, Customer Care, volume 7, #4, p. 7-8, 1996).
  • Guerilla testing involves ad hoc testing done by someone who is skilled at finding errors on the fly. It is one person’s best shot at finding bugs. This approach is typically time-limited. For example, to say that an area will be guerilla-level tested, you might mean that this area of the program will receive a total of two days of ad hoc testing, spread across the project. Normally, guerilla testing is done after (not instead of) mainstream testing.
  • Formally planned testing involves carefully thought through test cases that are intended to thoroughly test that area. Depending on your company’s philosophy of testing, this might mean a set of test cases that collectively trace back to (check) every item in a list of specifications. Or it might mean a set of test cases that cover all of the extreme values and difficult options and uses of the program (or other product component) in this area. Or something else. It is harsh testing, intended to expose those problems that a customer would find if the area were not tested and fixed.
  • Planned regression testing involves carefully thought through test cases that are run frequently, perhaps every build or every few builds. They are designed to recheck an area of the product that was well tested, to determine whether it is still as stable as it was previously. Developing this test suite takes much longer than the development of the first good plan for testing an area. Here you are selecting fewer tests (or automating many of them), searching hard for efficiencies and for test cases that would be particularly revealing of side effect problems.

Your test manager might use different names and different descriptions. There’s nothing magic about ours. You just need some descriptions that are clear, clearly different, and that cover the options that the test group will actually use.

Third, a list of the areas of the product. The test group will define the “areas” in its own way. You might help them with this or not. They have to be free to organize their work in a way that works for them, which might be different from how you would do it. However, you should be able to get from them a list of areas that together include all of the things that the testers will test.

Fourth, for each area of the product, a list of sub-areas or sub-tasks. This list should be detailed. It should include anything that takes a day (or even half a day) of work. Having things broken down this finely makes it possible to accurately estimate the size of the task and to accurately track how much work is remaining to be done. You might not be able to get this list–some people refuse to estimate tasks this finely, dividing the project into two week phases instead. We would want more than that, but you might not be able to force it.

Fifth, for every sub-area of the product, the list should specify how much time it will take to test that sub-area at the level of depth that it will be tested.

Sixth, a total across sub-areas. For every major area of the product, how much time will be spent testing, and, overall, what is the level of testing of this area?

This list of areas and sub-areas gives you something to review, to negotiate, and to track progress against. If the test group wants to spend too much time on one area, you can ask why they intend to test this area’s sub-areas at the levels they have chosen. What tradeoffs are they making? As you come to understand the test group’s tradeoffs, you might decide that they really do need more time and money. Or you might persuade them to test some areas less intensely (with your support). Or you might come away with a well-understood disagreement. In any case, the level of detail of the list is what makes possible the calm, task-oriented (rather than pointy-haired-manager-wants-to-save-money) discussion that safeguards the quality of the product by focusing the most resources on the most important tasks.

Original article can be found at: http://www.jrothman.com/Papers/Managingtestingresources.html

Johanna Rothman consults, speaks, and writes on managing high-technology product development. Johanna is the author of Manage It!’Your Guide to Modern Pragmatic Project Management’. She is the coauthor of the pragmatic Behind Closed Doors, Secrets of Great Management, and author of the highly acclaimed Hiring the Best Knowledge Workers, Techies & Nerds: The Secrets and Science of Hiring Technical People. And, Johanna is a host and session leader at the Amplifying Your Effectiveness (AYE) conference (http://www.ayeconference.com). You can see Johanna’s other writings at http://www.jrothman.com.

Share this article:
  • Digg
  • del.icio.us
  • Facebook
  • Google
  • blogmarks
  • LinkedIn
  • Reddit
  • StumbleUpon
  • TwitThis
  • Yahoo! Buzz

Related Articles

This entry was posted on Thursday, April 17th, 2008 and is filed under Project Plan Development, Quality Management. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

No comments yet.

feel free to leave a comment

Comment Guidelines: Basic XHTML is allowed (a href, strong, em, code). All line breaks and paragraphs are automatically generated. Off-topic or inappropriate comments will be edited or deleted. Email addresses will never be published. Keep it PG-13 people!

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

All fields marked with " * " are required.

Project Management Categories