The Perennial Time and Scope Dilemma

July 25, 2010 | Author: PM Hut | Filed under: Requirements Management

The Perennial Time and Scope Dilemma
By Literal Thinking

In one of our earlier articles, we introduced a fairly familiar project scenario where signoff points for scope and time overlap. This typically happens when deliverable (scope) signoffs are crammed into one time block, often also coinciding with the end of a particular phase (time) of a project. In some projects that we have seen and been involved in, this simplified approach had led to project managers choosing to either compromise on scope or on time (sometimes both), just to make the signoff point. We call this the perennial time and scope dilemma.

To illustrate this time and scope tension, we will walk you through some of the key sign-off decisions made in each major phase of a global IT outsourcing project by a large consumer goods manufacturer (hereafter we shall refer to as the client). The project was undertaken with support from a consulting partner (hereafter referred to as the vendor) that was also going to be the outsourcing organization after project completion.

Business Blueprint

The business blueprint document, arguably the single most important document in any IT transformation project, was signed off unconditionally with major sections of it indicated as “to be completed”. The vendor used strong arm tactics to force the client to sign off, arguing that the blueprint completion delay was singularly due to the client not being able to provide the required resources for on-time delivery. For the client, it was a matter of signing off on an incomplete blueprint, or paying a hefty delay cost.

The client cascaded the strong arm tactic down to its internal subject matter experts (SMEs). The SMEs were forced to sign off on the data migration design and plan without being given the opportunity to seriously assess project implications to both their retiring and retained systems and processes; or scrutinize two 150-page documents and four complex workbooks, which were to be used as primary tools for transferring data.

All custom development functional specifications were conditionally signed off, contingent on missing blueprint information being provided in the next phase of the project and proper assessment of proposed technical solutions. It was correct to conditionally sign off these deliverables, but their dependence on a completed blueprint document, when it was already unconditionally signed off, was a big risk.

The sign off requirement for a go live checklist was postponed as it was deemed unnecessary at such an early phase of the project. The project would eventually go live without a formal checklist being signed off.

Solution Build

It quickly became clear to both client and vendor that the business blueprint documentation did not contain adequate information to allow the vendor to build a complete enterprise solution. The vendor tried to push the client to raise change requests; while the client pushed back, arguing that while the business blueprint documentation may not have been completed, the enterprise solution requirements had always been clear.

Client and vendor met halfway, but a lot of effort was wasted on just scope renegotiations. The checkpoint was reached, and the enterprise solution was not yet completely built. To ensure the project continues on schedule, both client and vendor project managers agreed to conditionally sign off on the enterprise solution, with the proviso that incomplete components will be progressively delivered in the next phase.

The conditional signoff of the enterprise solution had a trickle down effect on the rest of the deliverables. For example, the data migration processes and most of the custom development requirements were reliant on the enterprise solution being completed, so these were also conditionally signed off.

Part of the conditional signoff of some custom development deliverables was also due to the lack of time to complete the developments. The original project schedule was made based on a number of assumptions, one of which was the number of custom development requirements. These requirements were significantly underestimated, but the broader scope did not deter both client and vendor project managers from sticking with an ambitious delivery schedule.

Additionally, some other requirements like super user training and cutover planning just could not be delivered with a half baked enterprise solution, so signoff for these were postponed to the next phase. Not surprisingly, these were not looked at again until late in the project.

Integration Testing

Integration testing is all about making sure that the to-be-implemented enterprise solution can replace retiring system functionality, including integration with mission critical retained systems and processes. With an incomplete enterprise solution, integration testing for this project was a shambles. In fact, proper integration testing processes were totally ignored, and activities during this phase were more to ensure that systems and processes were ready for user acceptance and parallel testing (the next phase).

Full end-to-end data migration testing was a mess. The client did not have enough time to analyze their source system requirements; while the vendor was still in the middle of building the solutions during data migration testing, so the target system requirements were also not yet properly defined. The whole data migration team struggled to get data loaded properly, and virtually every step of data migration had errors. In the end, the goal changed from end-to-end data migration testing to just making sure that data is loaded into the user acceptance and parallel test systems. There was no proper data validation process, and strong arm tactics were used to force the data migration SMEs to sign off on data migration testing.

A lot of the custom developments, most especially interfaces to mission critical systems, could also not be tested properly. This was due to the incomplete configuration of the enterprise solution, the lack of meaningful data to test against, and the still ongoing development activities. Without much headway and with the project still chasing its time requirements, a lot more of the custom developments were conditionally signed off, contingent on these passing user acceptance and parallel testing.

User Acceptance Testing

User acceptance testing (UAT) started with a big conundrum: who should do the testing? One train of thought was it should be the client’s key users, as they were the ones who knew the existing system and process requirements and should be doing the tests to ensure that the to-be solution met all of these requirements. Another view was it should be the vendor’s key outsourcing users, as they were going to be the ones using the system on a day to day basis, post-implementation. In the end, members of the vendor’s implementation team did the testing, the very people responsible for building the solution. It was a glorified unit test, and a farcical user acceptance test activity.

This was an unexpected requirement for the implementation team, so they were naturally not prepared and had not planned for it. UAT test scripts, which should have been completed and signed off in an earlier phase of the project were haphazardly done, often recycled from previous projects that had little relevance to the current one.

The project managers then decided that because of time and resource constraints, UAT was only going to be executed against core system functionality and maintained at a very high level; and all custom development testing was postponed to the next project phase. But even with this minimal requirement, it quickly became apparent that proper testing was going to be an arduous process, mainly because of the low quality of data loaded during integration testing and the still incomplete enterprise solution. So despite the significantly reduced scope, UAT could not be completed on time.

To ensure that the project remained on track, both client and vendor project managers decided to start parallel testing while UAT was still going on. UAT and parallel testing thus ended up running in parallel.

Parallel Testing

To make sure that the to-be solution met the client’s system and process requirements, the project simulated a parallel run over two months’ worth of transaction data. Parallel testing started at a later stage of the project, so the data migration team had more time to ensure that the quality of data loaded for parallel testing was of a much higher quality. But the good news ended there.

Virtually every process in parallel testing ended in error and data between the legacy and the new system just could not be reconciled. Various factors contributed to these errors: poor data quality coming from the source systems, incomplete configuration of the enterprise solution, and highlighted missing functionality in the to-be system.

The premature and forced deliverable signoffs done during the initial stages of the project was finally taking a heavy toll and putting tremendous pressure on the project team. Despite this, a very tight deadline for parallel testing completion was still followed, and strong arm tactics continued to be used by the project managers to ensure signoff. The parallel testing team was forced to manipulate data and introduce quick system configuration solutions just so parallel testing could be green-lighted on time.

The problems with the core solution encountered during parallel testing and the required efforts to address these also meant that signoffs of some solution components and functionality, and of various custom developments, were yet again postponed to the next phase of the project.

End to End Testing

End to end testing was identified as a key phase to be completed during project initiation to see whether the enterprise solution and the outsourcing organization could cope with the client’s end to end business process and service delivery requirements. Incredibly, some solution components, various custom developments, and UAT and parallel testing, were not yet completed before end to end testing was to start.

The end to end testing process itself was chaotic. There were yet again no test scripts written; and while the project had members who were experts in their individual work components, it had no identified business process experts. The project team did not have a clear understanding of an end to end business process to test, and no expert to look to for guidance.

Faced with time constraints and the lack of expert advise, both client and vendor project managers, not for the first time, decided to only take a helicopter view of the entire business process chain for end to end testing. And once more, even with this reduced scope, the to-be solution was found wanting.

Last-minute change requests to address process gaps were quickly approved, half-thought of solutions were developed and signed off, and an alarming level of testing variance against benchmarks was accepted and approved. All this was done for the singular purpose of meeting the project’s time requirements.

Go Live Cutover

The project reached go-live cutover without a cutover plan, with both client and vendor project managers still swamped with issues to close out: some custom developments were still not properly tested, let alone signed off; acceptance of testing variances were still being debated; and more change requests were being considered.

The big volume of issues still outstanding forced both client and vendor to bring in two additional project managers, with the sole brief of working together to take the project to go-live cutover. These two “cutover managers” had no experience in the project yet were given free reign to map out the go-live cutover plan.

A mad rush of signoffs and completions occurred in the last two weeks before going live. Changes to system configuration, with minimal testing, were introduced very late in the project because of even later solution gap identification. Critical custom developments with barely there testing were finally signed off, with some developments just getting completed during the last weekend before go live.

Data migration took two weeks to complete, with the data migration team losing count of the delta loads done. No one could swear on the integrity of data loaded into the new production system, and everyone crossed their fingers when go live cutover was signed off – on time.

We are deliberately stopping the case discussion at just before go live to let you, the reader, imagine how go live – and the immediate post go live – activities went. What is most amazing is this project was actually delivered on time and on budget; and with some creativity, it can even be reported to have been on scope.

Literal Thinking is a blog that aims to present real stories of workplace follies. While we may occasionally post some interesting high profile cases of corporate missteps and failures that we find in the news, our focus is more on typical working day experiences of typical working people.

Our goal is not to name and shame, but to share stories and experiences in the hope that the reader is able to take away some valuable, practical learning that they can apply in their own workplaces. So they can go to where others have previously gone to and failed, and succeed.

Share this article:

This entry was posted on Sunday, July 25th, 2010 and is filed under Requirements Management. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

No comments yet.

feel free to leave a comment

Comment Guidelines: Basic XHTML is allowed (a href, strong, em, code). All line breaks and paragraphs are automatically generated. Off-topic or inappropriate comments will be edited or deleted. Email addresses will never be published. Keep it PG-13 people!

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

All fields marked with " * " are required.

Improve Project Performance With Lessons Learned

Defining Program Management and How Agile Helps

Facilitating a Requirements Validation Meeting with Ease!

Decisions, Outcomes, and the Project Manager

Project Management - Can You Handle the Truth?

Eliminating the Reasons Organizations Choose the Wrong Projects

On the Interpretation of Probabilities in Project Management

Proactively Address Unpredictability about Project Resource Availability

Project Management Checklists

Ambiguity Kills Projects