Life After Waterfall

Changing a culture within a company is no easy task and the expected struggles popped up everywhere. From middle managers expecting certain types of reporting and documentation to infrastructure changes and adopting new templates during development. QC bugs vs. TFS bugs, TFS Backlog vs. the old faithful excel spreadsheet and the list goes on.

So assuming the technical design bits, contracts and resource allocations are out of the way, the first challenge we had was to get the team to buy into Agile. The only time an individual will buy into something is when they realize that that something will make their lives better, or in this case the software. As a programmer myself, and as most of us know, the general perception of programmers is that they don’t enjoy participating in any form of group interaction. Programmers are used to work being drip fed to them. “There you go Pete, please write the XYZ screen for us. We need it before the end of Friday.” Pete will go away to do this task and hopefully get it done before Friday. There is no emphasis on ‘Business Analysis’, ‘Quality’, ‘Design’, ‘How does it fit in with the bigger picture?’, ‘What’s the criteria for getting it 100% correct?’. This mentality is exactly why 9 out of 10 projects fail, fall behind of schedule or go over budget. Regarding the deadline of ‘before the end of Friday’, would you tell a Michelin Chef to make you an omelet but it needs to be done in 60 seconds? Whilst ducking and diving between the flying knives you’ll hear him screaming ‘It’s ready when it’s ready!’. In the same way the piece of software is ready when it meets the Acceptance Criteria. Or meet the agreed upon definition of done. And if you don’t trust your Developer’s estimation and work ethic there is a different dilemma which should be addressed.

We needed to break this mentality, for the programmer as well as the manager. Also we had to pull the testers and analysts into our team, to make them part of our team and to show them that we are all equally important to make this successful. No more throwing work over the fence to the other team, but rather small iterative processes between all members to deliver a high quality working piece of software. Initially the team didn’t speak up or interact a lot, they wanted instructions and did not want to change the way they work. But gradually the team started interacting and soon the team started to function together. They went through a painful process from no meetings and technical discussions, to long meetings and a lot of planning sessions. There were quite a few complaints about productivity in general due to these long meetings, and because it is hard to measure the value of quality early on this didn’t go down well. However, after Release 2 in 2011 we had 8 weeks assigned to us for bug fixing, and for release 3 the same thing. This was the norm. Long development periods followed by long testing and bug fixing periods. However, when that time came, we had almost zero bugs! The quality was spectacular, and because the product owner was involved all the way there were no surprises. A lot of times you would get stakeholders or end users saying, ‘this is not what we wanted’, or ‘8 months ago this made sense, but things have changed so this is no longer valid’. We used those bug fixing periods to work on the remaining tasks and to get ahead of schedule. This worked really well for us.

Product Backlog and Velocity

We also realized the value of a visible and prioritized Product Backlog. (A list of prioritized items of all the remaining work) And how that goes hand in hand with an active Product Owner. (The person who is responsible for the priority of the work and who can make decisions on the spot) Up until then we only had Product Backlog Items for the current work we were working on and very little visibility of what’s to come. We got an agile expert in from Hitachi consulting to come and help us to set this structure up for the remaining work. This took one person roughly 4 weeks to complete. Part of that process involved quite a few ‘Planning Poker’ sessions where we’ll have teams sitting together, from different roles and responsibilities, estimating how big a particular task is in relation to a predetermined task. So if creating a login screen was of size 5, how big would the Find Tool screen be? We would discuss this amongst everyone and break stories up into smaller pieces until we got some sort of consensus of how big a particular task would be…all relative to the Login Screen of size 5. Pretty soon we had an idea of how much work was left. And considering it took us x amount of time previously to complete 100 story points, we could now estimate how long it will take us to do the remainder. In theory this sounds great, but one mistake we made was to use the velocity of the previous team and apply that on the newly estimated points, estimated by different people. Consider this: Team xyz went through an estimation process where they assigned points to tasks. They used the same principle by giving a task points, relative to a task that was agreed upon initially. However, for team xyz a login screen was a 2 and not a 5. So if it took team xyz 6 days to complete a 2 story points task, you cannot say that it will take them 6 days to do a 2 story point task on the newly estimated work. Because of that mistake we thought that we’ll only complete the work in 2013(a year late) which was unacceptable for the business. Needless to say there were a lot of unhappy people. However, after a few sprints we could see that the velocity on the newly estimated work was radically different which placed things in perspective again. And soon we were back on track to deliver on time. This was an important lesson that I won’t forget.

Quality

Another rule we set in place (that is still helping us today) was that no developer will start to code if the Acceptance Criteria for that particular task is not in place. If John wants to start coding the login screen, but realized there were no acceptance tests, he’ll go over to the Business Analysts and ask them if they could sit together and get it on paper. However this situation should actually never happen. Before each sprint(2 week cycle of work), all the planed stories should have acceptance criteria already. This means however that the scrum master must plan ahead and notify the BA’s and testers of what’s in the pipeline and what we’ll work on next. Also, if the Scrum Master sees that there are a few difficult tasks coming up(8 points and above), he should raise this and notify them that the task will take longer to analyze, in effect giving them more time to come up with the acceptance tests. Unfortunately the only thing in this process that was and still is really hard to get, is dedicated time from the BA/Tester. Ideally a team of 4-5 developers need a dedicated tester, 8-10 will need at least 2 testers. But here we are lacking Testing and BA resources and when we find ourselves in that situation we count on the developers to fill that void. Unfortunately this is impacting our velocity and quality dramatically. This was formally raised as a risk but dismissed. Sometimes ‘real’ life clashes against agile principles but you can still align yourself to the methodology as best as possible. It will be harder on the scrum master, but if you get a team that talks a lot and works together most projects can be successful.

We are currently in testing phase of Release 1, 2012, working ahead on Release 2 work. We are ahead of schedule and our code quality is very high so we aren’t expecting too many bugs to come through. However there will always be bugs and there will always be room for improvement. One way we found really useful to perk up our day to day as well as our releases in general was to have regular retrospective meetings. Basically asking ourselves, what didn’t go well in the previous 2 weeks and why? How can we make it better? What did go well, and should we keep on doing it? Examples of this was how we realized early on how valuable ‘Code Reviews’ were and how bad we were at delivering regular pieces of testable code.

Johan Van Staden is a certified Scrum Master at Hitatchi Consulting.

This entry was posted on Tuesday, February 21st, 2012 and is filed under Agile Project Management. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.