The Waterfall I enjoyed in my Agile Journey

It is a Sunday afternoon. 

I remembered that I am scheduled to do an Agile Training for a group of 20 associates in my organization tomorrow morning.  For a moment I thought of looking at my slide deck to keep it ready for tomorrow’s session, immediately another powerful thought came in saying, “You have been doing this training for several years now!  Parroting this tomorrow won’t be a challenge.  Leave it!!”.  As usual I listened to my second thought that was favourable to me for a Sunday Afternoon.

Leaning back, I started reflecting.  “When this kind of mass class room trainings began for Agile?”.

Way back, we were part of the teams whether it is Scrum or XP or IID.  It was exactly the “Manager – Teacher within” principle of Toyota.  We never had any lengthy, formal, large group Training as we see today.  Training happened as and when a context emerged within the teams, around applying the principles to the specific scenario.  The teaching used to be informal and spontaneous where learning was given importance over teaching.  Importantly there was no feedback collected for the Trainer. Most of the times, we saw teams practicing the ones they learnt in the next couple of hours.

I recall that these mass class room trainings are usually conducted by a Trainer, who is considered to be an expert in the subject but has no clue about the team, their product, technology, etc,. I saw these kind of large Training practices have started for Agile only after large organizations began their journey towards Transforming their Enterprise to Agile. As usual these organizations went ahead and defined a set of step by step processes, as they normally do for any other enterprise transformation exercise, wherein each process step is associated with progress indicators.  For many organizations, one such a progress indicator – for Adoption of Agile – is the number of employees who have undergone this type of trainings.

The belief is – the number of people trained in Agile, is seen as one of the measures adding to the degree of Agility.

I personally designed my training format in a such a way that soon after the initial introduction, I always start with a general question, “Why Organizations would like to adopt Agile?”.  I use to write down the responses from the participants in a Flipchart, which helps me to revisit those responses at the end of the session to see if Agile is able to address those and how. 

From the time I started my first such formal class room training, almost all those responses to this question, invariably start with, “Because in Waterfall, ….”.  This experience is not something that is unique to me.  If you ask anyone, individually – the same question – “Why should we follow Agile?”, the response is highly likely to start with “Because in Waterfall …”.

Throughout my career, I never worked in such “Waterfall” model as referred by many and hence I became curious after hearing such responses in my first few such formal classroom Agile Training Sessions repeatedly.  Sometimes, I ask them back, “Do you know the sources of reference to the Waterfall that you are referring to? Something similar to Agile Manifesto – Values and Principles?”. 

None could respond.  It appeared to me as if people casually refer to “Waterfall” mostly with a view that it is believed to have everything that is opposite to that of what “Agile” is.

After some years, I got a pointer to the reference source, most often cited for “Waterfall”, – “Managing the Development of Large Software Systems” – by Winston Royce, in the Proceedings of IEEE Westcon, 1970.  This a paper was presumed to be presented in response to the ad-hoc, “code and fix” approach that existed in early 1960 and before.

After studying this article deeply and discussing with a handful of software process experts, I strongly felt that this article is incorrectly understood for unidirectional flow of work, that is irreversible and does not have a feedback loop.  However, it is interesting to note that Royce recommends a totally different approach from what is today popularly construed as “Waterfall” with its firm unidirectional sequence of Analysis, Design, Development, Test Phases.

Royce himself, in this same paper realizes and says that the unidirectional implementation is risky and invites failure.  Hence in order to mitigate this risk, he introduces a five-step approach.  In this, the first step is, iterative interaction between various sequential consecutive phases. (Figure-3 – as in the paper).  While realizing that the feedbacks from some of the phases are never confined to the successive steps, as recommended in Figure-4 (as in the paper), with “Program Design” phase,

Royce goes on to recommend, “Do it Twice”.  To Quote:

If the computer program in question is being developed for the first time, arrange matters so that the version finally delivered to the customer for operational deployment is actually the second version insofar as critical design/operations areas are concerned.

He goes on to say that, “It is simply the entire process done in miniature, to a time scale that is relatively small, with respect to the overall effort”.  This justifies its necessity when there are new elements and unknown factors (similar to what in Agile many today call as “Spike”).

To summarize, Royce, clearly identifies the risk in sequential flow of work (what is now popularly referred to as “Waterfall”), introduces iteration with feedback and further goes on to recommend operational deployment of the second version.

The anti-patterns are not proprietary to Agile; in my view, we can see anti-patterns of “Waterfall” even today. 

Unfortunately this incorrect understanding resulted in ill-treating “Waterfall” in the name of well-treating “Agile”.

After few years, I brought this topic, in one of my personal conversations with Craig Larman. I asked, “When many claims to be following Waterfall for years, why they have not clearly understood what Royce meant?”.

“………………………………………………………………” He gave a long pause.

“Knowing the name of something and knowing something are two different things!!”.

It took sufficiently long time for me to absorb and digest this statement. 

The Next morning!!

As usual I went early to set up my presentation, check the office supplies and arrangements in the Training Room. 

Participants started arriving. 

After the initial introduction, as usual, I began my session with the question –

“Why Organizations would like to adopt Agile?”.

Before I start walking towards the Flipchart to write down the responses, the answer came like a flash.

“Because in Waterfall ………“

Sairam