If we observe the practices in the Software industry, we can notice that many of them are strongly influenced by the Industrial era experiences. For example, the hierarchical organization structures, management practices, leadership styles, quality management systems etc. still have the legacy of 300 years of industrial influence which are predominantly derived from the mass-production context. Whereas, the Software industry is design/creation intensive and mass production is trivial. The following diagram illustrates this contrast.
The traditional practices in Software Engineering & Management are force-fitting mass-production learning into the design-creation intensive work. While the principles of those learning are still valid, they need to be reapplied in the Software context. Let’s ask a few questions to understand this paradigm shift.
What is the raw material for Software? – It’s the knowledge/idea.
Where does the transformation from raw material to finished product happen in this context? – It happens in people’s minds. It is not happening in computer, as some people may think; Computer is just a tool which captures whatever we type.
If transformation is happening in people’s minds, then how to know how the transformation is happening? – We can know this only if the person, in whose mind the transformation is happening, brings it out in different ways like through pictures, documents, models, prototypes, communication etc.
What is a measuring equipment to check whether this transformation is happening correctly or not? – It’s another knowledge-body like peer, Customer, Expert.
If we look at this paradigm, we are dealing with intangibles like knowledge/idea as the raw material and idiosyncratic human beings as the transformation engines, not stereotype machines. Unlike in other engineering disciplines like mechanics or civil engineering, in software engineering there are no laws of physics, objects are not physical in nature. This is a significant paradigm shift. The below table contrasts this paradigm shift from industrial era context to knowledge era context.
This understanding drives several significant paradigm shifts in the traditional software engineering & management practices. Some of them are illustrated below.
Paradigm Shift 1: Repeatable process to Contextual process.
What would be your response if someone asks you ‘what is a process?’. The following are the typical responses we get from hundreds of software professionals/teams: Process means “Set of steps to perform to get output from input”, “Standard way of doing things”, “Repeatable tasks to be performed”, “Clearly documented instructions to do a job”, …. As you can see, these patterns are coming from the industrial era thinking – like a shop floor context. The equation Input à Process à Output and Repeatable Process leads to Consistent Output worked in manufacturing/mass-production context, as the Input were fairly homogenous, variability was less and transformation engines were stereotype & repetitive. Where as in Software production context, the input has high variability – like people, technology, project context, and the transformation engines are idiosyncratic & constantly changing human minds … In a Function f(x) = y, if x is varying and we want to get consistent output y, then the Function f() must change, isn’t it? So, the appropriate equation in Software production context is Variable Input à Contextual Process à Consistent Output.
Paradigm Shift 2: Process driven to Principles driven
When variability and diversity are high how to get consistent output? Can we define rules, dos & don’ts for all possible situations? It will not be effective. A better way to deal with such situations is to have a broad framework of way of doing driven by values & principles – like Agile Values & Principles. Guided by these values & principles, teams will use appropriate contextual processes to produce consistent outputs. For example, a checklist approach can be used for preventing poor soldering because there are only a few causes for that – like soldering paste, temperature, surface cleanliness etc. Whereas the causes for defects in Software Code are many – like communication, requirements understanding, experience of programmer, motivation, distraction etc. 80% of Software defects are known to be caused due to human errors. Collaborative programming, iterative learning, quick learn & adopt cycles are some of the better ways of preventing programming errors.
Paradigm Shift 3: Touch-time of tools to Touch-time of Minds
In the industrial era ‘touch time of tools’ was the primary measure of productivity – how much the tools are touching the raw material to make the transformation happen. In knowledge era context, ‘touch time of minds’ becomes the key imperative for software productivity, quality, and efficiency. For example, a Software engineer may be sitting in front of the computer for 8 hours, but no output may happen if his/her mind is not engaged. The ‘Energy Project’ in Harvard shows that when it comes to knowledge professionals, it’s not so much about managing time, but managing energy that matters most for productivity, quality, efficiency.
Paradigm Shift 4: Supervised to Self-Managed
Since software happens in people’s mind, some one else trying to manage that from outside will not be effective. Primary onus of ‘Touch Time of Mind’ is with the software engineer as the transformation from raw material to finished product is happening in his/her mind along with many other though processes that are going on. Hence Software Engineers and Teams need to be self-managed.
Paradigm Shift 5: Measuring to Sensing
It’s a known fact that only 10% of iceberg is visible and 90% is submerged in the water and not so visible. It’s also known that icebergs move because of undercurrents in the water and not because of the perceivable wind outside. In a similar way if we observe what makes or breaks software projects, it’s the not so perceivable soft aspects that make the projects succeed or fail more than the visible hard aspects.
The visible measurements like effort, cost, schedule, defects etc. are typically lagging indicators and are not much useful in managing projects (they can be useful in analysis & improvement actions). Whereas the soft aspects like shared understanding in the team, collaboration, communication, motivation, team engagement, customer engagement etc. are leading indicators and have significant impact on the project outcomes.
Because of idiosyncratic human beings as the transformation engines and dealing with intangibles like knowledge, idea…etc., measurement is a challenge in software production. What we CAN measure are perhaps not very significant – E.g.: Defect Removal Efficiency, Effort/Schedule variance, … – Mostly Lagging Indicators. Those which are significant, are not so visible/measurable – E.g. People experience, Motivation, Collaboration, level of engagement of people (‘touch time’ of mind), … Early sensing of these soft issues and preventing them while they are small & young, provides a better way to manage projects. So, sensing and sense-making becomes more important than just relying on hard measurements.
Agile/Scrum methodologies provide an excellent scaffolding to build knowledge era practices & culture. They drive contextual team processes with a broad Scrum framework. Scrum practices are designed to increase ‘touch time of mind’ in professionals by bringing focus & time-boxing – e.g. Scrum Master’s role on preventing interference and removing obstacles to team, short iterations and time-boxed ceremonies, reducing multi-tasking and so on. Agile/Scrum practices bring simple measurements like Burn down charts (which drives leading indicator of remaining effort rather than effort already spent), visual task board, velocity & burn up charts to indicate real-time progress on amount of working software delivered against the release target.
In summary, in Software production we are dealing with Human beings as the transformation engines and Knowledge as the raw material; ‘Touch time of minds’ is the key imperative for higher productivity and quality; Capable people Collaborating with Common protocol (process) are the critical success factors; Agile/Scrum principles & practices harness these fundamentals effectively.
