A certainly debatable and may be controversial question. Scott Ambler (SA) debunks the Standish Group’s Chaos report that claims that only one third of all projects world-wide are successful. He uses his own survey to gauge success and claims that two-thirds of Agile projects are successful. One can question whether Scott’s survey and data are really valid, but based on my own experience of working with many organizations in an Agile coaching role for close to a hundred teams over the last 10 years, I believe that represents a reasonable reality.
Now the big question is how does one define Project Success? And who should define it? While we can endlessly debate on what defines success, let me take the easier route by using Scott Ambler’s definition:
- Successful if a solution has been delivered and it met its success criteria within a range acceptable to the organization;
- Challenged if a solution was delivered, but the team did not fully meet all of the project’s success criteria within acceptable ranges (for example, the quality was fine, the project was pretty much on time, but ROI was too low);
- Failed if the team did not deliver a solution.
Interestingly, Scott talks about ROI as an example of a criterion and not so much directly the cost or the budget involved. This seems sensible because cost and budget are constraints and only when you compare them to value delivered do we have a meaningful metric. So ROI becomes a more appropriate metric to look at. While normal projects also look at ROI, more often than not, success is measured by effort or schedule overshoot and not so much the ROI.
The next question is about who should define success. Is it the development team, the Product Owner (PO) or going one step further, the end user or customer involved? My own view is that success is measured by the Agile team and the PO (representing the customer) together, taking a holistic view of benefits / outcomes and making every effort towards achieving them including NOT doing some work, if the ROI does not justify that. Having said that, there must be a clear alignment on criteria for success across all concerned stakeholders and a mutual agreement on the way those will be measured. In a vast majority of projects, timeliness and quality of delivery, value delivered and ROI would be key determinants for success, and given the way teams work in a typical Agile project, it is fair to expect almost all Agile projects to succeed.
So are all agile projects successful? I am sure you are curious to know why and how – you will appreciate when you consider and understand the following are the key characteristics for an Agile project.
- Team working on a prioritized list of features, with priorities based on value delivered to users and a corresponding ROI
- A “lean” approach to cutting down on waste by focusing on just-in-time design and development of high-priority features (no huge specifications or massive architectures and designs, no un-tested code etc. etc.)
- A cross-functional team with all the required skills collaborating with the customer / customer proxy (PO) on an on-going basis through the course of the project
- Constant / frequent user feedback on what the team does and produces, with the team responsive to feedback and changes
- A shippable product increment ready to be deployed, virtually every few weeks or as required
If these characteristics were indeed true, I am sure you will agree that the probability that an Agile project will ever fail is close to zero. If that is indeed the case, why is it that one-third of Agile projects are still challenged or fall in the fail category (assuming Scott Ambler is indeed right with his survey data)? In my own experience, the following could be possible reasons:
- Lack of effective collaboration between development and test teams, and between the engineering team overall and the concerned PO. Given that we are dealing with human beings and the soft aspects involved in effective collaboration, the organization culture plays a huge role in this process. It takes a while, sometimes even a few years, before things fall in place in this regard even if the organization is fully aligned and supporting.
- A related point on development and test team collaboration is on organization structures and reporting relationships combined with Performance Management Systems. For true cross functional collaboration to happen, we need the two teams to be well-aligned in terms of Agile values & culture, effective matrix structures and a suitably aligned Performance Management System.
- Ineffective POs, with their lack of clarity on requirements, often end up confusing and frustrating engineering teams.
- Ineffective Scrum Masters whose role is to cushion the team from mid-sprint changes and interrupts, often do a poor job of this, allowing Product Management and their own delivery leadership teams to play around with the team’s time and priorities.
- Most organizations don’t have an effective balance when it comes to Product Management and Engineering leadership. They are either very strong in one of the areas – Engineering or Product Management – and weak in the other area. Strong Engineering (with weak Product Management) organizations may become irrelevant in the market place very soon as they are out of tune with reality. Where the Product Management is strong (and Engineering relatively weak), I have seen Product Management thrust decisions on engineering and mis-interpret Agile to mean that changes can be done at will and that planning is a meaningless exercise. In such organizations, ‘scope is flexible when schedule is rigid’ works only in theory.
So, in order for all Agile projects to really succeed, one needs to consider and address all the above mentioned aspects and challenges. And that does not happen over-night.
I will leave you with one final thought – it is still wonderful to see that two-thirds of Agile projects succeed (going by Scott Ambler’s survey). I believe that with better organizational maturity happening over time combined with effective Agile adoption coaching that drives improvements and the required transformation much faster, we should start to see even better numbers in the next couple of years.
What is your view / experience on success percentage of Agile projects?
Look forward to hearing from you!
