Most of my recent engagements with large corporates moving over to Agile or scaling with Agile have had major challenges in implementation. This has been in no small measure due to them continuing with traditional organization structure thinking even while moving to Agile for delivery. You could change processes, you could re-skill developers, even change their mindset to being Agile but changing organization structures to align with an Agile philosophy is a far bigger challenge. If you are a start-up trying to scale, it is not so difficult but if you have a few thousand engineers already, you are in for a real challenge.
Traditional organizations are structured around functional expertise where individuals with specific skills report into a functional manager. For example, while the development engineering organization is often responsible for delivery, test engineers report into a Test organization head, user experience design (UxD or UED) folks report into a similar functional head and Product Owners report into a Product Management organization – all of them supporting delivery. Every one of these functional organization units (OUs) service different product lines. Agile or Scrum teams are cross-functional teams drawing resources from each one of the functional OUs and work with either a Scrum Master or development manager co-ordinating delivery for each of these teams. This leads to major challenges in a dynamic marketplace where speed of delivery and innovation are crucial for success. Some of the challenges that I have seen include:
- While they may work with other engineers from other functional OUs, individual team members often look up to their functional head for issues or challenges they face – for example, test engineers look up to their test manager for delayed builds or quality issues with builds, and development and test engineers both complain about UED or UxD being a bottleneck for them to complete stories. There are perceived “hand-offs” between “sub-teams” even if everyone is supposed to be part of the same team working on the same goal
- Ownership for delivery is not uniformly felt across all the team members – very often, the development engineering manager is charged with responsibility for delivery and he/she is the one running around and who is also pulled up if there are delivery issues.
- The development engineering managers are responsible for delivery but there is often no accountability for other functional managers for delivery delays or quality issues.
- The resources are controlled by the respective functional OUs – if an engineer leaves the organization, there is a lot of running around to get replacements. The development manager responsible for delivery faces the heat but is at the mercy of the concerned functional unit head to get the replacement!
- All team members are not necessarily co-located – while most organizations have development and test engineers co-located, UED / UxD team members or Product Owners are often not co-located with the dev/test team members of a team. I have often seen UxD team members skip ceremonies in such cases.
Overall, there is frustration all around, major ownership issues, finger pointing amongst team members and managers – all of which lead to avoidable delivery issues that are significant from a customer or product perspective.
Why do organizations continue with such functional structures? Here are two major reasons:
- Functional OUs are competency centers that help to develop core competencies in specific areas – such as UxD or Testing. Functional units track emerging techniques, technologies and trends in these areas and group members are kept abreast of these so they are able to contribute at a higher level on an on-going basis.
- People Management aspects such as resourcing, performance management, competency development, and career planning are handled by the functional units since they have a good understanding of what individuals in these areas bring to the larger organization.
Even if organizations realize that they need to break boundaries between functional units by moving to a strong matrix structure for speed of product delivery, it is difficult to drive change in mindset needed for such matrix working at middle / senior management levels.
With Agile, delivery teams need to be nimble to respond to the dynamic characteristics of the marketplace. Innovation is almost a must for survival as an organization. Both these require a different mindset and culture at the leadership level first. They need to place the organization goals ahead of their functional unit goals. There is a need is for a delivery focused organization where cross-functional teams feel empowered and the collaboration so good that there is no time and energy wasted on hand-offs and perceived conflicts that hinder delivery. Leadership should facilitate this working and provide the right environment, infrastructure and support for teams to succeed.
To start with, a good approach to getting the right organization in place for Agile is for even large enterprises to behave like many small companies (ideally start-ups!) for their different business units (which could have revenue responsibility for specific product lines). Each of these business units should be self-contained with all required functions such as dev, QA (testing), UxD, Product Mgt etc. The leadership needs to provide an environment where cross-functional teams feel empowered and self-organize for delivery. Leaders should also help facilitate informal communities of practice (CoPs) that could even span across product lines / business units, for engineers to share learnings and develop competencies required to remain competitive.
One of the things to avoid doing in this process is to copy other organizations or blindly implement a model thinking that will solve all your problems. For example we have seen large organizations (unsuccessful with other scaling efforts) trying to implement the Spotify model ending up just renaming their Scrum teams to Squads, specific Product Engineering groups to Tribes and creating Chapters for testing, User Experience and architect communities of practice (CoPs). The underlying philosophy and leadership culture required for success were missing and teams suffered from lack of autonomy and environment support, resulting in frustration all across the organization. The first step is to eliminate strong functional silos and understand the spirit behind matrix working. The leadership should first become the change they want to see across the organization. When the leadership sets an example for effective matrix working with the appropriate culture, it invariably leads to success with Agile.
