What Organizations Should Use the Agile Methodology in 2019?
Welcome to my blog about organizational, program and project strategy, management and controls. Each week, I’ll post a blog on a new subject. Please, suggest one if there’s something on your mind. I’ll put it in queue for discussion in upcoming blogs. Last week, I created a brief, high-level video about the pitfalls of creating a strategy that can’t be executed. See previous blog posting.
This week the topic I’ve chosen is implementing the Agile methodology, which I think is pertinent, not just to software development, but to many other areas as well.
Numerous organizations are embracing massive system and process changes using the Agile Methodology to introduce scalability, enable repeatability of execution, cut costs and enable growth. Agile increases speed to market incrementally. Industries currently engaged in Agile projects include financial services, professional / business services, energy, manufacturing, retail, and consumer services. These include IBM, Intel, Deloitte, Exxon-Mobile, BP, Domino’s, Amazon, Accenture, 3M, Coca-Cola and Lesara, However, for all its benefits many organizations hit roadblocks during implementation that can have massive impacts on the culture. So why Agile and is it worth the effort?
For software development, integration, and application implementation projects, today’s swiftly developing and advancing technological environment poses significant challenges to organizations, has a huge impact on the required speed to adopt new technologies and reduces the long-term commitment companies have to any specific technological version. As technology advances, there is a lot of pressure to adopt the newest, next best thing to gain an advantage over competition. There is less time to adjust. By the time an organization completes a multi-year project, a new technology may have been born that is light years beyond the technology currently being implemented. The new available options may have improved well beyond the targeted deliverables or requirements. Organizations that are early adopters of technology may find themselves leading their industry with better margins, lower costs, greater revenues.
I had something similar happen in college. My peers and I had spent three semesters studying Basic programming language. At the end of those three semesters (one and a half years), my professor announced that the language was changing / obsolete based on the new release of Visual Basic written by Microsoft. Of course, VB was based on the original BASIC, so building on the knowledge of the original wasn’t difficult.
In business, we don’t have the time or money to constantly scrap multi-year programs mid-way, adopt new ones, and retrain, not if we want healthy margins. If a company’s Innovative Strategy is clearly defined, focused and aligned with mission, vision, values and goals, and Agile is being used as a tool to deliver products and services to the market quicker, more accurately with better response time, organizations will find themselves better positioned to forecast the direction they need to go. However, it is important to realize that Agile by its very nature requires transformation at the organizational level. Not many organizations have autonomous teams that are self-driven and self-managed. This sort of organizational change requires accountability, top-down leadership, setting expectations and building structures that enable teams to make decisions without the red tape of layered bureaucratic approvals.
Recently, I implemented a PMO for a Houston based construction company. This required development of tools, systems and processes to support the project team in addition to an innovation strategy that was used as a long-term guide for why, how and when the leadership team wanted to innovate. As we introduced new systems to the organization, one software company in particular continued to develop tools based on the feedback that the team provided – talk about an Agile company. As we rolled out each iteration, we would review the newest releases from the software company to determine if the newest versions or tools would have a significant, immediate positive impact, or if we should wait and roll them out in a package later if at all. Much of this was dependent on gauging the stage of acceptance within the organization for the current releases. Had the previous iteration and implementation been accepted, and what impact would introduction of a new tool or version have on the organization?
Timing the roll out of a new application, integration or software version shouldn’t depend solely on whether or not the technology meets business case requirements and deliverables, but should also depend on the impact to the organization. An Agile system that can develop and release software in smaller chunks will help overcome resistance to change. Implementation needs to be well planned and aligned with the organization’s Innovation Strategy. Rather than performing large multi-year projects that could require months or years of up-front planning and analysis and may be obsolete by the time implementation occurs, the Agile Methodology within the Scrum Framework plans, analyzes, designs, codes and tests, each incremental development as a separate potential release to the market in a 30-day window called a Sprint. That does not mean that there will be a release at the end of the 30-day Sprint, but that it’s developed and tested to the point that it could be released. The release is based on the whether or not it would be significant, how prepared the organization is, and the implementation plan. Whether the product is released or not, at the end of each increment, there is a product waiting to be released. This greatly reduces the risk that technology and need would have outpaced development. Incremental development and release allow organizations to analyze current business cases and determine priorities or shifts in direction.
So if Agile enables businesses to be more flexible, why is there such resistance to the Agile methodology?
The Agile methodology and related frameworks (Scrum, Kanban, Extreme Programming (XP), Feature Driven Development (FDD), etc) were developed as alternatives to traditional, project delivery methodologies and the perception that the traditional project management and software development delivery methodologies did not work well. The men and women who have executed projects using the traditional waterfall methodology (planning everything up front) have had it hammered home – fail to plan up front, plan to fail. The plan that is established in the beginning is the die-hard plan. Change to a project is supposed to be resisted to the nth degree – traditionally. Fixed scope and budget are red lines drawn in the sand, and woe to any who cross that line or suggest it move. The deliverable set in the beginning is the deliverable that must be delivered in the end, no matter that the project covers multiple years and changing economies of scale, technology and environments may indicate a need to take a different tack. Traditionally, a project would begin as a requirement analysis that leads to an overall general concept. The architecture would then be designed. Once designed, the programming code would be generated. Finally, came testing and deployment. Plan as much as possible on the front end. Design. Create and build. Test. Release / deploy.
Projects would have multiple managers involved in a project. Each discipline would have a department manager. The project would have a manager. Depending on the project, each discipline within the project might have a separate manager overseeing their work. This could lead to confusion. Opposing deadlines and priorities could cause stops and starts. Teams were often unclear about priorities. Results were not owned. The project manager was often a dictator of plans, and interpreter of priorities, and they still might compete with other project managers for resources, human and otherwise. Plans were set in stone. Most changes were considered negative.
The new Agile paradigm is quite different. Scrum, one of the most used Agile frameworks, uses self-organizing teams. Under the Scrum framework for Agile, a project owner is responsible for setting project priorities from the backlog list that is the high-level overview of the project broken down into 30-day Sprints to be executed and tested in 30 days or less by the team. Sprints are further broken down into tasks for which the team holds one another accountable. A “Scrum Master” ensures that there are no obstacles or impediments to progress. Teams meet daily. There is accountability – less folks at the top, but perhaps some challenges to overcome from middle management. The team consists of 6 to 8 people who are collocated ideally and accountable horizontally. Even the Scrum Master and Owner aren’t “bosses” or “managers” in the traditional sense. The project and related business cases / stories are reexamined with each new Sprint. The owner sets priorities and is the governing body of the project / program. The team executes. In some cases, there is an Agile Coach to help guide the team in Agile Practices. At the end of each Sprint, the team looks back at the Sprint and reviews lessons learned to take advantage of them in future Sprints. Most essential are the backlog, teams and tested products that work – seems simple.
So, why the resistance? I’ve heard many reasons.“It’s failed before.”“I was born agile. Don’t need a methodology.”“What other companies are doing it?”“I’m a developer. Why should I care about it?”“My role isn’t even included in the Scrum team definition.”“Meetings every day? Are you kidding?”“Who is going to approve team decisions?”“Sounds like budgets are going to be blown, if there are any.”“There’s too many dependencies between teams.”“There’s limited access to expertise in a timely manner.”“We have to get approval from too many people.”“Teams are dedicated to other tasks.”“Teams share resources or responsibilities to completion.”
The headwinds against implementation are strong. Why? It is a cultural change – a tectonic shift. Agile itself is a change. Organizational culture impacts the way that change is accepted and what types of changes are accepted. Many organizations approach change by introducing new methodologies and straining through training to adhere them to the culture. Confusion may include decision making around adherence to new processes and procedures, the right way versus the wrong way to move a work flow through a system, reasons for the changes etc. In my experience, this becomes especially true when the culture spans multiple larger social structures, for instance, the difference between US, German, Japanese and various Middle Eastern cultures, wherein values and norms differ. Globally, within the organization, they will have common accepted beliefs, practices and norms with similar structures, but perhaps different accommodations. Some examples of globally held beliefs might be, “The customer is always right”, “Safety first”, “treat each other equally without regard to race, gender, ethnicity, religion”, etc. Culture influences who interacts with who. That impacts process flow – another core belief that is ingrained in the culture. What is the right flow? What are the right checks and balances to ensure quality? When Agile is introduced, the historically accepted practices may change to accommodate a more horizontal, team centered approach.
Additionally, within the larger corporate culture there are smaller microcultures, which are also impacted by the introduction of new methods that vie with an organizations currently accepted practices. Implementation of Agile at the microlevel will be different than implementation across an organization. Also, the more dependencies between the microlevels or between the micro and macro levels, the more preparation and change can be expected in order to introduce Agile into an organization.
Once an organization has identified the business challenge(s) they are attempting to solve, and the magnitude of the scope, size and parts of the organization that will be impacted, then the leadership needs to decide whether or not the dependencies can be outsourced or dealt with internally. If the organization is small, the dependencies are probably not as large as those of a larger organization, though I’ve seen complexity in small organizations too.
To begin, the challenge is to answer the following questions and develop an action plan that addresses Agile implementation as part of the Innovation Strategy:
* What is the business challenge we are trying to solve?
* What is the magnitude of scope?
* What parts of the organization will be impacted?
* What is the size of impact on each portion of the organization?
* How many backlog items are there?
* How many intersections / nodes of dependency exist?
* Can any of these dependencies be outsourced, or do we need to keep them in house?
* Can the scope be broken down into 30-day increments that can be uniquely / singularly executed and released?
* Can teams be self-organized and given the latitude to make decisions, or will the hierarchy of the organization and culture create road blocks and bottlenecks?
* Does the organization have product owners who determine and prioritize the backlog and define the definition of completion?
* How will we measure and control progress?
* Is there a dedicated team that is able to execute the backlog?
* If not, can dedicated teams be created? For those who have attempted to use a part-time team to execute the backlog, you probably realize that a part-time team provides no guarantee as to when a backlog will be completed.
* How will we remove impediments to Agile?
To become an Agile organization is often a tectonic shift. When Agile fails in organizations, in my experience, it’s often because of one of the following reasons:
* There is no self-organizing, dedicated team.
* The backlog has not been whittled down to 30-day (approximately) Sprints.
* There is no progress measurement or definition of completion for each Sprint.
In summary, organizations must first determine if they are ready to embrace the Agile Methodology. An in-depth analysis of the business case and potential solutions should to be performed. While most businesses are moving toward an Agile methodology, not all cultures are prepared to make that move. Those that are prepared are more likely to pull ahead as industry leaders in their fields. Organizations that aren’t ready should consider what incremental changes they can make internally to equip their organizations for a more adaptable business model. The profitability of Agile to an organization will most likely be linked to how well the organization aligns itself for the impact. Those that are unable to make the shift may likely find themselves left in the shadows of their competitors. Organizations that aren’t ready should consider what incremental changes they can make internally to equip their organizations for a more adaptable business model.