Agile, What it is and Why it still Matters After 20 Years

Photo by Natalie Pedigo, Unsplash

Introduction

Recently, February 2021 marked the 20th anniversary of the creation of the Agile Manifesto. Two decades and It remains as valuable as it was then.

Given how fast the industry is changing, for a concept to maintain this level of enthusiasm over such a long time is remarkable. Many concepts in engineering and management faded away, but Agile has held strong.

As much as Agile has helped innumerable organizations thrive, some other organizations turned to Agile as a panacea to fix from engineering issues to management problems, but without proper understanding of it, and without appropriate culture shift, their Agile transformation failed.

This article will give you a quick introduction of what Agile is and why it matters.

Problems In The Old Days

Throughout history, people made many things: from small tools to mega constructions, from tangible hardwares to creative arts and intangible softwares.

Making things are basically problem solving activities in which the end product or service yields a specific value to one or more people: the customers.

In the old days, the problems were

• more stable: did not change that quickly.
• they were more predictable: People would usually know what would happen next year, and the year after, because they would follow a seasonal pattern.
• they were simple(r): the analysis of cause and effect were, in most cases, tractable; and
• they were well-known: the work could be defined in a formal way. Of course, except for research projects in labs.

In late 1880 and early 1900 a mechanical engineer called Frederick Taylor introduced Scientific management, a theory of management that analyzes and synthesizes workflows, with the goal of improving economic efficiency, especially labor productivity.

According to Taylor, the worker is unable to understand the science of doing his work.

“… the workman who is best suited to handling pig iron is unable to understand the real science of doing this class of work.” Frederick Taylor

Hence, the work should be broken down into pieces and for each piece of work, provide the worker with detailed instruction: what to do, and how to do it.

Yes, that will bring you the image of an assembly line.

“each man receives […] complete written instructions, describing in detail the task which he is to accomplish, as well as the means to be used in doing the work.” Frederick Taylor

The extrinsic reward was used to make them work harder.

“In order to have any hope of obtaining the initiative of his workmen the manager must give some special incentive to his men beyond that which is given to the average of the trade.” Frederick Taylor

Taylorism detaches the work from the management. Workers are considered instruction executioners and are given incentives and disincentives: carrots and sticks.

This method will probably not work on you. It will certainly not work on me, because I am in a business of building solutions, and solutions require creativity which cannot be framed in rigid instructions.

Today’s Problem

The problems we face nowadays are

• more volatile: What used to take years or months, now changes in days, even seconds. Take for example Bitcoin price.
• uncertain: As changes happen so often, it is hard to predict.
• way more complex: The cause and effect are all intermingled. A modification can produce unexpected consequences elsewhere; and
• ambiguous: Rarely a problem today is clear-cut. The problems are very vague.

For more literatures, you can search for VUCA the acronym for Volatility, Uncertainty, Complexity and Ambiguity.

Taking Cynefin framework — a conceptual framework used to aid decision-making — as reference, we would say most of our problems today are on the left side of the quadrant: Complex and Chaotic.

* Cynefin Framework Image from http://outsidethelines.ca/portfolio/cynefin/

Just consider Climate Change, Income Inequality, Gender Gap, or Genetic Engineering. They are all issues with multiple dimensions: social, environmental, economic, political, and ethical to name a few, intermixed and influencing each other.

In these problems the “Command and Control” model no longer works. By the time the command reaches the grassroots, it is no longer applicable. We need an alternative model.A model where everyone participates in the problem solving activity, while the leaders provide direction, context and help remove obstacles.

The data driven company, Google, conducted over 200 interviews and identified five keys to a high performing team. Those are:

1. Psychological Safety,
2. Dependability,
3. Structure & Clarity,
4. Meaning, and
5. Impact.

Many other similar studies showed similar results. For example, a well regarded consulting company, Deloitte, also presented a model for building High performance teams.

It includes:

1. Supportive environment,
2. Compelling direction
3. Diversity of composition
4. Effective practices, and
5. Psychological safety

Today you cannot force people to do something. Instead, you provide a correct setting that motivates them.

According to Daniel Pink, you must trigger intrinsic motivation supporting these three areas: Autonomy, Mastery and Purpose.

The software development epitomizes the creative endeavor. The frustrations of the old model were becoming more than apparent, it was intolerable. And that’s how the new model sprouted among software development practitioners.

Now that we have seen how problems today are much more complex, requiring different management models, let’s now get into the brief history of software engineering and Agile.

Software Engineering

Computer programming began in the mid 40s, used for military and science purposes, later applying in business in general.

Then came the “Software crisis” from the 70s to the 80s. The projects went over-budget and over-time. They were of low quality, and often did not meet the requirements. Taylorism was a major influencer to the waterfall software development model which made it heavy and bureaucratic, a major contributing factor to the crisis.

There were many efforts to enhance the Waterfall model, but the improvement in the results were nominal.

In the 90s, alternatives to the Waterfall started to emerge: notably Spiral and Iterative models yielded significantly better results, but wasn’t enough: they were still heavy with unnecessary overheads.

And on February 11–13, 2001, seventeen software practitioners interested in “uncovering better ways of developing software” gathered in Utah and the Agile Manifesto was born!

A decade later, in 2010, thanks to Agile, we saw the advent of DevOps reducing the customer feedback cycle and allowing new feature releases in matter of days if not hours.

And in 2020 came COVID, (but that’s out of this context.)

Agile, an Approach, a Mindset

This is the Agile Manifesto.

1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Twenty years and still relevant!

The manifesto is complemented with 12 principles, which you can read in more detail later.

In its core, Agile is more than just a development methodology. It is an approach, a mindset.

Agile Alliance describes Agile as, “an umbrella term for a set of frameworks and practices based on the values and principles expressed in the Manifesto for Agile Software Development and the 12 Principles.”

The principles in Agile are based on many lessons learned from practices in software engineering and other different disciplines.

Today, Agile has been propagated beyond software engineering, from self improvement to business management, manufacturing, architectural engineering, and more.

In the old models, workers rigorously followed a plan handed over to them. They were isolated from the changing reality. With Agile, every member knows the context. Everyone is empowered. And all the members work together to maximize the customer value.

While “doing” is a good way to start, ultimately for the Agile transformation to be successful, “being” agile is essential.

Doing agile is about tactics, ceremonies and techniques, whereas being agile is about mindset, behaviors, and focus on quality and performance of the team.

In software engineering there are several Agile methodologies, these are some of the most popular:

• XP: Disciplines team to produce quality software and adapt to changes
• SCRUM: Empowers creative, cross-functional teams
• Kanban: Visualizes workflows and limit work in process
• Lean Development: Eliminates waste from the system as a whole

Depending on the characteristics of your project or product, one may serve better than the others.No matter which flavor of Agile you use, the key lies in the four values and 12 principles.

The very essence of Agile is that it is not static, it is dynamic, it evolves over time, and adapts to the circumstances.

Martin Fowler, one of the signatories of the Agile Manifesto once said

“if you’re still doing XP in iteration 6 the way you were in iteration 1 then you’re not doing XP.” — Martin Fowler

It is worth reiterating, “doing agile” is a good start, but to fulfill the promise of Agile you need to move to “being agile”.

To tackle today’s ever changing complex problems, we must keep learning and adapting.

Now is a good time to revisit the essence of Agile and reflect on the way we work today.