PPI SyEN 62 A2

 

Why Agile Product Development Systematically Fails, And What to Do About It!

by

Kai Gilb

Value Delivery Expert

Email: Kai@Gilb.com

LinkedIn: www.linkedin.com/in/kaigilb/

Abstract

Agile has been proffered by its proponents as a superior development approach. However, the way it is taught and practiced today, Agile is not succeeding in facilitating product development. The issue is not why it might fail here and there; the issue is why it repeatedly fails. It has been clear to us for a long time that the decades-long IT project failure rate has not been ‘cured’ by the availability of agile methods. The 2015 CHAOS Report published by the Standish Group reports a 61% failure rate of agile projects. In this article, I will discuss why it fails and what can be done to make it successful.

Introduction

Agile is a methodology created to allow small groups to create high quality products in a relatively short amount of time. Agile has tried to evolve to apply to larger-scale IT projects and has always intended to be responsive to change and emerging needs. Agile has been advocated by its proponents as a superior development approach [1, 4, and 5]. However, the 2015 CHAOS Report published by the Standish Group [7] reports a 61% failure rate of agile projects. The purpose of this article is to share the experience of the author in serving as value delivery expert and project coach over the past 25 years; specifically, to address why agile development has systematically failed and what he has done for organizations to make it successful. Some Agile methods are not new, for example, Evolutionary Project Management (Evo)[1] [3, Chapter 10] and Cleanroom Software Engineering [13 and 14].

What Does It Mean to Succeed In Product Development?

People are willing to invest a specific amount of resources in a team, expecting to get something specific back. If something can be delivered to them, within the resources made available, then a level of success can be achieved.

The important question is, what is that something?

According to the 2015 CHAOS Report, experience is that those advocating Agile have not understood this question well nor the answer, and as a result, Agile is failing.

Reviewing the literature concerning Agile, it appears that it is obsessed with shippable code, minimum viable product, user-stories, burndown charts, Kanban, standup meetings, and so forth.

However, the people investing in the project are expecting that their jobs, tasks, products, and services get better, easier, faster, cheaper, and more efficient. In our experience, serving as project coach for many organizations, Agile does not prioritize these improvements. Although Agile does mention that value is important, typically “value” is not defined, specified, or quantified. Consequently, the logical next phase of creating success in product development has not happened. The important step is to prioritize all actions, decisions, and solutions towards creating the defined value.

Example Case

Let’s say a banker wants information faster. Then that is what should be delivered in order that invested effort is considered to be successful. We need to determine what type of information, under what circumstances, and how fast. To determine how fast, we need first to understand how fast the banker gets that information today. Let’s say that for the type of information desired, it takes 3 seconds today. Now we know what would be faster: faster than 3 seconds. Next, we need to set a target, most desirably together with the banker and the development team. Let’s say we set the target to 0.5-seconds. I will name it the “Banker-Fast.Goal”. Now we have determined the improvement the banker expects. Delivering the Banker-Fast.Goal, within the resources agreed upon, would be considered a successful product delivery.

To make the Banker-Fast.Goal happen, everyone needs to exercise extreme focus and prioritization towards reaching the 0.5-second goal. Every decision, every action of all people involved, needs to be directed towards shaving off the 2.5 seconds. For all decisions, we must ask: what will best improve the situation for the banker moving from 3 seconds to 0.5-seconds. For every prioritization, we must ask which choice will best improve the situation for the banker moving from 3 seconds towards 0.5 seconds. For every task we do, we must challenge ourselves concerning the degree to which this task improves the situation for the banker from 3 seconds towards 0.5 seconds. A manager should be inquiring of all members of the team, “Concerning the task you are doing right now, how will it improve the situation for the banker from 3 seconds towards the goal of 0.5 seconds?”

Measure

Then we must measure reality, and adjust accordingly. And repeat until we get to 0.5 seconds. That the task is ‘done’ is entirely irrelevant. The only relevant question is; how much faster does the banker get the information?

Agile has not prioritized actions, decisions, and solutions towards creating defined value. If one is concerned with user-stories and trying to organize and distribute tasks, creating burndown charts, and asking about the definition of done, this is not a success-oriented approach.  Until we realize this fundamental idea of succeeding in projects, we will keep on delivering code and be experiencing unhappy customers. You might call your project a success, but ask the users and the other stakeholders if they are positively thrilled with the delivery they are getting.

Video Tutorials

The author has made available at no charge for all readers of SyEN video tutorials that explain how one in practice can capture, specify, quantify, measure, prioritize, develop, and deliver value to stakeholders. The three tutorials are provided under the title, What Every Successful Project Manager should know!

Tutorial 1 of 3 – Your customer is “always” wrong! Tutorial 2 of 3 – How not to pay your contractors, unless you succeed! Tutorial 3 of 3 – How you can deliver value early, rapidly and continuously!

The tutorials are available at www.gilb.com/p/SyEN.

Conclusions

The success of agile development and implementation can be further strengthened and improved by clarifying what value is for customers. By defining, specifying, and quantifying what value is for customers, and through prioritizing improvement opportunities, the effectiveness of agile development efforts can result in better products that are provided easier, faster, less expensively, and more efficiently. Recent reports concerning project success rates are encouraging [2, 8]. It appears that we finally are applying the results of the lessons of the past 25 years.

 

References

[1] Douglas, Bruce Powell. Agile Systems Engineering. Burlington, Massachusetts USA: Morgan Kaufman Publishers (October 28, 2016). ISBN-13: 978-0128021200.

[2] Florentine, Sharon, “IT Project Success Rates are Finally Improving”. Framingham, Massachusetts USA: CIO, February 27, 2017. See https://www.cio.com/article/3174516/project-management/it-project-success-rates-finally-improving.html.

[3] Gilb, Tom. Competitive Engineering: A Handbook for Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage (2005). Available from https://www.gilb.com/p/competitive-engineering

[4] Grau, Rainer, “The Impact of Effective Agile Teams on Organizations”. Victoria, Australia: The Systems Engineering Newsletter, published by Project Performance International, Volume 57, September 18, 2017. See http://www.ppi-int.com/wp-content/uploads/2017/11/SyEN-57.pdf.

[5] Larman, Craig and Bas Vodde. Scaling Lean and Agile Development. Boston, Massachusetts USA: Addison Wesley, December 8, 2008. ISBN13: 978-0321480965.

[6] Larman, Craig and Victor R. Basili, “Iterative and Incremental Development: A Brief History”, Computer, vol. 36, no., pp. 47-56, June 2003, doi:10.1109/MC.2003.1204375. Available at http://www.craiglarman.com/wiki/downloads/misc/history-of-iterative-larman-and-basili-ieee-computer.pdf

[7] The Standish Group International. The CHAOS Report. Dennis, Massachusetts USA: The Standish Group International, 2015.

[8] Project Management Institute (PMI). Success Rates Rise: Transforming the High Cost of Low Performance. Pulse of the Profession 2017. Newtown Square, Pennsylvania USA: The Project Management Institute, February 2017. Available at https://www.pmi.org/learning/thought-leadership/pulse/pulse-of-the-profession-2017.

[9] Young, Ralph R. Effective Requirements Practices. Boston, Massachusetts USA: Addison-Wesley, 2001.

[10] Young, Ralph R. Project Requirements: A Guide to Best Practices. Vienna, Virginia USA: Management Concepts, 2006.

[11] Zalavadia, Sanjay, “Why Agile Fails in Large Enterprises”. InfoQ Weekly, December 1, 2015. See https://www.infoq.com/articles/agile-fails-enterprise.

[12] Gilb, Kai. Evo Book Manuscript. Project Management and Product Development: How Successful People Manage Projects Delivering Stakeholder Values and Product Qualities Competitively, Early, and Predictably. Available at https://www.gilb.com/store/kNgRApLA.

[13] Mills, Harlan D.; Dyer, M.; and Linger, R. C., “Cleanroom Software Engineering” (1987). The Harlan D. Mills Collection. Available at http://trace.tennessee.edu/utk_harlan/18

[14] Mills, H. 1980. The Management of Software Engineering – Part 1: Principles of Software Engineering. IBM Systems Journal 19, issue 4 (Dec.):414-420. Available at http://trace.tennessee.edu/cgi/viewcontent.cgi?article=1004&context=utk_harlan

Footnotes

[1] Surprisingly, many project cultures have little formal knowledge of Evolutionary Project Management (Evo) (also known as Iterative and Incremental Development [6]) as a modern practice, even though it has been in use since the 1960’s. Prominent software-engineering thought leaders from each succeeding decade supported Evo practices, and many large projects used them successfully. These practices may have differed in their details, but all had a common theme— to avoid a single-pass, sequential, document-driven, gated-step approach. See Larman and Basili, 2003 [6] for a brief history of Evo. See also [3], Chapter 10, Evolutionary Project Management, for an excellent explanation of how to implement it.

Author Biography

Kai Gilb works passionately in his quest to help project teams all around the world learn to succeed in product development. Kai has been the apprentice of his father Tom Gilb since 1992. He considers himself fortunate to have learned from the many brilliant people he has worked with all around the globe.

Kai coaches managers, product owners, and development teams; lectures; develops and facilitates workshops; consults worldwide, and writes articles. Currently, he is authoring a book [9] – ‘Evo – Evolutionary Project Management & Product Development.’

Kai was teaching Agile project management long before the term “Agile” existed, for example, in 1993, teaching at large corporations such as Ericsson and Nokia, and working on large projects including mobile phone base stations and networks.

Among Kai’s clients are Alcatel, BAA, Boeing, Bosch, Citibank, Confirmit, Credit Suisse, HP, Microsoft, NTNU, Philips, If Insurance, Qualcomm, Schlumberger, TomTom, UECC, and Statoil.

Kai resides in Oslo, Norway, is married, and the father of two children.

Kai welcomes feedback concerning this article – reach Kai at Kai@Gilb.com.

[4] Surprisingly, many project cultures have little formal knowledge of Evolutionary Project Management (Evo) (also known as Iterative and Incremental Development [6]) as a modern practice, even though it has been in use since the 1960’s. Prominent software-engineering thought leaders from each succeeding decade supported Evo practices, and many large projects used them successfully. These practices may have differed in their details, but all had a common theme— to avoid a single-pass, sequential, document-driven, gated-step approach. See Larman and Basili, 2003 [6] for a brief history of Evo. See also [3], Chapter 10, Evolutionary Project Management, for an excellent explanation of how to implement it.

 

PPA-006920-1

Scroll to Top