Development methodologies

Content:

1. Introduction
2. Waterfall
3. Iterative
   1. Unified Process (UP)
   2. Spiral
   3. Agile
      1. True Agile
      2. Boxed Agile
      3. Principles
      4. eXtreme Programming (XP)
      5. Scrum
      6. Kanban
      7. Scrumban
   4. Hybrids of UP and Agile
      1. Scaled Agile
      2. Disciplined Agile
      3. Large Scale Scrum
4. Comparison

Introduction:

Development methodologies are obviously an important part of software development. It is also an area where many developers have strong feelings but not necessarily much evidence about what methodoly is best.

This area is not an exact science, so even though I will try and describe many alternatives with pro's and con's, then this text will also have some subjective evaluations by me. And be aware that I am not an expert in development methodolgies or project management, so my subjective evaluations should not be taken as the one and only truth. But hopefully they will make you think about the problems, realize that this is a very complex topic and make your own conclusions.

The following text will show 4 phases of development:

Requirements

Finding out what the solution needs to do. Many possible artifacts are possible including traditional requirements documents, use cases, user stories etc..

Design

Deciding on how the the requirements can be met. This goes from high level architecture to low level software design. Possible artifacts include UML diagrams, design documents, pseudo code and in case of TDD also unit test code.

Implementation

Producing the solution: write the code, install infrastructure etc.. Writing the code include writing unit tests if that was not done as part of design. The artifacts are completed software and infrastructure.

Test

Test whether the solution meet requirements. Many different tests must be done: functiocnal test, performance test etc.. The artifacts are bug reports and test reports.

Waterfall:

The waterfall model simply do each of the 4 phases of development sequentially and each phase is completed before moving on to the next phase.

This model is very old. It was described by Herbert Benington in 1956 and Winston Royce in 1970. But it is still widely used.

Requirements are usually documented in a traditional and often very detailed requirements document.

It sort of has a bad reputation. Many developers especially those with little or no experience in larger technical complex projects consider it a major reason for project failures.

An obvious question is what happens if a new requirement show up. That question has two answers.

When someone trying to sell a training course or a book about iterative/agile methodologies describe it, then it is handled by starting all over:

In the real world it is handled by Change Requests (CR). A new or changed requirement show up. It get documented in a CR. It gets discussed by those managing CR's (usually some sort of project steering commitee). If it get approved, then it get done in a separate stream also following a waterfall model.

It is common with many CR's in larger projects. In that case it is normal to group CR's together in a few CR batches and treat each CR batch as separate waterfall instead of treating each CR individually.

So it is actually possible to handle some changes in requirements using waterfall model.

But the CR mechanism does have its limitations. If the number of CR's become too big and especially if the CR's start overlapping each other in functionality, then the project becomes a mess.

Evaluation:

Iterative:

Iterative models are a group of very different models just sharing the characteristic that the work is done in iterations.

Unified Process (UP):

Unified Process (UP) was defined by Ivar Jacobson, Grady Booch and James Rumbaugh in 1999.

It split up the development in multiple iterations. Each iteration contain all 4 phases of development.

A special characteristica of UP is that iterations are not necessarily complete. It is possible for a given feature to do requirements in iteration N, design in iteration N+1, implementation in iteration N+2 and test in iteration N+3. As a consequence early iterations contain more requiremenst and design while late iterations contain more implementation and test.

Usage of UP usually involve lots of architecture work and UML disgrams.

The distribution of functionality on iterations is not done based on customer priority but based on technical risk. High risk functionality get done early to allow for sufficient time for redesign in case of problems with design.

Requirements are documented in the form of use cases.

Iterations are typically 4-10 weeks.

The usage of UP sort of got tied to the use of a commercial product to manage the process: Rational Unified Process (RUP). The company Rational got acquired by IBM.

UP/RUP was rather popular back in the early 00's, but has since almost disappeared - replaced by agile and hybrid methodologies.

Iterations are grouped in 4 projects phases:

• Inception
• Elaboration
• Construction
• Transition

Evaluation:

Spiral:

The spiral model is an old development model first described by Barry Boehm in 1986.

Each cyklus goes through 4 stages:

1. Determine objectives
2. Identify and resolve risks
3. Development and test
4. Plan the next iteration

(last cyklus does not include step 4)

The cyklus get repeated a number of times like:

first prototype -> second prototype -> operational product

It is called the spiral model because it is usually drawn spiralling out from center.

The spiral model is more focused on how the business manage the development process than the actual development process.

It is not used today, so no need to go into much details.

Agile:

Agile methodologies are a relative broad family of methodologies that share some basic characteristics:

Many agile principles are old, but the it was defined using the term in 2001 by a group of 17 experienced software people including Alistair Cockburn, Martin Fowler, Kent Beck, Robert Martin and Jim Highsmith.

Common characteristics are:

• Acceptance that requirements will change
• Focus on informal collaboration instead of formal process
• Usage of CI/CD (Continious Integration/Continious Delivery)

Agile methods can actually be used in two different ways. Which I will label with the terms:

• True agile
• Boxed agiled

The two concepts are widely known, but the terms I use are not widely used.

True Agile

What I call "true agile" is where requirements is done for each iteration and there is no defined end goal. Stakeholder provide requirements for iteration, team implements requirements for iteration and that continue until stakeholders consider the solution complete.

Evaluation:

Boxed Agile

What I call "boxed agile" is where requirements are done up front and the end goal is defined. Each iteration implements a part of the requirements.

It is important to note that even though requiremenst are done upfront similar to waterfall, then the requirmenets are typical much less detailed for boxed agile than for waterfall.

Many believers in agile consider this boxed agile model to be a form for waterfall made to look like agile. But non the less it is widely used.

Evaluation:

Manifesto & Principles

The original manifesto:

The original 12 agile principles:

1. Customer satisfaction by early and continuous delivery of valuable software
2. Welcome changing requirements, even in late development
3. Working software is delivered frequently (weeks rather than months)
4. Close, daily cooperation between business people and developers
5. Projects are built around motivated individuals, who should be trusted
6. Face-to-face conversation is the best form of communication (co-location)
7. Working software is the primary measure of progress
8. Sustainable development, able to maintain a constant pace
9. Continuous attention to technical excellence and good design
10. Simplicity—the art of maximizing the amount of work not done—is essential
11. Best architectures, requirements, and designs emerge from self-organizing teams
12. Regularly, the team reflects on how to become more effective, and adjusts accordingly

eXtreme Programming (XP)

eXtreme Programming (XP) was defined by Ken Beck in 1999.

Requirements are done in the form of user stories.

XP does not use any upfront design. In case of doubt a POC called a Spike can be done.

Key characteristics of development include:

• Test Driven Design (TDD)
• Pair programming
• Striong focus on refactoring
• Collective ownership of code

Iterations are typical 2 weeks.

XP was very popular around 2000-2002, but is very rare today.

Scrum

The history of Scrum goes back to 1986 where Hirotaka Takeuchi and Ikujiro Nonaka described the process for physical product development. It was defined around 1995 for software by Ken Schwaber and Jeff Sutherland.

Scrum defines 3 primary roles:

Product owner

Stakeholder/customer voice

Development team

Team of 3-9 developers implementing solution

Scrum master

Process facilitator

Iterations called sprints are typical 2 weeks.

Requiremenst get into the requirement backlog and each iteration/sprint take a number of those.

A key element is the daily 15 minutes standing meeting called scrum where status is presented.

Scrum is very widely used today. In some circles agile and Scrum has almost become synonyms.

Kanban

Kanban history goes back to mid 1980's where Taichi Ohno developed it for manufacturing as part of lean/JIT manufacturing. In the late 2000's it became extended to software development. Kanban in some ways build on Scrum.

Kanban visualize workflow using a Kanban Board. For software development the columns can be the traditional:

• Backlog (waiting)
• Design
• Implementation
• Test
• Completed (done)

Kanban key principles are:

• Visualize workflow
• Limit the number of tasks under in progress
• move work items from state to state
• monitor, adapt and improve

Kanban had become used a lot in recent years.

Scrumban

Scrumban is a hybrid of Scrum and Kanban.

It uses a Scrumban board with:

• ToDo
• Doing
• Done

There are also 3 buckets:

• 12 months bucket with ideas
• 6 months bucket with projects
• 3 month bucket with tasks

There is a WIP (Work In Progress) minimum and maxium.

When number tasks in Doing status goes below WIP minimum planning is triggered and tasks are moved from ToDo to Doing status.

Iterations are short just a few weeks, but are not fixed.

The planning also moves tasks from 3 monbth bucket to ToDo status.

Projects in 6 months bucket becomes tasks in 3 months bucket when requirements are done.

Ideas in 12 months bucket becomes projects in 6 months bucket when business decide it is a go.

No special roles defined.

Hybrids of UP and Agile

Experience with agile for small projects has been very good.

But agile does not scale.

Many agile proponents may deny that, but some of the items in the agile manifest and principles simply work better for small teams than for large teams.

Specifically:

• value ... Individuals and interactions over processes and tools
• value ... Working software over comprehensive documentation
• value ... Customer collaboration over contract negotiation
• 4. Close, daily cooperation between business people and developers
• 6. Face-to-face conversation is the best form of communication (co-location)

It is true that developer A talking to developer B is more effective that written documentation.

But in the real world it works like:

So as a result several methodologies that are hybrids of agile and other methodologies especially UP has been created to try and handle this problem as good as possible.

They all try to keep as much of the good things from agile while adding more structure to work for larger teams.

The hybrid methodologies are usually closer to "boxed agile" than "true agile".

Evaluation (assuming boxed agile style):

Scaled Agile

SAFe (Scaled Agile Framework) was described by Dean Leffingwill in 2011.

SAFe comes in two flavors:

• 3 level for team size <= 100
• 4 level for team size > 100

3 level SAFe consist of:

1. Team of 5-10 people working as an XP team doing 2 week sprints
2. Program of 5-10 teams coordinates teams and own backlog doing program increments of 3-5 development sprints + 1 planning sprint
3. Portfolio of multiple programs managing budgets and using Kanban techniques for cross-program initaitives

4 level SAFe consist of:

1. Team of 5-10 people working as an XP team doing 2 week sprints
2. Program of 5-10 teams coordinates teams and own backlog doing program increments of 3-5 development sprints + 1 planning sprint
3. Stream of multiple programs providing coordination focusing on delivering value fast
4. Portfolio of multiple streams managing budgets and using Kanban techniques for cross-program initaitives

Critics consider it more UP than agile.

Disciplined Agile

DAD (Disciplined Agile Delivery) is created by Scott Ambler while working for Rational/IBM. It started as AUP (Agile UP) in 2006 and ended up in its current form as DAD in 2012.

It is a more discplined approach to agile suitable for larger projects.

DAD split project in 3 phases (UP has 4 phases as described above):

• Inception
• Construction
• Transition

Inception is focused on planning and architecture.

Construction is focused on development including test.

Transition is focused on final test and deployment.

DAD also like UP prioritize doing critical complex development early to reduce risk.

Again critics consider it more UP than agile.

Large Scale Scrum

LeSS (Large Scale Scrum) was defined by Craig Larman and Bas Vodde in 2013.

LeSS has two levels:

• "Smaller LeSS" with <= 8 teams
• "LeSS Huge" with > 8 teams

The teams in LeSS are simply Scrum teams. And the structure on top tries to follow Scrum principles. Each teams sprints are synchronized.

Smaller LeSS:

• 2-8 teams (one scrum master can support 1-3 teams)
• single product backlog owned by a single product owner
• each team have a team backlog
• sprint planning is done at two levels:
  • product owner and team representatives discuss product backlog and move items from product backlog to team backlogs
  • teams discuss and plan team backlog
• if necessary multi-team design workshops are done

Huge LeSS:

• 2+ requirement areas
• each requirment area consist of 4-8 teams (one scrum master can support 1-3 teams)
• single product backlog owned by a single product owner
• each area have an area product backlog owned by a area product owner
• each team have a team backlog
• sprint planning is done at three levels:
  • product owner team (product owner + all area product owners) define product areas
  • area product owners and team representatives discuss area product backlog and move items from area product backlog to team backlogs
  • teams discuss and plan team backlog
• if necessary multi-team design workshops are done

LeSS is less UP than the previous two.

Comparison:

Comparison of scores:

Article history:

Other articles:

See list of all articles here

Comments:

Please send comments to Arne Vajhøj