Dynamics of project success and failures

Let’s examine one aspect the complex nature of software projects, that lately has given me a hard time.

How do well-intentioned stakeholders sabotage  technical projects?

So many large technical projects suffer from stakeholders misunderstanding of complex, underground dynamics. In period of stress, or when the project releases turns out to be late, they sometimes hastily take decisions that dramatically lowers the odds of success, although they genuinely believe they act for the good.

Imagine this common case-scenario:

• A 5 year-old, pretty big, software project, with very few automated tests, and having slowly accumulated heavy technical and functional debts over the years,
  
• Early team members have nearly all quit (current members have all less than 2 year-experience on the project),
  
• The team is the only true asset of the project: technically experienced, they’re used to work extremely well together for several years with Agile methods,
  
• The required investments to clean up the project are huge, 
  
• The client, (rightly) focused on the roadmap of the next functional features, does not understand the poor team velocity, and every sprint trust less and less the team abilities to respond to demand,
  
• A vicious circle installs : the client requires to monitor and pilot the detailed team activities, with no understanding of the software development complexity... precipitating the death of the project.
  

I have attempted to draw a diagram of effects of this project, following roughly the notations of Jerry Weinberg (see next chapter for some theory).

Disclaimer: As many people before me have thought and written about this, it is very possible that other authors (or Jerry himself) already contributed similar diagrams.

Theory: Seeing the larger system through “Diagram of effects”

Some years ago, my friend and mentor suggested me to read a book from Peter M Senge, “The fifth discipline”, in order to learn “System Thinking”. Later on, I read further on the topic in Jerry Weinberg’s first volume of the “Quality Software Management” series. (I highly recommend these 2 books for anyone interested in explaining the dynamics behind a tough situation)

Don Gray provides an interesting first explanation for people unfamiliar with diagram of effects.

Jerry Weinberg also defines the following notations :

• arrows with squares stand for open choice for action from the management
• arrows with double pipes ( ----| |----> ) represent a time delay between a change and its consequence.

The concepts of delay and non-linearity may at first seem secondary when we look at or design a diagram of effects: balancing and re-inforcing systems indeed provides an interesting level of understanding, that may sometimes proves sufficient for acting wisely.

However, the combination of delays and non-linearities often explain the inability of some people to manage even simple systems :

• delays hide the effects of decisions : the consequences (positive or negative) of any action occur weeks (or months) later,
• non-linearities take people by surprise, by the dramatic magnitude of change (again, positive or negative) a minor change can produce

If your environment seems to run out of control, I highly recommend you take a sheet and a pen, and brainstorm for producing a diagram of effects, reflecting your system dynamics (read Senge and Weinberg excellent books for guidance). This analysis alone certainly won’t alone solve your problem, but it stands as a solid starting point before choosing your next actions.