XP in big organisations: why is it so difficult?

At first glance, this problem could look straightforward: by definition, big organisation means big inertia. Can you honestly expect a mammoth to change its course as fastly as a gazelle when the surrounding environment has brutally changed?

I admit this is simple common sense, but I would like here to think a little further than this apparent simplicity. I don't want Agility to become a buzzword with no sense, but I believe there is room for big companies to improve the way they handle lightweight methodologies.

Over the last 6 years, I have been involved in 3 XP-driven projects in big organisations, and each turned out to be frank successes. Don't misread my words though: I am still totally convinced  there is no silver bullet. XP adoption is not a guarantee to succeed, but merely a means to concentrate more efficiently on creating customer value.

For two of the projects, I was very lucky to witness their births (their father was Régis Médina). Conan Dalton (another agilist and good friend of mine) has told me how he initiated the third project, which I am currently working on as a coach.

Lately, I suspect the organisation I work for to act against its own interests on a few other projects: middle managers have been striving with all their might to reject the initiatives from Conan and Régis to make their projects healthier! Why that?

What makes an organisation decide to replace its traditional project management recipes, or at least give a try to something else? I have noticed that a genuine support from some part of the middle management was incredibly helpful to make the agile way a success. I now think it is a mandatory precondition.

While a (sometime harsh) resistance to novelty is expectable from people at first,  there are situations in which things go only worse with time. I consider it should be our responsability to try to understand the system behind this apparent blind resistance. Can we describe a reinforcing process somewhere? Can we find a leverage for inversing the tendency and create a virtuous circle instead?

I certainly don't claim to have the whole picture in mind and give a miraculous process for getting out of this kind of trouble. However I believe there must be some generic way to handle resistant organisations. Let's try and list a first set of concrete principles:

• Take the heat out of debates: it is OK to disagree, but let's be objective and focused
• Chase individual fears away (power loss, layoff...)
• Align personal interests with the project interests
• Understand the structural and cultural barriers to change of the organisation (system thinking)
  
• Do not reject everything: acknowledge on what is working well (strengths of the current approach)
• Identify and anwer only to concrete problems (a dogmatic presentation won't make Agile convincing)

The bottomline here is that everything must be set up to establish constructive collaborations with the management and the staff ; this is the basis for any collective success.

This list of principles is of course only a first draft ; there are plenty of litterature on the subject (The Covey "7 habits", the Arbinger "Leadership & Self-Deception", or DeMarco "Peopleware" are very good references on this matter, but I am sure they are not the only ones). Go and check them out!