An estimation exercise

Added to Community

There is still, over ten years after the movement began, a fuzziness about many quantitative aspects of Agile that bugs me more than a bit.

Here's one example - my friend Antoine Contal recently asked me: "how many Agile practitioners do you think there could be in France, or alternately how would you go about estimating the number?"

Offhand, it doesn't seem too likely that anyone would possibly have an exact answer (i.e a census), or even a well-supported approximate answer (i.e. a poll with a random or reliably corrected sample). We must take a guess.

By "take a guess", though, I don't mean "pull a number out of thin air". Ideally we would like an educated guess, something that we could name confidence bounds for. That is, I'd like to be able to say "the number lies between X and Y, and I would offer a bet at 10-to-1 odds on that estimate". If I'm wrong I lose $10, while if I'm right you only owe me $1. That's a guess with teeth. At 10-to-1 odds you would make your interval quite large; at even odds you could afford to be more precise.

For the purposes of estimating, let's define a practitioner as someone who could demonstrate competence in more than one Agile practice and actively uses it in a professional context. So "we have a daily standup" doesn't count, you'd need to show that you bothered to learn what distinguishes a good from a bad standup (demonstrating competence). Neither would "attended a two-day class and went back to work to do the exact same thing I was doing before". Learning TDD and using it on the job, reading the refactoring book and actually refactoring your code - those would count.

Hopefully this definition narrows it down enough that the estimate is doable. Now this has some characteristics of a Fermi problem. We can express the answer as some fraction of an overall population, e.g. "everyone working in IT".

This can be helpful for the first couple steps, for instance we can safely assume that the overwhelming majority of Agile users as of today are still programmers. Some might add, "unfortunately", but it is a useful simplification: if we only estimate the number of Agile developers we should not be too far off the mark.

Estimating the programmer population is non-trivial. Various sources place the total population of IT workers at 600,000 to 800,00 in France, of which we might estimate about half are developers. In the US, official BLS statistics count about 3 million employed in IT; adding up the subcategories for various kinds of programmers yields around 1.5 million.

We haven't even gotten to the hard step - estimating the fraction that has mastered more than one Agile practice. This is where the Fermi method breaks down somewhat. Instead of directly estimating a fraction, I looked for even moderately reasonable upper and lower bounds.

I am extremely reluctant to use something like the total number of certified Scrum Masters, which is known to some precision (around 140,000 as of this writing), but may not be really indicative of anything; it may both grossly over-estimate the number of people who attained competence (because not everyone taking the 2-day class may have subsequently applied the insights from it) and grossly under-estimate it (because not every Agile practitioner will have gotten there by following the Scrum path).

For the lower bound I would go with conference attendance: I estimate the total number of people who attended an Agile conference in France in the thousands, of which probably something close to half attended more than one, which I take as a relatively reliable indicator of striving for competence.

For an upper bound I looked at the job market in France; specifically, I looked at job postings with the keyword "developer" and compared them with postings that included the keyword "agile". On two unrelated job boards I found the same (to me surprisingly high) ratio of one "agile" posting for every four "developer" postings.

My next step is probably the shakiest in the chain - I applied this ratio to the turnover rate in the industry in France (estimated at 50,000 annually among IT workers, which translates to 25,000 developers) to obtain an upper bound on the number of people being selected on the basis of Agile skills - about 6000. This was probably less in previous years.

My reply to Antoine was that I estimated the number of Agile practitioners in France meeting the above criterion as being between 500 and 30,000 at a 90% level of confidence. My 50% confidence interval was between 1000 and 5000 - I'm pretty sure that any estimate in the tens of thousands is over-optimistic.

For the population in the US, I would expect that the same rough proportions apply as in the overall IT population - about five times as many practitioners as I calculated for France.

I have to admit that I'm slightly embarrassed putting these estimates out there - they look terribly speculative. But hopefully someone can comment on this, tell me what an idiot I am, and give me their argument for an improved estimate. Then we'll have gotten somewhere...

 

About the Author

No bio currently available.


This is an Agile Alliance community blog post. Opinions represented are personal and belong solely to the author. They do not represent opinion or policy of Agile Alliance.