If you’re really 88% sure (not 90%), does it matter?

I’m not a statistician and I don’t have a math degree; I have two business grad degrees, one in project management.  A statistician or math major will look at Statistical PERT and scowl.  “It uses a normal curve, not the beta distribution!  You’re using the wrong distribution for that uncertainty!”  But as a project manager, I’m intrigued with Statistical PERT’s simplicity and ease-of-use.

I’ve been running a lot of Monte Carlo simulations lately, comparing Statistical PERT with Monte Carlo simulation using the PERT (a type of beta) distribution, to find how much error exists with Statistical PERT.  This analysis makes several assumptions, among them, that the beta distribution is the better distribution to use for a given uncertainty.

In my analysis, I’ve come to realize that whatever errors Statistical PERT introduces to the estimation effort, they are, in most circumstances irrelevant.  The error is sometimes a difference of less than 1%, and often less than 2%.  I occasionally see differences arise in the 3% – 5% range, too, but that usually is with very skewed probability curves and looking at estimates at the high 90th percentiles.

I think Statistical PERT’s greatest value is how it can align expectations between project manager, project team, project sponsor, and other stakeholder groups.

If a team uses Statistical PERT and offers the project sponsor a comfortable estimate that has a SPERT-calculated 90% confidence level — but the Monte Carlo simulation of the same uncertainty shows that the estimate really has a 92% probability of being equal to or greater than the actual result — does it matter?  Maybe in a few situations it will, but I think in many or most situations, it’s an immaterial difference.  Everyone senses that the estimate has a high chance of working, of being successful.  If the project sponsor is willing to accept the high cost (in terms of money and/or time), he will be rewarded with a high confidence that the project team will finish on-time, on-budget.  Conversely, if a project team is forced into a low-ball estimate, and they calculate that the estimate has only a 28% chance of succeeding when the Monte Carlo simulation says that it’s really got a 30% chance, does that extra 2% make anyone feel any better about their chance of success?  No!

When a team tells a project sponsor that his top-down estimate has 28% chance of success, the question that the project sponsor ought to raise is obvious:  “Why do you have such low confidence in my estimate?”  Now, an authentic conversation can ensue, and now the project team can explain their three-point estimate, and the subjective opinion they used about how likely the most likely outcome really is.  Maybe the team will still be forced into accepting the sponsor’s own estimate, but Statistical PERT has shone light onto that decision and the high risk of failure that comes with using the project sponsor’s estimate.

Statistical PERT isn’t designed to be a mathematically bullet-proof estimation technique.  It’s designed to be a quick, easy, no-cost, reliable and reasonably accurate estimation technique that helps all project stakeholders communicate.  Statistical PERT helps everyone identify their chance of success — and failure.

Leave a Reply

Your email address will not be published. Required fields are marked *