Perspectives · Change Delivery

Challenges with Using PERT for Estimating

This post was modified from one I developed several years ago for the Mercure AACE 2013 program I took, called W5_PM_PERT-Challenges. I am still surprised that with the number of project managers I meet who recommend this technique, which effectively puts the Flaw of Averages on steroids. This post is not specifically about sustainable change delivery. It does provide useful considerations, tools and techniques for consideration though for project controls in sustainable projects.

Context

The PERT (Program Evaluation and Review Technique) has been around since the late 1950s, and is frequently mentioned in Project Management estimating, scheduling and costing references.¹  There appears to be several “camps” in regards to scheduling with PERT and presenting this information to senior management. This post will provide a high-level analysis of PERT and conduct an assessment of a simple MS Project schedule using both PERT and Monte Carlo to evaluate the differences in the response.

Development of the feasible alternative

This post will propose that for the purpose of identifying scheduling and costing information to be used for senior management reporting, one of these two alternatives is correct:

• PERT is a generally sound estimating approach
• PERT is a limited estimating approach

Development of the outcome

This post intends to provide a factual analysis of PERT to assist in making decisions on future usage.

Selection of criteria

The following selection criteria / attributes for reference materials were identified:

• Are there any statistical / mathematical constraints or concerns
• Are there situations where PERT is not appropriate

Analysis and comparison of the alternatives

Methodology

For the purposes of this analysis, a simple previous project was used with activities for Peter Milsom as Project Manager, plus an Analyst and a Developer all at $100 and hour for simplicity. We conducted this analysis using MS Project 2003 (yes this is from an older post), the PERT Entry Screen for MS Project 2003, the PERT Entry Screen for MS Excel, and MS Project 2003 using a Monte Carlo add on (using Risk+ and Barbecana Full Monte). In all instances for PERT and Monte Carlo the same optimistic, expected and pessimistic numbers were used.  Also for both the Beta distribution was used. The final plan of record results are provided at the end. 0. Sample MS Project schedule baseline for analysis: 1. PERT Entry Screen for MS Project 2003 (using the MS Project 2003 PA_PERT_Entry Table): 1a. PERT Entry Screen for MS Excel 2007 (only the critical path activities were included up to 3 Sigma): 2. MS Project Monte Carlo (using Risk+ and Barbecana Full Monte) Entry Screen (the curve is a beta curve distribution) with no conditional branching: Though the same data, activities, costing and scheduling were the same, the manner in which the activities and schedule are analyzed are completely different (i.e. Monte Carlo ran 10,000 simulations). The following are the results: 1. Resulting outcome of PERT Analysis (running the MS Project 2003 Calculate PERT Function): 2. Resulting Outcome of Monte Carlo Assessment: 2a. The following provides a range of the costs and percentage probability 2b. The following highlights those activities with the highest cost sensitivity or uncertainty variance.  Focusing on reducing the uncertainty in these activities would reduce the risk and cost variance 2c. Cumulative S-curves of Project Cost 2d. The following provides a range of the project early finish and percentage probability 2e. The following highlights those activities with the highest time duration sensitivity or uncertainty variance.  Focusing on reducing the uncertainty in these activities would reduce the time and cost variance 2f. Cumulative S-curves of Project Early Finish

Selection of Alternative

Once again the alternatives were:

• PERT is a generally sound estimating approach
• PERT is a limited estimating approach

The analysis has referenced several respected sources that have identified a number of statistical and mathematical constraints or concerns with the PERT approach.  Also, running the PERT calculations “properly” for the author was challenging at best. Though commonly referenced, and for simple schedules perceived as easily executed (though to do properly is not straightforward), there are many situations where PERT is not appropriate.

“Scheduling and managing the activities involved in completing a complex, large scale project can be overwhelming. Many managers use project management software that is built on some basic models, but they are not familiar with the underlying models.” “The results are appealing because they appear scientific…  However, the results are misleadingly precise.”⁷

Based on a very simplistic assessment of PERT and Monte Carlo, the following results came out: Based on the literature, and the “quick and dirty” test, for the purpose of identifying scheduling and costing information to be used for providing senior management information to make informed decisions, PERT is only recommended in limited situations.

Recommendations

This was particularly challenging finding straightforward guidance on employing PERT and Sigma for scheduling without concerns over validity or constraints. The recommendation is to use a Monte Carlo based approach. Please refer to the sister post "Sustainable Estimating" for recommendations on how to properly provide sustainable project estimating.  

References:

¹ Malcolm, Roseboom, and Fazar, (1959). Application of a technique for Research and Development Program Evaluation. Operations Research. 7 (5), pp.646 – 669 ² Galway, Lionel, (2004). Quantitative Risk Analysis for Project Management: A Critical Review. 1st ed. USA: Rand Working Paper, WR-112-RC. http://www.rand.org/content/dam/rand/pubs/working_papers/2004/RAND_WR112.pdf ³ Reginier, Eva, (2005). Activity Completion Times in PERT and Scheduling Network Simulation. DRMI Newsletter Naval Postgraduate School, Monterey CA. . 12, pp 1 – 9. http://edocs.nps.edu/npspubs/institutional/newsletters/DRMI/2005/Apr05.pdf ⁴ Reginier, Eva, (2005). Hidden Assumptions in Project Management Tools. DRMI Newsletter Naval Postgraduate School, Monterey CA. . 11, pp 1 – 4. http://www.nps.edu/Academics/Centers/DRMI/docs/1jan05-newsletter.pdf ⁵ Kerzner, Harold (2013-01-28). Project Management: A Systems Approach to Planning, Scheduling, and Controlling (Kindle Locations 15789-15792). Wiley. Kindle Edition. ⁶ Alleman, Glen B. (2009). Why PERT Has Problems. Available at: http://herdingcats.typepad.com/my_weblog/2009/08/why-pert-has-problems.html. . ⁷ Reginier, Eva, (2005). Hidden Assumptions in Project Management Tools. DRMI Newsletter Naval Postgraduate School, Monterey CA. . 11, pp 1 – 4. ⁸ Hubbard, Douglas W. (2009). The Failure of Risk Management: Why It’s Broken and How to Fix It. Wiley. http://www.amazon.com/Failure-Risk-Management-Why-Broken/dp/0470387955/ref=sr_1_1?s=books&ie=UTF8&qid=1442807076&sr=1-1&keywords=the+failure+of+risk+management Hubbard, Douglas W. (2014). How to Measure Anything: Finding the Value of Intangibles in Business, Third Edition. Wiley. http://www.amazon.com/How-Measure-Anything-Intangibles-Business/dp/1118539273/ref=sr_1_1?ie=UTF8&qid=1442806937&sr=8-1&keywords=How+to+Measure+Anything%3A+Finding+the+Value+of+Intangibles+in+Business Savage, Sam L. (2012). The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty. Wiley. http://www.amazon.com/Flaw-Averages-Underestimate-Risk-Uncertainty/dp/1118073754/ref=sr_1_1?ie=UTF8&qid=1442807005&sr=8-1&keywords=The+Flaw+of+Averages