Incorporating Risk Assessment into Project Forecasting

Author: Dione Palomino Conde Laratta, PMP

Company: ICF International - USA

Phone: +1 (858) 444-3969

Dione.laratta@icfi.com

Subject Category: Project Risk Management and Cost Estimating

Brief author profile

Dione Laratta (MBA, PMP) has 16 years of experience in the energy industry with the last 8 years focused on project management.

She currently serves as a Project Controller with ICF International on a $200 million environmental project involving mitigation for construction of a transmission line (TRTP).  She is responsible for overseeing budget forecasts, risk assessment, schedule controls, and subcontractor management using sophisticated budget managements tools that are integrated throughout the project.

Her additional project management experience includes five years as the Planning and Control Manager for a $30 billion refinery program for the third largest Oil and Gas Company in the world.

Acknowledgements

This paper presents the results of a risk assessment approach applied to an environmental project. The author wishes to express her gratitude to all who participated in the risk workshops conducted as part of this paper. The author also wishes to express my gratitude to the reviewers of this paper for their comments and suggestions.

Abstract

The science of project management was founded, in large part, to manage risk and prevent it from negatively affecting project objectives, schedules, and budgets.

Risk in any project is unavoidable. Fortunately, there are proven methods to identify and analyze potential threats so that appropriate risk responses are developed and the project's level of exposure is controlled.

Risk analysis has become an important discipline within the field of project management. It involves prioritizing risks and assessing each identified risk's probability of occurrence and potential impact, whether positive or negative.

This paper explores both qualitative and quantitative risk analysis techniques applied to the environmental industry. It explains how to incorporate risk assessment into forecasting and shows how a project was able to increase forecast accuracy from 50% to 95% by using the described approach.

Introduction

In a business world that can transform in the blink of an eye, complexity is the new norm. On top of that, add forces of nature and you are entering the environmental industry.

Forecasting for a project that can be impacted by rain, snow, drought and wild fires can be very challenging since there is always “something in the air.” To get an accurate gauge of these risks – and opportunities – across the project, project executives are appealing to risk management and incorporating its results into the forecast process.

The Risk Management Process

Risk is inherent in projects. One can never overcome all potential risks in a project, but preparation, planning and execution can mitigate much of the risk.

Successful management of a project’s risks gives you better control over the future and can significantly improve the chances of you reaching your project objectives, including scheduling and budget objectives.

Figure 1 shows a summary of the risk management process and its connection with the forecast.

Figure 1 – Risk Assessment Process/Forecasting Process

Phase 1 – Identify Risks

The first step is to identify all risks that could realistically affect the project. This activity is best performed by the project team rather than by one individual. Depending on circumstances, it can be useful to obtain input from your customers, subcontractors, vendors, and other stakeholders involved in the project. Engaging them in the process can help your stakeholders become more committed to the project.

The approach used to identify the risks for our example was a brainstorming session.

In a brainstorm session a team works together with the help of a facilitator. The facilitator encourages everyone to participate in a free-flowing conversation amongst a group of knowledgeable people without criticizing or rewarding ideas. S/He provides guidance throughout the meeting by using structured questions and templates to foster the discussion.

Ideally, all stakeholders should eventually participate in the brainstorming sessions, but the initial Risk Identification Workshop should be restricted to a small number. Choose those who will be full-time members of the project team, have key responsibilities, and cover critical technologies and processes. As the project moves along, new workshops should be performed to incorporate more stakeholders and update risks already mapped.

Prior to the Risk Identification Workshop, participants should receive support documentation such as the statement of work (SOW), the baseline budget and current forecast as well as the Risk Breakdown Structure (RBS) and Work Breakdown Structure (WBS).

The dynamic of the brainstorming session should be discussed between the project manager and facilitator. For example, deciding if discussions will be conducted according to the WBS or RBS, identifying the participants, and defining the number of participants per knowledge area.

During the session, all potential risks are captured by the facilitator and then condensed and refined in order to be validated and classified.

The template used to identify each risk has the following fields:

• Risk Description
• General Information
• Causes
• Potential Impact (Time-Days)
• Potential Impact (Cost- $)
• Period of Occurrence
• Risk Status (Active, New, Completed, Canceled)
• Category (RBS classification)
• Risk Classification as Opportunity (+) or Risk(-)

Phase 2 – Qualitative Analysis

The Qualitative Analysis consists of the methods used to prioritize the risks identified.

Each risk is classified on three variables: impact on cost and schedule, probability, and importance to the project. To simplify the analysis, the importance to the project was indicated by the multiplication of probability times impact.

Tables 1, 2 and 3 show the range used to estimate the severity of the impact and the probability of the risk occurring.

Table 1 – Schedule Impact

Table 2 – Cost Impact

Table 3 - Probability

Based on the complexity and size of the project, a risk tolerance with a lower limit of 4% and an  upper limit of  14% was used, which means that risks with a probability and impact higher than 15% are considered severe and therefore must have a risk response.

Phase 3 – Quantitative Analysis

The Quantitative Analysis consists of evaluating the magnitude of the risks previously classified. It incorporates cost and schedule impacts and evaluates pessimistic and optimistic scenarios.

Based on the forecast and the baseline budget, it was possible in our example to identify potential impacts on schedule and cost.

An expected monetary value (EMV) was calculated for the list of prioritized risks by multiplying the probability (P) times the potential impact (I) in cost.

EMV = P x I

Figure 2 – Matrix Probability vs Impact

Phase 4 - Risk Response

Risk response determines actions and responsibilities to keep track of each risk identified and prioritized.

The response should be aligned with one of the following risk strategies:

• Avoid: Change the project or some assumption to protect the project against the impacts.
• Transfer: Transfer the consequences of a risk to a third party.
• Mitigation: Aims to reduce the consequences or probability of happening.
• Acceptance: Incapability of pursuing another risk strategy or consciously assume the risk.
• Exploit:  Aims to foster the probability of an opportunity happening.
• Share: Used when a partner has a higher potential to capture the opportunity.

Phase 5 - Risk Monitor and Control

Risk monitor and control consists of identifying, evaluating and planning the risks and responses.

New risks can be identified during the project and should be included and tracked with a list of risks at the management team meetings.  Severe risks and their impacts, probability, EMV and responses should be reviewed monthly.

Phase 6 - Integrating Risk with Forecast

After quantifying the risks, the forecast will be split into 3 different scenarios: optimistic, most probable and pessimistic.

The pessimistic scenario incorporates the EMV of risks and the optimistic scenario incorporates the EMV of opportunities identified. The most probable uses the forecast as is.

Graph 1 shows the forecast scenarios distributed through the year.

Graph 1 – Forecast Scenarios

Graph 2 – Risk and Opportunities – Expected Monetary Values (EMV)

Graph 2 shows the distribution through the year of the expected monetary values for risks and opportunities quantified. These numbers were incorporated into the revenue forecast and can be detailed as shown in the table below.

Table 4 – Details on EMV

By using the results from the qualitative and quantitative analysis one can improve the accuracy of the forecast and also support the understanding of the nature of the work being executed. It demystifies the work and its associated risks to top executives and the project management team.

Before introducing the risk assessment into the forecast process, executives in our example project had a hard time understanding why there was often a large difference between the forecast and the actuals, as some months would vary by almost 50%.

Graph 3 shows how volatile and hard to control the environmental industry is, breaking down the risks according to its source.

Graph 3 – Risk Sources

Observing Graph 3 it can be seen that almost 10% of the risks were unforeseeable.  They were related to forces of nature such as rain, snow, wildfires, or accidents. Another 75% were related to external sources such as permits, taxes, regulations, other contractors participating in the project, clients, unions, competition, environmental conditions, the economy and market forces.

Being able to anticipate when most of these risks will happen, and to quantify and incorporate them into a regular forecast process, enables the project team to be more accurate and aware of potential risks and opportunities. Forecasts should be revised on a regular basis, and risk should be one of the topics discussed and updated.

Conclusion

It is widely accepted that risk management is a key contributor to project success, but integrating it into the forecast process can add even more value to the project.

There is no magic bullet to implement a process like the one presented in this paper, but there are some key guidelines you can follow. With that in mind, the following simple steps may help guide you as you begin planning your process to evaluate risk and improve your forecasts.

• Define what values your project and organization will gain from this approach. Examples: reduce volatility to enable a more efficient use of capital. Increase customer satisfaction and transparency. Obtain the project and company goals – a project on budget and on schedule and a more accurate company’s forecast for the market.
• Seek support and help. Get the involvement of your team and an executive sponsor. Explain the value of the process to the project and the organization.
• Use templates to keep the process simple and straight forward.
• Start in a small group with the core management team. Extend to other stakeholders once the process is more refined and established.
• Keep the ball rolling. Define regular meetings with the project team to revise the forecast and risk analysis. Look for unanticipated risks as you already mapped and decided how to deal with the expected ones. Explore the different scenarios.
• Support  the development of an organizational knowledge base. Create a database for the risks mapped and share with your team and organization.
• Develop a monthly report with the three scenarios forecasted and highlight the most relevant risks and upcoming opportunities.

Appendix  - Glossary

• Risk is an uncertain event that may result in a positive or negative impact on project objectives.
• Qualitative Risk Analysis is the process for prioritizing risks for subsequent further analysis or action by assessing and combining their probability of occurrence and impact.
• Risk probability and impact assessment is a method for "investigating the likelihood that each specific risk will occur" and a method for explicating their "potential effects" on the project which can be positive (risk is an "opportunity") or negative (risk is a "threat")
• Risk Severity is a risk classification in terms of impact and probability of occurring. The tolerance to risk will define the thresholds.
• Brainstorming is an information gathering technique used to collect requirements for the project. Uses the project team or experts to creatively identify risks, ideas or solutions.
• RBS: Risk Breakdown Structure
• WBS: Work Breakdown Structure
• Risk categorization is the act of linking identified and evaluated risks into the RBS or WBS.
• Quantitative Risk Analysis is the process for numerically analyzing the effect on overall project objectivities of identified risks. Based on the results of the Qualitative Risk Analysis the Quantitative Risk Analysis is performed on risks that have been prioritized.
• Expected Monetary Value Analysis (EMV) determines an overall ranking of risks multiplying the probability times the impact of the risk, creating different scenarios that may or may not occur.

References

-          Real-world Risk Management. White Paper. PMI. http://www.pmi.org/Business-Solutions/~/media/PDF/Business-Solutions/Risk%20Management_FINAL.ashx

-          Project Management Institute. (2015). A guide to the project management body of knowledge (PMBOK® guide) – Fifth edition. Newtown Square, PA: Author.

Primavera allows for top level planning as well as being ideal for managing the intricate details. This enables project managers, planners, planning controllers and other associated professionals to have instant access to all the project information they require at the touch of a button. It also means that all parties can be kept updated within one system, reducing duplicate information and keeping everyone in the loop.

There are number of other P6 features which makes it unique and recommended tool for planning and will make a planner or a project manager life easier by providing structure, access to information, monitoring progress and reporting mechanisms capability:-

Perfect for Project-focused Business functions - Primavera project portfolio management solutions are also suited for any company with project-focused business functions such as Construction program management, Capital asset maintenance,  Software system development and deployment, Enterprise investment portfolio management, Resource productivity and capacity planning, Product design, manufacturing, and implementation program management etc.

High performance project management - The recognised standard for high performance project management Primavera handles large-scale, highly sophisticated and multifaceted projects. Organise projects of up to 100,000 activities with unlimited resources and an unlimited number of target plans. Enhanced fiscal accountability to customers to identify common scheduling pitfalls and quickly remedy them. Track costs and gain insight into change orders and forecasts. P6 allows top level planning as well as being ideal for managing the intricate details. All the parties can be kept updated within one system, reducing duplicate information and keeping everyone in the loop. Primavera P6 Implement sound risk management principles, reducing the risks associated with projects such as delays in delivery and resource shortages. It reduces risks of schedule inconstancies, errors, or overrun issues. P6 helps identify and mitigate risks in the course of planning and controlling projects. 

Optimized Resources – It helps to carefully monitor resource availability and adjust scare resources to meet project demand. Furthermore, the software can help identify areas where resource cost may be reduced by analysing trends and costs. Primavera has resource levelling option which is very handy in optimizing resources.

Enhanced visibility – Everything is included in the programme so you can easily see what’s going on with a project at any time.  It allows all data to be entered, tracked and analyzed in one location. An unified project view for all participants and stakeholders.

Improved Forecasting – Having up to date information means that you are able to clearly see where there is likely to be overbooked or underutilized resources and can plan accordingly. As the project progresses, the project may require additional resources/activities to meet stakeholder demand. Within P6 schedulers can create forecasts for resources, activities and other project needs.

Instant and Quicker access – Keeping track of progress with time and resources and getting information whenever and wherever needed. Usage of only one system which gives you all the information needed which saves time and cost by not having to refer different tools.

Tracking Features – The tracking features of Primavera P6 allows to rapidly generates wide range of reports

Enhanced Communication – Projects spanning large geographic areas requires hundreds of workers, engineers and many contractors. Primavera P6 enables executive staff to communicate with other stakeholders easily. Furthermore, notes can be made to the schedule in the software to ensure all users see the message.

Improved Collaboration – Since communication is enhanced throughout a project, it’s easy to see how improved communication translates into improved collaboration. 

Gives Employees user access in schedule creation/update- Site Engineers can create schedule, turn in timesheets and update progress.

"This blog is writen by Vishal K Singh, ProjCon. ProjCon is exclusive, niche and rapidly growing Project Controls Consultancy offering innovative, advisory, implementation, technical and training support to our Clients. Our consulting services enable effective use of Project Controls technology, methods and practices for EPC (Oil & Gas, Power, Defence and Infrastructure) industry customers worldwide.

Our comprehensive array of products and services provide powerful, affordable solutions to virtually all aspects of the Project Controls. Contact us to find out how we can help you to build/strengthen your Project Controls capability. For more info, visit us on www.ProjCon.co.uk"

Engineering progress is commonly measured by assigning a weight, usually the required number of required manhours, to each task/deliverable. Once the task is performed/ the deliverable is issued, the corresponding manhours are earned.

The earned progress divided by the total number of manhours gives the % progress.

As each engineering task/deliverable is scheduled at certain dates, it is possible to anticipate the progress that should be earned at a given date. It is the planned progress.

At regular period, usually on a monthly basis, the actual progress of each activity/deliverable is measured against the planned progress. An actual progress less than the planned progress might show a lack of resources and a need for increased mobilization to get back on plan, following a (re-)forecast progress curve.

Although such progress measure is commonly used, it could be deceiving. It indeed reflects rather well the progress of engineering on its own but not how well is engineering supporting the Project schedule.

Let’s consider that engineering must issue 2 material requisitions, an urgent one for a Long Lead Item and another one which is required later on. Engineering will earn progress whatever requisition it issues, even if putting the Project in delay by issuing the non urgent requisition first.

One sees that the above measure of progress alone is insufficient. It must be complemented by monitoring that important Milestones are met.

These Milestones are first of all, the ones associated with the issue of the Requisition for the equipment. Long lead items have naturally to be purchased early. All equipment and packages also need to be purchased as early as their technical definition allows. Indeed, engineering development is highly dependent on information from vendors. The sooner the purchase orders are placed the sooner the vendor information will be available.

Next come the Milestones associated with Bulk Material Procurement to support construction, such as the Piping MTO and the Structural Steel MTO (for an off-shore Project).

Then come the Milestones associated with Construction. These are the IFC Plot Plan, a pre-requisite to start any site work, and the IFC P&IDs, a pre-requisite to the issue of Piping isometrics. The 50% IFC Piping isometric milestone comes next, which typically falls half way through the Project, as ensuing works, such as pre-fab and erection have a rather incompressible duration, due to site constraints (capacity of pre-fab shop, space constraints for erection limiting the progress).

Even if engineering deliveries are in sequence, the above engineering progress measure might still be deceiving, as it will only reflect the amount of engineering work completed and not the workfront made available to construction.

Let’s consider for instance that two foundations are to be cast. The first one is a very large foundation and the second one a small one. Issuing the drawing of either the large or small foundation will earn engineering the same progress, although it will open quite a different workfront to Construction.

One sees the necessity to measure the issued Workfront.

In the case of foundations, for instance, this will be done by monitoring the cumulative quantity of concrete (m3) of all issued IFC foundation drawings.

Producing an S curve, such as the one shown here, showing both planned and actually issued quantities will give a true picture of how well engineering is supporting civil works.

One will similarly monitor, for an On-Shore project, the cumulative quantity of steel (tons) of issued IFC Structural drawings.

The cumulative tons (or dia inch) of IFC issued Piping isometrics will show the available piping workfront.

Such progress curves, showing the actual versus planned available workfronts are instrumental to monitor engineering progress, identify shortage and take corrective actions (increase mobilisation).

It is not perfect however and can still be deceiving, in case of out-of-sequence issues: engineering may have issued drawings representing significant quantities, but that does not generate construction workfront as such works can not be performed at this time (due to lack of access or pre-requisite for another work to be completed before, for instance).

Such integrated model existed, up to the eighties, but has disappeared, Engineering companies having first cut their construction labour and equipment then their construction supervision.

The EPC Contractor one will find today is typically an association of two companies, one doing Engineering and Procurement (E&P) and the other one the Construction (C).

There are 3 types of associations between these two companies: a JV, a consortium or a sub-contract. The most frequent is the last one, the Construction contractor being sub-contractor to the Engineering Company, to which is awarded the EPC Contract.

Let’s look at the pro’s and con’s of each type of association to understand the paradigm leading to systemic delays in EPC contracts execution:

The Joint Venture seems the ideal: both parties share a common profit or loss. There is no conflict of interests.

The issue lies with how one party controls the costs charged by the other party. It is very difficult for an Engineering Company to control the manhours of manpower and equipment charged by a Construction contractor. The Construction contractor is likely to inflate the latter to make its own profit on these charges, regardless of the profit it could get from the JV.

The consortium has each party responsible for its scope, expenses and profit. This provides an incentive for each party to minimize its costs. There is a non recourse clause in the consortium agreement that prevents one party to claim to the other.

The issue lies with the impact that could be suffered by the Construction partner due to the delays in drawings and materials deliveries from the Engineering company. These delays will typically result in idle manpower and equipment. The Construction contractor will not be able to claim the resulting extra cost from the Engineering company. Knowing this, it will include such costs in its bid which will affect the price competitiveness of the consortium bid. This type of scheme is therefore not often seen…

Finally, the most commonly found type of association is the sub-contract. The Engineering company sub-contracts construction activities to a construction sub-contractor.

The construction contractor is commonly paid applying unit rates to installed quantities, e.g. so much for a cubic meter of concrete cast, so much for a ton of pipe erected  etc. This means that the construction contractor will be paid a fixed amount for a given amount of work done whatever its actual consumption of resources (manpower, equipment) is. In other words, the construction contractor bears its productivity risks.

The productivity of the sub-contractor is however highly dependent on timely deliveries of drawings and materials by the Engineering company. In case drawings and material deliveries are delayed, idle time of manpower and equipment will be suffered by sub-contractor, as sub-contractor will still be paid the same amount for each erected ton of steel and the manpower and equipment will require to be mobilized over a longer period.

In theory the sub-contractor could claim for such extension of time and related costs. Such claims are indeed made possible by the sub-contract type of association, contrary to the consortium.

In practice, the sub-contract usually contains difficult to match conditions to such claims. The claim might, for instance, be eligible only of there is a proven overall – not local – lack of workfront. The sub-contractor might also be required to prove that the delay impacts the schedule critical path etc.

As engineering and material deliveries are always subject to out-of-sequence and delayed deliveries, and the above claims are difficult to make, the sub-contractor will be careful not to mobilize too early. The sub-contractor will rather aim to always be a little under mobilized to achieve the best productivity.

On the other hand, the EPC contractor will not be fully transparent with expected engineering and material delivery slippage as its interest is construction progress rather than productivity.

Here, I believe, lies the systemic factor that leads to delays of EPC Projects organized under such contractual schemes.

As such scheme is the norm, one deducts that the owner is more concerned with price than schedule and has accounted float in its overall schedule for delay in the execution of the EPC Contract.

The scheme still entices the EPC contractor to complete as early as possible to avoid both Liquidated Damages and extra costs of prolonged presence at Site.

Introduction

Take your schedule, drop it into a Monte Carlo engine, apply some risks, press the button and in a couple of turns of the egg timer you have a set of confidence dates, some distribution curves and even a tornado chart or two. If you don’t get quite the answer you were looking for you can alter a maximum duration here, a risk likelihood percentage there and press the button again. Eventually you’ll get a result that supports your business case and doesn’t attract too many difficult questions. That’s the aim of SRA right? It’s a means to an end. 

It’s pretty obvious that the previous paragraph was designed to provoke the response “No! Of course that’s not right”. But be honest, how often is schedule risk analysis (SRA) completely unconstrained and unbiased, based on credible and technically accurate inputs and analysed by an experienced risk practitioner? Moreover when have you used the results to inform decisions on budgets, resource allocations and even the viability of the project?  

Regardless of the reason for undertaking SRA, every project manager must consider whether they and their project are ready to go through the unbiased process required to produce a beneficial SRA output and to accept the results. 

This paper doesn’t intend to discuss the benefits and process involved in undertaking an SRA, as these are already well documented. Instead the paper seeks to ask the question ‘are you REALLY ready for SRA?’ by examining the true components of a robust analysis and the potential impact of compromising any one of them. 

However, before you read on it must be stressed that this paper isn’t designed to turn you away from the idea of utilising SRA, as it’s a useful part of a project manager’s arsenal. The paper aims to give you a greater appreciation of how to plan for and conduct a quality SRA in order to gain the most benefit from it.

Components of SRA

It’s a common misconception that if you have a schedule and you have a risk register then you have all the components required to undertake an SRA. The fact of the matter is that there are a number other components that must be in place before an SRA will be close to meaningful. These components can be broadly grouped into three perspectives; 

It is the sum of all of these components that make up the SRA. The analogy of a house of cards is quite apt, if any one of these components is absent or has been compromised then the hard work involved in putting the individual cards together will be in vain. You will be left with a pile of cards with nothing meaningful to show for all your effort, or even worse, your house of cards will just about stand up, but on extremely weak foundations that may lead to key decisions being taken based on misleading analysis.

The following sections explore each of the perspectives above and provide the real questions you should be asking yourself to ensure you are ready to run an SRA. 

 Inputs

The inputs to an SRA are its foundations, if these aren’t solid then the outputs, analysis and decisions that come from the SRA will be baseless. I’m sure you’ve all heard of the phrase “Rubbish in; Rubbish Out”, or more specifically in the case of SRA, it will probably be “Bias in; Bias out”.

Before considering whether to undertake an SRA you as the project manager not only need to be assured that the inputs are well founded, but also, that you understand what it is you want to achieve by undertaking an SRA. 

The following four questions hope to prompt that thought process.  

1. 1) Do you fully understand the purpose of running this SRA?  

Reasons such as; “to get the senior management off my back”, “because we have to convince the scrutiny department or client that we know what we’re doing” or “we need to show that we are going to meet out deadline” are not good reasons for undertaking an SRA and indicate that the benefits of SRA are perhaps not fully understood. 

Understanding the context, stakeholder expectations and having a clear understanding of the decisions your SRA is intended to support, will make it easier to gather the necessary inputs and “sell” the results to the stakeholders. There is no point going through the SRA process only to provide an analysis that fails to answer the questions you and your stakeholders wanting answers for. 

Understanding the purpose also allows you to focus the SRA on particular areas of the project that are of interest. For instance, if the project is 20 years in duration, but stakeholders are only interested in the likelihood of achieving the first deliverable after two years there is no point in developing a risk network for the entire project. 

If you can’t answer the question “what is the purpose of running this SRA?” with a valid, focused and unambiguous reason, such as; “we are trying to identify the phase in our project that is most likely to affect the likelihood of meeting our contract deadline” or “the penalty clauses in our contract mean that missing our deadlines could prove very costly – how much money should we be spending up front to mitigate risk and protect our profit”, then there is no foundation to run an SRA. 

1. 2) Do you have estimates free of bias, obtained from multiple sources and which are considered credible?

The answer to the questions; “how do you know whether your estimates are free from bias and are credible” is that you can’t, but you can take action to reduce bias and increase the credibility of your estimates.  

Only if you have consulted as many people as is practical, with the expertise and experience required, on an individual basis (to avoid ‘groupthink’), can you say for certain that your estimates are as free of bias as possible and therefore, as credible as possible. 

As tempting as it is to believe, putting poor estimates through a modelling tool does not make them any more accurate and certainly does not validate them. Referring back to the ‘House of Cards’ analogy, if the foundations are weak, you cannot be sure that the structure they are supporting will not collapse under even the lightest challenge.

1. 3) Does your risk network contain sound and tested logic?

If the risk network is constructed using any scheduling technique other than left to right with complete and free flowing logic, the answer to the above question is ‘no’ and the SRA will fail to accurately portray the impacts of estimating uncertainty and event risk. 

The risk network forms the backbone of the SRA. Regardless of the level of the risk network and the tasks it includes, it must allow delays to honestly and fully propagate through without interference (i.e. constraints, lags) to provide a meaningful output. 

1. 4) Have all assumptions upon which the risk model is based been clearly articulated and documented?

 Any analysis is only as good as the assumptions with which it is presented. It’s highly likely that some information needed to undertake an accurate SRA is either not available or is unstable at the time required. In these cases planning assumptions should be made in order to complete the SRA. 

These need to be documented to allow you to understand the results of the SRA when revisiting it at a future date. If the answer to the above question is “no” then revisit it to understand what factors may invalidate the SRA if they were to change in the future. 

Remember that an SRA will never provide ‘the answer’. Even the best quality SRA will never end with a statement saying, “the answer is X”.  Project management, as with life, is never that cut and dry; and it is part of the responsibility of the analyst to ensure that the results they present are not divorced from the assumptions and context with which the analysis was carried out.

It is clear that to ensure the results of the SRA are credible and provide value, time must be taken upfront to ensure that the inputs are meaningful and well thought out. Without credible inputs to the SRA, the results should not be trusted. 

 Enablers

Enablers are the things that allow a successful SRA to take place, free of interference, at the appropriate level and with the right analysis to answer the required questions. 

This paper is focusing on organisational enablers such as; knowledge, availability of resources, appropriate governance and organisational maturity. 

The following questions are intended to challenge whether you are in the position to make the most of the SRA and its outcomes. If you’re not, you must ask “why am I doing it?”

1. 1) Do you have the right level of knowledge, experience and impartiality within the organisation to properly analyse the SRA results?  

It’s not too difficult to throw a few risks together with a high level schedule and click a button. Similarly it’s not hard to read results from a graph. However, would you or any of your team be comfortable explaining to senior management the detailed results of an SRA, the context and assumptions that underpin them, how they were achieved and what they do (and importantly, don’t) tell you about the project? 

If you want to get valid and impartial results that provide a meaningful insight to the project then you need people with specialist competence, training and experience. If you expect to run a meaningful SRA without the specialist skills, you run the risk of making decisions about your project based on un-informed analysis.  

1. 2) Have you allowed enough time to fully engage with the SRA process, analyse the results and put actions in place?

Running an SRA is not a simple process; from experience an SRA invariably takes longer than you initially expect. Rushing it can result in poor quality analysis and can invalidate the whole outcome. 

An important point to remember is that SRA needs to be done to an appropriate level of granularity and should be iterative. SRA takes time and depending on the reasons for undertaking an SRA it may not be necessary to undertake it on the entire project. Consider what is appropriate to you and weigh the costs, time and effort against the potential benefits. 

1. 3) Are you opened minded about the outcome and unconstrained by pre-conception? 

This question speaks for itself; if you already know the answer you want the SRA to provide, aren’t open to alternatives and are willing to manipulate the model to get the answer you want, then it’s a fairly futile exercise. SRA is not flawless; referring back to the second sentence of the paper; 

“If you don’t get quite the answer you were looking for you can alter a maximum duration here, a risk likelihood percentage there and press the button again”

Doing this defeats the object of the SRA and invalidates the process, analysis and any decisions made based on the outputs. 

It’s strongly recommended that an impartial third party is utilised to assure that the process is undertaken correctly, regardless of the result. If you are the project manager or senior stakeholder commissioning an SRA then it is your responsibility to ensure that the analyst is not unduly influenced by yourself or other stakeholders. If you think you know what you want the ‘answer’ to be, then don’t tell your analyst!

It is clear that enablers are a key aspect of running an SRA. Without aspects such as the knowledge, time and right intention of running an SRA, results of worth and value are impossible. 

Outputs

Outputs in the context of this paper are not referring to the technical analysis or various graphs that an SRA produces, these are produced regardless of whether the SRA is based on solid foundations or not. Instead, this paper is looking at the actions of the organisation and project team as a result of the SRA outputs.  

The three simple questions that you should ask yourself with regard to outputs may be difficult to answer. However, they need to be considered. 

1. 1) Is your SRA analyst independent and free of un-due influence from the project team or senior management? 

If the answer is no, how can you trust the results and base decisions on them?

To ensure credible, valid and impartial outputs the operator needs to be independent of the project team or senior management. Without this you cannot assure yourself or your stakeholders that the outputs have been free of any influence that could have altered the results. 

1. 2) Is the governance and culture in your organisation prepared to understand and act upon the outcomes of the SRA? 

“Prepared to understand”, what does this mean? Fundamentally, will you or your senior management accept the results of the SRA and try not to influence them to make a political point, or to ensure the continuation of the project. What other information will you be taking into account when considering the results?

As for “acting”; is the organisation ready to make the decisions that a SRA may highlight? For instance; “where shall we spend the £100K budget for risk mitigation?” or more contentiously, “should we cancel this project?” 

It’s also important to remember that SRA is just one of many tools used to inform decisions. There is nothing inherently wrong with basing decisions on the project manager’s experience, or ‘gut instinct’, but SRA can provide the evidence based analysis and perspective to support your gut instinct or indeed challenge it. SRA is another tool for the armoury, and should be used as such – not the magic bullet, but extra ammunition!

If the organisation is not mature enough to accept an outcome or ready to take action then ask why are you doing an SRA, what is the benefit to the organisation and the project?

1. 3) Do you have the time and resources to act on the outputs of the SRA? 

This question goes hand in hand with its predecessor. Whilst you may have the intention to act, depending on what may be required, do you have the time or resource to actually do it. 

Identifying the reasons for undertaking the SRA, and conducting the analysis at an appropriate level, is key. If resources are not available, expectations must be managed at the outset of the SRA process so that the reasons for undertaking the SRA are not undermined.  

The actions of an organisation following the SRA are fundamental to success. If no action will be taken following the SRA, what value has it added? The effort put into the SRA must be matched by the effort put into the results to ensure that the right direction is taken by the organisation or project following the results.

Finally

Looking back at the question the paper is trying to answer, ‘are you REALLY ready for SRA?’, put simply; if you can’t answer ‘yes’ to all of the questions asked through the paper, and compiled in table 1, then realistically you’re not ready to get the best from an SRA. 

However, before you think, “well I just won’t bother then, as it all seems a bit too hard to do properly”, everything discussed can be overcome or managed. The key is to understand the weaknesses of the SRA to ensure you get the most benefit, or tailor the process to an appropriate level. 

The key points to remember are that in order to get the best from SRA you must: 

• Understand the reasons for undertaking an SRA
• Assure yourself that the process is impartial and unbiased
• Apply it an appropriate level 
• Be confident that the outcomes can be used to take decisive action for the good of the organisation. 

SRA is an extremely powerful tool that can provide huge benefit to projects and organisations when ‘done right’. So now, ask yourself, are you REALLY ready for SRA?

Table 1: “The Complete SRA Readiness Quiz”

Acknowledgements

The author would like to thank the following individuals for their assistance in developing this paper;

 Laura Smith – BMT Hi-Q Sigma

Russell Tarver – BMT Hi-Q Sigma

Jo Langley – BMT Hi-Q Sigma

Michelle Glasgow – BMT Hi-Q Sigma

Georgina Jones – BMT Hi-Q Sigma

Author:

Tom Olden
T: +44 (0) 1225 820 980 
E: tom.olden@bmt-hqs.com

Disclaimer

 

I hereby declare that the content of this paper does not infringe any copyrights and is owned by the author. 

 

 

Signed:  …………………………………………

 

                                              T. Olden

Key Words

Progress. Position. Forecast. Earned value. Change. Linear regression. S-curve. Schedule.

Abstract

Current methods for assessing activity progress, calculating project position and forecasting project completion (including the use of earned value analysis) are examined in this paper. The disadvantages of current forecasting methods are discussed with special reference to single data point extrapolation and the difficulty for non-specialists in analysing s-curves. A different method of forecasting project completion using simple linear regression and time series analysis is proposed which has real practical applications for project managers and allows them to easily and rapidly produce position and forecast data in a format that is understood by layman and specialist.

Introduction

The UK Construction Industry does not have a good record for completing projects on time. Constructing Excellence’s data from its 2012 report (the latest available) shows that the actual out-turn time taken for the construction phase of projects compared with the length of time agreed at the start of that phase dropped sharply in 2012, with only 42% of projects delivered on time or better, compared with 60% the year before.  Whilst this is the first time since 2000 that the KPI was below 50% the data shows that, on average, less than 40% of projects finish on time, Figure 1. 

Figure 1.  Predictability Time - Construction

The production of a project schedule is the first step in project control.  In some cases, particularly on undemanding and straightforward projects, this initial planning and programming is sufficient and the project manager will be able to determine the status of the project without rigorous examination.  More often schedules are required to assist with the active management of time by regular monitoring, examination and modification.  Active management of time comprises three steps; Progress, Position, and Prediction, these are all relative to a point in time when the measurements or calculation are made; ‘time now’.  Without accurate measurement of progress it is not possible to establish the position of the project and without knowing the current status of the project, predictions about the completion of the project are likely to be little more than guesswork.  Without knowing when the project is likely to be complete it is impossible to determine what action must be taken to bring in the project on time.

Current Techniques

Progress – how much has been done

This is reasonably straightforward and usually involves assessment or calculation of the percentage of work completed on individual schedule activities.  Progress can also be attributed to the project as a whole but unless the project is relatively simple a single measure of how much of the project is complete is somewhat superficial.

Position – what is the current status

Position is usually stated as time ahead, on schedule or time behind and relates to an individual activity or the project as a whole is a comparison of where the activity, or project, is compared to where it was planned to be.  

For a single activity, assessment or calculation of position is also relatively straightforward. To calculate activity position:

if S  TN  F, then  if (% < 100, P = S + (D x %) – TN, else P = 0), else

if S  TN  F, then P = S + (D x %) – TN, else

if S  TN, then  if (% > 0, P = S + (D x %) – TN, else P = 0)

Where: P is the activity position,

S is the planned start of the activity,

F is the planned finish of the activity,

TN is time now,

D is the planned duration of the activity, and

% is the percentage complete of the activity at time now.

However, for a single activity, position is more readily demonstrated graphically using the bar chart ‘drop line’ method (see figure 2).

Figure 2.  Drop line activity progress and position

Determining the project position (or project status) accurately is more difficult.  Many project managers will use their skill, judgement and experience to assess the project position.  However, such visceral and subjective techniques are open to suggestion of bias and manipulation for commercial or other ends.

A number of objective techniques have been developed:

• Averaging. This method averages the position of all the activities that are ahead or behind schedule.  Although simple and apparently reasonable this method is mathematically unsound.
• Planned Progress Monitoring (PPM). This method compares the planned work content (based on activity duration) with that achieved.  This method does not depend on the schedule being a critical path network and is predominately the underlying method adopted to ‘roll up’ progress in summary and expanded type bars of project management software. 
• Critical Path Methods. When the progressed project is rescheduled with the variance in the end date of the project can be interpreted as the project position.  This method depends on a fully linked and logical network.
• Earned Value Analysis (EVA).  The parameter SV (scheduled variance) is a measure of the current status of the project.  This is similar to standard cash flow analysis; income -v- expenditure or cash weighted PPM.

Prediction – when will the project end

Predicting is the estimation or forecasting of some future event or condition of the project as a result of the study and analysis of available data on the basis of observation, experience or scientific reason.  Generally this will relate to a project milestone and particularly a forecast of when the project will be complete.

By its nature the prediction of future events with any degree of accuracy is difficult.  Project managers tend to rely on their experience and analysis based upon the current position of the project and with the assistance of the project schedule will envisage, or more formally reassess, the schedule for the remaining work.  If the reason for delay in the schedule was merely unrealistic durations and sequences then updating and amending the schedule to predict the completion date may be viable.  Unfortunately the time taken to complete activities generally conforms to Parkinson’s Law and the Student Syndrome so, unless the underlying causes of delay are confronted, there is inherent risk of overrun of the reassessed schedule too.

The result of analysing a critical path network taking into account current progress is often erroneously referred to as a forecast of completion.  For instance, where the project is in delay the rescheduled end date of the project would be delayed, this will only be a forecast of the completion date if the uncompleted remaining work were to be carried out in accordance with the schedule.  It is more likely that if past work was not carried out in accordance with the schedule then, unless something changes, nor would future work.  As stated previously the result of rescheduling a network taking account of current progress is a measure of the project position.

EVA attempts to formalise the forecasting of completion of projects using the parameter EAC (Estimate at Completion).  The unnecessarily complex acronyms render the technique virtually unusable for all but the ardent enthusiast.  In relation to time alone the technique can be simplified using the rate of progress to date and the time outstanding on the original schedule; for example, see Figure 3.

Planned completion date = 22.0 (BAC_t)

Time now = 12.0 (BCWS_t)

Current position = 10.0 (BCWP_t)

Rate of Progress = 10.0 / 12.0 = 0.83 (SPI)

Time not yet completed=22.0 – 10.00=12.0

Forecast time to complete = 12.0 / 0.83 = 14.5 (ETC_t)

Forecast completion date = 12.0 + 14.5 = 26.5 (EAC_t)

Figure 3.  Forecast completion using Earned Value Analysis

Whilst the estimated completion date can be calculated, plotting of the remaining forecast to completion curve is problematic, but without it, it is difficult to envisage the remaining progress of the project and to determine if future work is proceeding to the forecast plan.  In his booklet ‘EVA in the UK’, Steve Wake says:

The estimates to complete can be plotted (or hand-drawn by “experienced professionals”). …

The prediction of potential EACs (Estimates at Completion) has become increasingly accurate by using performance statistics from similar projects.  These statistics become templates that are overlaid onto the existing cost curves of a project and provide an independent and objective estimate of the final cost and completion date.  Something that everyone is interested in.

Blythe and Kaka take a different view and appear to suggest that advanced mathematical modelling is required (or at least beyond the capabilities of most project managers) and that the accuracy of the models is questionable:

There have been many attempts in the past to develop cash flow forecasting models.  They were mainly part of more comprehensive models aimed at assisting contractors or clients forecast their cash flow on an individual project level or on a company level.  The majority of these models were based on the idea of developing standard S-curves to represent the running value or cost of different types of construction projects.  Typically this was achieved by collecting data relating to the monthly valuations and the projects’ general characteristics.  These projects would then be classified and distributed into groups and average S-curves would then be fitted on the individual groups.  Several mathematical models were used to fit the S-curves (e.g. alpha-beta cubic equation, Weibull function, DHSS model etc.).  These models could be used, given that the total value and duration of the projects to be constructed are known, to forecast the cumulative monthly (or at any other time interval), value/cost of that project.  The accuracy of these previous models is in question.

Using PPM similar shaped graphs to EVA’s BCWS and ACWP for planned progress and actual/as-built progress are generated.  Whilst PPM is a useful method for determining the position of a project the s-curves that are typically produced are not easy for most practitioners to assimilate and to use for forecasting, see Figure 4.

Extrapolating the rate of progress, planned compared to actual at ‘time now’ can be used to predict the project completion date without the need for considering EVA or PPM. The only data require is the original project duration (D), the project position (P) and the ‘time now’ date (TN).  The forecast completion date is:

Completion = D x TN / P

Using the previous example:

= 22 x 12 / 10 = 26.4

Figure 4.  PPM - planned, actual and forecast curves

The Proposed Method – Simple Linear Regression

Statistical analysis of project data has, up to recently, been the preserve of the financial analysts, be they corporate accountants or project accountants.  The data produced, when graphed, tends to resemble an s-curve.  As described previously, in connection to EVA, it is not easy to estimate the path of a partially completed curve.  It is possible, theoretically at least, to assign a mathematical formula to most curves but these can be extremely complicated (at least for the layman) and there is no certainty as to the shape, and hence formula, of that a predicted curve will, or should take.

The data for the graph at Figure 3 was based on SPI of 0.8 and further randomized on a monthly basis between 80% and 120% to model variances in progress.  The graph at Figure 5 illustrates the difference between the forecast data (from Figure 3) and the modelled ‘actual’ data.  The forecast completion is 27.6 months which is very close to 27.5 months that would be expected for a 22 month project (22/0.8).  Whilst this apparent accuracy is as much to do with the coincidence randomness of the data it illustrates the primary flaw in SPI type forecasts that they use a single data point as the basis for extrapolation rather than a longer term trend.

Figure 5.  EVA - forecast and actual

To overcome this weakness and the limitations of projecting unsystematic curves the method described below is based upon simple linear regression and time series analysis.  Whilst the components of the method are not novel the author is not aware of it being used to forecast project completion, particularly at least in the UK construction and engineering industry.

The planned model

The position of a project is usually stated as being ahead, on schedule or behind but it can also be stated as the number of schedule weeks achieved.  For instance, a project at week 20 which is 2.5 weeks behind schedule can be said to have achieved week 17.5, similarly a project at week 20 that is 2.5 weeks ahead of schedule can be said to have achieved week 22.5. The importance of the proposed method is recognising that for all projects there is a simple straight-line relationship between the planned position of a project and project time such that, for instance, at week 20 the project is planned to have achieved week 20.  The planned position line for all projects will be a ’45 degree’ line which straightens at the project completion date; see Figure 6.  In terms of EVA the planned line is similar to the BCWS curve.

Figure 6. The actual/as-built model

The data for the actual/as-built model (BCWP in EVA parlance) is generated by calculating the project position by whatever method is appropriate as outlined above.  It is not recommended that different methods of determining the project position are used for each period but it may be good practice to generate multiple datasets based on different methods of calculation which then may give a range of estimates of project position.

The actual/as-built position data can then be plotted against the planned data; see Figure 7.  As the planned model is based on a straight line it is easier to appreciate the deviation of the actual position compared to the planned position.

Figure 7. The actual/as-built model

The forecast model

The forecast of completion is made on the premise that, if nothing changes, if progress carries on in the future as it has in the past, the project completion date will be thus.  Previous forecast models have used a single data point; the last measured project position.  However trends are not absolute and there is likely to be some waxing and waning, positive and negative deviations from the general trend.  Using the last measured progress position may exaggerate or understate the general trend.

As the planned position is based on a linear model it is acceptable to consider that the actual model, unless it is subject to wild fluctuations due to specific delaying events, will also follow a linear trend and hence forecast can be made using simple linear regression which will take account of all the past progress not just the last project position.

Figure 8 shows the simple linear regression line plotted for the actual project positions.  The linear equation enables the trend line to be extended to the completion position (month 22) and for the date to be calculated, in this case 26.62 months.

Figure 8.  The forecast model

The position trends are easier to assimilate as straight lines and progress and trends, should future performance match past performance, can be readily seen as can  changes in progress required if the project is to not lose any further time or to be completed on time, see Figure 9.  It is submitted that such trends are not readily accessible using s-curves.
 

Figure 9.  Change required

Most emphasis in this paper has been on projects that are behind schedule. Figure 10 shows typical regression plots for projects that are on schedule and for projects that are ahead of schedule.

Figure 10.  Ahead and on schedule

Conclusion

Forecasts of completion dates are almost always wrong.  Forecasting completion of projects is not about estimating when a project will be complete but more about when it will be complete if progress continues in the future at the same rate that was achieved in the past.  Only by knowing what the potential overrun (usually) will be if nothing changes can the project manager determine what needs to be done to bring the project back on schedule.  The reason forecasts are wrong is that, hopefully, project managers will have taken steps, with the knowledge of the effects of doing nothing, and have pulled the project around.

Current methods of forecasting completion mostly depend on extrapolating the last known project position to forecast project completion.  Earned Value methods also use a single position measure but depend on s-curves to illustrate the work flow. S-curves are difficult to assimilate and difficult to mathematically predict.

The proposed method depends on simple linear regression taking account of all the position data and presenting it in simple straight-line graphs that are more readily understood by non-specialists.  Trends are easier to understand and the amount of action to bring the project back on schedule is straightforward to see.

Like all current methods of forecasting, including earned value methods, specific and exceptional delaying events can skew the forecast.  Progress trends tend to be influenced by leadership, management, resources, experience and strategy decisions.

Acknowledgements

Anneka Wilson, Driver Group’s Group Marketing Executive has been constant with her help and encouragement even though, like most planners, I have always been behind schedule.

My colleagues at Driver Group; Stephen Lowsley, Keith Strutt, David Wileman, Philip Allington and Janus Botha have provided technical critique of my paper – any errors, however, remain mine.

Dr Chris Chatfield of the Department of Mathematical Sciences at the University of Bath and author of ‘The Analysis of Time Series: An Introduction, Sixth Edition’, kindly took time to reply to my emails and responded to my very basic time series questions.

Whilst every effort has been made to ensure the accuracy of the information supplied herein, Driver Group plc, its subsidiaries and the author cannot be held responsible for any errors or omissions. Unless otherwise indicated, opinions expressed herein are those of the author and do not necessarily represent the views of Driver Group plc and/or its subsidiaries.

The author warrants that he is the copyright owner and that all sources are acknowledged and referenced and that as far as it is possible to ascertain this work does not infringe any existing copyrights.

All rights reserved. No part of this document may be reproduced or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without prior written permission the author or Driver Group plc.