Risk Management

A Leading UK Multi-Utility

Probabilistic management of risks, contingencies and benefits

Introduction

Our client, a major multi-utility with several million customers, engaged Barnsnape Consulting to help to manage the implementation of a major IT enabled business transformation programme for residential customers. One of the key areas of focus was to analyse the risks involved in the programme delivery.

Such risk analysis is key to the successful implementation of large scale projects, as it highlights, in quantified terms to the delivery team, key issues and risks and allows time to develop innovative solutions reducing impact and likelihood of such risks occurring. It also provides a robust basis for assessing the contingencies, that are managed at each stage of the project.

There were three major areas of risk for the programme; technical risk associated with the product fit to the UK market, programme risk in implementing the solution and external risks which could add to the requirements, such as change in legislation. These three areas of risk are standard to many projects that organisations carry out.

One key issue that faced the Barnsnape team is that such a project is evolving over time. It is therefore vital to review the risks, making sure that they’re up to date and relevant and the figures used to quantify them are correct for the current stage of the project. Risks can be monitored on a basis that suit the clients needs, e.g. weekly, monthly, quarterly, annually.

1. How do Barnsnape identify risk?

Barnsnape Consulting uses a model to help identify and manage risks for different projects. The initial phase is risk identification; Figure 1 illustrates the process adopted to identify risks and characterise them.

Potential impact of risk

For the purpose of ranking the risks we used the score and rank based on impact. Where:

Impact = probability (1-3) x severity (1-3)
This is illustrated in the risk matrix (below) and allows the Barnsnape team to see which risks need addressing most urgently and appear the biggest threat to the project.

Raw Risks - identified by an initial risk assessment and in a risk workshop involving the project team, this is an initial draft of all the risks. The individuals are asked to personally state what they feel the risks involved in the project are and to rate the risk between 1 and 3 in terms of probability and severity. (With 1 meaning that particular risk is highly unlikely to occur and 3 being highly likely to occur). This scoring system was used on this project but can be altered if desired.

Rationalised Risks - collated raw risks. The raw risks are grouped together, as many of them are very similar or implying the same thing. For each risk the average score is taken from probability and severity, which were identified in the raw risks section.

Risk Matrix - maps the Top Level Risk Identification codes on a matrix of risk Likelihood against Impact on the Project. The Risk Matrix illustrates the relationship between the probability of a risk occurring and the impact it will have on the project. It clearly shows which risks need addressing most urgently. It is vital to minimise the likelihood of such an occurrence and be financial able to recover if they do occur.

Top Level Risk Register - tabulates the Top Level Risks together with their unique identifying codes, likelihood of occurrence, impact on the project and associated mitigating actions. After identifying risks, the next stage is to provide solutions (mitigating actions) to reduce the likelihood of them occurring and the impact they will have if they can’t be totally prevented. Each risk is given a mitigating action to help reduce the likelihood of occurrence. One benefit Barnsnape bring is that these might range from simple programme actions through to sector knowledge based. An example is “Minimise risk through business scenario planning”.

2. How do Barnsnape quantify risk?

Monte Carlo simulation is a technique used to understand the impact of risk and uncertainty in project management, cost and other forecasting models. In this case, the Barnsnape team used the Monte Carlo technique to recommend the appropriate level of project contingency required to confidently mitigate the programme risks. Contingency planning is applied to any risks identified enabling the organisation to be able to provide an appropriate level of contingency funding for such occurrences.

The risks in the model are then reviewed periodically as to their current likelihood and impact. New risks identified during the course of a project are also added to the model. This update process enables the contingencies to be re-evaluated periodically throughout the project.

The main purpose of carrying out a Monte Carlo assessment is to:

Build a model of all identified risks, probabilities and inputs.
Run a computer based simulation of the model which explores a statistically significant number of alternative risk scenarios, described by the computer model.
Derive the probability distribution of all identified risks.
Derive the cumulative probability distribution of all identified risks.

Resulting in a mapping of programme contingency to the cumulative probability distribution. The contingency level used is dictated by management confidence levels. The Barnsnape team in this case calculated a maximum, minimum, median and a range of percentile contingencies e.g. 80%, 85%, 90%, 95% and maximum contingency. Complying at maximum contingency presumes all the risks will occur and covers for such an event and is a very conservative stance. In this case we judged that the balance between risk and opportunity was struck with an 80% level. However, this will vary from project to project depending on confidence levels and the clients appetite for risks.
The Barnsnape approach offers an overall risk picture which highlights the maximum, median, standard deviation and a range of percentile contingencies. Then we break down the overall picture to show the separate analysis of risks e.g. Technical, programme and external risks. This makes the breakdown of the contingencies clear for the client.

Also contingencies can be broken down in the same structure as the cost structure for the programme, for example, external, delivery, hardware and software, migration, rollout, total of program and technical risks and other. Again this break down makes it clear where the contingencies are being derived from.

As the programme evolves, the major risks are identified and appropriate solutions developed. This in turn helps bolster confidence levels, allowing the programme to release contingency funds in a phased manner.

Sensitivity analysis

The risk scenarios can be explored using the ability to run the Monte Carlo simulation with a greater or lesser weighting on the ‘most likely’ values for probability and financial impact. This has a direct impact on the results and the contingency values. For example, the results using a lesser weighting can give a much greater standard deviation and a longer ‘tail’ for the maximum values. If the differences are significant this must be considered when deciding the appropriate level of contingency.

3. Other programme applications of Monte Carlo

As the programme proceeds, major checkpoints such as the appointment of a systems integrator will be reached. The understanding of the system integrators capability and associated risks can be input into the assessment, enabling the different risk profiles to be compared as part of the selection process, allowing Barnsnape to draw conclusions as to which integrator is more beneficial in a number of factors including being lower risk and requiring a lower contingency.

There are also advantages to assessing the programme benefits by Monte Carlo.
The ultimate purpose of any programme is to deliver measurable business benefits. Often these arise from many different streams of activity, each of them with associated uncertainties concerning the minimum and maximum values, the likely distribution and the confidence attached to the benefits. Benefits management is a standard work stream within a programme which attempts to ensure that the delivered benefits are maximised. Usually this is based on the tracking of likely outcome by experienced change managers, and is largely a judgemental process. Barnsnape has developed this further and now builds Monte Carlo models of the benefits in the same way it does the risks. Findings to date have offered clients clear advantages in terms of:

A confident prediction of the lower bound benefit is established – this is often found to be greater than would be judged.
A confident prediction of the upper bound benefit is established – this is often found to be lower than would be judged.

The two findings together confirm the often made observation by business managers that major programmes under deliver. By adopting the approach described, a clear model of the basis of the benefits is established, enabling management focus to ensure that the lower bound is achieved, and more importantly to give the quantified tools to attempt to optimise the upper bound.

Outcome and benefits

The Barnsnape team brought to the programme a single unified approach to managing risks, contingencies and benefits throughout the delivery.

Individual risks were identified using a repeatable process that enabled them to be checked as needed throughout the programme. The quantification of the magnitude and probability of the risks came from the same process.

Adoption of the Monte Carlo modelling approach enabled the programme to take a view of the confidence level associated with the contingency allocated to the project, and to judge when the balance occurred between risk and opportunity in the programme. In this example that was judged to be at the 80th percentile of all risks being covered. Reducing contingency down from the maximum level enabled the programme to make a significant saving in contingency allocated.

This modelling approach was continued throughout the project, enabling other key decisions to be rigorously assessed for risk and contingency implications.

In a final, and in our experience novel extension of the approach, the uncertainties and confidence levels associated with the programme benefits were treated in a similarly comprehensive fashion.

Overall, allowing our client to adopt a comprehensive and rigorous approach to risk, contingency and benefit management. Enabling management attention to be focussed as needed.