Perspectives · Benefits Management

Sustainable Risk Management – Four Current Challenges with Solutions

This post focuses on the current challenges of sustainable risk management within sustainable change delivery, with useful perspectives, tools, and techniques.  This blog post is part of a series that provides the foundation for understanding sustainable change delivery.

“Primum non nocere – First, do no harm” - Auguste François Chomel (Hooker, p. 219, 1849)

“The golden axiom of Chomel that it is only the second law of therapeutics to do good, its first law being this – not to do harm – is gradually finding its way into the medical mind, preventing an incalculable amount of positive ill” (Hooker, p. 219, 1849).

(Quote borrowed from Douglas Hubbard’s The Failure of Risk Management)

"Risk aversion" is a common phrase.  Despite the familiarity, it inappropriately explains people's response to risks.  Resources are seldom 'risk averse' though they are frequently 'change averse'.  Individuals have different risk thresholds under different circumstances, such as times of the day.  A more accurate phrase is that people are simply 'risk ignorant.'   People often take dreadful and unnecessary risks without even realizing it (i.e. delaying, making uninformed decisions, etc.) (Bacon, Kindle Location 3744, 2013).

One of the challenges with risk management is that people evaluate risk by replacing the scientific discipline with experience.  The challenges and flaws with this approach include the following:

• "Experience is a nonrandom, nonscientific sample of events throughout our lifetime.
• Experience is memory-based, and we are very selective regarding what we choose to remember.
• What we conclude from our experience (or at least that part we choose to remember of it) can be full of logical errors.
• Unless we get reliable feedback on past decisions, there is no reason to believe our experience will tell us much.
• No matter how much experience we accumulate, we seem to be very inconsistent in its application." (Hubbard, p. 96, 2009)

Sustainable Risk Management Current Challenges

Risk Management Including Opportunities

One challenge deals with the recent inclusion of positive opportunities as a risk along with negative threats.  Douglas Hubbard does not include opportunity with risk analysis.  Instead, Hubbard postulates that risk management is a subset of management.  As well, risk analysis is a subset of decision analysis just as opportunities are a subset of decision analysis... each a different discipline with its own tools and techniques (Hubbard, p. 92, 2009). Hubbard also points out that "including positive outcomes as part of risk is a significant departure from how the term is used in the decision sciences, insurance, probabilistic risk analysis in engineering, and most other professions that had been dealing with risks for decades" (Hubbard, p. 89, 2009).  To clarify Hubbard presumes the following:

• "Risk has to include some probability of a loss.
• Risk involves only losses (not gains).
• Outside of finance, volatility may not necessarily entail risk—this excludes considering volatility alone as synonymous with risk.
• Risk is not just the product of probability and loss. Multiplying them together unnecessarily presumes that the decision maker is risk-neutral. Keep risk as a vector quantity where probability and magnitude of loss are separate until we compare it to the risk aversion of the decision maker.
• Risk can be made of discrete or continuous losses and associated probabilities. We do not need to make the distinctions sometimes made in construction engineering that risk is only discrete events."  (Hubbard, p. 91, 2009).

I like the following definitions from Douglas Hubbard:

• "Uncertainty.
  • The lack of complete certainty—that is the existence of more than one possibility. The 'true' outcome/ state/result/value is not known.
• Measurement of uncertainty.
  • A set of probabilities assigned to a set of possibilities. For example, 'There is a 60% chance it will rain tomorrow, and a 40% chance it won’t.'
• Risk.
  • A state of uncertainty where some of the possibilities involve a loss, injury, catastrophe, or other undesirable outcomes (i.e., something bad could happen).
• Measurement of risk.
  • A set of possibilities each with quantified probabilities and quantified losses.
  • For example, 'We believe there is a 40% chance the proposed oil well will be dry with a loss of $12 million in exploratory drilling costs.'
• Ignorance.
  • In the state of ignorance, we don’t even know the possible outcomes, much less their probabilities. This state of ignorance is what former U.S. Secretary of Defense Donald Rumsfeld and others would have meant by the term 'unknown unknowns.'
  • In effect, most real-world risk models must have some level of ignorance, but this is no showstopper toward better risk management." (Hubbard, p. 80, 2009)

  As a trained and certified risk manager I fully endorse Hubbard's beliefs.  With sustainable change delivery and the integration of risk, value, benefits, asset, portfolio and business case management there is an advantage, however, to incorporating the notion of opportunity within risk management to help align the disciplines. For this reason, within the GPM Global sustainable change delivery risk management framework we are including positive opportunities as part of the risk management framework.

Ordinal vs. Cardinal Number Models

This is one of my personal pet peeves: the usage of ordinal numbers in calculations and tools and techniques, specifically in risk management matrices such as multiplying impact / severity by probability / likelihood to get the value of the risk or severity. Subjectively ranked ordinal scales are often used dangerously in arithmetic calculations (intended for cardinal numbers).  At this stage it is useful to review the difference between ordinal and cardinal numbers:

• Ordinal scale: i.e. first, second, third, fourth, fifth
• Cardinal scale: i.e. 1, 2, 3, 4, 5

Most of the assessment models on the market use some sort of simple ordinal scale that indicates a relative order of what is being assessed, not actual units of measure. The ordinal scale that many assessment scoring systems use might be a 1-to-5-point system or simply a high/medium/low rating system. Ordinal numbers are different than cardinal numbers where 2 is twice as much as 1 and 4 is four times 1 or twice 2.  When a scale is applied, there is an assumption that the numbers used in the scale at least roughly approximate the relative magnitudes of those items.  This is what causes models like exhibit 2 to be even worse. This tool has nominal value.  What is an improbable likelihood or probability compared to probable?  What is low impact severity versus moderate impact?  Also what is the difference between undesirable, unacceptable, and catastrophic risks? These models provide information that is 'worse than useless'. Ineffective methods / models / tools like this are used with great confidence even though they add error to the evaluation. These 'sophisticated' methods are far worse than doing nothing or simply wasting money on ineffectual methods. They cause erroneous decisions to be taken that would not otherwise have been made. Note that, in this spectrum, doing nothing with this information is not actually the worst case. It is in the middle of the list. Those firms invoking the infamous “at least I am doing something” defense of their management decision processes are likely to fare worse. Doing nothing is not as bad as things can get for management. The worst thing to do is to adopt a soft scoring method or an unproven but seemingly sophisticated method (what some have called “crackpot rigor”) and act on it with high confidence. Dealing with this requires a few strategies including avoiding the Flaw of Averages and Calibrating resources to provide useful estimates.  From there, applying a variation of Hubbard's Applied Information Economics model is also useful.  These will be outlined in the following sections.

Flaw of Averages

Managing probability, severity, or proximity for risk requires an actionable response.  Responses can be a point value where there is almost no uncertainty. . . but this would not be a risk as there is no uncertainty.  Going back to Hubbard, a risk is a state of uncertainty where some of the possibilities involve a loss, injury, catastrophe, or other undesirable outcomes (i.e. something bad could happen).  Risks are measured by a set of possibilities each with quantified probabilities and quantified losses.  The key is to not offer a single value, or worse, an average due to the Flaw of Averages.  As outlined in Exhibit 3 a classic example of the Flaw of Averages involves the statistician who drowned crossing a river that was on average three feet deep. The estimated probability, severity, or proximity for a risk is not just one number.  It is a whole bunch of numbers, each with its own probability of being met.  A classic example is describing a risk that will occur in three months and have an impact of $250K with a 75% probability.  This leaves out critical information and usurps the executive's right to manage risk.

"We're also not telling them what a change in the estimate would do to that probability.  In other words, we're not showing the full range of choices in our estimates; we're usurping the decision maker's authority to decide how much risk to take on and what they'll accept as the probability of success or failure" (Thibault, 2010).

Most things we DO know are better represented by ranges and probabilities – we don’t have to assume anything we don’t really know. This is represented as a “threshold confidence." Giving decision makers a complete picture of estimate uncertainties is vital to effective risk management. It'll take some extra tools: tools for analyzing and presenting uncertain data, tools for modelling and calculating with uncertainty, and tools for capturing and quantifying input uncertainty.  A few examples are outlined next.

Incorporating Sustainability Risks

During the preliminary phase of identifying risks, risks around sustainability are often neglected.  During change delivery initiatives, these include risks that can affect the new assets lifecycle costing (such as unplanned for rising operating and maintenance costs).  Or it can affect supply and value chain risks that can affect your brand (are your suppliers using child labour?).  Or it can affect your public perception because of your environmental policies (are you damaging the environment and a risk to your customers environmental committments?).

Solutions to the Sustainable Risk Management Current Challenges

Bow Tie

Note: this section was largely borrowed from a presentation by Syed Zaiful Hamzah. The bow tie methodology originated as a technique for developing a “Safety Case”in the oil and gas industry after the Piper Alpha Incident in 1988. It is exceptionally helpful for visually analyzing and visually representing risk causes, events, and impacts. By linking hazards, or risk cause, and effects on an objective, or consequences to a risk event, it is possible to develop the relationship to include the causes, or ‘threats’, and the ‘prevention’ and ‘recovery measures.’  Further understanding can be gained by examining the means by which these defences can fail, and identifying the key components which demonstrate the integrity of these control objectives of hazard identification and assessment.

Terminology

Hazard: Potential source of harm to people, assets, the environment, and company reputation

Top Event: The incident that occurs when a hazard is realized

Threats: What could cause the top event to occur?

Consequences: What could happen if the top event occurs?

Barrier: What directly prevents or reduces the likelihood of a threat?

Recovery Measure: What prevents, minimizes or helps recovery from the consequence?

Escalation Factor: What could prevent the barrier or recovery measure from working as intended?

Escalation Factor Control: What prevents or minimizes the chance of barriers or recovery measures becoming ineffective?

To employ this methodology it is important to ensure that hazards and potential effects are known, understood and properly managed.  This is referred to as risk being reduced to As Low As Reasonably Practicable (ALARP).

Managing Hazards through Risk Reduction

The general structure of a bow tie representation is outlined below.

Bow Tie Connections The flow and connectivity for a bow tie analysis is outlined below:

Bow Tie Concept

The following is a graphical representation of what a bow tie analysis would look like.

Sampling History

The best source of information for input estimates is experience. If the records and experience exist, we can examine similar activities and use what we learn to build a suitable estimate. If there's a lot of history, we can use a resampling approach; if not, we'll need an informed estimate, preferably by calibrated estimators.

Calibration Training

For all estimates, teach subject matter experts and risk owners to provide an upper and lower bound that they are 90% certain contains the correct answer.  The challenge is that decades of studies show that most resources are statistically “overconfident” when assessing their own uncertainty.  Curiously, studies have shown that bookies were great at assessing odds subjectively, while doctors were terrible as were young professionals just out of school.  Fortunately studies also have shown that measuring your own uncertainty about a quantity is a general skill that can be taught with a measurable improvement.  Training can “calibrate” people so that of all the times they say they are 90% confident, they will be right 90% of the time. Unfortunately, almost nobody uses those methods. Calibration aids:

• “The Equivalent Bet”: for 90% Confidence Interval questions, which would you rather have?
  • A: Win $1,000 if your interval contains the correct answer
  • B: A 90% chance to win $1,000
• Think of a couple of pros and cons for your range. How could it be wrong? Why do you think it is right?
• Are you focusing on being “right” instead of realistically representing your uncertainty?
• Are you hanging on to traditional expectations of “+/- 10%” or similar narrow ranges? Are you resisting wider ranges because you think they are “too wide”?
• Are you actually trying an equivalent bet on each activity?
• Try ways to avoid “anchoring” – Don’t think of one number then add and subtract an error. Instead, treat each bound as a separate binary question (e.g. are you 95% certain the value is less than the upper bound?)

Douglas Hubbard's book The Failure of Risk Management provides training on calibration, and he also provides a course on this topic.

Applied Information Economics (AIE)

Although specifically intended for decision support modelling and measurement, the AIE provides an exceptional methodology and toolkit that are employable in business case development as well.  The AIE method is elaborated in Exhibit 7.0.

Sustainable Risk Management Analysis

There are numerous ways to integrate sustainability into risk management. One straightforward approach is to employ the P5 Standard (outlined below), facilitate a P5 impact analysis concerning the various risks (threats and opportunities), incorporate the high-impact areas into a Sustainability Management Plan (SMP) and use the organizational Sustainability Management System (SMS) to mitigate/enhance the risks.

Conclusion

A number of common challenges with risk management have been identified, with some known tools and techniques to counter act them.  It is important to realize that with sustainable change delivery the focus is on longer term asset operations and maintenance costing, asset benefit lifecycles, benefits realization, and protecting the organizations brand.  In short, proper risk management and risk mitigation.

Series Objectives

This series is all about raising awareness of sustainable change delivery and the integral elements, disciplines and competencies associated with it.  In the graphic below, each of these elements is identified in terms of its integration in empowering for sustainability.  These elements form the basis of the GPM® Global’s P5™ Standard for Sustainability in Project Management, the GPM® Global Training Programs, and the GPM® Global Portfolio, Program, & Project Sustainability Model (PSM3™) for organizational assessment.

REFERENCES

Please refer to Sustainable Risk Management - 1 Overview.