Judgments and Biases Fundamentals
About the Skill Module
This skill module introduces judgments and biases. Analysis quality depends mainly on the quality of inputs, and some of the inputs may be highly subjective. We rely upon subject matter experts (SMEs) to judge input probabilities and input distributions. We also ask SMEs to describe relationships (perhaps physical laws) so that we can model correlations. The following topics are also discussed.
Calibration Exercises
Most people are poorly calibrated and overestimate the quality of their information and knowledge. It is common for outcomes to miss the 80% confidence range. With practice and feedback, most people greatly improve their calibration.
Judgment Elicitation
Typically, one or two interviewers will elicit a distribution or probability judgment from one or several subject matter experts (SMEs). Thinking through what questions you would ask as an interviewer will prepare you for the role. Similarly, thinking about what questions someone might ask you will help if the role is reversed. The exercise assumes that you are interviewing an oil price expert to assess the oil price (WTI, Brent, or other index) three years from now.
Target Audience
Geologists, engineers, geophysicists, managers, team leaders, economists, and planners.
You Will Learn
You will learn how to:
- Describe intuition and its strengths and weaknesses in decision-making
- Name the several motivational biases the most important of the many cognitive biases
- Describe the Delphi process tool for anonymous forecasting and estimation
- Plan an elicitation interview to elicit an assessment from one or multiple SMEs
- Use calibration exercises for improving judgment
- Explain some of the causes of correlation and ways to discover this from data
- Explain how to calculate deterministic and stochastic variances to explain the difference between forecast and actual values
- Describe two or more auction types
- Describe tail estimate, winner’s curse, and optimizer’s curse biases