Know-how

Cognitive bias & technology development

“In terms of its consequences for decisions, the optimistic bias may well be the most significant cognitive bias. Because optimistic bias is both a blessing and a risk, you should be both happy and wary if you are temperamentally optimistic.”– Daniel Kahneman

Kahneman won the 2002 Nobel Prize in economics for his work on the psychology of judgement and decision making. In his book Thinking Fast and Slow, he introduces the concept of cognitive biases – systematic errors in our judgement or deviations from rationality (2). These ideas have found application across a range of fields, from economics, to medicine, to avalanche safety.

One symptom of the ‘optimistic bias’, as Kahneman calls it, is our tendency to overestimate benefits and underestimate costs. We’ve all experienced this. How often do cost estimates go down as a project proceeds? Not often.

A dated, but still relevant, RAND study examined cost estimate growth for forty process plants built between the late 1960s and early 1980s (3). The graph below summarizes the results nicely. On average, plant cost estimates made during R&D were less than fifty percent of the actual plant cost.

Kahneman coins the phrase 'what you see is all there is' to help explain this bias. Our minds tend to focus on the known knowns, giving little thought to the known unknowns and ignoring the unknown unknowns. In new technology development, this manifests itself when we consider only the obvious and easily estimated factors, and our cost models end up looking like this:

profit = product value – raw material costs from www.alibaba.com

We tend to neglect or underestimate factors that are less obvious or more difficult to estimate, like costs for utilities, waste disposal, and transportation. Capital costs and recycle stream implications can also fall into this category.

The RAND study identified four key factors contributing to cost growth: use of unproven technology, lack of project definition, lack of estimate detail, and plant complexity. This puts technology developers in a tough spot; we have at least three out of four strikes against us from the outset. On one hand, we need realistic cost estimates to effectively evaluate our technologies' economic potential. On the other hand, we want to avoid getting prematurely bogged down in details and unnecessarily repeating work.

There is a balance to be found here though. We can make our estimates as detailed as possible, while still maintaining the flexibility to adjust process parameters as R&D progresses. This begins with some preliminary process design work to approximate a commercial implementation of the technology. (Direct scaling of laboratory procedures isn't normally realistic, due to inefficient use of reagents and other resources at small scale.) With a material balance and a list of major equipment in hand, we can use factored methods based on historical data to develop capital and operating cost estimates. If we then incorporate these techniques into an automated techno-economic model, we have a tool for quickly assessing profitability.

Techno-economic modeling helps efficiently mitigate optimistic bias by forcing us to adopt a holistic, quantitative view of the process economics. It allows us to objectively assess profitability, measure progress, and direct resources toward important parameters. Since 'optimistic bias is both a blessing and a risk', build a model to keep it in check.

References

Kahneman, D. (2011, Oct. 24). Bias, Blindness and How We Truly Think (Part 1). Bloomberg View.
Kahneman, D. (2011). Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.
Merrow, E.W. (1983). Cost Growth in New Process Facilities. RAND Corporation.