The Global Warming Challenge

Evidence-based forecasting for climate change

longines,tissot,fake rolex for sale,rolex day date,zenith,rolex datejust,cartier,omega,replica watches,u boat,rolex milgauss,patek philippe,rolex masterpiece,montblanc,rolex replica,a lange sohne,panerai,tag heuer

Scott Armstrong featured in “The Unwisdom of Solomon”

without comments

Does the Wisdom of Solomon Extend to the Science of Forecasting?

by J. Scott Armstrong

SOLOMON ET AL. (2009) share their opinions with us about what will happen to the climate over the next 1,000 years. I am willing to accept that Susan Solomon and her coauthors are experts in their fields and that they are doing their best to provide useful long-term forecasts. For the purposes of discussion, let’s assume that they are the smartest people in the world and that, like King Solomon, they have great wisdom. But are their forecasts of any value? Evidence on the value of experts’ forecasts began to be published in the 1930s. I summarized the evidence in Armstrong (1978). When I found that people resisted the evidence, I proposed the seer-sucker theory: “No matter how much evidence exists that seers do not exist, seers will find suckers” (Armstrong 1980).

And then, along came Philip Tetlock (2005) with findings from an ambitious experiment. He had recruited 284 people whose professions included “commenting or offering advice on political and economic trends.” He asked them to forecast the probability that various outcomes would or would not occur, picking situations within and outside their areas of expertise. Over a 20-year period ending in 2003, he had accumulated 82,361 forecasts. He then evaluated the experts’ predictions against the outcomes, and compared these with predictions from simple statistical procedures, uninformed non-experts, and well-informed non-experts. The experts barely if at all outperformed informed non-experts and neither group of forecasters did well against simple rules and models. What can we conclude about the value of predictions from experts who are unaided by scientific forecasting principles? Here is how we summarized the findings in Green and Armstrong (2007):

For long-term forecasts for complex situations where the causal factors are subject to uncertainty (as with climate), unaided judgmental forecasts by experts have no value. This applies whether the opinions are expressed in words, spreadsheets, or mathematical models. It applies regardless of how much scientific evidence [about the domain of interest] is possessed by the experts. Among the reasons are:

a) Complexity — People cannot assess complex relationships through unaided observations.

b) Coincidence — People confuse correlation with causation.

c) Feedback — People making judgmental predictions rarely receive unambiguous feedback they can use to improve their forecasting.

d) Bias — People have difficulty obtaining or using evidence that contradicts their initial beliefs. People who view themselves as experts are particularly prone to this problem.

In sum, speculation about the future, even by the cleverest and most well-informed people, should be spurned by policymakers. Instead, policymakers should make decisions based upon forecasts from scientific forecasting procedures.

References:

Armstrong, J.S. (1980). “The seer-sucker theory: The value of experts in forecasting,” Technology Review, 83 (June-July), 16-24.

Armstrong, J.S. (1978). Long-Range Forecasting: From Crystal Ball to Computer. New York: Wiley-Interscience.

Green, K. C. & J. S. Armstrong (2007). “Global warming: Forecasts by scientists versus scientific forecasts,” Energy and Environment, 18, 997-1021.

Solomon, S., G. Plattner, R. Knutti & P. Friedlingstein (2009). “Irreversible climate changes due to carbon dioxide emissions,” Proceedings of the National Academy of Sciences, Feb 10, 2009.

Tetlock, P.E. (2005). Expert Political Judgment: How Good Is It? How Can We Know? Princeton, NJ: Princeton University Press.

Written by mzfeldm

April 6th, 2009 at 1:02 pm

Posted in Uncategorized

Leave a Reply

You must be logged in to post a comment.