Archive for June, 2007
At one of the concluding sessions of the 27th Annual International Symposium on Forecasting (ISF) titled “Auditing Public Policy Forecasting: Climate Change, Gun Control, and Other Issues”, Scott Armstrong, Paul Goodwin, and John Lott each gave presentations during the hour-long session. Below is information on the speakers as well as talk summaries.
Scott Armstrong on Climate Change
Professor of Marketing, University of Pennsylvania
Click for a brief video of Armstrong’s talk.
Presented his and Kesten Green’s findings from the paper, “Global Warming: Forecasts by Scientists Versus Scientific Forecasts,” where they applied forecasting principles to Section 8 of the 2007 IPCC report, revealing that scientists did not adhere to proper forecasting procedure. These critical violations of forecasting principles led them to conclude that they have been unable to identify scientific forecasts to support global warming. The presentation concluded with an explanation of the challenge to Al Gore, as well as theclimatebet.com.
Paul Goodwin on Business Forecasting
Professor of Management Science, University of Bath (UK)
How do models fit in with business forecasting? Goodwin explained how the use of ‘the system’ and high-tech software analytics creates models that are merely used to support desired forecasts, instead of the back-and-forth relationship that should exist between models and forecasters’ interests. He left the audience with the question: in a world where billions of dollars are at stake, how to firms stay afloat if they only see what they want to see?
John Lott on Gun Control
Senior Research Scientist, University of Maryland
Controversial economist John Lott concluded the session by highlighting the strength of emotion against the facts, present in most major public policy debates. His issue? Gun control. Lott explained why we are much more likely to see violent gun crimes in the mass media, and why we rarely hear of the use of guns in self-defense or to ward off criminals. Though the first is much more memorable, the latter is far more common. These facts must be used when making public policy decisions that are emotionally charged.
The session concluded with questions and discussion, as well as this impromptu photo of Armstrong and Trenberth encouraging scientists to facilitate discussion across opinions:
Dr. Kevin Trenberth, Head of the Climate Analysis Section at the National Center for Atmospheric Research, gave a keynote presentation June 26th, 2007, at the 27th International Symposium for Forecasting. He was a lead author of the 1995, 2001, and 2007 Intergovernmental Panels on Climate Change (IPCC) Scientific Assessment of Climate Change. For a full list of credentials, please visit Dr. Trenberth’s homepage.
Trenberth’s keynote presentation was titled Climate and weather forecasting: Issues and prospects for prediction of climate on multiple time scales. His full presentation is available at his website. The following is an abstract excerpt:
In contrast to weather, which involves the evolution of the atmosphere, climate involves the entire climate system and its forcings. The climate system includes the atmosphere and oceans as fluid components, and all aspects of the land surface including vegetation and hydrology, and the cryosphere. The forcings include influences external to the climate system, such as the sun or human activities. While prediction of the atmosphere is inherently limited by chaotic developments, its systematic interaction with other components of the climate system and forcings enables patterns or the statistics of atmospheric behavior to have some predictability.
Updated: Gore’s office responded on Thursday, June 28th, 2007.
“Meet the Ivy League professor and expert on forecasting who is challenging Al Gore to a $20,000 bet that he is wrong on global warming.“
This Monday June 25th, 2007, the article “Put your money where your ‘myth’ is” by Brendan O’Neill was featured on spiked-online.com, an online magazine with a global and critically minded readership. Armstrong was interviewed for the article by O’Neill. Read on for two excerpts from the article.
Tuesday afternoon, June 19th, 2007, a letter with the terms of the challenge attached was both faxed and mailed to Al Gore. The full contents of the letter are published below:
June 19, 2007
Honorable Albert Gore
2100 West End Avenue
Nashville, TN 37203
Dear Mr. Gore,
A “Global Warming Challenge” is attached. I think the challenge serves our mutual interest in developing better public policy. The terms of the challenge can be easily changed upon mutual agreement.
The primary objective is to improve the application of scientific methods in forecasting climate change and, thus, to use better forecasting methods. In addition, it may provide funding for one of our charities.
The objectives are attainable no matter which of us would “win” the challenge. The fact that we would be joining together in this challenge should draw the attention of scientists to the need for using the best forecasting methods and conducting proper validation tests.
Might you be able to respond by the time of my International Symposium on Forecasting talk on the morning of Wednesday June 27? This could be something as simple as “accept,” “decline,” or “contemplating.” Or it could be a longer response. You, or one of your representatives would be welcome, of course, to be a guest at this conference.
I believe that you already know the chairperson, Kajal Lahiri, from his days in Arlington when your sons played on the same soccer team. If you can attend, Kajal will make provisions for you to respond to this challenge. The conference runs from Monday through mid-day Wednesday.
Would you be interested in receiving a copy of my Principles of Forecasting book? I will be happy to send you a copy if you tell me what address to use.
J. Scott Armstrong
Professor of Marketing
The Wharton School
University of Pennsylvania
Philadelphia, PA 19104
armstrong at wharton dot upenn dot edu
Today, the University of Pennsylvania newspaper “The Daily Pennsylvanian” covered Armstrong’s challenge on the front page (pdf). Click here to view the online version of “An Inconvenient Bet?”
Of note in the article is the author’s interview of an MIT climate scientist:
Carl Wunsch, a professor of physical oceanography and climate-change expert at the Massachusetts Institute of Technology, said that though different forecasting models show different outcomes for the planet as a result of global warming, most which are taken seriously by climate-change experts do show the world overall becoming warming and wetter because of human activity.
“They should be regarded as possibilities … that need to be taken seriously,” he said.
Wunsch also questioned whether someone who is admittedly unversed in the scientific principles of meteorology and geology could provide a better forecasting model than experts in these fields.
“Hopefully other competitors would join to show the value of their forecasting methods.”
As stated in the terms, we are actively looking for other participants to take part in the challenge. All will be documented right here at theclimatebet.com for our readers. Maybe you think you can do better than Armstrong or Gore.
Have a model you wish to have tracked? Interested in promoting the proper use of science in formulating public policy?
Send Scott Armstrong an email: armstrong at wharton dot upenn dot edu.
Scott Armstrong will present the Armstrong and Green paper, Global Warming: Forecasts by Scientists versus Scientific Forecasts, at a session at the International Symposium on Forecasting on Wednesday June 27 at 11:45 in the Olmstead Room of the Marriott Hotel in Times Square. The session will introduce the Public Policy Forecasting site. Following in the tradition of “Julian Simon’s Bet,” Scott Armstrong will announce a $20,000 challenge (each side to post $10,000 to go to a charitable cause nominated by the winner) that he will be able to more accurately forecast climate change than can any currently available fully disclosed climate model.
The following terms were sent to Al Gore Tuesday, June 19th, 2007.
Al Gore has claimed that there are scientific forecasts that the earth will become warmer and that this will occur rapidly. University of Pennsylvania Professor J. Scott Armstrong, author of Principle of Forecasting: A Handbook for Researchers and Practitioners, and Kesten C. Green, of the University of South Australia (and Armstrong’s Co-Director of forecastingprinciples.com), have been unable to locate a scientific forecast to support that viewpoint. As a result, Scott Armstrong offers a challenge to Al Gore that he will be able to make more accurate forecasts of annual mean temperatures than those that can be produced by current climate models.
The general objective of the challenge is to promote the proper use of science in formulating public policy. This involves such things as full disclosure of forecasting methods and data, and the proper testing of alternative methods. A specific objective is to develop useful methods to forecast global temperatures. Hopefully other competitors would join to show the value of their forecasting methods. These are objectives that we share and they can be achieved no matter who wins the challenge.
Al Gore is invited to select any currently available fully disclosed climate model to produce the forecasts (without human adjustments to the model’s forecasts). Scott Armstrong’s forecasts will be based on the naive (no-change) model; that is, for each of the ten years of the challenge, he will use the most recent year’s average temperature at each station as the forecast for each of the years in the future. The naïve model is a commonly used benchmark in assessing forecasting methods and it is a strong competitor when uncertainty is high or when improper forecasting methods have been used.
Specifically, the challenge will involve making forecasts for ten weather stations that are reliable and geographically dispersed. An independent panel composed of experts agreeable to both parties will designate the weather stations. Data from these sites will be listed on a public web site along with daily temperature readings and, when available, error scores for each contestant.
Starting at the beginning of 2008, one-year ahead forecasts then two-year ahead forecasts, and so on up to ten-year-ahead forecasts of annual “mean temperature” will be made annually for each weather station for each of the next ten years. Forecasts must be submitted by the end of the first working day in January. Each calendar year would end on December 31.
The criteria for accuracy would be the average absolute forecast error at each weather station. Averages across stations would be made for each forecast horizon (e.g., for a six-year ahead forecast). Finally, simple unweighted averages will be made of the forecast errors across all forecast horizons. For example, the average across the two-year ahead forecast errors would receive the same weight as that across the nine-year-ahead forecast errors. This unweighted average would be used as the criterion for determining the winner.
Terms of the challenge can be modified by mutual agreement.
If Al Gore accepts the challenge, each party would invest $10,000 in a mutually agreed Charitable Trust stock account on December 1, 2007. The charity will receive the total value in the fund when the official award is made at the annual International Symposium on Forecasting in 2018. Should Scott Armstrong win, the gift would be made to the Institute for Justice, in Arlington, Virginia. Should Al Gore win, he will designate the charity.
Details on the challenge and related materials such as the paper on climate forecasting by Armstrong and Green are provided at publicpolicyforecasting.com. To facilitate open discussion, a blog is provided at theclimatebet.com.