Statistical Analysis of Forecasts, 1999-2005

[Regit]

There has been a lot of talk about season forecasts lately. With some free time today and good knowledge of dealing with statistics, I decided to look at some of Dr. Gray's past forecasts vs. actual numbers for the season. All forecasts were those issued on or around June 1 of the same year. The percentages suggest by how much Dr. Gray missed his numbers for the year. Brackets suggest his numbers were too high. Lack of brackets suggest they were too low. I have converted them into percentages, as missing by 1 in a season with 20 storms is statistically very different than missing by 1 in a season with 10 storms. I have done this for only the "named storms" category. Statistics currently include 1999-2005, but will expand it in the future and include other forecast sources.

1999 - [14%] Moderately Successful Forecast
2000 - 14% Moderately Successful Forecast
2001 - 20% Adequate Forecast
2002 - 8% Very Successful Forecast
2003 - 13% Moderately Successful Forecast
2004 - 6% Very Successful Forecast
2005 - 44% Poor Forecast

Conclusion: With only one outlier, the past seven June forecasts have proven to be statistically strong and this suggests future forecasts can serve as good measures of the future season. They would be considered excellent measures if not for the fact that the single outlier also happens to be the most recent forecast.

_____________________________________________________________


The strike probability data is posted in a large and detailed chart which I can't replicate in this forum. However, I will still present a discussion.

I will discuss three ways to interpret the data of Dr. Gray's strike probability forecast to predict the season's landfalls. One way is to translate the data simply as "Point A is more likely to get hit than Point B." Another way to read it is to use the actual spread of percentages (Point A is n percent more likely to get hit than Point B). Another way is odds-based (the odds are Point A will get hit this year; the odds are Point B will not get hit this year). There are more complicated interpretations too, none of which are discussed much in this forum. There is also, of course, the idea that because percentages are higher than in past forecasts, there will be more strikes. I discuss this at the end.

I think that the way most people in this forum read strike probabilities is the first example. That is, the place with the highest percentage is the place more likely to get hit or the place that will get more storms. Obviously, Dr. Gray's strike forecasts are much more complicated than this, but the forum is my audience, and a more detailed discussion about probability is not needed.

Conclusion: When we take the past seven forecasts and use the forecast to predict which coast is more likely to get hit, we find that we are right 3 times, or 43% of the time. So for those of you who are using Dr. Gray's forecasts to predict which coast will get hit, the forecasts are statistically useless. There is also no correlation between the percentages being higher and the actual number of storms being higher.

[Hybridstorm_November2001]

Conclusion: When we take the past seven forecasts and use the forecast to predict which coast is more likely to get hit, we find that we are right 3 times, or 43% of the time. So for those of you who are using Dr. Gray's forecasts to predict which coast will get hit, the forecasts are statistically useless. There is also no correlation between the percentages being higher and the actual number of storms being higher.

Very true. I could not have put it better myself.

[Stratusxpeye]

Brackets suggest his numbers were too low. Lack of brackets suggest they were too high.

1999 - [14%] Moderately Successful Forecast
2000 - 14% Moderately Successful Forecast
2001 - 20% Adequate Forecast
2002 - 8% Very Successful Forecast
2003 - 13% Moderately Successful Forecast
2004 - 6% Very Successful Forecast
2005 - 44% Poor Forecast

I may be misreading this but in 1999 he didnt forecast ENOUGH storms and there were more than he forecast but all other years including 2005 he forecast that much less? I belive he forecasted 15 or so last year and there were 28 which would make 44% more storms than he predicted not less. Correct me ifI am wrong.

[Regit]

Brackets suggest his numbers were too low. Lack of brackets suggest they were too high.

1999 - [14%] Moderately Successful Forecast
2000 - 14% Moderately Successful Forecast
2001 - 20% Adequate Forecast
2002 - 8% Very Successful Forecast
2003 - 13% Moderately Successful Forecast
2004 - 6% Very Successful Forecast
2005 - 44% Poor Forecast

I may be misreading this but in 1999 he didnt forecast ENOUGH storms and there were more than he forecast but all other years including 2005 he forecast that much less? I belive he forecasted 15 or so last year and there were 28 which would make 44% more storms than he predicted not less. Correct me ifI am wrong.

That was a typo. I've fixed it. Thanks for pointing it out.

[Stratusxpeye]

Your Welcome. Just wanted to make sure I wasn't crazy.

Nice post though makes it easier to see in percentages. so far for 5 years there has been about 15% or so more storms than what was predicted (On Average). If this holds true we could see approx 15 -19 Storms this season. Or this would be the first season since 1999 more storms were predicted than actually occured, given the period we are in I would doubt this highly. Statisticly speaking.

[Regit]

A bump, now that the June forecast is out.