jlauderdal wrote:boca wrote:Which model is that are they new or changed their name to something else.
Euro AI model, and it's been all over the place; discount it for now.
Hey! (first post of the season for me, but I've been reading all along)
For one of my last courses on atmospheric modelling at uni, I studied the performance of an AI-driven model (PanguWeather) on hurricane Ian, and compared it with ECMWF and two resolutions of WRF. While it is nothing peer-reviewed, I thought it may be useful to share my insights. And maybe this is not the right thread to post this, if so, moderators can gladly move this.
Overall, AI-driven weather models tend to score very well on large-scale patterns on the medium- to long term. This can also enable them to make potentially better track forecasts. While I only studied one AI-driven model and one situation (hurricane Ian), I found much more accurate track forecasts by the AI-driven model compared to ECMWF, or any of the NHC's numerical and statistical models. It's quite difficult to summarize it in a few sentences, but I found an average track error for the forecasts initialized between September 23rd to September 28th of only 138 km (or 75 nm) at +96 hours for PanguWeather, compared to an average error of 248 km (134 nm) for ECMWF. The post-season report of the NHC showed that almost all models had an track error of between 140 and 230 nm for that time range. This is a huge difference at 96 hours ahead. I added both a comparison of the tracks by PanguWeather, ECMWF and WRF (9/27 km) and the absolute track error as images at the end.
And keep in mind that this was on another AI-model, namely PanguWeather rather than the AIFS. However, the skill-scores for upper-level patterns have been consistently higher in favour of the AIFS compared to PanguWeather (and in fact all numerical models). Also, almost all official verifications of AI-models show that they can significantly outperform NWP models in terms of the track forecasts of tropical cyclones. That is why I would suggest not to discard this model whatsoever. And of course, ensembles remain very important (the AIFS also has one since a month or two actually), but at least in terms of operational models I would even prefer the AIFS.
This is the abstract on the small research for a bit more context:
Hurricanes are powerful natural disasters. In September 2022, hurricane Ian struck part of the Atlantic basin, including Cuba, Florida and South Carolina. With the introduction of artificial intelligence (AI) driven weather models, this case study compares the performance of these models to classical, numerical weather prediction (NWP) models. The AI-driven model PanguWeather outperforms the NWP-models ECMWF and WRF for track forecasts. It has an average track error of about 230 kilometers at a lead time of 120 hours. ECMWF reaches this error level at a lead time of 90 hours, while WRF already reaches it at 60 to 72 hours ahead. On the other hand, the NWP-models outperform PanguWeather on intensity forecasts, both for the minimum central pressure and the maximum sustained wind speeds. PanguWeather tends to strongly underestimate hurricane Ian’s intensity. ECMWF does best regarding the minimum central pressure, while WRF scores best on most verification statistics for the maximum sustained wind speeds. The physical representation of hurricane Ian at its peak is represented best by the NWP-modelsas well. PanguWeather is much less accurate in showing the key characteristics of a hurricane like a relatively warm core with high wind speeds around it, and the areas with high moisture don’t even make sense from a physical point of view. Overall, the introduction of AI-driven weather models may help to improve the forecasting of tropical cyclones, especially in terms of their future track. However, NWPmodels remain crucial to give an indication of how strong the hurricane will become, and for understanding the physical properties.
21-year old meteorologist and student MSc Climate Studies from The Netherlands. Interested in all fields of meteorology and climatology, including tropical systems like hurricanes.
