Network News

X My Profile
View More Activity
The new Washington
Post Weather website
Jump to CWG's
Latest Full Forecast
Outside now? Radar, temps
and more: Weather Wall
Follow us on Twitter (@capitalweather) and become a fan on Facebook
Posted at 10:45 AM ET, 02/ 3/2010

This past Saturday's snow forecast: Subpar

By Capital Weather Gang

Weather Checker

* Weather breaks 'til Friday: Full Forecast | Big snow coming? SLCB *

Weather Checker is an outsider's analysis of CWG's forecast accuracy. See previous Weather Checker posts.


More snow than initially predicted by most forecast outlets, including CWG, falls in Georgetown on Saturday. By CWG photographer Kevin Ambrose.

By Jamie Yesnowitz

A few notes on CWG's forecasting performance for the surprisingly potent storm this past weekend:

Good pre-storm coverage. Coverage of a potential threat in the days leading up to the storm was as usual, extensive, and covered the run-to-run fluctuations of the forecast models. Jason Samenow mentioned in his Monday forecast last week the threat of a storm occurring between late Thursday and early Saturday, and consistent updates followed as the event approached. The first Snow Lover's Crystal Ball was posted early Monday morning as well, with a probability chart, which leads to the next point . . .

The probability tables are extremely helpful. They provide adequate warning for the outlier event. It is interesting to note that in the accounting profession, practitioners are required to measure for uncertainty in income tax positions by using probability tables remarkably similar to those used by CWG.

Keep reading for more analysis of CWG's coverage of the past weekend's snowstorm...

It's a good topic for a future column, but suffice to say, the most likely outcome is not necessarily the outcome that is used for accounting purposes -- rather, the outcome that occurs at the 50% probability line is what matters (I characterize this as the "50% rule"). And the issue of how to read a probability map was raised by CWG and readers alike in the comments section in the days prior to the storm. A good example was the probability chart in last Tuesday's SLCB, which read as follows:

Here is the current (VERY EARLY) probabilities on accumulations: 40% chance: Less than 1"; 20% chance: 1-4"; 20% chance: 4-8"; 20% chance: 8"+

If one looked at the probability chart using the most likely option listed, a prediction of less than 1 inch would have been made. Using the 50% rule, on the other hand, would yield a 1-4 inch prediction. While both these predictions ultimately failed to verify, the early call that there was a 40% chance of a 4+ inch snowfall four days before the event was noteworthy. Clearly, there was optimism early on that some accumulating snow would make it to the D.C. area.

Issuing the snow map is in many respects "the end game." Probability tables are great, especially when gauging a potential outlier threat, but when the snow map comes out, there's a level of finality inherent in the map which doesn't come through in the probability charts. The very first map issued by CWG on Friday showed D.C. in the "dusting to 1 inch" category (backtracking from the 1-4 inch prediction above, if one considers the 50% rule to be in effect), with a 1-4 inch band extending from just south of D.C. to about Fredericksburg, deeper snows south of Fredericksburg and flurries north of DC. Obviously, this map missed by a wide margin, with only Richmond's 8+ inch forecast verifying. Later maps -- late Friday morning and Friday afternoon at least had the magnitude of the snow moving in the right direction, though the substantial accumulations north of D.C. never really got incorporated into the forecast until CWG entered the "nowcasting" phase. Other local forecast outlets didn't do much better through late Friday night.

A whole lot of nowcasting. The forecast was well off until the flakes were flying throughout the metro area. By late morning Saturday, CWG had updated its snow prediction to 4-8 inches, which did mostly verify throughout the region (though some spots to the south attained higher amounts). The good news is that CWG then stuck with its 4-8 inch call through the afternoon and evening, rather than continuing to tweak the forecast with each snowflake, thereby avoiding the "double-miss" phenomenon that occurred in the early March 2009 snowstorm.

The bad news is that, for the most part, CWG's overall forecast missed on two levels, underplaying the snow depth in the immediate metro area until the day of the storm, and underestimating the additional push of snow to the northern suburbs.

Post-mortems in the comments section are getting better and more extensive. It's getting increasingly difficult to add much to a the conversation about what CWG could have done to improve a particular forecast, and that's a good thing -- CWG and their readers are engaging in discussions on this topic even before a forecast may be going awry, and continue the discussion long after the event is past.

Incorporating this winter's storms into future snow predictions? This year, the D.C. area has had two very unusual snowstorms, the Dec. 19 megastorm that was in part enhanced by unusually warm waters off the Atlantic, and last Saturday's "cold southern storm," with temperatures hovering in the teens and a southern track that, contrary to most early predictions, moved much farther north than initially expected (accumulating snows all the way up to southern New Jersey). Hopefully, the experiences gleaned from these two events will be incorporated into future forecasts in an effort to improve the long- and short-range predictions that often fall short of the mark during snow season.


Final totals from Saturday's snow, as compiled by the National Weather Service. Larger image.

Jamie Yesnowitz has been interested in the weather since he rooted for school-closing snowstorms while growing up in Brooklyn and East Rockaway, N.Y. After graduating from Dartmouth College with a bachelor's degree in economics and government, his focus on the accuracy of weather predictions took hold when he moved to Coral Gables, Fla., to attend the University of Miami School of Law. Class was scheduled to begin on August 24, 1992. Hurricane Andrew had other ideas, however, shutting down the school for weeks. But what stuck in Jamie's mind was the final unpredicted swerve of the eye that saved those living in Miami and points north, and completely devastated areas about 20 miles south of Miami.

Undeterred by the hurricane, Jamie ultimately served as editor-in-chief of his law school newspaper, and earned both a juris doctorate and master's degree in taxation. Following law school, Jamie practiced corporate and securities law in New York before shifting to the state and local tax consulting world. Jamie moved from New York to the Washington area in 2003, and he is presently a state and local tax senior manager at a major accounting firm. Jamie lives in Potomac with his wife, Sandra, and their two daughters, Sarah and Carly.

By Capital Weather Gang  | February 3, 2010; 10:45 AM ET
Categories:  Weather Checker  
Save & Share:  Send E-mail   Facebook   Twitter   Digg   Yahoo Buzz   Del.icio.us   StumbleUpon   Technorati   Google Buzz   Previous: Forecast: Weather takes a break until Friday
Next: Models stay consistent with major snow potential

Comments

First. Office is buzzing with chatter about the upcoming storms!

Posted by: Hanz1 | February 3, 2010 10:48 AM | Report abuse

The probability tables are definitely helpful. This is the most informative weather blog around -- thanks for that.

A small aside: I hope for a day when people can be banned from saying "first" in the comments section. The "firsters" epitomize zero-value added.

Posted by: aircardinal | February 3, 2010 10:54 AM | Report abuse

Geez, CWG! You scared me. I see the headline "Saturday snow forecast - subpar" and I think you're talking about this upcoming Saturday.

I'm getting way too emotionally involved with these storms!

Posted by: teezee210 | February 3, 2010 10:58 AM | Report abuse

LOVE the fact that you do postmortems! I remember years ago when TV was forecasting a big (march 2001?) storm.

That storm never materialized, and I felt totally duped and cheated by the TV forecasters. I waited with great anticipation for their explanation of what happened that evening.

That night, they basically made NO MENTION of the huge bust, and went on to forecast the next few days as if all was normal. I felt so... so... used...

Posted by: SJ43560 | February 3, 2010 11:04 AM | Report abuse

I had the same reaction as teezee210! My heart sank when I read the headline, until I realized it was about last weekend's storm. :-P Givin' the snowlovers a heart attack!

Posted by: megamuphen | February 3, 2010 11:07 AM | Report abuse

@ SJ43560 - Are you referring to the Feb. or Mar. 2000 storm. For days leading up to the weekend, forecasters were warning of a huge coastal. NBC4 was calling for 12+ inches as little as 12 hours before predicted onset. Those amounts quickly dropped, and I think we ended up with a flurry. That was painful to watch.

Posted by: ParkerGP | February 3, 2010 11:11 AM | Report abuse

Made a tweak to this post's headline, so we don't scare too many people or sink too many hearts :)

Posted by: Dan-CapitalWeatherGang | February 3, 2010 11:13 AM | Report abuse

Fascinating. I live in Baltimore and was unpleasantly surprised by the burst of snow on Saturday (I love snow, but not when I'd been planning to run errands and shop for food). However, most of the casters in my area whiffed pretty hard on that norther spread as well, and none of them are doing postmortems. I appreciate the self-critique and the attempts to improve forecasting. And hey- if we knew exactly what was going to happen, this wouldn't be called weather "prediction".

Posted by: Bawlmer51 | February 3, 2010 12:12 PM | Report abuse

I am a bit curious if we will be effected at work on Monday

Posted by: Bious | February 3, 2010 12:15 PM | Report abuse

First: CWG Rock On!!! You have possibly the finest, most interesting and creative product in meteorology.

My question relates to the probability tables, which I find confusing. You wrote:

"Here is the current (VERY EARLY) probabilities on accumulations: 40% chance: Less than 1"; 20% chance: 1-4"; 20% chance: 4-8"; 20% chance: 8"+
If one looked at the probability chart using the most likely option listed, a prediction of less than 1 inch would have been made. Using the 50% rule, on the other hand, would yield a 1-4 inch prediction."

I, for one, do not see how the 50% rule yields a 1-4 inch prediction. I guess what confuses people (we've discussed this at work)is that "less than 1 inch" means ZERO to less than 1 inch," but never reaching 1 inch. So we can interpret that as 60% chance of no snow to 4 inches of snow.

Maybe it works for accountants, but I would prefer another system :)
Still, you guys are great and an ADDICTION in my workplace! Maybe you can clear things up a bit or us...

Posted by: Hoyas4Ever | February 3, 2010 12:20 PM | Report abuse

what did the headline originally say?

Posted by: snowlover | February 3, 2010 12:40 PM | Report abuse

It didn't have the "This past" :-)

Posted by: megamuphen | February 3, 2010 12:51 PM | Report abuse

I was hard on you guys yesterday, but over all you do a great job and CapWx is my go to for DC weather.

Posted by: johnnyd2 | February 3, 2010 12:53 PM | Report abuse

I think what bothered me about this last storm's forecast was that it seemed like your forecast was entirely based on what the computers were telling you. There seemed to be no other work put into the predictions. Whatever the models said, is what your forecast was. In one of the pre-Saturday posts somebody posted a link to I think it was Joe Bastardi's forecast where he referenced to a storm from 1967 and noted all of its similarities to the what the conditions of Saturday's storm were. To me that is what I want from a meteorologist. If all you are doing is looking at a model and repeating what it forecasts, that doesn't seem like much what a meteorologist is supposed to do. That is just my impression though, so feel free to prove me wrong. I expect you will. When forecasting a storm what other criteria goes into the forecast other than the model runs? All of that being said, I do enjoy CWG and all of the updates you give.

Posted by: nojY | February 3, 2010 1:16 PM | Report abuse

@nojY: That's because the average age of the mets at CapWx is like 35. Meteorlogists who have been around a long time and can sometimes use their experience to over ride what the models are predicting. Like it or not, that is a weakness of this team.

Posted by: johnnyd2 | February 3, 2010 1:27 PM | Report abuse

I agree with Hoyas4Ever - "Using the 50% rule, on the other hand, would yield a 1-4 inch prediction" just isn't correct. (Better go back and check those tax tables too.) There was a predicted 60% chance of 4 inches or less - you can't say anything beyond that. If you assume some linear relation to the spread, you could speculate a 10% chance of 1 to 2.5 and a 10% chance of 2.5 to 4. That would give you a "50% rule" for the prediction of 2.5 inches or less. Alternatively, the 40% for an inch or less suggests a little more than an inch was far more likely than 4 inches (non linear). Yes - it's a small point, but what else do geeks have to debate. The 50 % rule also falls apart when you predict a 50% chance of 4" or less and a 50% chance of 4" or more. So while it might look more "scientific" to put percentages into a forecast, I wouldn't call it "extremely helpful."

Posted by: manatt | February 3, 2010 1:35 PM | Report abuse

And after looking more closely at their bios, looks like most of the regular contributors have only been out of school in the last 5-10 years and have never worked as professional forecasters, save for a few internships.

I'm not being a hater, just statn' the facts.

Posted by: johnnyd2 | February 3, 2010 1:45 PM | Report abuse

ParkerGP - yes thats the one (feb or Mar 2000). It was indeed painful as I can still recall it a decade later.

Posted by: SJ43560 | February 3, 2010 1:45 PM | Report abuse

Hoyas4ever and Manatt -- good points both. I plan to address the 50% rule in a later column -- came to me in a flash when I saw the prob tables. Believe me, the accounting implication of such a rule can be, and is aggravating. Having said that, I like the linear explanation even better than the range -- 2.5 inches being the "50%" result. Still a work in progress -- amazing that tax accounting and weather can be so similar sometimes.

--Jamie Y (Potomac) aka Weather Checker

Posted by: JamieYPotomac | February 3, 2010 1:51 PM | Report abuse

@johnnyd2

Our team is generally young though we do have a couple "veterans" on board. Collectively, we have a fair amount of experience and we collaborate.

@nojY

While we try not to just read models and waffle back and forth, they are critical tools all forecasters use, irrespective of their experience. As a matter of fact, even when the models suggested the region would get nothing, we were highly skeptical of that and kept the possibility of accumulating snow in the forecast (when other, much more "experienced" forecasters on TV said there would be no major snowstorm this weekend). Yes, we lowered probabilities, because the models do have skill and to have completely ignored them would have been foolish. Leveraging one's intuition and experience is important when forecasting as well as understanding model biases -- but at the same time, completely disregarding the models is very risky business and against the odds.

Posted by: CapitalWeatherGang | February 3, 2010 2:10 PM | Report abuse

When might we see Winter Storm Watches be posted for the region?

Posted by: jcmcgrath1969 | February 3, 2010 2:15 PM | Report abuse

@johnnyd2

Folks who have been following this blog since its inception in 2004 can vouch that we've been as accurate or more accurate than other local outlets (i.e. TV and NWS) in forecasting snow on average.

Posted by: CapitalWeatherGang | February 3, 2010 2:22 PM | Report abuse

I've been here since then (oh the days of that old school indi website you had!) and agree never said you guys aren't good. But do you have hard data, not just anecdotal, to support your claims? Just askn' since you brought it up.

Posted by: johnnyd2 | February 3, 2010 2:41 PM | Report abuse

I love the post-mortems on CWG, although I disagree with the 'subpar' in the headline. It's true CWG did do a lot of now-casting, but they also did a lot better in preparing their readers for a big storm than any of the TV forecasters. If you compare CWG to the idea of the perfect, then the forecasting was subpar. But that's an absurd comparison to make. If you instead compare CWG to other forecasters, than they did well above average.

Posted by: nlcaldwell | February 3, 2010 3:04 PM | Report abuse

@johnnyd2

It's anecdotal, but we look at these things after each storm and either self-assess and/or (as of late) get an independent assessment.

Posted by: CapitalWeatherGang | February 3, 2010 3:13 PM | Report abuse

Cool, love the transparency of this site. Truly forecasting in the 21st Century!

Posted by: johnnyd2 | February 3, 2010 3:54 PM | Report abuse

Glad you do some post-analysis. Next time it would be cool to add what happened to cause the storm to come more north.

Dunno if you've ever heard of them, but footsforecast were the only ones to it right for this storm.

Posted by: SkinsFan132 | February 3, 2010 5:57 PM | Report abuse

dude, shut up. do you impress chicks with these long winded useless post?

Posted by: makplan20002 | February 3, 2010 7:53 PM | Report abuse

ParkerGP: I remember the big miss in 00 well. I played golf in Frederick County, it was a beautiful day with warm sun in the mid 40's. Seemed improbable and it then never appeared.

Posted by: drc231 | February 3, 2010 10:08 PM | Report abuse

I'm a noob to this, and I'm loving it all. The Joe Bastardi forecast was interesting: he put together top notch analysis that made sense and used it to support a bold prediction. (Better than ESPN breaking down the Super Bowl IMHO.) And there has been a lot of discussion about FootsForcast (the response from CWG about not doing Balt. Cty. struck me as overly defensive, particularly since WaPo covers the Orioles and sells papers in the Balt. area). And the sniping back and forth -- it trips me out to see this happening over _weather forecasts_. _Very_ entertaining and informative. I'll never go back to Weather Channel again! A week ago I never HEARD of footsforecast, now they're bookmarked...

Posted by: gbooksdc | February 4, 2010 12:08 AM | Report abuse

@gbooksdc

Fair point that we shouldn't neglect Baltimore given its proximity and we do pay it some attention. The point I was trying to make, and maybe I was overly defensive, is that we expend a lot more effort analyzing the D.C. metro region compared to surrounding cities like Richmond and Baltimore so our forecasts may not be as careful for those areas.

Posted by: Jason-CapitalWeatherGang | February 4, 2010 2:29 AM | Report abuse

The comments to this entry are closed.

 
 
RSS Feed
Subscribe to The Post

© 2012 The Washington Post Company