Do weather forecasts exaggerate the chance of rain? A chapter in Nate Silver’s book on prediction considers weather forecasts and the incentives of the forecasters.
The National Weather Service shoots straight: when they forecast a 30% chance of rain, it rains 30% of the time, as affirmed by this old-school technical report from the Office of Naval Research.
The Weather Channel bases its predictions on public NWS data combined with their own proprietary methods. A research paper published by the American Meteorological Society examined their track record. In his book, Mr. Silver shows a simplified version of the key result. Here’s the original:
The daily forecast chance of precipitation (“PoP”) is compared to the actual frequency of precipitation. Perfect prediction would fall along the solid 45-degree line. The wiggly solid lines above and below the perfection-prediction line delineate a window of acceptable variation, based on the number of forecasts. The gray area shows, as an aside, how often each forecast is issued.
The Weather Channel has a “wet bias” at the extremes. When precipitation is unlikely, the forecast exaggerates the chance of rain. When the likelihood is moderate, the forecast is honest. When precipitation is virtually certain, the forecast rounds up from 90% to 100% more often than it should.