Skip Navigation Links weather.gov 
NOAA logo - Click to go to the NOAA homepage National Weather Service   NWS logo - Click to go to the NWS homepage
The Weather Prediction Center

Search WPC
 

biastext
SUBJECTIVE LIST OF MODEL PERFORMANCE CHARACTERISTICS

1. Introduction
The NCEP model suite is upgraded numerous times annually.  Since each "model" is actually a system of integrated components, even slight “tweaks” to any of the components can drastically effect the model's performance characteristics.  For example, the Global Forecast System (GFS) consists of the initialization scheme (Global Data Assimilation Scheme - GDAS), the Global Model itself, and the post processed grids that are made available for use in grid and grib format.   Even a slight modification to any one of these components can drastically effect the perceived performance of the model.  As a result it is not only difficult to isolate consistent model performance characteristics (loosely referred to as “bias”) across the model upgrades, but also the source of the bias.

Examples of modifications to a model system that can effect model performance characteristics include:

  • Modification of initial data ingested by model (volume, type, density, accuracy)
  • Modification of model structure (horizontal/vertical resolution, time step increments, domain, grid point vs spectral wave model, vertical coordinate, hydrostatic vs non hydrostatic models)
  • Physics packages used by the model (radiation schemes, diffusion, land/surface representations)
  • Parameterization schemes used by the model (convective parameterization, frictional components)
  • Post Processing of model data (precip type algorithms, resolution of the grid the model data is displayed upon)


2. Model Performance Characteristics
Model performance is a function of model error - which can be split into two components (systematic error and random error).
The random component of error is that which we are unable to easily attribute a cause (and therefore not easily correct).
Systematic error can be automatically removed from model output to correct or minimize the amount of error in the model solution.

A subjective list of model performance characteristics (biases) have been obtained via forecasters and is available in the next section.  This information is useful for both forecasters and by the modelers.

The majority of the biases listed below are from WPC.  However, in order for EMC to gain a more complete picture of model performance, all users of NCEP model output should provide their subjective observations of model performance by submitting their subjective observations of bias.

This information will be conveyed to WPC and EMC and potentially added to this web page.

As a reminder, results of OBJECTIVE methods to quantify the systematic error in model output are found at these sites
EMC home site http://www.emc.ncep.noaa.gov/modelperf/
WPC interactive model bias site /mdlbias/index.html
 

3. Table of Subjective Model Performance Characteristics
Click on a model name to see the latest subjective bias list.
 

Global Models (operational) Mesoscale Models (operational) Ensemble Prediction Systems
CMC AFWA MM5 CMC
ECMWF NAM ECMWF
NCEP GFS CMC GEM NCEP GFS
NOGAPS RUC NCEP SREF
UKMET NOGAPS

4. Submit a subjective bias
Click these words to submit a bias.
 

5. Other model information
Want to know the latest upcoming plans for model development and system resources as it relates to modeling at NCEP ???
Click this link /html/model2.shtml#synergy

Want a quick summary of model attributes ?  Click this link http://meted.ucar.edu/nwp/pcu2/index.htm
 


OPERATIONAL MODELS
 

AFWA MM5

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Over predicts QPF at times in regions of moderate synoptic scale forcing
Southeast and east coast
Anytime
NCEP WPC
2000
.
Initial Analysis Scheme ?

CMC GEM

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.

CMC Global

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.

ECMWF

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.

NAM

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far

GFS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
GFS too ambitious with strength and speed of systems crossing Sierras after fhr 36  
SW US
Cool season
USGS
Dec 2005
Too progressive and strong with systems crossing Sierras
Model resolution of topography
Convective Feedback 
Primarily east of front range and west of Appalachians
Warm season, any time of model day
NCEP WPC
Spring of 1998
When specific thresholds in the mass fields are met, convective scheme is triggered and then dumps a large amount of QPF over a grid point - releasing so much latent heat over the grid point that the model is forced to adjust the mass fields by producing a local vertical motion max in the mid troposphere (~ 500mb), a corresponding upper level jet max over the vertical motion max - an intense and small scale vort max in the mid levels (MCV).

The model scales up the mesoscale circulation at mid levels and holds onto it as a real feature for as long as 3 days. 

The model can produce precipitation in association with the feature as it tracks along in the flow.

GFS Convective Parameterization Scheme
Dry bias north of areas where over 2" of QPF has been produced in a 6hr period
Primarily east of front range and west of Appalachians
Warm season, any time of model day
NCEP WPC
Spring of 1998
QPF produced from convective feedback blocks northward advection of moisture
Result of GFS Convective Parameterization Scheme
QPF verification historically better than Eta
CONUS
Cool Season only
NCEP WPC
1999
Rely more heavily on QPF from GFS - especially beyond 36 hours
GDAS better than EDAS ?
Aerial coverage of QPF and mass fields over done (QPF at low thresholds .01" and .10")
CONUS
Anytime
NCEP WPC
Since mid 1990's
Over forecast of aerial coverage of precip can lead to high bias in PoPs
Model resolution (the lower the resolution the more geography a QPF pattern can get spread over)
Slightly ambitious with magnitude of high amplitude patterns
North America
Cool season so far
NCEP WPC
Since fall 2002
Prediction of southward progression of cold air over done

Model a bit too extreme in temp patterns beyond 84 hours

Precip Type Algorithm off of GFS  too eager to depict snow 

?
Ambitious to phase northern and southern stream systems in fast and spit flow patterns beyond fhr 84
North America
Cool season
NCEP WPC
Cool season 2001 (not noticed yet in 2002)
Over forecast of cyclogenesis east of Rockies
Suspect related to model resolution and lack of dense obs data where associated systems originate in forecast cycle
Major difference in QPF forecast than ETA 
CONUS
Warm season
NCEP WPC
Since mid 1990's
Lack of run to run continuity in QPF
Different convective parameterizations between models result in different QPF forecasts (primarily in areas where synoptic scale forcing is weak)

NOGAPS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far

Prior to recent upgrade was typically the model that exhibited lowest AC scores. 

Seemed very angular in its depiction of 500mb heights (extreme gradients beyond 48 hours).

.
.
.
.
.
.

RUC

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.

UKMET

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Seems better than GFS with forecast of phasing of systems in northern and southern branch of jet
North American middle latitudes
Anytime
NCEP WPC
Since fall of 2001
When GFS is showing phasing of systems beyond 84 hours, check UKMET to see if solution is consistent
GDAS ?


ENSEMBLE PREDICTION SYSTEMS

CMC EPS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.

ECMWF EPS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.

NCEP GFS EPS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Ensemble mean maxima/minima much more subdued compared to actual verification

 

Anywhere
Anytime
NCEP WPC
Fall 2000
Position forecast decent, but actual strength of a system watered down
This is normal for ensemble output with significant spread as it takes the average of numerous solutions thereby flattening the amplitude of "ridges or valleys in mass fields
The ensemble mean does well, in situations where operational GFS is a bit too aggressive, the GFS ENS MEAN seems to verify better at 500mb beyond 96 hours.
North America
Anytime
NCEP WPC
Fall 2001
More run to run continuity in EPS output, than operational runs.
Because EPS output is average of numerous solutions, it is less sensitive to extreme solutions.

NCEP SREF EPS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Ensemble mean maxima/minima much more subdued compared to actual verification
Anywhere
Anytime
NCEP WPC
Fall 2000
Position forecast decent, but actual strength of a system watered down
This is typical for ensemble output with significant spread as it takes the average of numerous solutions thereby flattening the amplitude of "ridges or valleys in mass fields

NOGAPS EPS

Subjectively Observed Bias
Geographical location of bias
Annual/Diurnal attribute
Submitted by
Date Submitted
Operational Implication
Suspected Cause
Non submitted so far
.
.
.
.
.
.


Submit a bias
Fill out the form and click SUBMIT and it will be reviewed by WPC personnel.
We will then need to contact you for more specifics, so please leave an email address.

Name
E-Mail Address (if you don't supply an email address, we won't be able to contact you for specifics on the bias)
Affiliation (NWS, private, commercial, university, public, Govt., military - be as specific as you can)


What model are you noticing a bias (GFS, Eta, ECMWF, etc.) ?

Enter the general description of bias below:

What model parameter are you noticing the bias (example -heights, temp, wind) ?
What level do you note the bias (example - 500mb, surface, 1000-500mb layer) ?
Any particular forecast hour this occurs (example - F96 to F108, anytime after fhr 36) ?
What geographical area do you notice the bias (example- Northeast US, CA, tropical Atlantic, front range of Rockies in CO) ?
Any seasonal or diurnal signal (example - nocturnal, cool season, warm season) ?


Do you have an idea why this occurs ?

What if any operational implication does this have ?

Thanks for taking the time to submit this information,  WPC personnel may contact you for more information.


 
 

NOAA/ National Weather Service
National Centers for Environmental Prediction
Weather Prediction Center
5830 University Research Court
College Park, Maryland 20740
Weather Prediction Center Web Team
Disclaimer
Credits
Glossary
Privacy Policy
About Us
Career Opportunities
Page last modified: Thursday, 12-May-2022 19:34:37 UTC