PROGRAM filter
Overview
Main program for driving ensemble filter assimilations.
filter
is a Fortran 90 program, and provides a large number of options for controlling execution behavior and
parameter configuration that are driven from its namelist. See the namelist section below for more details. The number
of assimilation steps to be done is controlled by the input observation sequence and by the time-stepping capabilities
of the model being used in the assimilation.
This overview includes these subsections:
Program Flow
Filter Types
Getting Started
Free Model Run after Assimilation
Evaluate a Model State against Observations
Compare Results with and without Assimilation
DART Quality Control Values on Output
Description of Inflation Options
Detailed Program Flow
See Welcome to the Data Assimilation Research Testbed for more documentation, including a discussion of the capabilities of the assimilation system, a diagram of the entire execution cycle, the options and features.
Program flow
The basic execution loop is:
Read in model initial conditions, observations, set up and initialize
Until out of observations:
Run multiple copies of the model to get forecasts of model state
Assimilate all observations in the current time window
Repeat
Write out diagnostic files, restart files, final observation sequence file
The time of the observations in the input observation sequence file controls the length of execution of filter.
For large, parallel models, the execution loop is usually wrapped in an external script which does these additional steps:
Link to an observation sequence file which contains only observation times within the next assimilation window
Link any output inflation files from the previous step to be the input files for this step
Run filter, which will exit after doing the assimilation without trying to advance the model
Save the output diagnostic files for later
Advance the N copies of the model using the model scripts or whatever method is appropriate
Repeat until all data is assimilated
For large models filter is almost always compiled to be a parallel MPI program, and most large models are themselves a parallel program using OpenMP, MPI, or both. MPI programs usually cannot start other MPI programs, so the external script submits both the filter job and the N model advances to a batch system so all run as independent parallel jobs.
The same source code is used for all applications of filter. The code specific to the types of observations and the interface code for the computational model is configured at compile time. The top level directory has been simplified from previous versions to look like :
README
COPYRIGHT
assimilation_code
build_templates
diagnostics
documentation
models
observations
the assimilation_code contains all module and program source code for all of the main programs including filter.
Specifically in the modules directory there is a filter_mod.f90
which contains the source for the filter main
program. Each model has a separate directory under DART/models, and under each model is a work directory where the code
is compiled and can be run for testing. Generally when a full-size experiment is done the executables are copied to a
different location - e.g. scratch space on a large filesystem - since the data files for 10s to 100s of copies of a
model can get very large.
Directories expected to be modified
DART is distributed as a toolkit/library/facility that can be used as-is with the existing models and observations, but is also designed so that users can add new models, new observation types and forward operators, and new assimilation algorithms.
The locations in the DART code tree which are intended to be modified by users are:
- New Models
Add a new directory in the
models
subdirectory. Copy (recursively, e.g.cp -r
) the contents of thetemplate
directory and modify from there. Note that themodel_mod.f90
file in the template dir is appropriate for small models; for large geophysical models see thefull_model_mod.f90
file and also examine other model directories for ideas. See additional documentation in the MODULE model_mod documentation, and the DART web pages on adding new models.- New Observation Platforms
To convert observations from other formats to DART format, add a new directory in the
observations/obs_converters
subdirectory and populate it with converter code.- New Observation Types and Forward Operators
Define a new type (a measurement from an observing platform) via a file in the
observations/forward_operators
subdirectory. If the forward operator is more complicated than directly interpolating a field in the model state, this is where the code for that goes. See additional documentation in the MODULE obs_def_mod documentation, and the DART web pages on adding new types. Adding a new type may require adding a newgeneric kind
, which is documented in MODULE obs_kind_mod.- New Assimilation Algorithms
If you want to try out a different filter type modify the filter code in the
assim_tools_mod.f90
file. See the MODULE assim_tools_mod documentation.
Detailed program execution flow
The Manhattan release of DART includes state space output expanded from the
previous two stages (Prior and Posterior) to up to six (input, forecast, preassim,
postassim, analysis, and output). This makes it possible to examine the states with
and without either kind of inflation, as described below. In addition, the state
space vectors are each written to a separate NetCDF file:
${stage}_mean.nc, ${stage}_sd.nc, ${stage}_member_####.nc
.
The detailed execution flow inside the filter program is:
Read in observations.
Read in state vectors from model netcdf restart files.
Initialize inflation fields, possibly reading netcdf restart files.
If requested, initialize and write to “input” netcdf diagnostic files.
Trim off any observations if start/stop times specified.
Begin main assimilation loop:
Check model time vs observation times:
If current assimilation window is earlier than model time, error.
If current assimilation window includes model time, begin assimilating.
If current assimilation window is later than model time, advance model:
Write out current state vectors for all ensemble members.
Advance the model by subroutine call or by shell script:
Tell the model to run up to the requested time.
Read in new state vectors from netcdf files for all ensemble members.
Apply prior inflation if requested.
Compute ensemble of prior observation values with forward operators.
If requested, compute and write the “preassim” netcdf diagnostic files. This is AFTER any prior inflation has been applied.
Compute prior observation space diagnostics.
Assimilate all observations in this window:
Get all obs locations and kinds.
Get all state vector locations and kinds.
For each observation:
Compute the observation increments.
Find all other obs and states within localization radius.
Compute the covariance between obs and state variables.
Apply increments to state variables weighted by correlation values.
Apply increments to any remaining unassimilated observations.
Loop until all observations in window processed.
If requested, compute and write the “postassim” netcdf diagnostic files (members, mean, spread). This is BEFORE any posterior inflation has been applied.
Apply posterior inflation if requested.
Compute ensemble of posterior observation values with forward operators.
Compute posterior observation space diagnostics.
If requested, compute and write out the “output” netcdf diagnostic files (members, mean, spread). This is AFTER any posterior inflation has been applied.
Loop until all observations in input file processed.
Close diagnostic files.
Write out final observation sequence file.
Write out inflation restart files if requested.
Write out final state vectors to model restart files if requested.
Release memory for state vector and observation ensemble members.
Namelist
See the filter namelist page for a detailed description of all
&filter_nml
variables. This namelist is read from the file input.nml
.
Modules used
mpi_utilities_mod
filter_mod
Note that filter_mod.f90 uses many more modules.
Files
See Detailed Program Flow for a short description of DART’s new ‘stages’. In addition, the Manhattan release simplifies some namelists by replacing many user-settable file names with hardwired filenames. Files can then be renamed in the run scripts to suit the user’s needs.
input ensemble member states; from &filter_nml :: input_state_files or input_state_file_list
output ensemble member states; to &filter_nml :: output_state_files or output_state_file_list
input observation sequence file; from
&filter_nml :: obs_sequence_in_name
output observation sequence file; from
&filter_nml :: obs_sequence_out_name
output state space diagnostics files;
${stage}_mean.nc, ${stage}_sd.nc,
where stage = {input,forecast,preassim,postassim,analysis,output}input state space inflation data (if enabled); from
input_{prior,post}inf_{mean,sd}.nc.
output state space inflation data (if enabled); to
${stage}_{prior,post}inf_{mean,sd}.nc.
, where stage ≠ “input”input.nml, to read &filter_nml
References
Anderson, J. L., 2001: An Ensemble Adjustment Kalman Filter for Data Assimilation. Mon. Wea. Rev., 129, 2884-2903. doi: 10.1175/1520-0493(2001)129<2884:AEAKFF>2.0.CO;2
Anderson, J. L., 2003: A Local Least Squares Framework for Ensemble Filtering. Mon. Wea. Rev., 131, 634-642. doi: 10.1175/1520-0493(2003)131<0634:ALLSFF>2.0.CO;2
Anderson, J. L., 2007: An adaptive covariance inflation error correction algorithm for ensemble filters. Tellus A, 59, 210-224. doi: 10.1111/j.1600-0870.2006.00216.x
Anderson, J. L., 2007: Exploring the need for localization in ensemble data assimilation using a hierarchical ensemble filter. Physica D, 230, 99-111. doi:10.1016/j.physd.2006.02.011
Anderson, J., Collins, N., 2007: Scalable Implementations of Ensemble Filter Algorithms for Data Assimilation. Journal of Atmospheric and Oceanic Technology, 24, 1452-1463. doi: 10.1175/JTECH2049.1
Anderson, J. L., 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus A, 61, 72-83. doi: 10.1111/j.1600-0870.2008.00361.x
Anderson, J., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Arellano, 2009: The Data Assimilation Research Testbed: A Community Facility. Bull. Amer. Meteor. Soc., 90, 1283-1296. doi: 10.1175/2009BAMS2618.1
Anderson, J. L., 2010: A Non-Gaussian Ensemble Filter Update for Data Assimilation. Mon. Wea. Rev., 139, 4186-4198. doi: 10.1175/2010MWR3253.1
Anderson, J. L., 2011: Localization and Sampling Error Correction in Ensemble Kalman Filter Data Assimilation. Submitted for publication, Jan 2011. Contact author.