MODULE filter_mod¶
Overview¶
Main module for driving ensemble filter assimilations. Used by filter.f90, perfect_model_obs.f90, model_mod_check.f90, and a variety of test programs. See the PROGRAM filter for a general description of filter capabilities and controls.
filter_mod
is a Fortran 90 module, and provides a large number of options for controlling execution behavior and
parameter configuration that are driven from its namelist. See the namelist section below for more details. The number
of assimilation steps to be done is controlled by the input observation sequence and by the timestepping capabilities
of the model being used in the assimilation.
See Welcome to the Data Assimilation Research Testbed for more documentation, including a discussion of the capabilities of the assimilation system, a diagram of the entire execution cycle, the options and features.
Namelist¶
This namelist is read from the file input.nml
. Namelists start with an ampersand ‘&’ and terminate with a slash ‘/’.
Character strings that contain a ‘/’ must be enclosed in quotes to prevent them from prematurely terminating the
namelist.
&filter_nml
single_file_in = .false.,
input_state_files = '',
input_state_file_list = '',
init_time_days = 0,
init_time_seconds = 0,
perturb_from_single_instance = .false.,
perturbation_amplitude = 0.2,
stages_to_write = 'output'
single_file_out = .false.,
output_state_files = '',
output_state_file_list = '',
output_interval = 1,
output_members = .true.,
num_output_state_members = 0,
output_mean = .true.,
output_sd = .true.,
write_all_stages_at_end = .false.,
compute_posterior = .true.
ens_size = 20,
num_groups = 1,
distributed_state = .true.,
async = 0,
adv_ens_command = "./advance_model.csh",
tasks_per_model_advance = 1,
obs_sequence_in_name = "obs_seq.out",
obs_sequence_out_name = "obs_seq.final",
num_output_obs_members = 0,
first_obs_days = 1,
first_obs_seconds = 1,
last_obs_days = 1,
last_obs_seconds = 1,
obs_window_days = 1,
obs_window_seconds = 1,
inf_flavor = 0, 0,
inf_initial_from_restart = .false., .false.,
inf_sd_initial_from_restart = .false., .false.,
inf_deterministic = .true., .true.,
inf_initial = 1.0, 1.0,
inf_lower_bound = 1.0, 1.0,
inf_upper_bound = 1000000.0, 1000000.0,
inf_damping = 1.0, 1.0,
inf_sd_initial = 0.0, 0.0,
inf_sd_lower_bound = 0.0, 0.0,
inf_sd_max_change = 1.05, 1.05,
trace_execution = .false.,
output_timestamps = .false.,
output_forward_op_errors = .false.,
write_obs_every_cycle = .false.,
allow_missing_clm = .false.,
silence = .false.,
/
Particular options to be aware of are: ens_size, cutoff (localization radius), inflation flavor, outlier_threshold, input and output state filenames, obs_sequence_in_name, horiz_dist_only, and the binary or ascii controls for observation sequence file formats. Some of these important items are located in other namelists, but all are in the same input.nml file.
The inflation control variables are all dimensioned 2, the first value controls the prior inflation and the second controls the posterior inflation.
Item 
Type 
Description 

single_file_in 
logical 

input_state_files 
character(len=256), dimension(MAXFILES) 
A list of the NetCDF files to open to read the state vectors. Models using multiple domains must put the domain and ensemble numbers in the file names. The order and format of those is to be determined. NOT SUPPORTED as of March, 2017. 
input_state_file_list 
character(len=256), dimension(MAXFILES) 
A list of files, one per domain. Each file must be a text file containing the names of the NetCDF files to open, one per ensemble member, one per line. 
init_time_days 
integer 
If negative, use the initial days read from the state data restart file. If positive, override the initial days read from state data restart files. Days since 1 Jan 1601. 
init_time_seconds 
integer 
If negative use the initial seconds read from the state data restart file. If positive, override the initial seconds read from state data restart files. Seconds since midnight. 
perturb_from_single_instance 
logical 

perturbation_amplitude 
real(r8) 
Standard deviation for the noise model
used when generating ensemble members.
This value is available to the model_mod
for use in the required interface

stages_to_write 
character(len=10), dimension(6) 
Controls diagnostic and restart output. Valid values are: ‘input’, ‘forecast’, ‘preassim’, ‘postassim’, ‘analysis’, ‘output’, and ‘null’. Input is caseinsensitive. 
single_file_out 
logical 

output_state_files 
character(len=256), dimension(MAXFILES) 
A list of the netCDF files to open for writing updated state vectors. Not supported when using multiple domains. 
output_state_file_list 
character(len=256), 
A list of files, one per domain. Each file must be a text file containing the names of the netCDF files to open, one per ensemble member, one per line. 
output_interval 
integer 
Output state and observation diagnostics every ‘N’th assimilation time, N is output_interval. 
output_members 
logical 

num_output_state_members 
integer 
Number of ensemble members to be included
in the state diagnostic output for stages
‘forecast’, ‘preassim’, ‘postassim’ and
‘analysis’. 
output_mean 
logical 

output_sd 
logical 

write_all_stages_at_end 
logical 
For most cases this should be

compute_posterior 
logical 
If 
ens_size 
integer 
Size of ensemble. 
num_groups 
integer 
Number of groups for hierarchical filter. It should evenly divide ens_size. 
distributed_state 
logical 

async 
integer 
Controls method for advancing model:
Ignored if filter is not controlling the model advance, e.g. in CESM, WRF, etc 
adv_ens_command 
character(len=256) 
Command sent to shell if async is 2. 
tasks_per_model_advance 
integer 
Number of tasks to assign to each ensemble member advance. 
obs_sequence_in_name 
character(len=256) 
File name from which to read an observation sequence. 
obs_sequence_out_name 
character(len=256) 
File name to which to write output observation sequence. 
num_output_obs_members 
integer 
Number of ensemble members to be included in the output observation sequence file. 
first_obs_days 
integer 
If negative, don’t use. If nonnegative, ignore all observations before this time. 
first_obs_seconds 
integer 
If negative, don’t use. If nonnegative, ignore all observations before this time. 
last_obs_days 
integer 
If negative, don’t use. If nonnegative, ignore all observations after this time. 
last_obs_seconds 
integer 
If negative, don’t use. If nonnegative, ignore all observations after this time. 
obs_window_days 
integer 
Assimilation window days; defaults to model timestep size. 
obs_window_seconds 
integer 
Assimilation window seconds; defaults to model timestep size. 
All variables named inf_* are arrays of length 2. The first element controls the prior, the second element controls the posterior inflation. See PROGRAM filter for a discussion of inflation and effective strategies. 

inf_flavor 
character(len=32), dimension(2) 
Inflation flavor [prior, posterior] see Inflation Options below. 
inf_initial_from_restart 
logical, dimension(2) 
If 
inf_sd_initial_from_restart 
logical, dimension(2) 
If 
inf_deterministic 
logical, dimension(2) 

inf_initial 
real(r8), dimension(2) 
Initial value of inflation if not read from restart file. 
inf_lower_bound 
real(r8), dimension(2) 
Lower bound for inflation value. 
inf_upper_bound 
real(r8), dimension(2) 
Upper bound for inflation value. 
inf_damping 
real(r8), dimension(2) 
Damping factor for inflation mean values. The difference between the current inflation value and 1.0 is multiplied by this factor and added to 1.0 to provide the next inflation mean. The value should be between 0.0 and 1.0. Setting a value of 0.0 is full damping, which in fact turns off all inflation by fixing the inflation value at 1.0. A value of 1.0 turns inflation damping off leaving the original inflation value unchanged. 
inf_sd_initial 
real(r8) dimension(2) 
Initial value of inflation standard deviation if not read from restart file. If ≤ 0, do not update the inflation values, so they are timeconstant. If positive, the inflation values will adapt through time. 
inf_sd_lower_bound 
real(r8), dimension(2) 
Lower bound for inflation standard deviation. If using a negative value for inf_sd_initial this should also be negative to preserve the setting. 
inf_sd_max_change 
real(r8), dimension(2) 
For 
trace_execution 
logical 

output_timestamps 
logical 

output_forward_op_errors 
logical 

write_obs_every_cycle 
logical 
For debug use; this option can significantly slow the execution of filter. True means to write the entire output observation sequence diagnostic file each time through the main filter loop even though only observations with times up to and including the current model time will have been assimilated. Unassimilated observations have the value 888888.0 (the DART “missing value”). If filter crashes before finishing it may help to see the forward operator values of observations that have been assimilated so far. 
allow_missing_clm 
logical 
Some models are allowed to have MISSING_R8
values in the DART state. If 
silence 
logical 

Inflation Options¶
The value for the inf_flavor
is a character string. For backwards compatiblity
(it was an integer code), the specification of the integer is still supported.
Inflation values (for flavors other than 0) will be timevarying
only if inf_sd_initial
> 0.
inflation option 
description 

0
‘0’
‘NO_INFLATION’

no inflation 
2
‘2’
‘VARYING_SS_INFLATION’

spatiallyvarying statespace (gaussian) 
3
‘3’
‘SINGLE_SS_INFLATION’

spatiallyfixed statespace (gaussian) 
4
‘4’
‘RELAXATION_TO_PRIOR_SPREAD’
‘RTPS

Relaxation To Prior Spread (Posterior inflation only) 
5
‘5’
‘ENHANCED_SS_INFLATION’

Enhanced spatiallyvarying statespace (inverse gamma).
Refer to 
Create an initial ensemble from a single file¶
If the default pert_model_copies
routine is used, random noise values drawn from a
gaussian distribution with the standard deviation specified by perturbation_amplitude
will be added to the data in a single
initial ensemble member to generate the rest of the members. This option is more frequently
used in the low order models and less frequently used in large models. This is in part due
to the different scales of real geophysical variable values, and the resulting inconsistencies
between related field values. A more successful initial condition generation strategy is to
generate climatological distributions from long model runs which have internally consistent
structures and values and then use observations with a ‘spinup’ period of assimilation to
shape the initial states into a set of members with enough spread and which match the current
set of observations. Each model_mod is required to provide a pert_model_copies routine
which can be used to either passthrough to the default routine or can be customized for
that specific model.
Modules used¶
types_mod
obs_sequence_mod
obs_def_mod
obs_def_utilities_mod
time_manager_mod
utilities_mod
assim_model_mod
assim_tools_mod
obs_model_mod
ensemble_manager_mod
adaptive_inflate_mod
mpi_utilities_mod
smoother_mod
random_seq_mod
state_vector_io_mod
io_filenames_mod
forward_operator_mod
quality_control_mod
Files¶
See the filter overview for the list of files.
Error codes and conditions¶
Routine 
Message 
Comment 

filter_main 
ens_size in namelist is ###: Must be > 1 
Ensemble size must be at least 2. 
filter_main 
inf_flavor= ### Must be 0, 2, 3. 
Observation Inflation is no longer supported (i.e flavor 1). 
filter_main 
Posterior observation space inflation (type 1) not supported. 
Posterior observation space inflation doesn’t work. 
filter_main 
Number of processes > model size. 
Number of processes can’t exceed model size for now. 
filter_generate_copy_meta_data 
output metadata in filter needs state ensemble size < 10000, not ###. 
Only up to 10000 ensemble members with state output for now. 
filter_generate_copy_meta_data 
output metadata in filter needs obs ensemble size < 10000, not ###. 
Only up to 10000 ensemble members with obs space output for now. 
filter_setup_obs_sequence 
input obs_seq file has ### qc fields; must be < 2. 
Only 0 or 1 qc fields in input obs sequence for now. 
get_obs_copy_index 
Did not find observation copy with metadata observation. 
Only 0 or 1 qc fields in input obs sequence for now. 