Welcome to the Data Assimilation Research Testbed
The Data Assimilation Research Testbed (DART) is an open-source, freely available community facility for ensemble data assimilation (DA).  DART is developed and maintained by the Data Assimilation Research Section (DAReS) at the NSF National Center for Atmospheric Research (NSF NCAR).
Ensemble Data Assimilation
Ensemble DA is a technique for combining observations with numerical models to estimate the state of a physical system.
It enables modelers, observational scientists, and geophysicists to:
Generate initial conditions for forecasts.
Create a retrospective estimate of the state of a system, a practice known as producing a reanalysis.
Assess the relative value of specific observations on forecast skill, a practice known as conducting an observing system experiment (OSE).
Estimate the value of hypothetical observations in order to inform the design of an observing system, a practice known as conducting an observing system simulation experiment (OSSE).
Determine a model’s systematic bias in estimating the state of a system, a practice known as diagnosing model error.
The DART software environment makes it easy to explore a variety of data assimilation methods and observations with different numerical models. It provides powerful, flexible DA tools that are easy to use and customize to support efficient and reliable DA applications. While DART is primarily oriented for DA research, it has also been used in operational settings.
A comprehensive tutorial introducing the concepts of ensemble DA.
Extensive documentation of its source code.
Interfaces to a variety of models and observation sets that can be used to introduce new users or graduate students to ensemble DA.
Nonlinear and Non-Gaussian DA Capabilities
DART is also designed to facilitate the combination of assimilation algorithms, models, and real or synthetic observations to allow increased understanding of all three. It provides a framework for developing, testing, and distributing advances in ensemble DA to a broad community of users by removing the implementation-specific peculiarities of one-off DA systems.
These tools are intended for use by the full range of geosciencies community: beginners and experts; students and teachers; national centers and university research labs.
Nonlinear and Non-Gaussian Data Assimilation Capabilities in DART
One of the historical drawbacks of ensemble data assimilation techniques is the assumption that the quantities being assimilated obey a normal distribution. While this is often a safe assumption – distributions of temperature and pressure can be approximated using a normal distribution – many quantities such as precipitation, snow depth and tracer concentration, as well as many model parameters aren’t normally distributed.
Applying traditional ensemble data assimilation techniques in situations where assumptions of gaussianity are invalid can lead to poor forecast skill and inconclusive results.
To overcome these problems, DART now implements a novel data assimilation technique that no longer requires quantities to be normally distributed. The Quantile-Conserving Ensemble Filtering Framework (QCEFF) provides a general method of computing increments for the prior ensemble of an observed quantity by allowing the use of arbitrary distributions for the prior and the observation error. For a detailed description of the QCEFF, see Anderson (2022). 
While the QCEFF for computing observation increments can lead to significant improvements in analysis estimates for observed variables, those improvements can be lost when using standard linear regression of observation increments to update state variables. The QCEFF also implements a capability to do regression in a probit probability integral transformed space. Doing the regression of observation quantile increments in the transformed space guarantees that the posterior ensembles for state variables also retain the advantages of the observation space quantile conserving posteriors. For example, if state variables are bounded, then posterior ensembles will respect the bounds. The posterior ensembles also respect other aspects of the continuous prior distributions. For a detailed description of this process, see Anderson (2023)  and Anderson et al. (2023). 
Inflation and localization, methods that improve the quality of ensemble DA, can also negate the advantages of the QCEFF methods. For this reason, both localization and inflation can be done in the probit-transformed quantile space as well. Combining these new methods can significantly improve data assimilation for non-Gaussian quantities in Earth system models by extending the capabilities of ensemble DA to general non-Gaussian and nonlinear distributions. Transformative improvements in DA can result for many applications. The largest improvements are found for bounded variables like tracer concentrations, snow and ice depth, soil moisture, and similar quantities in other parts of the Earth system. Model parameters can also be estimated with DA and large improvements can occur for bounded parameters. Variables that have distinctly non-Gaussian prior distributions can also see large improvements. Examples can include atmospheric quantities like moisture and cloud amount in the presence of convection, and many land surface variables.
For instructions on how to use these tools, see Quantile-Conserving Ensemble Filter Framework.
For step-by-step examples of the QCEFF tools, you can work through examples with the Lorenz 96 tracer model
Organization of the documentation
Because of DART’s extensive scope, this documentation is detailed and carefully organized, enabling you to easily find the information you need. If you have any questions or suggestions for improvements, please contact DAReS staff by emailing firstname.lastname@example.org.
The documentation is partitioned into three parts:
a user guide that explains how to install DART and perform data assimilation
source code documentation that provides a detailed description of the programs and modules in the repository
a comprehensive description of data assimilation theory
The current version of DART is the Manhattan release. Email email@example.com for advice if you are interested in a model which has not been converted from the previous Lanai release.
DART is available through GitHub. To download the latest version of DART, use:
git clone https://github.com/NCAR/DART.git
Go into the
build_templates directory and copy over the closest
mkmf.template._compiler.system_ file into
Edit it to set the NETCDF directory location if not in
/usr/local or comment
it out and set $NETCDF in your environment. This NetCDF library must have been
compiled with the same compiler that you use to compile DART and must include
the F90 interfaces.
models/lorenz_63/work and run quickbuild.sh nompi.
$ cd models/lorenz_63/work
$ ./quickbuild.sh nompi
If it compiles, run this series of commands to do a very basic test:
If that runs and you have Matlab installed on your system add
DART/diagnostics/matlab to your matlab search path and run the
plot_total_err diagnostic script while in the
directory. If the output plots and looks reasonable (error level stays around 2
and doesn’t grow unbounded) you have successfully installed DART and completed
your first assimilation with it.
If you are planning to run one of the larger models and want to use the Lorenz
63 model as a test, run
./quickbuild.sh. It will build filter and any
other MPI-capable executables with MPI.
./quickbuild.sh help will print out the quickbuild.sh usage.
mpif90 command you use must have been built with the same version of
the compiler as you are using.
If any of these steps fail or you don’t know how to do them, go to the DART project web page listed above for very detailed instructions that should get you over any bumps in the process.
Quick-start for developers
To create a fork of DART for your own development you will need a GitHub account.
fork the NCAR/DART repo on GitHub
clone your (new) fork to your machine - this will set up a remote named ‘origin’.
git clone https://github.com/USERNAME/DART.git
where USERNAME is your GitHub username.
create a remote to point back to the NCAR/DART repo. Convention dictates that this remote should be called ‘upstream’
git remote add upstream https://github.com/NCAR/DART.git
Use ‘upstream’ to keep your fork up to date with NCAR/DART. GitHub has documentation on working with forks.
Download one of the tar files (listed below) of ‘large’ files so you can test your DART installation.
If you want to contribute your work back to the DART community, create a feature branch with your work, then issue a pull request to propose changes to NCAR/DART.
There are several large files that are needed to run some of the tests and examples but are not included in order to keep the repository as small as possible. If you are interested in running bgrid_solo, cam-fv, or testing the NCEP/prep_bufr observation converter, you will need these files. These files are available at:
Download the appropriate tar file and untar it into your DART repository. Ignore
any warnings about
tar: Ignoring unknown extended header keyword.
Cite DART using the following text:
The Data Assimilation Research Testbed (Version X.Y.Z) [Software]. (2019). Boulder, Colorado: UCAR/NCAR/CISL/DAReS. http://doi.org/10.5065/D6WQ0202
Update the DART version and year as appropriate.
- Introduction to ensemble data assimilation
- The Lorenz 63 model and its relevance to data assimilation
- Data assimilation in DART using the Lorenz 63 model
- What is DART?
- The benefits of using DART
- A brief history of DART
- High-level data assimilation workflows in DART
- DART’s design philosophy
- Important capabilities of DART
- Quantile-Conserving Ensemble Filter Framework
- Can I run my model with DART?
- Instructions for porting a new model to DART
- DART build system
- Assimilation in a complex model
- Message Passing Interface
- Required model_mod routines
- Suggestions for a “simple” model
- Suggestions for a “complex” model
- How to test your model_mod routines
- Controlling which files are output by filter
- Advice for models with multiple vertical coordinate options
- Data management in DART
- Programs included with DART
- Adding your observations to DART
- How DART supports different types of observations: the preprocess program
- How DART stores observations: observation sequence (obs_seq) files
- Detailed structure of an obs_seq file
- Creating an obs_seq file of synthetic observations
- Creating an obs_seq file from real observations
- Available observation converter programs
- Manipulating obs_seq files with the obs_sequence_tool
- The difference between observation TYPE and QUANTITY
- Adding support for a new observation TYPE
- Checking your initial assimilation
- Computing filter increments
- Computing filter increments using a complex model
- DART missing data value
- DART quality control field
- Examining the obs_seq.final file
- MATLAB observation space diagnostics
- DART Tutorial
- Conditional probability and Bayes’ theorem
- DART_LAB Tutorial
- CLM-DART Tutorial
- Tutorial Overview
- Step 1: Download CLM5
- Adding CLM5 SourceMods
- Compiling CLM5
- Step 2: Download DART
- Step 3: Navigating DART Scripts
- Step 4: Compiling DART
- Step 5: Setting up the atmospheric forcing
- Step 6: Setting up the initial conditions for land earth system properties
- Step 7: Setting up the observations to be assimilated
- Step 8: Setting up the DART and CLM states
- Step 9: Set the spatial localization
- Step 10: Set the Inflation
- Step 11: Complete the Assimilation Setup
- Step 12: Execute the Assimilation Run
- Step 13: Diagnose the Assimilation Run
- Matlab Diagnostics
- WRF-DART Tutorial
- Step 1: Setup
- Step 2: Initial conditions
- Step 3: Prepare observations [OPTIONAL]
- Step 4: Creating the first set of adaptive inflation files
- Step 5: Cycled analysis system
- Step 6: Diagnosing the assimilation results
- Visualizing the observation locations and acceptance rate
- Quantification of model-observation mismatch and ensemble spread
- Generating the obs_diag_output.nc and obs_epoch*.nc files manually [OPTIONAL]
- Generating the obs_epoch*.nc file
- Generating the obs_diag_output.nc file
- Additional materials from previous in-person tutorials
- More Resources
- QCEFF: Examples with the Lorenz 96 Tracer Model
- DART Manhattan Differences from Lanai Release Notes
- Forward Operators
- Approaches for Common Situations
- Parallelism Implementation Details
- Other Parallelism Options
- Netcdf Inflation Files
- State Stucture
- Filter async modes
- Distributed State