PROGRAM COSMOS_development
Overview
Trial COSMOS text file to DART converter
COSMOS_development
reads Rafaels data streams and converts them to DART observation
sequence files. Since these data streams are not widespread, we recommend using PROGRAM COSMOS_to_obs.get the site metadata and enter it in the
input.nml
&COSMOS_development_nmlacquire the development observation data and prefix the filename with the station name (or else they all get named
corcounts.txt
) and enter the filename into &COSMOS_development_nmlmake sure the station soil parameters and COSMIC parameters are contained in the
observations/COSMOS/data/COSMIC_parlist.nc
(more on this in the section on COSMIC parameters)run
COSMOS_development
to generate a DART observation sequence file for the station and rename the output file if necessary (you can explicity name the output file via the namelist).repeat steps 1-4 for this converter to generate a DART observation sequence file for each station.
use the program obs_sequence_tool to combine the observations from multiple sites
Data sources
The COSMOS data portal can be found at: http://cosmos.hwr.arizona.edu/Probes/probemap.php The development observation data for each station is generally not available. The metadata for each station (location, height, etc) is also available from the data portal. The Level 2 Data is most suited for use with DART. We recommend using PROGRAM COSMOS_to_obs. An example of the development observation data follows:
month,day,hour,doy,neutron_fluxAVE,neutron_fluxSTD,neutron_fluxQC
1, 1, 0, 1,-9999,9999,3
1, 1, 1, 1,-9999,9999,3
1, 1, 2, 1,-9999,9999,3
1, 1, 3, 1,-9999,9999,3
...
Programs
The COSMOS_development.f90
file is the source code for the main converter program. At present there is an
uncomfortable assumption that the order of the columns in the Level 2 data is fixed. I hope to relax that requirement in
the near future. COSMOS_development
reads each text line into a character buffer and then reads from that buffer to
parse up the data items. The items are then combined with the COSMIC parameters for that site and written to a
DART-format observation sequence file. The DART format allows for the additional COSMIC parameters to be contained as
metadata for each observation.
To compile and test, go into the COSMOS/work
subdirectory and run the quickbuild.sh
script to build the
converter and a couple of general purpose utilities. The
program obs_sequence_tool manipulates (i.e. combines, subsets) DART
observation files once they have been created. The default observations supported are those defined in
DART/observations/forward_operators/obs_def_land_mod.f90
and
DART/observations/forward_operators/obs_def_COSMOS_mod.f90
. If you need
additional observation types, you will have to add the appropriate obs_def_XXX_mod.f90
file to the input.nml
&preprocess_nml:input_files
variable and run quickbuild.sh
again. It rebuilds the table of supported
observation types before compiling the source code.
COSMIC parameters
COSMOS_development
reads the site name from the input namelist and the known station information from
COSMIC_parlist.nc
. The simplest way to add a new station to COSMIC_parlist.nc
is probably to:manually enter the information into the “data” section of
COSMIC_parlist_station.txt
then use
ncgen
to convertCOSMIC_parlist_station.txt
to a netCDF file.That netCDF file can be concatenated onto
COSMIC_parlist.nc
with a simplencrcat
command.
Listing the sites already supported is easy:
observations/COSMOS/data % ncdump -v sitenames COSMIC_parlist.nc
netcdf COSMIC_parlist {
dimensions:
nsites = UNLIMITED ; // (42 currently)
strlength = 21 ;
variables:
char sitenames(nsites, strlength) ;
sitenames:long_name = "COSMOS Site Names" ;
double longitude(nsites) ;
longitude:long_name = "Longitude" ;
longitude:units = "degrees" ;
double latitude(nsites) ;
latitude:long_name = "Latitude" ;
latitude:units = "degrees" ;
double elevation(nsites) ;
elevation:long_name = "Elevation" ;
elevation:units = "m" ;
double bd(nsites) ;
bd:long_name = "Dry Soil Bulk Density" ;
bd:units = "g cm{-3}" ;
double lattwat(nsites) ;
lattwat:long_name = "Lattice Water Content" ;
lattwat:units = "m{3} m{-3}" ;
double N(nsites) ;
N:long_name = "High Energy Neutron Intensity" ;
N:units = "relative counts" ;
double alpha(nsites) ;
alpha:long_name = "Ratio of Fast Neutron Creation Factor (Soil to Water)" ;
alpha:units = "-" ;
double L1(nsites) ;
L1:long_name = "High Energy Soil Attenuation Length" ;
L1:units = "g cm{-2}" ;
double L2(nsites) ;
L2:long_name = "High Energy Water Attenuation Length" ;
L2:units = "g cm{-2}" ;
double L3(nsites) ;
L3:long_name = "Fast Neutron Soil Attenuation Length" ;
L3:units = "g cm{-2}" ;
double L4(nsites) ;
L4:long_name = "Fast Neutron Water Attenuation Length" ;
L4:units = "g cm{-2}" ;
// global attributes:
:website = "COsmic-ray Soil Moisture Observing System (COSMOS) -
http://cosmos.hwr.arizona.edu" ;
data:
sitenames =
"ARM-1 ",
"Austin_Cary ",
"Bondville ",
"Brookings ",
"Chestnut_Ridge_NOAA ",
"Coastal_Sage_UCI ",
"Daniel_Forest ",
"Desert_Chaparral_UCI ",
"Fort_Peck ",
"Harvard_Forest ",
"Hauser_Farm_North ",
"Hauser_Farm_South ",
"Howland ",
"Iowa_Validation_Site ",
"Island_Dairy ",
"JERC ",
"Kendall ",
"KLEE ",
"Manitou_Forest_Ground",
"Metolius ",
"Morgan_Monroe ",
"Mozark ",
"Mpala_North ",
"Neb_Field_3 ",
"P301 ",
"Park_Falls ",
"Pe-de-Gigante ",
"Rancho_No_Tengo ",
"Reynolds_Creek ",
"Rietholzbach ",
"Rosemount ",
"San_Pedro_2 ",
"Santa_Rita_Creosote ",
"Savannah_River ",
"Silver_Sword ",
"SMAP-OK ",
"Soaproot ",
"Sterling ",
"Tonzi_Ranch ",
"UMBS ",
"UVA ",
"Wind_River " ;
}
The observation sequence files will look something like the following, the attributes in yellow are the information
from COSMIC_parlist.nc
:
obs_sequence
obs_kind_definitions
1
20 COSMOS_NEUTRON_INTENSITY
num_copies: 1 num_qc: 1
num_obs: 3840 max_num_obs: 3840
observation
COSMOS QC
first: 1 last: 3840
OBS 1
1048.0000000000000
1.0000000000000000
-1 2 -1
obdef
loc3d
4.154723123116714 0.7997185899100618 0.000000000000000 -1
kind
20
cosmic 0.88500000000000001 5.84099999999999966E-002 336.95696938999998 0.31918025877000000 161.98621864285701 129.14558984999999 55.311849408000000 3.8086191933000002 1
77340 150034
1225.0000000000000
...
Namelist
This namelist is read from the file input.nml
. Namelists start with an ampersand ‘&’ and terminate with a slash ‘/’.
Character strings that contain a ‘/’ must be enclosed in quotes to prevent them from prematurely terminating the
namelist.
&COSMOS_development_nml
site_metadata_file = 'COSMIC_parlist.nc'
text_input_file = 'textdata.input',
obs_out_file = 'obs_seq.out',
sitename = 'missing',
year = -1
maxgoodqc = 3,
verbose = .false.
/
Contents |
Type |
Description |
---|---|---|
site_metadata_file |
character(len=256) |
The netCDF file containing the parameter values for each site. |
text_input_file |
character(len=128) |
The text file containing the raw observations for each site. |
obs_out_file |
character(len=128) |
The output observation sequence file for DART. |
sitename |
character(len=128) |
The name of the site. Must match one of the site names in the
|
year |
integer |
The year of the data. |
maxgoodqc |
integer |
left for future implementation. |
verbose |
logical |
A switch to specify the amount of run-time output. |
COSMOS development namelist
&COSMOS_development_nml
site_metadata_file = '../data/COSMIC_parlist.nc',
text_input_file = 'SantaRita_corcounts.txt',
obs_out_file = 'SantaRita_obs_seq.out',
sitename = 'Santa_Rita_Creosote',
References
Franz, T.E, M. Zreda, T.P.A. Ferre, R. Rosolem, C. Zweck, S. Stillman, X. Zeng and W.J. Shuttleworth, 2012: Measurement depth of the cosmic-ray soil moisture probe affected by hydrogen from various sources. Water Resources Research 48, W08515, doi:10.1029/2012WR011871
Franz, T.E, M. Zreda, R. Rosolem, T.P.A. Ferre, 2012: Field validation of cosmic-ray soil moisture probe using a distributed sensor network. Vadose Zone Journal (in press), doi:10.2136/vzj2012.0046
Future Plans
Implement a routine to automatically determine the column indices of the columns of interest.
Implement a QC encoding that reflects the uncertainty of the measurement. Presently, all Level 2 data have an incoming QC of 1.