EBAS Configuration Directory

This directory contains the files controlling NASA-Ames data generated for export to EBAS by data.aggregate.ebas. The station specific instance is located at $DB/etc/$STATION/ebas, while the defaults are located at $DB/etc/ebas. Files within the station specific directory take the form of $TYPE.$STATION.conf, while those in the default location are of the form $TYPE.conf.

The general structure of the system is a series of file defining configuration values (over a time range) that are overlaid on top of one another. The default files is loaded first, followed by the station specific one, so any values set in the station specific one are used if they exist for a time, otherwise the default one is used.

data.aggregate.ebas first loads the type “global” then the files corresponding to the “record” it was called with, so calling it with “neph_L0_cut” would cause the global file “neph_L0_cut.conf” then the station specific “neph_L0_cut.$STATION.conf” to load. So the final loading sequence in this example would be:

  1. $DB/etc/ebas/global.conf
  2. $DB/etc/$STATION/ebas/global.$STATION.conf
  3. $DB/etc/ebas/neph_L0_cut.conf
  4. $DB/etc/$STATION/ebas/neph_L0_cut.$STATION.conf

Each of those files may trigger additional loading of other types, which follow the same default then station specific pattern.

The full specifications that EBAS requires are defined at:

Conventional Loading Sequence

The required loading sequence is as defined above, however most already defined instrument/record types use additional conventions. The standard overlay sequence (items further down the list take precedence) for the neph looks like:

  1. $DB/etc/ebas/global.conf - This contains global defaults, not specific to any instrument or station (e.x. default contact information).
  2. $DB/etc/$STATION/ebas/global.$STATION.conf - This contains station specific defaults not specific to an instrument (e.x. station location and contact information).
  3. $DB/etc/ebas/neph_L0_3w.conf - The definition of the contents of a 3-wavelength neph at level 0. This contains the actual variable definitions.
  4. $DB/etc/ebas/neph_L0.conf - The global default for neph_L0, which is to load the L0 definition of a 3-wavelength neph.
  5. $DB/etc/$STATION/ebas/neph_L0.$STATION.conf - This is often empty (that is, no effect), but when present it is used to define the times that the “contents” of an instrument where changed (e.x. it would be where the times a neph was changed from 4-wavelength to 3-wavelength).
  6. $DB/etc/ebas/cutsize.conf - Enables cutsize splitting.
  7. $DB/etc/ebas/neph_L0_cut.conf - All this does is trigger the loading of the neph_L0 and cutsize types.

Note that usually $DB/etc/$STATION/bas/neph_L0_cut.$STATION.conf is empty, as the global default is accurate (but that global default triggers the loading of the station specific neph_L0, which is used to set the times the neph changed).

Simple Example

The detailed format of the files is defined below. A common task for a station is to set the location and contact information. This is usually done by changing the station specific “global” type in $DB/etc/$STATION/ebas/global.$STATION.conf. For example, BND's global.conf looks like:

,,BLOCK
    Laboratory,US06L
    Station,US0035R
    Platform,US0035S
    StationWDCAID,GAWAUSILBND
    StationWDCAName,"Bondville, Illinois"
    StationState,Illinois
    Latitude,40.050
    Longitude,-88.367
    Altitude,213m
    LandUse,Agricultural
    StationSetting,Rural
    GAWType,R
    WMORegion,4

    InletHeightAGL,10m
    InletType,Hat or hood
END

What this specifies is a “block” of applied values, that apply to all time. That is, the first line defines a start and end time (both empty which is what the “,,” part is from) that are infinite, all values until the “END” are applied to that time range; so all those values are applied to all time. Each line defines a comma separated list, with the first value (leading whitespace is ignored) being the name of the value being set, and subsequent items are the value (most types only take one, but there are a few that accept multiple). The exact meaning of each line is defined below, but most are self explanatory. Take care that a few have a limited set of values (for example “StationSetting” and “WMORegion”). Others must follow certain conventions to be within the WDCA requirements, even if those are not strictly enforced (e.x. “Station” must be as supplied by EBAS).

In this case, there is no contact information present, since the global default (in $DB/etc/ebas/global.conf) is sufficient. A portion of that file looks like:

,,BLOCK
    OriginatorName,Ogren,John
    OriginatorEMail,John.A.Ogren@noaa.gov
    PrincipalInvestigatorName,Ogren,John
    PrincipalInvestigatorEMail,John.A.Ogren@noaa.gov

    DataSubmitterName,Ogren,John
    DataSubmitterEMail,John.A.Ogren@noaa.gov

    OrganizationName,"National Oceanic and Atmospheric Administration/Earth System Research Laboratory/Global Monitoring Division"
    OrganizationAcronym,"NOAA/ESRL/GMD"
    OrganizationUnit,
    OrganizationAddress1,"325 Broadway"
    OrganizationAddress2,
    OrganizationZIP,"80305"
    OrganizationTown,"Boulder, CO"
    OrganizationCountry,"USA"

    Projects,"GAW-WDCA NOAA-ESRL"
END

Which follows the same structure defined above, but this specified information that applies to all stations. Note that the various “…Name” fields take multiple arguments, in this case the last name followed by the first name.

The individual instrument configurations are more complex, but are generally sufficient to use as is with no direct changes. The only common change needed is to set the times a PSAP was transitioned between one and three wavelengths (this is needed because the system needs additional variable definitions on the 3-wavelength PSAP). The transition time is usually set by creating override files in $DB/etc/$STATION/ebas/psap_L[012].$STATION.conf, where “[012]” is one of the numbers 0, 1 or 2. All these files need to do is tell the system to overlay the correct PSAP loading definition at the correct times. For example, BND's psap_L0.bnd.conf looks like:

,2006:53.82,LOAD,psap_L0_1w
2006:53.82,,LOAD,psap_L0_3w

This tells the system to use the 1-wavelength definition up until 2006:53.82 (that is, the first time field is empty, so it is treated as undefined or infinite), then to use the 3-wavelength definition from that time onwards (the second time field is empty, so the end is undefined/infinite).

In the rare case when new variables or record types need to be defined, the overlay loading chain must eventually define a set of variable blocks. Each variable will be resolved to a column in the output file, and must define the needed parameters for that (see below). For example, the standard CPC level 0 concentration and source looks like:

,,GENERATOR,base,"data.get ${STATION} N61a,N61m,N71a,N71m ${START} ${END} raw"

,,VARIABLE,conc
    Source,N
    Format,%11.3f
    MVC,9999999.999
    Description,"integral aerosol particle number concentration at internal instrument conditions, 1/cm3"
    CheckCompleteness
END

As usual, the first two fields of the block name are start and end time, once again undefined so this applies to all time. The first line defines the “generator” or the program called to create the CPD2 data stream that the final output will be constructed from, in most cases this will only need to be changed to set the record(s) and data sources. The variable block begins with the standard infinite start and end, and has a unique ID of “conc”. This is an internal name used to allow further overlay by variable name. The Source key defined the variable source, this is resolved to everything that matches in the input data. In this case that would generally be N_N61 and/or N_N71. The format is the output format in printf(2) form. The MVC is the code written when data is unavailable, which should match the format. The Description is the description field in the output format, note that this is usually required to be a specific value. The “CheckCompleteness” line flags this variable as being used to determine the completeness flag in the output.

Another common task is to set the assumed temperature and pressure for the CPC stp correction. This is done by creating cpc_L1.$STATION.conf and cpc_L2.$STATION.conf files for the station and adding the following:

,,Comment,"STP-Nominal-T=25C,STP-Nominal-P=880hPa"
,,VARIABLE,conc
    ProcessingData,25,880
END

The cpc_L2 file needs that and additional lines for the statistics:

,,Comment,"STP-Nominal-T=25C,STP-Nominal-P=880hPa"
,,VARIABLE,conc
    ProcessingData,25,880
END
,,VARIABLE,16_perc
    ProcessingData,25,880
END
,,VARIABLE,84_perc
    ProcessingData,25,880
END

The temperature can also be specified in Kelvin. Since these variables are already defined to use the fixed input STP correction, all that needs to be done is set the input values for it. These values should be calculated base on the nominal station pressure and temperature, modified by the specific CPC model in use.

Format

Lines beginning with '#' are treated as comments. The format consists of a CSV separated list of key and value sets. There are two general structures in the file: blocks and single values. Both these take a start and end time in any convertible time format or may be left empty/“0” to specify unlimited. The start and end time are the first two fields, then the “type”. The type determines how the line is treated. If it is a block, then it may have additional arguments to that block, then any values withing that block (which ends with the line “End”) are applied for the time range of the block. If it is a single value then that value is applied for the given time range. The following types are recognized:

Definition Block

This specifies a set of definition keys that are applied for the entire time range of the block. Continues until the key “End” is received. See below for the available definition keys.

Variable Block

This specifies the definition of a variable in the output file. Variables are overlaid based on their internal name. So a variable with the internal name “BaG” will replace any others loaded before with that same name, even if they have different parameters. The order in the output data is the order in the file that they appear in. The following keys are valid:

In addition to the global substitutions for the keys mentioned above the following are also valid:

The following processing types are recognized:

neph_rayleigh

Calculate Rayleigh scattering for a neph zero. The processing data consists of a set of “key:value” pairs, one for each CSV argument to it. The following keys are recognized:

neph_zero

Calculate the neph zero scattering for a neph zero from the background scattering. The processing data consists of a set of “key:value” pairs, one for each CSV argument to it. The following keys are recognized:

fixed_stp

Adjust the value to STP as if it was at the given fixed temperature and pressure. The processing data consists of a comma separated temperature (in Celsius if less than 150, kelvin otherwise) and pressure (hPa).

General Keys

In all these the global superstitions listed below are valid.

Global String Substitutions

As noted above various keys can have specific strings replaced in them with values loaded from other sources, the following replacements are available: