The configuration file used by data.aggregate.humidograph. It is located in $DB/etc/$STATION/humidograph.$STATION.conf or in the default location $DB/etc/humidograph.conf. If the station specific file does not exist then the default one is used.
Lines beginning with '#' are treated as comments. Each line is treated as a set of comma separated values with standard quoting applied. The file is divided into “blocks” that are nested together. A block begins with a specification and ends with a single line containing “END”. All key names are case insensitive.
The top level block is a profile block specifying the name of the profile being modified and the time range the modification is for. All profile blocks for the same profile at the same time are overlaid on top of each other, meaning that later blocks in the file can modify parts of the profile without re-defining all of it. The first field of the profile block is the name of the profile, and subsequent fields (if any) are the time range the modification is effective for.
Dry and Wet blocks
Within the profile block, there are “Dry” and “Wet” blocks defining the dry and wet instruments.
The instrument key within the block defines the general instrument (e.x. “S11”) to look up data based on. In most cases setting this is all that is necessary.
All occurrences of the records key are added to the records requested. This is only needed if any variables used are not contained in the “a” record for the instrument.
This sets variables the fit is done on. Each occurrence is a regular expression and all matching variables are added to the fit candidates. Variables are paired between wet and dry and only those that exist on both are fitted. Wet variables are adjusted the same wavelengths as the dry variables, if applicable.
This key takes two parameters both value specifications (see below). The first is a temperature and the second is a humidity. A dewpoint is calculated using the temperature and humidity then the internal RH of the instrument is recalculated using this dewpoint and its internal temperature.
This key takes a single value specification and overrides the internal RH for the instrument.
This key takes a single value specification and overrides the internal temperature for the instrument. This is generally only used in conjunction with the dewpoint calculation above.
This block contains a list of sub-blocks that are identified by an internal name (not visible outside the configuration). Each sub-block contains the specification for an instance of the humidogram fit to run. The common case is one sub-block for the global fit, one for the low RH and one for the high RH.
This is a simple description of the fit being run. It it used to generate the metadata description of the variables produced.
Require and Exclude
These keys (multiple instances allowed) set the conditions that data is filtered on before entering into the fit. All “require” conditions must be met for data to be included. If any “exclude” condition is met then the data point is not included.
Each condition consists of two value specifications separated by one of “<”, “⇐”, “>”, “>=”, “==”, “!=”, or “&”, “!&”. The condition evaluates to true if the first value is less than, less than or equal to, greater than, greater than or equal to, exactly equal, not equal, a bitwise AND evaluated to non-zero, or evaluates to zero.
The special values “true” and “false” are also supported causing the require or exclude to always evaluate to the given boolean value.
Set the function used to fit the data. One of:
- GammaOne - (1-RH/100)^(-gamma)
- GammaTwo - a*(1-RH/100)^(-gamma)
- Linear - a + b * RH (testing only, not representative of real data)
- LinearGamma - a + b * (1-RH/100)
- Quinn - a*( (100-RHwet)/(100-RHdry) )^(-gamma)
Defaults to “GammaTwo”.
This defines the output parameters generated. All fits are to the ratio of wet over dry scatterings.
Set the fitting routine used. One of:
- LevenbergMarquardt - Levenberg-Marquardt_algorithm
This defines the goodness of fit generated.
The internal priority that the fit has. This is used primarily for generating the ordering of variables. Fits with a smaller priority generate lower numbered instances of the variables. Fits with no priority set always sort to the end and fits with equal (or both absent) priorities are sorted alphanumerically.
These parameters can be set in either the profile block or on a per-fit basis in a fit block. If they are set in the profile block, they override the default for all fits.
The instrument code used in the output records. Defaults to “XH”.
The output record generated. Defaults to the “a” record for the instrument.
The minimum number of valid points required before a fit is attempted. Defaults to 14.
The minimum value for the highest RH achieved before a fit is attempted. Defaults to 60. That is, a fit will not be done if the highest RH reached is less than 60.
The maximum value for the lowest RH achieved before a fit is attempted. Defaults to 64. That is, a fit will not be done if the lowest RH reached is greater than 64.
The minimum difference between the highest and lowest RHs achieved before a fit is attempted. Defaults to 30. That is, a fit will not be done if the difference between the highest and lowest RH reached is less than 30.
The minimum average green scattering before a fit is attempted. Defaults to 5. That is, if the average green scattering for the entire fitting period is less than 5, a fit will not be done.
Exclude all points for which the wet RH is less than this value. Defaults to 40. That is, points with a wet RH of less than 40 will not be considered in the fit.
Set the length of the scan in seconds. Defaults to 3600. That is, a fit will be done once per hour on the hour.
A value specification normally consists of a variable and an optional calibration. The components are separated by “:”, “;”, or “,” (be sure to quote comma containing elements, however). The variable comes first and must be present in one the input records (see above to request additional records). The calibration starts at the offset, then the first order term and so on. For example:
Is the sensor “U_V11” offset by +0.5% RH. The special variables “T_wet”, “T_dry”, “U_wet”, “U_dry”, “scan” are available to represent the temperature, humidities, and seconds into the scan.
The specification can also be a simple decimal constant (e.x. “1.5”) or specification of the form “time” or “time:modulus” where the time is the number of seconds after the start of the scan, optionally modulo the given number.
Aerosol Dry Instrument,S11 END Wet Instrument,S12 Records,X1a,X2a Td,T_V13,U_V13 END Fits Global Priority,0 Description,"all data" END Low Priority,1 Description,"RH < 65 percent" Require,U_wet<65 RequiredPoints,7 MinimumSpanRH,15 MinimumHighRH,55 END High Priority,2 Description,"RH > 65 percent" Require,U_wet>65 RequiredPoints,7 MinimumSpanRH,15 MaximumLowRH,70 END End END
This configuration defines the fit as using “S11” for the dry instrument with all defaults. “S12” for the wet instrument but recalculating the internal RH using an external sensor for the dewpoint and the internal temperature sensor. It defines three fits, one for all data using all defaults, one for low (<65%) and one for high (>65%) RH. These fits override the default filtering condition to be in line with the data they contain. The RH used is the one calculated from the dewpoint.