Jakob, C., R. Pincus, C. Hannay, and K.-M. Xu, 2004: Use of cloud radar observations for model evaluation: A probabilistic approach. J. Geophys. Res., 109(D3), D03202, 10.1029/2003JD003473, 12pp.
The use of narrow-beam, ground-based active remote sensors (such as cloud radars and lidars) for long-term observations provides valuable new measurements of the vertical structure of cloud fields. These observations might be quite valuable as tests for numerical simulations, but the vastly different spatial and temporal scales of the observations and simulation must first be reconciled. Typically, the observations are averaged over time and those averages are claimed to be representative of a given model spatial scale, though the equivalence of temporal and spatial averages is known to be quite tenuous. This paper explores an alternative method of model evaluation based on the interpretation of model cloud predictions as probabilistic forecasts at the observation point. This approach requires no assumptions about statistical stationarity and allows the use of an existing, well-developed suite of analytic tools. Time-averaging and probabilistic evaluation techniques are contrasted, and their performance is explored using a set of "perfect" forecasts and observations extracted from a long cloud system model simulation of continental convection. This idealized example demonstrates that simple time averaging always obscures forecast skill regardless of model domain size. Reliability diagrams are more robust, though scalar scores derived from the diagrams are sensitive to the forecast probability distribution. Forecasts by cloud system and weather forecasting models then provide examples as to how probabilistic techniques might be used in a variety of contexts.