Return to: December 1996 FSL Forum FSL Home Page

Introduction

The extent of bird contamination in low-level hourly profiler winds is difficult to detect because it generates clusters of coherent, erroneous winds. Since the winds mutually agree, the wind-only quality control (QC) method (Weber et al. 1993; Miller et al. 1994) applied to the hourly measurements of the NOAA Profiler Network (NPN) was never expected to reliably locate bird-contaminated winds. To date, however, not enough has been known about the degradation of the winds in the presence of bird migration to accurately evaluate the QC method, or even, for that matter, to determine if QC for bird contamination was necessary. Examples of both good and bad agreement have been shown in preliminary studies in which profiler-measured winds were compared with winds measured by other instruments in the presence of bird contamination (Wilczak et al. 1995). In all cases, the velocity variance (second spectral moment) data associated with the profiler winds showed the typical large values indicative of the presence of birds. Since, in some of these cases, the profiler winds agreed with the rawinsonde winds, the large velocity variance would seem to be a necessary but not sufficient indicator of bad hourly winds. The sample sizes in these studies (Jungbluth et al. 1995; Warnock et al. 1995) were too small to be conclusive.

To more fully understand the issues, we designed an experiment to compare a large number of rawinsonde and profiler observations taken both during periods of bird migration and periods when the profiler data should not be susceptible to bird contamination. The goals of the experiment were to build a statistical database that could more definitively determine how seriously bird migrations affect the quality of the NPN hourly winds, and how accurately a velocity-variance-based QC algorithm could identify bird-contaminated winds. The results of the experiment follow.

Dataset Description and Comparison Techniques

Rawinsonde data from four stations collocated with profiler sites were obtained from the Department of Energy's Atmospheric Radiation Measurement (ARM) archive (Lesht 1995). These stations are located at Lamont, Vici, and Haskell, Oklahoma, and Hillsboro, Kansas. At the Lamont site, rawinsondes are launched five times a day under normal conditions and eight times a day during intensive operating periods. At the remaining sites, launches are performed once a day, except during intensive operating periods, when eight rawinsondes are launched. Because of their high temporal frequency, the ARM soundings are uniquely valuable for the study of bird contamination in profiler data. They can overcome the limitations of the 0000 and 1200 UTC National Weather Service (NWS) soundings, which occur at times of the day when heavy bird migration is not expected. The Table (next page) shows data from the four sites for four three-week intervals corresponding to four intensive operating periods during 22 January-14 November 1994. The two-second rawinsonde data were checked for gross wind speed errors (DiMego et al. 1985) and then averaged over the depth of the profilers' range gates (250 m).

Corresponding profiler data were also collected and quality controlled using the NPN wind-only QC check. Comparisons were then made whenever both the rawinsonde and profiler winds passed the QC checks. As shown in the Table, sufficient data were available to stratify the comparison statistics by height, season, and time of day. The stratifications were designed to group comparisons into datasets in which bird contamination might be expected to appear, and into datasets in which it would not.

Extent of Bird Contamination

Past studies, summarized by Wilczak et al. (1995), have suggested several characteristics of bird migration that should be identifiable in the comparison statistics, given the appropriate stratification. Significant migration is expected to occur only at night and to appear only in the spring and fall datasets. The errors caused by bird contamination should be more pronounced in the lower atmosphere (below 4500 m MSL (mean sea level)), and the errors should generally be in the direction of the migration, because the birds tend to fly with favorable tailwinds. In other words, the effect should be seen mainly in the v (north-south) component of the wind measured by the profiler, and the sign of the error should be compatible with the seasonal bird migration direction.

Table



Counts of comparison pairs for winds below 4500 m MSL obtained from four collocated profiler and rawinsonde stations during four 3-week periods during 1994: winter, 22 January-11 February; spring, 12 April-2 May; summer, 20 July-9 August; fall, 25 October-14 November. For the winter and fall seasons night was defined as 0000-1300 UTC; for spring and summer night was defined as 0200-1200 UTC.

Root-mean-square (rms) errors, absolute differences, biases (rawinsonde-minus-profiler), and vector differences were calculated as a function of height, for daytime and nighttime data collected in each season. The statistics indicated larger errors at night in the spring and fall than during the day in those seasons; and larger errors in the spring and fall nighttime data than in the summer and winter nighttime data. As would be expected for bird contamination, the effect was more pronounced in the v component than in the u component, but the errors appeared to affect both components. The most distinguishable difference between spring and fall night datasets and other datasets can be seen in the v component bias shown in Figure 1 for comparisons below 4500 m MSL. The solid lines are for spring and fall where errors from bird contamination might occur (at night), and the dashed lines are for winter and summer, where little or no such contamination is anticipated. In the spring night data, the bias is negative, indicating that the profiler winds are more southerly than the rawinsonde winds. The opposite is true in the fall night data, where the profiler winds are more northerly than those measured by the rawinsonde. In both seasons, the direction of the bias matches that expected for the seasonal bird migration direction, and is consistent with the hypothesis that the bias is due to errors in the profiler data caused by bird contamination. In addition to the bias being in the migratory direction, Figure 1 shows that while the errors in the spring are confined to the lower 3 km, the anomalous bias is present up to 4 km in the fall. This, too, is consistent with hypotheses about bird contamination reported in past work (Coulter and Holdridge 1995; Wilczak et al. 1995).

The comparison statistics were also accumulated for all profiler levels below 4500 m MSL (but not shown). For each category of statistics, the spring night dataset exhibited the poorest profiler-to-radiosonde fit, followed by the fall night dataset. However, with the exception of the v component bias for the spring night and fall night datasets (which were more than 10 times the overall v component bias), the degradation in each statistic when compared to the total calculated over all datasets was not that dramatic.

The statistics were also stratified by wind direction as well as by dataset. For all datasets other than spring and fall night there was virtually no difference in the statistics for northerly and southerly winds. This can be contrasted with the spring and fall night datasets, where almost all of the statistical degradation occurred when the wind was favorable for bird migration (southerly in the spring, northerly in the fall). With this stratification, it was evident that the magnitude of the degradation was about equal in both seasons. Although the gross statistics appeared to be worse in the spring than in the fall, this was a result of the winds being southerly almost 60% of the time, whereas in the fall the occurrence of northerly winds was less than 30%. Even stratified by direction, however, the level of additional differences was not that large. For example, the mean vector difference at night in spring and fall, in the direction of bird migration, was only 2 m s-1 greater than the total calculated over all datasets.

In order to determine the amount of error possibly attributable to bird contamination, and not simply part of the expected differences when comparing rawinsonde and profiler winds, all of the datasets other than spring and fall night were combined into a control dataset. By examining the differences in gross statistics between the control and the spring and fall night datasets, as well as through analysis of the distribution of the differences, we attempted to quantify the extent of the additional error that might be caused by bird migration. To that end, the relative cumulative frequency distributions for absolute u component difference, absolute v component difference, and vector difference were calculated. The probability of errors of a given magnitude could then be determined from examination of these curves. For example, with the control dataset, 90% of the v component differences were > 2 m s-1, 7% of the differences are between 2 m s-1 and 4 m s-1, etc.



Figure 1. Mean bias (rawinsonde-minus-profiler) for the v wind components obtained with four collocated profiler and rawinsonde stations, stratified by season of year and time of day. Solid lines are for the fall (a's) and spring (c's) seasons, dashed lines are for the summer (b's) and winter (d's) seasons: v component - daytime (left); v component - nighttime (right).

Subtracting the curves for the spring and fall night datasets from those for the control dataset produced functions representing the increased likelihood of errors that occurred at night during the spring and fall. These functions are shown for the vector differences in Figure 2, where the increased likelihood of errors at night during the spring is shown with a dashed line, and the increased likelihood of errors at night during the fall is shown with a solid line. This figure shows, for example, that the increased likelihood of errors > 4 m s-1 was 15% in the spring, and 8% in the fall.



Figure 2. Relative cumulative frequency distributions of absolute differences for wind vector differences obtained with four collocated profiler and rawinsonde stations at night during the spring (dashed lines) and fall (solid lines) bird migration seasons, subtracted from the distributions for the control dataset. Curves show the increased likelihood of errors at night during the spring and fall seasons.

In summary, we found that the comparison statistics for the spring and fall night datasets did in fact exhibit characteristics that differed from all of the other datasets, and that these characteristics were all consistent with the hypothesis that the increased errors in these datasets were primarily the result of bird contamination in the profiler winds. The statistics were also used to quantify the extent of the contamination:

• Overall, the accuracy of profiler winds at night in the spring and fall was slightly degraded. Compared to the control dataset, which was used to establish a "normal" amount of error, the mean vector differences were about 1 m s-1 greater during these seasons. Almost all of the degradation in the statistics was found when the winds were in the direction of the bird migration. When stratified in the direction of the migration for each season, the overall errors were about 2 m s-1 more than those found in the control dataset, and also about 2 m s-1 more than those found at night in the spring and fall for winds in the direction opposite that of the migration.

• The maximum vector differences (apparently caused by bird contamination) were on the order of 15 m s-1, although errors this large were rare. The increased likelihood of errors > 8 m s-1 was only 5% at night in the spring, and 4% in the fall.

Quality Control Algorithm Evaluation

A check for bird contamination in the hourly winds was devised by F.M. Ralph and D.W. van de Kamp (personal communication; Chadwick 1995), based on work reported in Wilczak et al. (1995). The bird contamination check is a simple, six-step test that is primarily based on identifying the large velocity variance signature that is thought to accompany the backscattered radar return from birds in profiler spectral moment data. The other steps in the test are meant to reduce the likelihood of false alarms, as other factors besides bird contamination can cause large velocity variance. The check flags an hourly wind observation as contaminated if all of the following criteria are met:
• The time of year is appropriate for bird migration ("spring": 15 February-15 June or "fall": 15 August-30 November).

• The time of day is appropriate ("night": 0200-1200 UTC in the spring, 0000-1300 UTC in the fall).

• The height is appropriate (<4500 m MSL in both seasons).

• The wind direction is favorable for migration (90-270o in the spring, 0-90o or 270-360o in the fall).

• The velocity variance signature is appropriate (north antenna beam only, value > 1.0 m2 s-2).

• The large velocity variance is not a result of rain (magnitude of downward vertical velocity < 3.0 m s-1, based on Ralph 1995 and Ralph et al. 1995.

To examine the performance of the bird contamination check, an algorithm was developed that used the rawinsonde observations to independently quality control the hourly profiler winds in the spring and fall night datasets. Based on the comparison statistics from the control dataset, the algorithm flagged profiler winds as bad when either the u or v component differed from the rawinsonde value by more than N standard deviations away from the mean difference at that height. Verification statistics for the bird contamination check were calculated by comparing its results with those produced by the statistical algorithm. The statistics are as follows: Probability of Detection (POD = d/(c+d)), False Alarm Rate (FAR = b/(b+d)), and Critical Success Index (CSI = d/(b+c+d)), where b is the number of good winds incorrectly labeled as bad (or contaminated), c is the number of undetected bad winds, and d is the number of detected bad winds. A perfect performance by a QC algorithm would be characterized by POD = 1, FAR = 0, and CSI = 1.

The verification statistics for spring and fall, and the statistics totaled over both seasons were calculated for four different values of N in order to determine how sensitive the results would be to the chosen thresholds. With the exception of the results for N = 1.0, which was clearly too restrictive, the actual scores may have changed depending on the thresholds in use, but the general patterns they revealed were the same. The POD was generally good, with the bird contamination check flagging 60 to 70% of the bad data when totaled over both seasons. The slope of the POD scores as a function of N indicated, however, that there was a cap to how much bad data could be detected. The FAR was also high, ranging from .44 with N = 2.0 to .62 with N = 3.0. There were more bad data in spring than in fall, and the algorithm successfully located more of these data in the spring (higher POD), but it also generated more false alarms in the spring, thus the check exhibited better overall skill (as seen in the CSI) in the fall.

Using the verification results for N = 2.0 standard deviations, case study analyses were performed to verify the statistical results, and to gain further understanding of the strengths and weaknesses of the algorithm. The check worked particularly well when the contamination was most severe, as illustrated by the case shown in Figure 3. In Figure 3a, and in the following case, boxed winds are bad (as indicated by the rawinsonde comparisons), and circled winds indicate winds that were flagged bad by the contamination check. The data plotted in Figure 3b are the difference vectors produced by subtracting the rawinsonde wind from the profiler wind. (Note that wind speeds < 5 kts are plotted with variable length shafts to provide finer resolution, and, although hourly profiler winds were available, only those hours with rawinsonde data are plotted.) This case from the fall dataset appears to be severely affected by bird contamination: the directions of the vector differences are almost due north, and their speeds are 15 to 25 kts. In fact, this was one of a handful of dates during the experiment when severe contamination occurred simultaneously at all four profiler sites - the algorithm worked equally well at all sites during this event.

Other case analyses, however, revealed that although most of the bird-contaminated winds were successfully flagged, a few POD misses (bad winds not flagged) occurred around the edges of the main areas of contamination, where the errors were not in the due north-south (migratory) direction. These types of errors, which can be large, cannot be detected by an algorithm which examines the variance from only the north antenna beam.



Figure 3 (a) and (b). Annotated time-height cross section of winds (kts) from the Vici, Oklahoma, profiler from 0000 UTC 1 November 1994 to 1200 UTC 1 November 1994: (a) above, profiler winds; (b) difference vectors produced by subtracting the rawinsonde wind from the profiler wind. The boxed winds are bad. The circled winds were labeled as bad by the bird contamination check. The check worked well when the contamination was most severe.

Another limitation revealed by the case studies, is the requirement that the wind direction be in the migratory direction for the season. Systematic errors consistent with bird migration, but not flagged because they are not in the direction of the migration, did appear in the datasets. Birds, it seems, sometimes fly against the wind (as noted in Wilczak et al. 1995).

Overall, however, the probability of detecting bird contamination using the velocity variance-based check was high. Lower POD scores were not a result of missed bird contamination, but rather a matter of other phenomena (e.g., ground clutter, inhomogeneities found during rain, etc.) causing undetected errors.

Cases of false alarms (good winds incorrectly flagged bad) were also examined. In most situations, the cases involved small regions of large bird-induced errors surrounded by a sprinkling of good winds flagged as bad. Some of these false alarms may indeed have a small amount of bird-induced error, but others exhibit very small differences from the rawinsonde data. As with the POD misses, the bird contamination check tends to make more mistakes on the edges of the contaminated area. An example of poorer performance is shown in Figure 4a (profiler winds) and 4b (vector differences) from the Haskell profiler in the spring. Here only a few profiler winds have large errors, but there are large areas where winds that have been identified as good are being flagged as bad.



Figure 4 (a) and (b). Annotated time-height cross section of winds (kts) from the Haskell, Oklahoma, profiler from 0300 UTC 27 April 1994-1200 UTC 27 April 1994: (a) above, profiler winds; (b) difference vectors produced by subtracting the rawinsonde wind from the profiler wind. The boxed winds are bad. The circled winds were labeled as bad by the bird contamination check. There are only a few profiler winds that have large errors, but there are large areas where good winds are being flagged by the check as bad.

The verification statistics showed that the performance of the algorithm varied quite a bit depending on the profiler site, and in the case of Haskell, depending on the season (Figure 5). While some possible explanations were explored, more work is needed to determine whether these differences are the result of climatology, bird migration patterns, or other instrument-related or atmospheric-related phenomena.

The overall impact of the bird contamination check on the spring and fall night datasets can be seen in the remaining increased likelihood of errors shown in Figure 6, and in summary comparison statistics (not shown) after processing with the contamination check. Only a small amount of additional error was identifiable in the spring and fall night statistics:

• Overall, the mean vector difference of the fall night dataset (2.00 m s-1) became indistinguishable from that of the control dataset (1.97 m s-1); the spring night vector difference remained only slightly higher (2.25 m s-1).

• The increased likelihood of large errors was also greatly reduced: vector differences > 8 m s-1 occurred only 1% more often at night in the spring than in the control dataset; the probability of large errors in the fall night dataset became virtually identical to the control statistics.


Figure 5. Verification statistics for the bird contamination check performed on four profilers at night in the spring and fall of 1994, using 2.0 standard deviations for the verification test (see text). The statistics are stratified by site and by season, and totaled over both seasons. The performance of the check varied quite a bit depending on the profiler site, and in the case of Haskell, depending on the season.

Unfortunately, quite a number of good winds were also eliminated:

• At night in the spring, 46% of the southerly winds below 4500 m MSL failed the bird contamination check; at night in the fall, 43% of the northerly winds were flagged. Given the high false alarm rate of the check, it is likely that many of these winds were good.

• Over 70% of the u and v components that failed the check at night in the spring differed < 2 m s-1 from the rawinsonde data. Of data failing the check in the fall night dataset, these small differences accounted for over 70% of the u component comparisons, and about 50% of the v component comparisons.


Figure 6. Relative cumulative frequency distributions of absolute differences for wind vector differences obtained with four collocated profiler and rawinsonde stations at night during the spring (dashed lines) and fall (solid lines) bird migration seasons, subtracted from the distributions for the control dataset, after quality control with the bird contamination check. The curve shows the residual increased likelihood of vector difference errors at night during the spring and fall seasons after application of the contamination check.

Conclusions

An experiment has been described that uses rawinsonde observations to determine the extent of contamination in hourly profiler winds caused by bird migration, and to evaluate a new Quality Control algorithm designed to flag the contaminated winds. It was found that during the spring and fall migration seasons, many of the errors probably caused by birds had magnitudes smaller than expected, although the profiler winds could be in error by up to 10 to 15 m s-1. The QC algorithm, called the bird contamination check, was successful in eliminating the large errors to the extent that the remaining profiler winds obtained during bird migrations were almost as accurate as those taken at other times. However, this was achieved at the cost of a high false alarm rate. The decision was made to implement the bird contamination check beginning 15 August 1996 while continuing work to improve the accuracy of the algorithm.

References

Chadwick, R.B., 1995: NOAA Profiler Network Facts Bulletin Fax, 17 March 1995.

Coulter, R.L., and D.J. Holdridge, 1995: A three-month comparison of hourly winds and temperatures from collocated 50-MHz and 915-MHz RASS profilers. Preprints, Ninth Symp. on Meteorological Observations and Instrumentation, AMS, 195-199.

DiMego, G.J., P.A. Phoebus, and J.E. McDonell, 1985: Data processing and quality control for optimum interpolation analysis at the National Meteorological Center. NMC Office Note 306, 38 pp.

Jungbluth, K., J. Belles, M. Schumacher, and R. Arritt, 1995: Velocity contamination of WSR-88D and wind profiler data due to migrating birds. Preprints, 27th Conference on Radar Meteorology, AMS, 666-668.

Lesht, B.M, 1995: An evaluation of ARM radiosonde operational performance. Preprints, Ninth Symposium on Meteorological Observations and Instrumentation, American Meteorological Society, 6-10.

Miller, P.A., M.F. Barth, D.W. van de Kamp, T.W. Schlatter, B.L. Weber, D.B. Wuertz, and K.A. Brewster, 1994: An evaluation of two automated quality control methods designed for use with hourly wind profiler data. Annales de Geophysique, 12, 711-724.

Ralph, F.M., 1995: Using radar-measured radial vertical velocities to distinguish precipitation scattering from clear-air scattering. Journal of Atmospheric and Oceanic Technology, 12, 257-267.

Ralph, F.M., P.J. Neiman, D.W. van de Kamp, and D.C. Law, 1995: Using spectral moment data from NOAA's 404-MHz radar wind profilers to observe precipitation. Bulletin of the Amer. Meteor. Soc., 76, 1717-1739.

Warnock, J.M., W.L. Ecklund, W.L. Clark, T.E. VanZandt, W.M. Angevine, and W.R. Evans, 1995: Multiple frequency profiler studies of echoes observed during bird migration. Preprints, 27th Conference on Radar Meteorology, AMS, 284-286.

Weber, B.L., D.B. Wuertz, D.C. Welsh, and R. McPeek, 1993: Quality controls for profiler measurements of winds and RASS temperatures. Journal of Atmospheric and Oceanic Technology, 10, 452-464.

Wilczak, J.M., R.G. Strauch, F.M. Ralph, B.L. Weber, D.A. Merritt, J.R. Jordan, D.E. Wolfe, L,K. Lewis, D.B. Wuertz, J.E. Gaynor, S.A. McLaughlin, R.R. Rogers, A.C. Riddle, and T.S. Dye, 1995: Contamination of wind profiler data by migrating birds: Characteristics of corrupted data and potential solutions. Journal of Amospheric and Oceanic Technology, 12, 449-467.

(Patricia A. Miller is Chief of the Model Verification and Production Assistance Branch, within the Aviation Division, headed by Michael Kraus. Michael F. Barth is Chief of the Software Development and Facility Management Branch, within the Demonstration Division, headed by Russell B. Chadwick. John R. Smart is a scientist in the Local Analysis and Prediction Branch, headed by John A. McGinley. Leon A. Benjamin is a program analyst in the Demonstration Division; he is on contract with the System Technology Associates, Inc., Colorado Springs, CO. Patricia A. Miller can be reached by e-mail:miller@fsl.noaa.gov.)