Scenario: Binary Data from a Variety of Instruments (OWLeS Field Study)

CASE STUDY UNDER CONSTRUCTION

Tools at a Glance

This case study describes an experiment that gathered data from a range of atmospheric sensors, collected mostly in ASCII format. Data files are converted to netCDF format using Unidata's Rosetta data transformation tool, and data are made available using Geode Systems' RAMADDA data server.

Project Background

A team at the University of Wyoming led by PI Dr. Bart Geerts particpated in the regional OWLeS (Ontario Winter Lake-effect Systems) study conducted December 2013 through January 2014. Additionally, the University of Wyoming is home to the King Air Research Aircraft (UWKA), which was used to collect data for the OWLeS study; Project Manager Dr. Larry Oolman participated in the data collection phase.

The OWLeS project, which involved nine universities and the Center for Severe Weather Research (CSWR), had funding from the National Science Foundation (NSF) and the National Center for Atmospheric Research's Earth Observing Laboratory (EOL). The OWLeS project was conducted to better understand the factors that determine the severity of lake effect snow events; Dr. Geerts' portion of the experiment specifically investigated the role of long fetch winds under the prevailing westerly wind belt.

Data collected as part of the OWLeS project are governed by the OWLeS Data Policy.

OWLeS flight track from UWKA.

Data Collected

The Wyoming Cloud Lidar (WCL) and the Wyoming Cloud Radar (WCR) instruments collect data in an instrument-specific binary format. After a flight by the research aircraft, data are transferred to computers at the University of Wyoming for processing and analysis.

Tools Used for Data Processing and Analysis

The raw binary format data is processed by software written at the Univeristy of Wyoming, converting it into a level 2 dataset. The data is also adjusted according to internal conventions including atmospheric pressure corrections for the WCL and ground mass corrections for the WCR. The final format is netCDF, with a metadata READme file attached that defines all variables contained in the netCDF file in human-readable format. While most researchers prefer to use this level 2 dataset, some also prefer to use the raw dataset. In addition, the in-situ data collected via probes on the research aircraft are also stored in netCDF format, with a READme file attached.

Visualization of data collected by UWKA.

The processed WCL and WCR data are not compliant with the Climate and Forecast (CF) conventions for metadata, which do not currently define appropriate metadata for radar and lidar moments data in radial (i.e. polar) coordinates. (Note that efforts are underway to include the NCAR-developed CfRadial format in the CF standard, which would solve this particular problem.)

The primary software used to analyze the data is the Interactive Data Language (IDL). A Python-based tool set called the Airborne Weather Observations Toolkit (AWOT) for visualizing the data is also under development at the University of Wyoming.

Data Storage Strategy and Tools Used

Collected data are stored within the instrumentation until the end of a flight, then transferred to an internal data server at the University of Wyoming for the project team to jointly access. Data files are also mirrored in deep storage at NCAR.

Data Access Stragegy and Tools Used

Data from the OWLeS project are availble from a project page at the NCAR Earth Observing Laboratory. Access to some items in the Field Catalog require authorization from the university that collected the data; other items are freely available, based on the individual institutions' data access policies.

The DOI will point to a projects page, where there will be a place for users to request a particular amount of data files along with a specific file type. Three Digital Object Identifiers (DOI) will be created to access the OWLeS project data. Each DOI will correspond to either the WCL, WCR, or the in situ data. This request is performed on a web based interface, and will transfer selected files to a FTP which will be emailed to the user.

Data Archiving Stragegy and Tools Used

The OWLeS team is currently archiving their data on their own internal data servers at the University of Wyoming, as well as mirroring their data in deep storage at NCAR.