Unidata Users Committee Meeting Summary

May 17-18, 2007
Boulder , Colorado

 

Members	UPC Staff
Gary Lackmann, North Carolina State Univ, Chair	Tom Baltzer
Sean Arms, Univ of Oklahoma	Tina Campbell
Elen Cutrim, Western Michigan Univ	Steve Chiswell
Chris Herbster, Embry Riddle Aeronautical Univ	Ethan Davis
Leigh Orf, Central Michigan Univ	Ben Domenico
Scott Rochette, Suny College at Brockport	Steve Emmerson
Tom Whittaker, SSEC - Univ. of Wisconsin-Madison	Jo Hansen
	Yuan Ho
	Linda Miller
	Terry Mitchell
	Jeff McWhirter
	Don Murray
	Jennifer Oxelson
	Mohan Ramamurthy
	Russ Rew
	Jeff Weber
	Anne Wilson
	Tom Yoksas

Please Note: Proposed Fall Meeting Date: 4-5 October 2007

Meeting Agenda

Meeting Action Items

Thursday, 17 May 2007

Administrative:

• Introduction of Sean Arms-first Users Committee student member is pursuing his PhD at OU
• Committee and Unidata staff introductions
• Review of Action Items from 25-26 September 2006 meeting
  • forum sub-committee formed and met with Jen Oxelson during the meeting
• 2007 Equipment Award themes-IDV and community broadening
  • next year focus should be on THREDDS
• Survey needs to be completed by the Users Committee members
  • Committee members should post their survey results before the meeting to allow for responses from staff and other committee members
    

Policy Committee Summary-Gary Lackmann

• Steve Businger, Univ of Hawaii is taking over as Chair from Melanie Wetzel, DRI
• April 13, 2008 is the start date for the management of NCAR
• The Metrics and Assessment report was discussed – significant outcome for UPC-transformative effect
• The new Strategic plan was discussed and will be modified and finalized for the next meeting
• LEAD-there was a major discussion about how LEAD would link with Unidata
  • funding cycles; a separate proposal, rather than with Unidata core proposal was recommended
  • Microsoft was interested in LEAD project

Resolution: The Policy Committee recognizes the strategic long term value of the LEAD project for the Unidata community and encourages the UPC to take a key role in coordinating the development of proposal(s) for evolution of LEAD, to an extent that will not compromise the core activities and resources.

• 5-year funding proposal to NSF/ATM-guideline for submission –change in polcomm summary
• would like to get community needs from UserComm
• Users Committee--querry members of Unidata to establish the pulse of the community

Director’s Report-Mohan Ramamurthy

Mohan reported on the switch between Melanie Wetzel and Steve Businger as chair of the Policy Committee. Steve will take chair at the next meeting in July 2007. Unidata has had administrative staffing changes. Ginger Emery and Tina Campbell are new members of the staff, replacing Sandra Petrie and Emily Doremire. We welcome Sean Arms-new student member of the Unidata Users Committee.

Other issues of importance included:

• DeSouza award was presented at the AMS Annual Meeting in San Antonio. It recognized-Jim Moore, posthumously. Jim was Scott Rochette's Advisor at SLU. Scott accepted the award on behalf of Jim's wife, Kathy, who was unable to attend.
• Data and data flows—As of 16 May 2007, 395 hosts and 210 unique network domains reporting stats
  • TIGGE represents 8-10GB/hr in bursts-data flowing to CMA, Beijing – the data are embargoed for RT data distribution.-but NCEP is providing the data in real time via CONDUIT, but other data centers have implemented a 48 hr delay
                The data consist of global ensembles
  • Hydrofeed-a new Level II data stream-is being used for data mining, rainfall, value added metadata. The question is what tools are needed for analysis and visualization for the data?
• IDV-Feb 2007-IDV was used over 300 times per weekday with 75 sites running the IDV 20 or more times. 
              IDV now able to work with Google Earth
              GEON-large NSF ITR project is using IDV as their analysis/visualization application
• GEMPAK-1479 unique users downloaded GEMPAK from the UPC during the last 6 months
              Ensemble-tracking hurricanes capability in GEMPAK
              GEMPAK can read TIGGE data, spaghetti diagrams, etc
• McIDAS use is decreasing in U.S. but increasing in global use-ADDE is playing a major role
• NetCDF-IPCC standard-netCDF is data standard for climate – Ray Pierrehumbert-major role
• Metrics & Assessment Study is complete and the report had recommendations for the UPC to consider as we move forward
• Training workshops-2006-72 participants, 52 from traditional community., 4 from private sector and 16 international
  2007 training workshops have been announced with registration still underway
• Unidata has participated in several meetings, i.e., AGU, AMS, EGU meetings-several talks, booth at AMS
              AGU/ESSI (Earth &Space Science Informatics) has become a sanctioned session. Unidata played a role in the decision
              A move is underway to try to do this for EGU
• Univ of Oklahoma Regional workshop (Fred Carr) will be held Sept 10-12 2007 -- ongoing coordination with Unidata staff
• Equipment Awards--5th year of Unidata administration
  • Received 276K-total of 100K available with focus of broadening use of IDV and broadening reach of Unidata
• CADIS-Cooperative Arctic Data and Information Service - Unidata is collaborating with NSIDC, NCAR/CISL, NCAR/EOL-moving data from Arctic observing sites-remote access will use THREDDS and OPeNDAP
• GOES-10 South Project
  • It took $30K to bring GOES-10 South data to the community through the set up of a remote access server
• Next Generation Case Studies-case studies will be living and will leverage resources available at Unidata, COMET, and other places
  • Question of QC - this is something to consider
  • Brian Etherton and Shelley Holmberg working with Jeff Weber
• LEAD-students at 13 university sites participating in National Collegiate Forecast Contest Weather Challenge.
  • Storm Prediction Center-using LEAD-10 member ensemble CONUS forecast at 4km grid spacing
  • Also, dedicated runs over tornado watches
  • LEAD beyond Year 5
  • One scenario-Unidata would be home for collaborative project. The PIs are discussing
  • Question of ADAS-can use THREDDS catalog to access; not the LDM
  • All data from Greensburg tornadoes are captured on LEAD server.

Discussion

The Next Generation Case Studies-unlike WikiPedia, no one can destroy the case study because you own the case study. WikiPedia-can be tweaked by others, so this is not a good way to go.

There is a total change in the NSF/GEO management with Margaret Leinen’s departure, there is a new director being sought to replace Margaret Leinen and for various other positions at NSF. Office of CI-Dan Atkins is Director-(Atkin’s report, Executive Summary)

Status Reports

The status reports were shorter which facilitated reading the reports by the committee members. The following items constitute discussions between the Committee and staff:

• Equipment Awards-can we get additional funding from other divisions or groups at NSF to create a larger pot of money for the equipment awards?

• GEMPAK-A question raised of GARP support and development, or has Unidata considered a GUI for GEMPAK? NMAP2 has a lot of functionality that GARP has had.  Case study issues with GARP. We need a list of comparisons between GARP and NMAP. GARP is being used at SUNY College at Brockport-(Scott). National Centers are using NMAP. Question of GEMPAK Fortran library-will the libraries be lost when GEMPAK is written in C? The FORTRAN wrapper will help.
• IDV-Leigh Orf or Marty Baxter should be on the IDV steering committee. The IDV developers encourage feedback. Other items:
  • Moving IDV lists to Forums
  • Should there be a focus on ATM needs rather than multi-disciplinary development for the IDV? IDV has been very successful in GEON. The lessons learned through these type of collaborations have been very useful to the developers and they have been able to leverage from the partnerships.
  • The question of IDV replacing GEMPAK—there was a decision made with PolComm and UserComm that Unidata should maintain multiple packages -- not all programs are created equal. A white paper was created by Russ Rew dealing with the issues of supported packages.

• LEAD-Weather Challenge-survey and feedback from users who participated - metrics study of system performance
  External Advisory committee-NOAA has embarked on LEAD-type functionalities for internal needs—focus on educational – forecast NOAA/GSD/FSL system - the Storm Prediction Center is using LEAD
• NetCDF - The Intergovernmental Panel on Climate Change (IPCC) adopted-netCDF 4 which does compression
• Case Study project—very good effort-positive response-IDV can save data in a bundle.  Also has remote grids, unzipped—can create mini-case studies.

Discussion: We should allocate one hour for status reports and discussion at future meetings. Telephone survey results should be scheduled with the status reports.

Strategic Plan—Mohan-The last strategic plan was developed in 2002. Mohan has been gathering input for the new strategic plan for one year, and the draft was presented to Policy Committee two months ago.  This plan will become the backdrop to the proposal. It covers:

• Who we are-Environmental Scan
  Science
  Education
  Technology
• Focus areas
  Community
  Data Services
  Tools
  Communication and Support
  Cyberinfrastructure Leadership
  Diversity

Some items still need to be completed. Mohan would like the new strategic plan to have a shelf life of five years.

THREDDS-Ben Domenico

• Motivation--seamless access to historical data and to faciliate the use archives from all data sites.
• OPeNDAP and ADDE technologies important for remote access capabilities
• Creation of metadata with catalogs
• Makes searching for data easier and more comprehensive
• In the second proposal, a goal was to make data and catalogs available to other disciplines, e.g. GIS – open standards and interoperability
• Want to break down stovepipes from GIS and Unidata-THREDDS applications by providing standard protocols: WMS, WFS, WCS, CSW, SWE, GML
  Data access is available via OPeNDAP, Web Coverage Service
  NetCDF subset service
  Common Data Model---combines best characteristics of netCDF, OPeNDAP and HDF data models
              Provides common model for data exchange
• Partners include:
  • UPC, data providers, interoperability partners, and community implementation of servers

Discussion:
Question of ADDE servers and THREDDS servers combining – idea is to help community members make their data holdings available to others. We need ideas of how to get others to share their data via THREDDS. This raised the issue of what expertise is needed to set up a THREDDS server. The question of a steering committee came up.  There has been a technical task force (John and Ben).  The idea for a steering commitee would be to have community members, i.e., faculty, researchers, et al, glean input to make deployments easier. Another idea was a forum for THREDDS. The role of "Data Guru" was mentioned as an issue that has been discussed and noted as a need within the UPC.

How do we promote the use of THREDDS data servers? It is important to point to information on the Web site on how to create a THREDDS data server.

We need to explore options for training, such as workshops, and testimonials from users. Concrete examples from community members on how and why THREDDS is useful and needed. This type of information would go far toward facilitating use of THREDDS servers by the community. Community members will need the assistance from Unidata technical staff to get underway. Next year, consideration should be given toward the THREDDS focus for Equipment Awards. Sean Arms is going to set up a THREDDS server at OU and will provide feedback from his experience.

Workshop Follow up

The Users Committee has the first Grad student representative to serve on the committee. Some of the activities for Sean Arms to undertake include:

• presentation to students about Unidata
• seminar series
• expose students to the data and tools provided by Unidata
• submit an abstract to the AMS Annual Meeting student fest

Active campaigning will be needed to make things happen. The Users Committee can be helpful by providing the information during their community phone contacts.

Jeff Rosenfeld, AMS, indicated that multiple articles about the workshop might be featured in BAMS.
The articles will appear in Nowcast. Rosenfeld liked the Mark Chandler presentation, and it was suggested that a "point-counter point" article be done by Leigh and John Snow, Stella model presentation.

Regional Workshop-Norman, Oklahoma

The Regional Workshop in Norman Oklahoma is schedule for September 10-12, 2007. The main emphasis of the workshop will be IDV, but it will include presentations on THREDDS, LEAD, and WDDS. There will be a Meet the Developers session.

Friday, 18 May 2007
Discussion on optimization of survey input collected by the Users Committee on various topics for the Unidata Program Center. It was agreed that starting telephone calls earlier would facilitate the process. There are times when people would rather communicate via email, so it might be wise to begin with an email introduction. Also, this could lead to having a discussion with the right person, if the inquiry is stated appropriately. The original intent was to ensure that the Uses Committee was in touch with the community, and to provide community awareness of the mission of the Users Committee. It was suggested that each member could make one call /month to avoid the activity as an onerous task.

The use of Google spreadsheet for updates was raised by Tom Whittaker. This would require asking everybody to sign up for a G-mail account, but it would pay off, due to the ease of reporting the calls for each of the Users Committee members.

DeSouza Award venue options were considered. A suggestion was made that the DeSouza Award presentation be made at the Spring Users Committee meeting. The early morning award prior to the AMS IIPS session does not have enough audience participation to warrant the presentation. By setting aside time during the Users Committee meeting, the award could be made into a Web cast seminar and the entire community could be invited to view the presentation. It was further suggested that during the tri-ennial workshop, the award could be made there. It was agreed that these steps would lead to a broader community reach. The award presentation will be advertised at the AMS annual meeting.

CONDUIT-Steve Chiswell-The server was replaced at the Web center at NOAA and the data are available from the NCEP server rather than NWSTG server. It's been very reliable.

• 75GB of CONDUIT data per day is flowing.
• An increase in the number of ensembles is now available.
• Use of GRIB2 has not officially been announced.
  • Currently GRIB1 and GRIB 2 are running in parallel.
  • RUC model (RUC model is running at 13 km.  Hourly to 9 hrs)  is already out in GRIB1 and GRIB2.
  • GRIB2 provides better compression and allows for additional bandwidth. Planning to add GDAS.
    

TIGGE-Adding products required by TIGGE is in progress.

• TIGGE has 78 parameters making up full data set-currently has 66 parameters available
• Precip increment for model-will not do a lot of extra processing
• A forum is being developed for TIGGE users - CISL and Jen are working on this
• THORPEX needs an audit trail of who is using data, so registration is required.
• There will be a community announcement about data availabilitly
• Follow on community workshop to bring TIGGE users together to discuss data needs and use
• TIGGE has been sanctioned as WMO pilot project

Miscellaneous:
WMO is planning to transition its GTS data system to something new-WMO Information System team is considering options—Unidata technologies are being considered; LDM, THREDDS, etc.

COSMIC GPS data to Unidata community - Unidata is working with COSMIC to move the COSMIC data to the Unidata community using LDM technology. Currently registration is required through a Web interface. A short community survey will be developed to assess the community desire for the data. A COSMIC users workshop is being planned for October in Boulder.

An NWSTG outage was brought to our attention by Unidata user, Gilbert Sebenste. He questioned if Unidata could set up a separate feed at NWS to use as a back up to the GTS. Data was missing for over 17 hours.

UniWisc datastream: data from South America will be added to UniWisc. It will add 32-45 mb/hr of additional data. Sites would need to indicate whether or not they want the additional data. This would need to be clearly stated on any announcement to the community of the additional data.

Chris-USPLN-Time of Arrival
WSI is new distribution group for USPLN data. 
Peter Neilley
Mark Miller
Unidata would be providing the decoders
ADDE can limit access to data sets, so this should be covered regarding restrictions.

Chris will continue to monitor the situation.

The source code and data from the 2006 summer workshop were not archived. Gary and Chris have archived. Some of the data and will contact others about providing their data for use by the broader community.

 

Linda Miller
Community Services - Unidata
University Corporation for Atmospheric Research
P.O. Box 3000
Boulder, CO 80307-3000
303 497-8646 fax: 303 497-8690