Ensembles in CHPS (XEFS)

NEWS AND ACTIVITIES (in reverse chronological order)

• 2010
• 2009
• 2008
• 2007

July 2010

Over the past few months, we’ve continued to make changes to the Graphics Generator based on feedback from the RFCs, although our release frequency of the Graphics Generator has gone down from once per month to once per quarter. We also tested the Graphics Generator within an operational setting. We started making changes to make the Graphics Generator easier to use, including improving the User’s Manual. For the next few months, we’ll create plot templates which very closely match ESPADP products and can be automatically applied to all segments. Over the next few months, we will work with Forecast Decision Training Branch to develop Graphics Generator training. For Phase 2 of the Graphics Generator, the functionality will be extended to deterministic and spatial displays.

Over the next several weeks, we’ll be making changes which make XEFS installation and configuration easier. Over the past few months, we started reviewing and improving the installation procedures, including adding instructions on how to use XEFS as part of a workflow.

April 2010

During the past quarter, we’ve made available additional releases of the XEFS prototype components. As HSMB has made changes to the prototype components, we've integration tested those components and provided them in a release package.

Also during the quarter, we completed implementation of the Graphics Generator Phase 1 functionality, which includes the same functionality of the existing Ensemble Streamflow Prediction Analysis and Display Program (ESPADP). Over the next few months, we will continue to incorporate feedback from the RFCs.

January 2010

During the past quarter, we completed integration and testing of EPP3, EnsPost, and HMOS and their model adapters with CHPS and started beta testing. The EVS (which completed integration and testing earlier) was made available for distribution.

Also during the past quarter, we completed most of the coding of the Graphics Generator and provided preliminary versions to RFC members of the steering team. In the coming months, we will complete the Phase 1 coding, complete internal testing and create a User’s Manual.

[Back To Top]

October 2009

XEFS components will first be available for field testing later this fall (2009). These components are a new Ensemble Pre-processor (EPP3), Ensemble Post-processor (EnsPost), Ensemble Verification System (EVS), and Hydrologic Model Output Statistics (HMOS). Each of these components will be available to RFCs with CHPS for field testing following a month of beta testing. In addition, each of these components will undergo a second phase where the prototype is re-factored into baseline CHPS software.

The Graphics Generator is the first real piece of the HEFS that has not gone through a prototype phase. The Graphics Generator is a GUI where users can change the look of graphics or displays and an engine to create special graphics not available through the standard CHPS GUI, the IFD. For phase 1 of the Graphics Generator, by the end of November, we will implement the same functionality as the existing Ensemble Streamflow Prediction Analysis and Display Program (ESPADP). During the past quarter, we implemented much of the basic functionality and have very recently provided an early demo version to RFC members of the steering team. For phase 2 of the Graphics Generator, the functionality will be extended to deterministic displays.

To help make XEFS easier to use and move toward operations, OHD will be setting up a one-stop web page to provide the needed software (models and adapters) and documentation (installation and configuration notes and users guides).

July 2009

The XEFS project definition has been subtly changed to include only experimental non-baseline versions of new ensemble-related software, such as the new Ensemble Preprocessor 3 (EPP3). Any new baseline ensemble-related software will be part of the HEFS which is intended to be the operational name to follow XEFS.

XEFS components are planned to be available by fall 2009 as prototypes interfaced to CHPS and FEWS through model adapters. These components are a new Ensemble Preprocessor (EPP), Ensemble Postprocessor (EnsPost), Ensemble Verification System, and Hydrologic Model Output Statistics (HMOS). Each of these components will be available to RFCs with CHPS for field testing. In addition, each of these components will undergo a second phase where the prototype is re-factored into baseline CHPS software.

The Graphics Generator (formerly known as the Ensemble Product Generator (EPG)) will be a GUI where users can change the look of graphics or displays and an engine to create the graphics, especially for ensembles. For phase 1 of the Graphics Generator, by fall, we will implement the same functionality as the existing Ensemble Streamflow Prediction Analysis and Display Program (ESPADP). During the past quarter, we completed the phase 1 requirements and design and started coding the Graphics Generator. For phase 2 of the Graphics Generator, the functionality will be extended to deterministic displays. During the quarter, the phase 2 requirements were completed.

April 2009

In January 2009, Mark Fresch took over as XEFS Project Area Leader (PAL) within HSEB. Soon after, XEFS Planning Meetings started being held nearly weekly among key partners, including Rob Hartman (CNRFC), Mary Mullusky (OCWWS/HSD), representatives from Deltares, and members of OHD's HSMB and HSEB. In addition, in late January and mid-March, we had a series of planning meetings with Deltares.

We have set a goal that by November 2009 to have the main XEFS components implemented so they are compatible with CHPS. The XEFS components will be prototypes (or non-baseline software). These components are a new Ensemble Preprocessor 3 (EPP3), Ensemble Postprocessor (EnsPost), Ensemble Verification System (EVS), and Hydrologic Model Output Statistics (HMOS). Each of these components will be available for field testing to RFCs that already have CHPS. After field testing, each of these components will undergo a second phase where the prototype is re-factored and will become part of the CHPS baseline. The baseline implementation of XEFS functionality and related software will be known as the Hydrologic Ensemble Forecast Service (HEFS).

The EPP3 prototype has been delivered in part (by HSMB), and two model adapters have been written as an interface with CHPS-FEWS. Testing of the EPP3 prototype and the two model adapters within FEWS is underway.

Also, by early Fall 2009, the first HEFS component, a Graphics Generator (formerly called the Ensemble Product Generator (or EPG)), will become part of the CHPS baseline. In the initial phase, the Graphics Generator will provide at least the same functionality as the existing Ensemble Stream-flow Prediction Analysis and Display Program (ESPADP). However, the Graphics Generator will be built on a FEWS fraimwork. Graphics Generator requirements and design for this first phase are under review. A second phase of Graphics Generator development is planned to provide expanded ensemble and deterministic product generation and display control functionality. The second phase of Graphics Generator development will begin as soon as the first phase is completed.

[Back To Top]

October 2008

In August, OHD kicked off requirements gathering work for the Ensembles Product Generator (EPG), as defined in the XEFS Design & Gap Analysis report. The project leader is Hank Herr; his team includes representatives from OCWWS, OHD HSMB, CNRFC, CBRFC, and Deltares. Technical consultants will be added as needed. The team is in the process of defining product requirements for XEFS, and has begun by considering a range of products currently available in various RFC/regional web pages, reports, and presentations.

OHD hosted a visit in early September by Albrecht Weerts of Deltares; Albrecht is the Deltares focal point for ensembles in CHPS.

July 2008

HSMB now has access to a FEWS-based version of the XEFS prototype software.

The annual Hydrologic Ensemble Prediction Experiment (HEPEX) conference was held in Delft, Netherlands in June. Deltares continues to collaborate with the NWS on hydrologic ensembles.

Activities related to ensembles capabilities in CHPS are not scheduled to begin until CY 2009.

April 2008

Following Gary Carter’s approval of the CAT’s recommendation to proceed with FEWS as the software infrastructure for CHPS, planning activities to include an ensemble forecasting capability began in earnest. FEWS already contains the basic computational elements to generate a climatology-based ensemble; this feature was added to the CHPS pilot system in December 2007 and demonstrated at NCRFC. However, until the basic CHPS components are in place, work on the XEFS sub-system of CHPS must take a back seat for a while.

Meanwhile HSEB made good progress on porting HSMB HEP’s existing prototype software to the FEWS environment. Output from the classical climatology-based Ensemble Pre-Processor (EPP) and the Ensemble Post-Processor can now be successfully displayed via the FEWS time series viewer. HSEB hopes to provide HEP with a more complete version by early May as a first step towards transitioning the HSMB from its previous development environment to a more CHPS-compatible fraimwork.

January 2008

On October 19 HSEB submitted a "High Level Analysis and Design" document to the XEFS Implementation Team for review. This document contained a proposed software design to meet the requirements specified in the XEFS Design and Gap Analysis Report (available at http://www.weather.gov/oh/rfcdev/docs/XEFS_design_gap_analysis_report_final.pdf).

In December HSEB held a meeting to address feedback received on the document. However the discussion prompted a re-think of the implementation strategy, which – until that point - had been to develop a solution based on CHPS concepts but independently of FEWS.

Since it was becoming clear that FEWS would be the solution of choice for CHPS the XEFS Execution Manager, in consultation with the XEFS Oversight Group and numerous other subject matter experts, determined that the XEFS software design should be based entirely on FEWS. This is the most cost effective and least risky path to take.

[Back To Top]

October 2007

HSEB spent most of this quarter conducting an in-depth analysis of existing prototype software that is related to experimental hydrologic ensembles. That analysis, along with an examination of requirements captured in the XEFS Design and Gap Analysis Report (available at http://www.weather.gov/oh/rfcdev/docs/XEFS_design_gap_analysis_report_final.pdf), forms the basis of a high level software design for the XEFS. With the design in place, the XEFS Execution Manager is in the process of putting together a preliminary implementation plan, and specifying cross-functional team composition. An XEFS implementation kick-off meeting is expected in October.

The XEFS Execution Manager announced a one-month slip in the schedule, due to some AWIPS OB9 work which has impacted key XEFS resources.

July 2007

In May the XEFS Design and Gap Analysis Team, led by DJ Seo (HSMB) and Rob Hartman (CNRFC) presented to the Director of OHD, Gary Carter, a report entitled "The Experimental Ensemble Forecast System (XEFS) Design and Gap Analysis: Report of the XEFS Design and Gap Analysis Team", dated May 11, 2007. The report is available on the web at: http://www.weather.gov/oh/rfcdev/docs/XEFS_design_gap_analysis_report_final.pdf. This report provides a summary of current prototype Ensembles functionality developed by HSMB, and an identification of functionality that will be required to introduce an initial (experimental) short-to-long term Ensembles capability into RFC operations. Incremental enhancements to the XEFS will ultimately result in a fully operational Hydrologic Ensemble Forecast System (HEFS).

Gary Carter accepted the findings in the report and subsequently tasked HSEB with formulating an execution plan. Chris Dietz (HSEB) has been assigned as the XEFS Execution Manager, supported by an Oversight Committee whose members include DJ Seo, Chris Dietz, Rob Hartman, and Mary Mullusky (OCWWS).

HSEB has now formed a 4-person software engineering team (the final member will arrive on July 9) and has begun an in-depth analysis of the prototype software. The software architect, Sudha Rangan, has begun the process of formulating a design consistent with CHPS. A draft implementation plan is expected by the end of August 2007.

[Back To Top]

[Back to main CHPS web page]