The goal of the Sruth project is to preserve the the best features of the current LDM while addressing its shortcomings. Sruth will focus on minimize bandwidth consumption while supporting dynamic routing, data sharing, arbitrary data-products and modern application conventions.
The LDM has grown to be a robust, reliable and portable base on which to build data distribution networks. As design or implementation limitations were identified, innovative developments have been employed to keep the LDM viable.The current implementation of the LDM when coupled with cluster technology may be able to effectively relay all of the data desired by the expanding Unidata community for several more years. However, input on the limitations in the current LDM design from you, the user community, has convinced us that new, alternate approaches to data distribution must be thoroughly investigated and pursued. The need for a more scalable technology is apparent from several trends:
- The volume of useful data appears to be growing faster than increases in network bandwidth;
- The explosion in the number of types of data that don't conviently fit into a limited number of feedtypes;
- Approaches to setting priorities for bandwidth by "shaping" its use, charging departments for bandwidth, or other institutional data policies may disrupt the model of essentially free bandwidth currently in use for the IDD; and
- Competition for network bandwidth from the explosive growth in video distribution may demand more dynamic routing than is now possible with current LDM technologies
To avoid interruption in data delivery, future needs must be anticipated and reliable software must be developed well before the need is apparent. Our experience suggests the current LDM-6 may have advanced about as far as it can given the constraints of its architecture.
Please be assured that the development of Sruth does not imply that incremental improvements to the current LDM-6 will cease!
However, further major advances in the LDM may require a new protocol that is not tied to the client/server approach but instead uses peer-to-peer concepts. A new protocol and implementation could allow for the following improvements:
- More dynamic creation and destruction of data-product streams;
- Support for access to "one-time" data products (i.e., data-products that are not continuously generated);
- Better load balancing of communication links;
- More adaptive and flexible dynamic routing of data-products with steady-state results that are relatively independent of the configuration of initial connections;
- A better interface for selecting data-products. For example, reception of a data-product stream could be started by clicking on a hyperlink in a web page; and
- Support for use on computing platforms beyond those being used in the community for data relay tasks (e.g., Windows).
Naturally, minimizing disruption — both to individual sites and to the flow of data — is a major concern of any new implementation and deployment. The LDM software is open source, so no one will be forced to upgrade to different and incompatible software. In fact, there could be two competing distribution systems running in parallel for some time, during which the new system will have to prove itself capable of satisfying users of the current LDM as well as providing solutions to problems the current LDM cannot handle.