Calimero NG
Calimero NG
Calimero NG
Wolfgang Kastner
Calimero is a Java library for access to KNX systems introduced in 2005. Its next generation provides numerous additional features. An improved architecture ensures continued ease of use. Increased modularity is provided by a feature bundle concept. Licensing options were extended. This paper presents Calimero Next Generation, discussing its design and benets. Special emphasis is placed on key parts of the new API.
1 Introduction
Calimero is a Java library for KNX access. It was rst presented to the public at the KNX Scientic Conference 2005 as a part of the KNXLive! project [1]. This project offers easy access to KNX systems, middleware, and development tools based on a Linux Live-System (Knoppix, [2]). From the beginning, Calimero was also available on SourceForge (versions 1.x, [3]). The packages included in this distribution provided a KNXnet/IP (EIBnet/IP) [5] Discovery and Tunneling client, APDU (application layer protocol data unit) type translation services essential for runtime interworking, and a simple group address storage using XML together with a GUI based demo application. Ease of use was a key design goal. Client applications were enabled to communicate with KNX devices via a high level API hiding network protocol details. Since then, development on Calimero continued. When considering new functionality to be included, the question arose how these features could be added while maintaining both ease of use and a compact footprint. Therefore, a considerable reorganization effort was undertaken, leading to a redesign of the user API as well as internal architectural aspects. The new Calimero NG API, in the following only referred to as Calimero, now supports additional connection protocols, management, property access, high level convenience methods for common tasks, and more.
The Calimero library now only requires the Java 2 platform, Micro Edition (J2ME) APIs. Supporting J2ME enables deployment on resource constrained devices, such as embedded systems. Modularity was improved further by introducing feature bundles grouping related functionality. These bundles are loosely coupled, interdependencies were eliminated wherever possible. This concept helps the user in understanding the API. Moreover, it enables tailoring the library to a specic task since only bundles relevant to the application need to be included. Ease of use is maintained by a bundle offering high level convenience methods as a single point of access for common user tasks. Extensibility of functionality continues to be an important aspect. The architecture was designed to allow new features or alternative implementations of services to be added seamlessly. Besides KNXnet/IP Tunneling, Routing is now provided as an additional connectivity option. Network Address Translation (NAT) is supported as specied in the KNXnet/IP documentation. A generic KNX network access link abstraction was introduced to easily accommodate further types of network access units and the protocols they use. This abstraction also shields higher level services from connection and medium specic details. Support for network and device management was added. This includes support for KNXnet/IP Local Device Management as well as KNX application layer property services. The Calimero property client provides a unied interface for accessing Interface Object properties, both via local device management or remote property services. An API for buffering incoming KNX network messages was added, considering the difference between state semantics (caching, to improve performance) and event semantics (queueing, e.g., to support client polled applications). Logging is now supported throughout Calimero, using a generic logging API for various kinds of events such as errors, status or progress information. To introduce certain features in Calimero and give a survey of the library, example console based tools are published. Each of these covers a specic use case (such as reading a group value), highlighting the required APIs in tutorial fashion. Most Calimero functionality is now backed up by unit tests. Their aim is to provide an automated way to test if the corresponding specications are implemented correctly, to check for regression issues when extensions or changes are made, and to trace problems in a controlled environment. This paper presents the architecture behind these features and shows how to use the new API by way of short code examples.
2 Architecture
When writing software that accesses a KNX network, it is necessary to bridge the gap between the application and the KNX medium. This can be done with a variety of protocols, like working over an IP network interface or using a serial cable connection to a bus coupler. Additionally, the KNX network can use different communication media, requiring appropriate medium specic formats. It would be benecial for higher level services in Calimero to rely on a well dened interface abstraction that encapsulates and hides those kinds of differences. As a consequence, the entire design of Calimero follows an architecture also known as the waist-line architecture. Figure 1 shows how, in this architecture, functionality is divided into three layers. Implementations of basic services in our case, the range of network access protocols are located at the bottom. In the middle, the network link abstraction offers a homogeneous, standard interface for communication with KNX networks, hiding the chosen basic service. This link, with its slim, stable API, is considered the waist in the architecture. Building upon the link abstraction, high level services ease development, and users create their applications depending on a link, a high level service, or both.
Calimero follows a non-strict layering approach. Even if components build on one another, the user is not forced to interact with those at the top level only. Thus, a user is able to choose the desired level of API abstraction. Corresponding to particular application requirements, he may opt for convenience or detailed options. On the one hand, he could use a high level API such as the Process Communicator that provides common tasks out of the box. On the other hand, he could opt for the low level KNXnet/IP interface if a non-blocking communication mode is required. High level services using the link abstraction can immediately and transparently make use of new network access protocol implementations as these are added. Also, most relevant interfaces have been designed to allow extensibility, for example by adding new cEMI types, additional application layer services to be used in the process communicator, or datapoint types.
no connection specic parameters are necessary at all, just an open link implementing the KNXNetworkLink interface. The same link object can also be supplied to other higher level services for shared use. Although the key intention of the network link interface is to provide a medium independent abstraction, passing medium related details specic to single KNX messages should not be prevented. Therefore, a link supports two ways of sending a frame. First, it will accept a network layer service data unit (NSDU) plus a KNX destination address. In this case, the link creates the remaining KNX message frame structure, using its knowledge about the medium and specied defaults. All Calimero higher level components use this NSDU API. As an alternative, the link also accepts link layer cEMI messages. For this purpose, Calimero provides types which are aligned with cEMI message structures (described below). The cEMI message format (common external message interface, [4]) is leveraged as a standard way of accommodating all possible medium specic aspects. This way, a user can control most details of an outgoing message. He could even add additional information records to a message without making changes to Calimero. Messages incoming from the KNX network are always provided in cEMI format through the link abstraction. Since cEMI encapsulates medium-independent information in a generic way
independent of any medium-specic details which may be present message receivers are free to choose the abstraction level appropriate for them.1 Since KNX devices in bus monitor mode do not allow outgoing transmissions, an additional link interface is available for this purpose. In contrast to KNXNetworkLink, KNXNetworkMonitor only supports receiving bus monitor indications. Such a Calimero monitor optionally decodes incoming raw frames from the medium. Medium dependent message types parse control information up to (and including) the network layer. Since TPDUs (Transport layer PDUs) are medium independent, the same parts of Calimero that are used for standard communication can be used to decode them.
While medium dependent aspects can be ignored when receiving a message, this is not the case for sending it. Therefore, it would not make sense to let high level services create medium independent cEMI messages for transmission, since they would have to be re-packed into a medium dependent type anyhow.
To reduce memory consumption, the reader uses a pull parser that reads the input piece by piece, with the application processing data as it is parsed. The logging framework allows ltering according to multiple severity levels. Logging to a stream, a le, or a network socket is possible. It is designed to be generic enough to be useful as the single logging facility for smaller user applications. On the other hand, log output can easily be integrated into any preferred application global logging facility. Besides keeping the required runtime environment compact, another design goal was to allow the user to reduce the size of the Calimero library itself. He should have the ability to actually remove unused library parts by merely deleting class les, without touching the source code. The rst step towards achieving this goal was to arrange functionality in a highly modular way. Services related to each other were grouped to feature bundles. Each feature bundle addresses a specic task; dependencies between bundles were kept to a minimum. Bundles form a dependency hierarchy, with high level services relying on basic services. Cyclic dependencies are avoided. This particularly clean design makes it easy to grasp which bundle dependencies are present. Within the hierarchy, bundles with no incoming dependencies can be safely removed. Such a removal just reduces the library by the features of the respective bundle. On the other hand, when a bundle (let us call it A) depends on another (let us call this one B), it usually makes no sense to remove B since A would not be able to do anything useful without the functionality provided by B anyway. However, there are cases where A would just lose a nice-to-have feature, but could still provide its core function. In Calimero, this is the case with XML handling. XML handling is used widely throughout the library for data persistence. Usually, this would create static dependencies between the XML bundle and many others. To remove this dependency, the bundles top package (in the Java package hierarchy) only contains interface declarations, with the actual service implementation moved to a sub package. Other bundles only reference these interfaces and obtain the actual implementation classes from a factory, which is also included in the top package. Thus, the bundle can be removed by eliminating the sub package, with only the small bundle stub containing the interfaces remaining.
package tuwien.auto.calimero.xml; // stub, contains XML interfaces and factory package tuwien.auto.calimero.xml.impl; // the actual implementations are contained in subpackage(s)
This approach allows the user to remove the XML bundle when no XML data needs to be handled (e.g., in an embedded system) or persistence is managed in the user application. Additionally, it enables Calimero to switch to a different (e.g., validating) parser in the future. Logging is another classic example of a cross-cutting concern. However, the Calimero logging bundle is lean enough that the complexity added by separating implementations and interfaces would outweigh the benets. Therefore, it is considered an integral part of Calimero and cannot be left out without leading to compile time errors. Figure 2 shows the static dependencies present between Calimero feature bundles, with arrows representing the direction of the dependency. A chain symbol indicates a dependency, while a scissors symbol indicates that no dependency is present. For example, the sixth row
Management
DPT Translators
KNXnet/IP
Datapoint
Process
Link cEMI KNXnet/IP Process Datapoint DPT Translators Management XML Logging
shows that the DPT translator bundle only depends on the logging bundle. As another example, the rightmost column shows that every bundle in the library depends on the logging bundle. Besides footprint issues, the feature bundle concept has the benet of giving the library an easily comprehensible structure. It allows a developer to focus on features that are of particular interest to him. The new console tool examples take the same line, focusing on a single aspect of the API at a time (besides, they also work in a J2ME environment).
Logging
cEMI
XML
Link
The Discoverer offers KNXnet/IP server discovery and self description. The discovery API can be used in nonblocking or blocking mode, optionally with a timeout limit for remote discovery endpoints to respond. It is possible to use a specic outgoing local network interface for search requests. A search can also automatically be performed on all available network interfaces. KNX messages are handled using appropriate cEMI message types as described in Section 2.2. For outgoing messages over KNXnet/IP connections, three different kinds of blocking modes are available: non-blocking, wait-for-acknowledge and wait-for-cEMI-conrmation. In non-blocking mode, no response is waited for; the user has to mind any protocol tasks. With wait-for-acknowledge, the send method blocks until the KNXnet/IP service acknowledge response, handling all necessary KNXnet/IP protocol tasks such as retransmission. With waitfor-cEMI-conrmation, the send method blocks until the conrmation is received, implementing cEMI ow control [4, p. 10]. Since Routing is an unreliable protocol by design, only non-blocking mode is supported here.
4 DPT translators
DPT translators bridge the gap between low-level data representations on the KNX network (KNX datapoint types, DPTs [7]) and Java data types. A KNX datapoint type consists of a data type and a dimension. The data type species the format and encoding, while the dimension describes a range and unit associated with a particular data type. Datapoint types are identied using a main number referring to the data type, and a sub number referring to the dimension. One Calimero DPT translator handles a single KNX main number. It also provides a list of the available KNX datapoint types (i.e., dimensions) for that data type. The translator encodes and decodes data items according to the additional semantic information provided by one of these DPTs. New or modied DPTs can be added by the user. Available DPTs could also be removed. A datapoint type is represented by the DPT type. A DPT stores semantic information about one KNX data point type behind a unied interface, consisting of the unique DPT identier, a general description, unit of measure and a lower and upper value representation in String format, specifying the value range. The general description largely follows the DPT names in [7], but is edited for readability. For example, assume that dpt refers to the KNX DPT 5.003 from the 8-bit unsigned translator:
dpt.getID(); dpt.getDescription(); dpt.getLowerValue(); dpt.getUpperValue(); dpt.getUnit(); // // // // // returns returns returns returns returns "5.003" "Angle" "0" "360" " "
DPTs may subtype DPT for offering additional type specic information. This applies to composite DPTs in particular. For example, B1 U3 (3.007 Control Dimming) provides the B1 control Bit type information. To congure a translator to use a particular DPT, the DPT is supplied on translator creation:
DPTXlator t = new DPTXlator8BitUnsigned(dpt);
The API of each DPT translator supplies Java data types suitable for the data type (main number) it covers. For example, the Java type boolean is provided for the KNX data type B1 . All translators accept and return value representations in String format. Both for string output as well as for dimensionless input, the dimension information of the DPT is applied automatically. Thus, a user can rely on real-world value representations exclusively, without even being aware of the data type encoding. The string representation (e.g., 250.4 lx) is interpreted by the translator using DPT information and offered as a KNX data byte array (and vice versa). The XlatorTypes interface maintains a mapping between all available KNX data types (identied by their main number) and their corresponding translator implementation classes, together with an optional description. It can be used for lookup or to request a specic translator given a main number and sub number or a DPT ID. These map entries can be added, modied, or removed. When working with KNX Interface Objects, property values need to be translated. The DPT translator bundle also addresses this requirement. A Property Data Type (PDT) describes the type of such a property. PropertyTypes can be used to create and obtain an appropriate translator for a given PDT. It maintains PDT to DPT map entries, initially associating PDTs with a DPT suitable for canonical representation (and sufcient for most translation purposes). Again, the map entries can be modied. A property value might not only consist of a single element, but also of an array of elements of the same encoding type. This is taken into account in DPT translators by supporting containment and translation of multiple items of the same type, for example by supplying a Java byte array with KNX data containing multiple elements.
5 Process communication
Process communication refers to the exchange of process-related data (e.g., sensor values or actuator commands). In KNX, process communication de facto exclusively uses group communication. A number of steps are necessary for sending a message to the KNX network to engage in process communication: The data value (or command) has to be transferred to its KNX representation. This is done by the DPT translator bundle already presented. The appropriate application and transport layer control information has to be added to create a TPDU. Control information for the lower layers has to be obtained. At the least, this information consists of a destination group address. All this information has to be supplied to a KNXNetworkLink, which assembles the frame and transmits it to the network. The same procedure has to be performed the other way round for receiving. Each of these tasks can be performed on different levels of abstraction via specic APIs available in Calimero. However, Calimero also offers a simplied one-stop API. In addition, a sophisticated framework for buffering data incoming from the network is provided. Both make use of a network conguration database.
5.1 Datapoints
The Datapoint bundle enables Calimero to store information about the KNX network conguration for process communication purposes. In KNX, process communication endpoints are identied by group addresses. Thus, the Datapoint type holds semantic information associated with group addresses for use by the user application or high level services within Calimero. The set of Datapoints is provided through a simple interface offering datapoint lookup, enumeration and persistence in XML format. A Datapoint is uniquely identied by its associated main group address. It also holds a user-supplied friendly name as well as the KNX DPT main number and sub number. Thus, a Datapoint possesses the knowledge to identify KNX messages belonging to it and to interpret them correctly. With Calimero NG, Datapoints store further useful information. This includes transmission priority and the possibility to distinguish between state based and event based datapoints. State based datapoints describe a process value (switch is off). Such states persist over an interval of real time. Event-based datapoints transport commands (turn 20 degrees). These events are associated with a single point in time. Often, a state based datapoint can be assumed to hold its value until an update request for this datapoint appears on the network. However, it may also change its value in response to time passing (e.g., the status output of an actuator implementing a timer switch) or communication activity on other datapoints (e.g., the output level of a dimmer when a "dim up" command is received). Depending on the network conguration, these changes may or may not be announced over the network. For this purpose, a state based datapoint may optionally contain a time-out interval and a list of invalidating group addresses. Incoming group value write indications with their address in the invalidating list will invalidate the current data point value, i.e., the value is marked to not be up-to-date anymore. In special cases, the state based datapoint may be known to take over a value from another group address (e.g., the status feedback output of a switching actuator being deactivated by an all-off command). To cover these situations, a state based datapoint may contain a list of updating group addresses.
The ProcessCommunicator offers a blocking API with a collection of convenience methods for reading and writing group values. It is also possible to listen for incoming group value write indications. If required, a user dened response timeout for read-methods to complete can be specied. All APDU and type translation issues are hidden. For example, reading a boolean from group address 1/2/4 is as easy as
10
will provide a human-readable String describing the value held by the datapoint referenced by aPoint.
11
12
Property Interface
1. Create property access adapter
Network access
PropertyClient PropertyAdapter
192.168.0.10
..0010110..
network KNX
new(
KNXnet/IP Local DM
3.1.1
..1010011..
Optionally, it is possible to load and save property denitions to be used by the property client for lookups and automatic property type translation. Such a denition contains useful information like the PID, PID name, a friendly name of the property, object type, PDT and DPT (not all of it required). The denitions can also be used for providing a more descriptive user interface. These denitions may be loaded from (and saved to) any resource, such as a le, network stream, temporary storage or similar. By default, a le handler for loading and saving property denitions using XML is provided. Calimero supplies all property denitions in XML format listed in [8]. Nevertheless, a user may choose to specify a different resource handler for loading and saving denitions.
13
bution can now obtain an additional license. This license allows including Calimero in a project without having to release its entire source code, but still requires modications to Calimero to be made available. These licenses are individually negotiated. Regarding future work, efforts are already being made to at least partially support the draft KNX XML schema [9]. This data encoding will be highly valuable for the exchange of conguration data with ETS. Also, additional console tool implementations will be provided. Although the existing codebase would be very well suited to implement USB connectivity, this is currently not an option since communication via USB is not supported in current Java runtime environments. However, a BCU 2/FT 1.2 connection would be an interesting extension (although it would probably require the Java Standard Edition). In any case, we are looking forward to seeing Calimero develop in close collaboration with its user community.
References
[1] KNXLive! project Website, https://www.auto.tuwien.ac.at/a-lab/knxlive.html [2] KNOPPIX Live Linux Filesystem On CD, http://www.knopper.net/knoppix/ [3] Calimero EIBnet/IP Tunnelling (and more) for Java, http://calimero.sourceforge.net [4] KNX AN033, Common EMI Specication, v03 AS and KNX AN066, cEMI Adaptations, v03 AS [5] KNX Handbook, Vol. 3, Part 8, Ch. 15, System Specications: EIBnet/IP, v1.2 DV [6] KNX Handbook, Vol. 3, Part 8, Ch. 2, System Specications: EIBnet/IP: Core, v1.2 DV [7] KNX Handbook, Vol. 3, Part 7, Ch. 2, System Specications: Interworking: Datapoint Types, v1.2 DV [8] KNX KSG259: Identiers, 20061003a [9] KNX Handbook, Vol. 3, Part 8, Ch. 10, System Specications: KNXnet/IP: XML Data Encoding, v0.7 DP
14