SDXF

SDXF (Structured Data eXchange Format) is a data serialization format defined by RFC 3072. It allows arbitrary structured data of different types to be assembled in one file for exchanging between arbitrary computers.

The ability to arbitrarily serialize data into a self-describing format is reminiscent of XML, but SDXF is not a text format (as XML) — SDXF is not compatible with text editors. The maximal length of a datum (composite as well as elementary) encoded using SDXF is 16777215 bytes (one less than 16 MB).

Technical structure format

SDXF data can express arbitrary levels of structural depth. Data elements are self-documenting, meaning that the metadata (numeric, character string or structure) are encoded into the data elements. The design of this format is simple and transparent: computer programs access SDXF data with the help of well-defined functions, exempting programmers from learning the precise data layout.

The word "exchange" in the name reflects another kind of transparency: the SDXF functions provide a computer architecture independent conversion of the data. Serializations can be exchanged among computers (via direct network, file transfer or CD) without further measures. The SDXF functions on the receiving side handle architectural adaptation.

Structured data is data with patterns predictable more complex than strings of text.

Example

A commercial example: two companies want to exchange digital invoices. The invoices have the following hierarchical nested structure:

INVOICE
│
├─ INVOICE_NO  
├─ DATE
├─ ADDRESS_SENDER
│    ├─ NAME
│    ├─ NAME
│    ├─ STREET
│    ├─ ZIP
│    ├─ CITY
│    └─ COUNTRY
├─ ADDRESS_RECIPIENT
│    ├─ NAME
│    ├─ NAME
│    ├─ STREET
│    ├─ ZIP
│    ├─ CITY
│    └─ COUNTRY
├─ INVOICE_SUM
├─ SINGLE_ITEMS
│    ├─ SINGLE_ITEM
│    │    ├─ QUANTITY
│    │    ├─ ITEM_NUMBER
│    │    ├─ ITEM_TEXT
│    │    ├─ CHARGE
│    │    └─ SUM
│    └─ ...           
├─ CONDITIONS
...

Structure

The basic element is a chunk. An SDXF serialization is itself a chunk. A chunk can consist of a set of smaller chunks. Chunks are composed of a header prefix of six bytes, followed by data. The header contains a chunk identifier as a 2-byte binary number (Chunk_ID), the chunk length and type. It may contain additional information about compression, encryption and more.

The chunk type indicates whether the data consists of text (a string of characters), a binary number (integer or floating point) or if the chunk a composite of other chunks.

Structured chunks enable the programmer to pack hierarchical constructions such as the INVOICE above into an SDXF structure as follow: Every named term (INVOICE, INVOICE_NO, DATE, ADDRESS_SENDER, etc.) is given a unique number out in the range 1 to 65535 (2 byte unsigned binary integer without sign). The top/outermost chunk is constructed with the ID INVOICE (that means with the associated numerical chunk_ID) as a structured chunk on level 1. This INVOICE chunk is filled with other chunks on level 2 and beyond: INVOICE_NO, DATE, ADDRESS_SENDER, ADDRESS_RECIPIENT, INVOICE_SUM, SINGLE_ITEMS, CONDITIONS. Some level 2 chunks are structured in turn as for the two addresses and SINGLE_ITEMS.

For a precise description see page 2 of the RFC or alternatively here.

SDXF allows programmer to work on SDXF structures with a compact function set. There are only few of them:

To read Chunks, following functions has to be used:

init

To initialize the parameter structure and linking to the existing Chunk.

enter

To step into a structured Chunk, the 1st Chunk of this structure is ready to process.

leave

To leave the current structure. This structure is already current.

next

Goes to next Chunk if exists (otherwise it leaves the current structure).

extract

To transfer (and adapt) data from the current Chunk into a program variable.

select

To search the next Chunk with a given Chunk ID and make it current.

To build Chunks, following functions has to be used:

init

To initialize the parameter structure and linking to an empty output buffer for to create a new Chunk.

create

Create a new Chunk and append it to the current existing structure (if exists).

append

Append a complete Chunk to an SDXF-Structure.

leave

To leave the current structure. This structure is already current.

The following pseudocode creates invoices:

  init (sdx, buffersize=1000);   // initialize the SDXF parameter structure sdx
  create (sdx, ID=INVOICE, datatype=STRUCTURED); // start of the main structure
  create (sdx, ID=INVOICE_NO, datatype=NUMERIC, value=123456); // create an elementary Chunk
  create (sdx, ID=DATE, datatype=CHAR, value="2005-06-17"); // once more
  create (sdx, ID=ADDRESS_SENDER, datatype=STRUCTURED); // Substructure
  create (sdx, ID=NAME, datatype=CHAR, value="Peter Somebody"); // element. Chunk inside this substructure
  ...
  create (sdx, ID= COUNTRY, datatype=CHAR, value="France"); // the last one inside this substructure
  leave; // closing the substructure ADDRESS_SENDER
  ...
  leave; // closing the substructure INVOICE

Pseudocode to extract the INVOICE structure could look like:

  init (sdx, container=pointer to an SDXF-structure);   // initialize the SDXF parameter structure sdx
  enter (sdx); // join into the INVOICE structure.
               //
  while (sdx.rc == SDX_RC_ok)
  {
      switch (sdx. Chunk_ID)
     {
         case INVOICE_NO:
           extract (sdx);    
           invno = sdx.value;  // the extract function put integer values into the parameter field 'value'
           break;
           //
         case DATE:
           extract (sdx);    
           strcpy (invdate, sdx.data); // sdx.data is a pointer to the extracted character string
           break;
           //
         case ADDRESS_SENDER:
           enter (sdx);  // we use 'enter' because ADDRESS is a structured Chunk
           do while (sdx.rc == SDX_RC_ok) // inner loop
            ...
           break;
        ...
     }  
  }

SDXF is not designed for readability or to be modified by text editors. A related editable structure is SDEF - Structured Data Editable Format.

See also

• External Data Representation
• Protocol Buffers
• Abstract Syntax Notation One
• Apache Thrift
• Etch (protocol)
• Internet Communications Engine
• Comparison of data serialization formats

External links

• Introduction chapter of a more detailed description