Network Protocols and Communications Utilities Under XMSF

There are a number of network protocol and network communications frameworks managed under the XMSF project. The include no less than three Distributed Intractive Simulation (DIS) implementations, and a framework that turns text XML into a more compact, more easily parsed binary format.

Contents

DIS Implementations

Why three? Over the years we've developed them to meet different needs. A quick description of each of them is below.

DIS-XML. This is a more modern implementation and the prefered Java implementation.

The objective of DIS-XML is to introduce another format in which to represent PDUs. Where DIS-Java had two formats--the Java object format and the IEEE-1278 binary format--DIS-XML introduces a third way to represent PDUs, namely XML format. This means that PDUs can be read from the wire in binary format, turned into Java objects, and then written out in XML format. An XML representation of PDUs opens up the DIS world to all the tools used in the XML world, include web services, archiving, XML database tools, and so on.

While DIS-Java used hand-written code, DIS-XML is created using a mix of automatically generated code and some hand-written code. Sun's Java API for XML Binding (JAXB) is used along with an XML schema written to describe the DIS protocol to automatically generate Java language classes that describe the PDU. For example, this is a fragment from the schema that describes DIS:

<xsd:complexType name="Vector3Float">
        <xsd:annotation>
            <xsd:documentation>Denotes 32-bit floating-point triplet</xsd:documentation>
        </xsd:annotation>
        <xsd:attribute name="x" type="xsd:float"/>
        <xsd:attribute name="y" type="xsd:float"/>
        <xsd:attribute name="z" type="xsd:float"/>
    </xsd:complexType>


Sun's JAXB tool will read this schema description and turn it into a Java class that has a class name of "Vector3Float", class comments as described by the xsd:documentation tag, and three instance variables declared as floats with the names "x", ,"y", and "z".

While most of the work of writing the class is done for you, it is still required that the programmer write code to write the PDUs into IEEE binary format, and read them from IEEE binary format into object format. JAXB can automatically read and write Java objects to an XML format compatible with the XML schema, so no programmer effort s required to implement this feature. About twenty of the most used PDUs have been fully implemented. About fifty PDUs have been described in the XML schema file, but not all of those have had IEEE binary serialization/deserialization methods written for them.

DIS-XML is easier to maintain than DIS-Java, and has better memory performance; not as many objects that must be GC'd are generated.  This makes it a better choice for real time applications. The ability to read and write XML also makes it the preferred solution for archiving PDUs.

The features of DIS-XML include:

A page that describes how to download DIS-XML is here. This includes instructions for downloading a pre-built distribution with jar files, and instructions for checking out the latest code from CVS.

XMLPG. XML Protocol Generator (XMLPG) is a third implementation of DIS. This implementation arose from a need for a C++ implementation of DIS for the Delta3D game engine produced by NPS. A hand-written C++ implementation would have had all the same problems as the hand-written Java implementation, namely the volume of code that would need to be maintained. Ideally we would have used the XML schema from DIS-XML plus a C++ tool equivalent to JAXB to generate the C++ classes. Unfortunately, all the C++ schema-to-code implementations I looked at generated C++ code that was a horror show, even by C++ standards.

The solution was to come up with what is essentially an XML template language that describes PDUs. This custom XML file is parsed into what amounts to an abstrct syntax tree (AST), and that AST can then be used to generate source code in any language. I've implemented C++ and Java language output languages.

The process looks like this:

+----------+      +-----+     +------------------+
| XML file |----->| AST | +-->| Java source file |
+----------+      +-----+ |   +------------------+
                          |   +-----------------+
                          +-->| C++ source file |
                              +-----------------+

The programmer writes an XML file that describes the protocol. This is converted by some code into an AST, and that AST can then be used to output either Java or C++ source code. Note that the concept is not limited to just DIS; you can write many network protocols by simply writing the correct XML file and then generating C++ and Java code.

Why not use the XML schema file rather than the special-purpose XML file described above? It turns out that intelligently parsing all the special cases in XML schema is a pain. There's a reason JAXB is big and complex. Rather than deal with the complexity of XML schema I simply punted and came up with a very simple XML file format that did what I needed.

The XMLPG impelmentation of DIS features:

If you want to use C++ DIS, XMLPG is your only choice. It's the only open source C++ DIS implementation I've found. The Java implementation generated by XMLPG is reasonably OK,  has a smaller footprint in terms of the supporting jar files needed, and is reasonably efficient in terms of memory use. But it also gives up the ability to read and write XML, and is unlikely to be maintained as rigorously as DIS-XML.

You can find out how to download a pre-compiled version of XMLPG that includes the generated DIS code, plus instructions for downloading the latest from CVS, here.


DIS-Java The original. DIS-Java is a hand-coded implementation of the DIS (IEEE-1278) protocol written in the Java programming language. Full source code and javadocs are provided with a modified BSD open source license. This is from a conceptual standpoint perhaps the easiest distribution to understand for a new programmer, because it is Just Source Code, but it also has some well-known defects. It features:


As you can guess, this code is somewhat elderly and isn't really maintained any more. While conceptually simple it's also a pain to maintain all the hand-written code. Also, it tends to produce a lot of objects for garbage collection to handle, which makes it not ideal for real time applications. Most of the getter methods for PDU fields are written to return a copy of the field object, rather than the object itself; this is nifty from a programming perspective, because encapsualtion is not violated, but also churns a lot of memory.

The download page for DIS-Java is here. This includes links for downloading a pre-built DIS-Java implementation and instructions for downloading the latest code from CVS.


XSBC

Extensible Schema-Based Compression (XSBC) is a system for converting text XML files into a more compact and more easily parsed binary format. It is similar to several of the candidates in the W3C's Efficient XML Interchange (EXI) working group. The overall objective is to create XML files that are smaller than text files, ideally smaller when gzipped than a text XML file when gzipped, and is faster to parse and in data binding than text XML. This last criteria is not always immediatly obvious. When a piece of XML like this is parsed:

<Position x="1.0" y="2.0" z="3.0"/>

The parser must extract the text representation of 1.0 from the XML, and then convert it to an IEEE floating point representation and associate it with a programming language variable. This process is called "data binding" With XSBC and most other EXI formats, the value 1.0 is already in IEEE floating point format, and the bits can simply be referenced by the name.

Instructions for downloading a pre-built version of XSBC, or checking out the latest version from anonymous CVS, can be found here.