Proposal 4.0: Simple XML Layer Lists

Author: Frank Warmerdam, 3i, warmerdam@pobox.com
Proposed: 1999/6/20
Last Edited: 2000/11/15
Status: Withdrawn

Summary

A simple proposal for querying OGDI datasources for a list of available layers, and information on how to form layer names via the cln_UpdateDictionary() call returning an XML document.

Introduction

At the last meeting I agreed to dig up my old XML layer list proposal and redistribute it. My intention is that it offers a relatively simple fallback position if Edric's "GetCapability" based approach turns out to be unsuitable.

I have prepared an initial draft for an XML DTD for returning layer selection information. The DTD, and example returned XML for GeoTIFF and VRF is attached.

I will include the entire GeoTIFF XML for a datastore with access to two GeoTIFF files named "irvine" and "eltoro" for a simple introduction, and then I will use the VRF example to show a more complicated case.

<?xml version="1.0" ?>
<!DOCTYPE OGDILayerInfo SYSTEM "ogdilayerinfo30.dtd">

<OGDILayerInfo>

  <layersyntax>
    <layertoken type="item">layername</layertoken>
  </layersyntax>

  <layergroup>
    <layer>
      <itemname item="layername">irvine</itemname>
      <defaultname>irvine</defaultname>
      <family>Matrix</family>
      <family>Image</family>
    </layer>
    <layer>
      <itemname item="layername">eltoro</itemname>
      <defaultname>eltoro</defaultname>
      <family>Matrix</family>
      <family>Image</family>
    </layer>
  </layergroup>

</OGDILayerInfo>

The returned information is broken down into two basic sections, the <layersyntax> section and the <layergroup> section.

<layersyntax>

The <layersyntax> information indicates how the layer name is constructed from underlying tokens. This section performs the same function as the SEL_STRING metadata in the Infomar examples, and would feed into the Global Geomatics cln_GetToken() function.

VRF Format gives us a sophisticated example:

<layersyntax>
  <layertoken type="item">class</layertoken>
  <layertoken type="separator">@</layertoken>
  <layertoken type="item">coverage</layertoken>
  <layertoken type="separator">(</layertoken>
  <layertoken type="expression" format="sql_where" default="*" 
              desc="Attribute Expression or *">expression</layertoken>
  <layertoken type="separator">)</layertoken>
</layersyntax>

The <layersyntax> section contains one or more <layertoken> items. Each layer token has a type attribute which must be one of:

 item: The contained text (such as class or coverage) is a variable value
       which is translated to an actual piece of text based on the 
       <itemname> tokens within the <layergroup>s below.  All possible values
       for items will be implicit in the following <layergroup>s. 

 separator: The contained text is literally included in the resulting layer
            name.  While the examples here all show single characters, 
            that isn't necessarily the case. 

 expression: This is a variable piece of text which must be supplied by the
             user but which isn't fully defined within the XML metadata.  The
             most common example of an expression is the attribute query that
             can be passed to the VRF and similar drivers.  A number of 
             additional attributes can be meaningfully attached to an 
             expression token, but they are all optional.
 
               format: A name defining the exact format of the expression
                       from a list of well known formats.  One might be 
                       "sql_where" indicating that an SQL WHERE statement
                       attribute syntax is legal.  

                       While the III should define some well known formats, 
                       it would also be legal for vendors to add their own
                       which might only be meaningful to their own clients. 
                       
	       default: A reasonable default value that can be used for this
                        expression.  In the VRF case a value of "*" for the
                        expression means all features.  

               desc: A text string that could be displayed to the user
                     as a clue to enter values for this expression.

<layergroup>

The <layergroup> and <layer> elements allow description of a heirarchical tree of layers available from this data store. The following shows what a VRF datastore with one coverage (named "col") with two classes (buildnga and landmrka) would look like.

<layergroup>
  <itemname item="coverage">col</itemname>
  <desc>Cultural Landmarks</desc>
  <layer>
    <itemname item="class">buildnga</itemname>
    <defaultname>buildnga@col(*)</defaultname>
    <family>Area</family>
  </layer>
  <layer>
    <itemname item="class">landmrka</itemname>
    <defaultname>landmrka@col(*)</defaultname>
    <family>Point</family>
  </layer>
</layergroup>

<layergroup> / <layer>

The <layergroup> elements can be nested arbitrarily deep though one and two levels of depth would be the most common. The deepest level of <layergroup> will contain one or more <layer> elements which describe actual layers that can be selected with cln_SelectLayer().

<desc>

The <desc> element is a user readable description for a layergroup or layer that could be presented to a user navigating through the available layers. It is not required, and though not shown in this example, could apply to <layer> items as well as <layergroup>s.

<itemname>

The <itemname> element is used to establish the value of one of the <layertoken type="item"> tokens in the syntax string. The item= attribute of the <itemname> relates the value back to the particular token with the syntax string that it applies to. By the time a <layer> has been processed it should be possible to track back through the <layer> and it's parent <layergroup> elements to find values for all "item" syntax tokens in the layer syntax.

The <layer> elements can also have the <defaultname> and <family> elements.

<defaultname>

The <defaultname> element provides a default layer name that could be used with cln_SelectLayer() to access some form of the layer. Simple clients will simply walk the <layergroup> and <layer> tree finding <defaultname>'s so their provision is important. For some layers such as the grid computation layers, there may be no meaningful default name value in which case it would not be provided. Such layers might be inaccessable to simple clients.

<family>

The <family> element may appear one or more times, and lists the feature families that can be selected for this layer. In the above example only one family is supported for each layer, but the geotiff example allows the raster layers to be selected as family <Matrix> or family <Image>. The only family text supported at this time are "Area", "Point", "Text", "Line", "Matrix" and "Image" ; however, clients should be prepared to gracefully ignore unknown family types as new feature families can be expected to be added to the protocol in the future, or as vendor extensions.

<OGDILayerInfo>

The whole regular portion of the document is contained within an <OGDILayerInfo> element, and the document is always prepended with the standard header. The OGDILayerInfo should always have one <layersyntax> tree, and at most one <layergroup> tree. For a datastore such as the computation grid datastore, there may be no <layergroup> at all indicating that layer names can only be constructed by clients which understand the <layersyntax> methodology, and can fill in the expressions.

<?xml version="1.0" ?>
<!DOCTYPE OGDILayerInfo SYSTEM "ogdilayerinfo30.dtd">

<OGDILayerInfo>

  <layersyntax>
    ...
  </layersyntax>

  <layergroup>
    ...
  </layergroup>
</OGDILayerInfo>

Note that DTD (ogdilayerinfo.dtd) isn't downloadable from the driver, but will be available from within the III OGDI specification. In the future, when the DTD is extended the version number embedded within the DTD can be changed.

Clients should generally be written to ignore the DTD document name returned, and to proceed on their best knowledge of the format. They should be permissive (ignore) elements and element attributes they don't understand to be as compatible as possible with future versions of the XML DTD.

Some Issues

XML is case sensitive. I have made almost evertthing but the <OGDILayerInfo> element lower case, but it isn't clear what would be preferred. Perhaps everything should be upper case?
I have not been specific on character encoding. For the time being I think we should limit it to UTF-8, with extension to unicode being desirable in the future. I don't really understand all the implications of this.
I have not addressed the extra layer information that Gilles suggested during the call. I recall the issue of per-layer bounds and resolution information, and a feature count. If the general approach is acceptable I think these should be added as additional optional elements within <layer>.
The current assumption is that all tokens in the <layersyntax> are required. While this appears to follow the Infomar and Global Geomatics approach, I think this is a bit weak. It might be desirable to add a way of indicating that some tokens in the layer syntax are optional. However, this might require adding the ability to group tokens into logical groups that are optional, and other difficulties. I think this would substantially complicate the specification, and client side implementation so I would like to avoid it for now. However, substantial changes in the future are likely to break ``layer syntax aware'' clients written now.
The current approach assumes that one layer syntax will apply to all layers in the datastore. Possibly we should move the <layersyntax> specification within the <layergroup> definition, potentially allowing different kinds of layer syntaxes for differe layer groups. If we intend to move in this direction, but don't want to do so now we could move the layer syntax definition within the root <layergroup> but not allow more than one root layergroup at this time, and not allow layer syntax at other levels.
The global geomatics API function cln_GetDictionary() (which is quite different than the OGDI 3.0 cln_GetDictionary()!) is essentially a function to walk a tree like the <layergroup>/<layer> tree. However, it requires an ecs_Family argument at each request. On the other hand, the data model I am defining doesn't give family information until the leaf notes (<layer>) are encountered. Should the whole layer trees be segregated out into families from the top level?
The Global Geomatics cln_GetRequest() function essentially formats a layername for passing to cln_SelectLayer() based on a the equivelent of a <layer> node (an ecs_DictionaryItem) and an expression value. This implicitly seems to limit the syntax to one expression token at most.
Do we wish to encode such a restriction into this data model?
I have not yet implemented Doug's tag indirection to keep the base tags ``culturally neutral''. Not only am I uncertain of the value of this, I am also not clear on how it is implemented without significantly complicating client implementation. Hopefully Doug can clarify how this is done since the working group did agree that this would be done.
It might be worthwhile to develop a UML model of the above data model. I think it would look alot like the one in Gilles power point document, but with some additions and changes in terminology. I am not a UML guru, and don't have any UML modelling tools, so I will bow out on that. In any event I don't think we should hold things up waiting for this.

Implementation Details

In addition to defining the above XML return format, I would like to see a ``convenience API'' written for the OGDI client library that encapsulates parsing of the format. My original thought was that this new API should exactly follow the definition of the Global Geomatics cln_GetDictionary(), cln_FreeDictionary(), cln_GetToken(), and cln_GetRequest() functions.

However, I am slightly nervous about this due to the different meaning given to cln_GetDictionary() from that in the old API (even though I know the old function was deprecated).

Furthermore, the Global Geomatics API doesn't quite model the generality of what the XML encoding can define, which may be a problem.

Nevertheless, I would like to see a client side API written in upcoming weeks to facilitate adoption of the XML encoding. Furthermore, it would be nice if that API also had fallback rules based on the driver name for drivers that don't implement the new XML metadata (via cln_UpdateDictionary( "LAYERLIST")) properly.

There are a few simple XML parsers available on the net (public domain or equivelent) which could be used for this. A quick look around suggests to me that the traditional expat XML parser from James Clark would be suitable to our purposes, and not significantly bloat OGDI.

I am also happy to prepare additional examples if there are drivers that people don't understand how they would map into this format.

GeoTIFF Example

<?xml version="1.0" ?>
<!DOCTYPE OGDILayerInfo SYSTEM "ogdilayerinfo.dtd">

<OGDILayerInfo>

  <layersyntax>
    <layertoken type="item">layername</layertoken>
  </layersyntax>

  <layergroup>
    <layer>
      <itemname item="layername">irvine</itemname>
      <defaultname>irvine</defaultname>
      <family>Matrix</family>
      <family>Image</family>
    </layer>
    <layer>
      <itemname item="layername">eltoro</itemname>
      <defaultname>eltoro</defaultname>
      <family>Matrix</family>
      <family>Image</family>
    </layer>
  </layergroup>

</OGDILayerInfo>

VRF Example


<?xml version="1.0" ?>
<!DOCTYPE OGDILayerInfo SYSTEM "ogdilayerinfo.dtd">

<OGDILayerInfo>

<layersyntax>
  <layertoken type="item">class</layertoken>
  <layertoken type="separator">@</layertoken>
  <layertoken type="item">coverage</layertoken>
  <layertoken type="separator">(</layertoken>
  <layertoken type="expression" format="sql_where" default="*" 
              desc="Attribute Expression or *">expression</layertoken>
  <layertoken type="separator">)</layertoken>
</layersyntax>
  
<layergroup>
  <itemname item="coverage">col</itemname>
  <desc>Cultural Landmarks</desc>
  <layer>
    <itemname item="class">buildnga</itemname>
    <defaultname>buildnga@col(*)</defaultname>
    <family>Area</family>
  </layer>
  <layer>
    <itemname item="class">landmrka</itemname>
    <defaultname>landmrka@col(*)</defaultname>
    <family>Point</family>
  </layer>
</layergroup>

</OGDILayerInfo>