Schema Formatters and Metadata Views

Date	Oct 15, 2014	Contacts	Jesse Eichar
Status	In Progress	Release	3.0
Resources	Jesse Eichar	Ticket #	650
Source code	https://github.com/jesseeichar/core-geonetwork/tree/formatter-functions
Funding	Swisstopo/Camptocamp

Overview

This proposal will change the formatter api:

So that it will look in schema plugins formatter directory as well as the data directory's formatter directory in order to find all the registered formatters.
So that formatters can be grouped by putting several formatters together in a sub-directory of the root formatters directory. In this case the formatter id will contain a / separator. IE package/identification. Note: A group directory cannot contain a formatter view file.
So that config.properties files can be shared by putting them in the root formatter directory. This makes the most sense when it comes to schema plugins since they will likely share most if not all configuration properties.
By adding a groovy based alternative formatter dialect. This will permit more terse and readable formatter files.

This proposal will also provide default metadata views based on the formatters framework for the default schemas.

Resources on formatters:

Technical Details:

Schema Plugin Formatters

Currently the <data_dir>/data/formatter directory is searched for directories containing a view.xsl file, each of these directories are considered to be a formatter plugin. This proposal will also search each schema plugin's formatter directory for formatters as well. These formatters will be the metadata viewers available for each schema.

Grouped Formatters

In some cases it makes sense to group related formatters. For example ISO19139 original viewer had a group of views based on the sections of the ISO19139 schema: metadata, identification, distribution, etc... This proposal will allow formatters to be grouped in a similar way by putting them together in a directory:

schema_plugins
    iso19139
        formatter
            package
                metadata
                identification
                spatial_rep
                distribution
                maintenance
                ...

Each formatter in a group will be assigned an id with a / separator separating the parts: package/metadata or package/identification for example.

Shared files

Since most views in a schema plugin will share much of its configuration configuration files like config.properties and localization files can be put in the root of the formatter directory of the schema plugin and all formatters in that schema plugin will share those files. In the case of config.properties a particular plugin can override the common properties by including its own config.properties file. The two files will be merged with the formatter specific file take priority.

Groovy Based Formatter

XSLT is rather verbose and many people find it extremely intimidating to approach. As an alternative syntax this proposal will provide a Groovy based DSL for defining the formatter.

A note about Groovy. Groovy is based on Java and virtually all Java syntax is also Groovy syntax so if you are a Java developer it should be extremely easy for you to write Groovy code. Groovy adds to Java many forms for syntax sugar like not having to declare variable types. In the case of XML processing, the dynamic nature of Groovy makes for a very clean API for creating and parsing XML. In addition it is a fairly popular language in the Java community.

The Groovy formatter API will at its most simple allow one to define element handlers and when the element is parsed the handler will be called to produce HTML. The handler can either return a file, a string or html using the HtmlBuilder class.

For parsing the metadata the Groovy XmlSlurper will be used and the syntax for selecting elements and processing elements will be similar, although extra options will be added to make the task of implementing a formatters simpler and cleaner.

The following is a simple example of a formatter. Remember view.groovy files are script so all programming techniques apply (if, when, methods, classes).

// the roots method allows you to select the elements in the document where the recursive processing
// of the xml should begin.  If this is not present then there will be a single root which is
// root element of the metadata.
roots (md.'gmd:identificationInfo', md.'gmd:distributionInfo', md.'gmd:referenceSystemInfo')

// According to Groovy the brackets in a method call are optional (in many cases) so roots can be written:
// roots md.'gmd:identificationInfo', md.'gmd:distributionInfo', md.'gmd:referenceSystemInfo'

// a root can also be added by calling: root md.'gmd:distributionInfo'

// the handle method registers and handler for the identified elements
// there are a few ways to specify what a handler can handle.  
// The first and most performant is to specify the name of the element
// in the following any time a gmd:abstract element is encountered the 
// function/closer will be executed passing in the selected element
// as mentioned above a handle function can return a string, file or XML
// in the case below it returns a string which will be parsed into XML
// groovy has multi-line strings with interpolation (see below)
handle 'gmd:abstract' { el -> 
    // Don't need a return because last expression of a function is 
    // always returned in groovy
    """<p>
         <span class="label">${translate('gmd:abstract')}</span>
         <span class="value">${el.'gco:CharacterString'.text()}</span>
       </p>"""
}

// Methods can be defined which can used anywhere in the script.
// This method will take an element which has gco:CharacterString and/or gmd:PT_FreeText
// children and finds the translation that best matches the UI language
// the UI language is a global variable available to the script as env.lang3 and env.lang2
// where they are the 3 and 2 letter language codes respectively
def isoText = { el ->
    def uiCode = "#$env.lang2.toUpperCase()" // using iterpolation to create a code like #DE
    def locStrings = el.'**'.'gmd:LocalisedCharacterString'
    def ptEl = locStrings.find{it.'@locale' == uiCode}
    if (ptEl != null) return ptEl.text()
    if (el.'gco:CharacterString') return el.'gco:CharacterString'.text()
    if (!locStrings.isEmpty) return locStrings[0].text()
    ""
}
// A second way to define a handler is to provide a function as the first parameter which returns a 
// boolean. (Again you don't need a return because return is implicit) 
//  In groovy functions there is a _magic_ it variable which refers to the single parameter
// passed in.  You can either simply use it in the function or define a parameter like el ->
handle {el -> el.'**'.find{it.name() == 'gmd:PT_FreeText'} { el, html ->
    html.p {
      span ('class' : 'label', translate(el.name()) // translate is a method provided by framework
      span ('class' : 'value', isoText(el))
    }
}

// A handler can also be assigned a priority.  The handlers with a higher priority will be evaluated before
// handlers with a lower priority.  The priority only makes sense when dealing with handlers that have a 
// selector function (as opposed to handlers with element id).
// this example also shows the parameter called processChildren by default it is fail but if it is true
handle select: { it.name().children.size > 0 }, priority: 10, processChildren: true { el, html, children ->
    // children is Future which will contain the children once they have been processed
    // this allows the children to be inserted into the results
    // we are returning a FileResult which has a path to the file as first parameter and takes a map
    // of String -> Object which are the replacements.  When this is returned the file will be loaded
    // (UTF-8 by default) and all parts of the file with the pattern: ${key} will be replaced with the
    // the value in the replacement map.  So in this example ${label} will be replaced with the 
    // translated node name and ${children} will be replaced with the children XML.
    new FileResult("block.html", [label: translate(el.name()), children: children]) 
}

Groovy Strings A note about strings in Groovy. both ' and " create strings in Groovy (like Javascript). However ' creates a literal string where " creates a string allowing interpolation. Interpolation is where the variables and methods are executed within the string and the results are placed in the string. For example if there was a variable count = 5 you could make a string: "There are $count boxes" which would result in There are 5 boxes. It is important to realize that if you use ' instead of " you will not get interpolation.

To call methods you can use the ${...} form around the code and the result from the code execution will be put in the string. For example "The max is ${Math.max(2,3)}" will result in The max is 3.

Multiline String: 3 " or 3 ' together starts a multiline string. As with normal strings """...""" allows interpolation where '''...''' will not use interpolation and what is written is literally what you get.

Shared Functions Since there are likely to be many shared functions between formatters, any functions.groovy or 'functions-*.groovyfiles defined in the rootformatter` directory will automatically be included on the classpath when executing a groovy formatter

XmlSlurper Resources: