Rubbish data in, rubbish data out – it is an inescapable fact that you can’t do a great deal with rubbish data, no matter how clever your axioms are. What you can do however, is work out how rubbish and why that is rubbish. After the usual checking of data structure, schema, basic geometry and syntax – you can now use business rules to check that the data content makes sense for your domain and usage. The reason for this post? Well, perhaps unsurprisingly, the team here at Snowflake has been quietly working on a set of configurable business rules based XML and GML validation web services.
Business Rules for XML Validation
The business rules are encoded as XML and are based on a series of constraints and domain specific statements, linking attributes and values together to provide a contextual checking of data – resulting in a validation report that can be stored or dealt with automatically. An example business rule, taken from our rendering of the ICAO legislation for aeronautical data quality, is shown below:
<sch:pattern> <sch:rule context="//aixm:ElevatedCurve"> <sch:assert test="not(./aixm:elevation and not(./aixm:elevation[@xsi:nil='true'])) or (./aixm:verticalDatum) or (./aixm:verticalDatum[@xsi:nil='true'])" diagnostics="std_err"> [Data consistency rule] ICAO Annex 10 - Volume 1 chapter 1 Definitions - generic rule 53 - ElevatedPoint_elevation_verticalDatum :: Each ElevatedCurve that has a elevation must have a verticalDatum. </sch:assert> <sch:assert test="not(./aixm:elevation) or (./aixm:elevation[@xsi:nil='true']) or (./aixm:elevation[@uom])" diagnostics="std_err"> [Coding rule] AIXM Model - Template - Mandatory UOM :: Each ElevatedCurve that has a elevation must have elevation.uom. </sch:assert> <sch:assert test="not(./aixm:horizontalAccuracy) or (./aixm:horizontalAccuracy[@xsi:nil='true']) or (./aixm:horizontalAccuracy[@uom])" diagnostics="std_err"> [Coding rule] AIXM Model - Template - Mandatory UOM :: Each ElevatedPoint that has a horizontalAccuracy must have horizontalAccuracy.uom. </sch:assert> </sch:rule> </sch:pattern>
The practical use of these rules is via a schematron engine running within a web service workflow, and in this way many rules can be developed and built up corresponding to a specific GML based exchange model enabling the service providing the data to meet data quality legislation or community driven rules for data quality. This example uses aviation, the AIXM model and the ICAO rules, but the same concepts can be applied to maritime for S-100 and INSPIRE for national data sets.
To talk to one of the Snowflake team about our business rules based validation web services, drop us a line at firstname.lastname@example.org
About The Author
Chartered Engineer with the IET and Principal Consultant with Snowflake Software Ltd, Alexis James Brooker is based in the UK, with extensive experience of technical project delivery, business development and consulting in the software and open standard web services domain. Alex’s current business development role spans the Defence and Maritime sectors of Snowflake’s business.
You can follow Alex on twitter @alexisbrooker or search for him on LinkedIn and G+