Question

Need help with GML translation

  • 21 January 2019
  • 5 replies
  • 18 views
drejkzet

Hello everyone reading this. I am new to gml conversion, so I seek help here.

I am trying to read gml files provided with xsd schemas, validate them, insert those data into PostGIS database, do some database processing, then exporting data from database into gml files using same xsd schema (meaning output file should be more or less copy of input gml file, with less or more features)

Importing gml files into PostGIS database is going fine, however I have several problems with exporting data, as the files differ in several places and I am unable to figure out how to duplicate the structure.

Unfortunately I cannot provide xsd file and have to use modified gml, sorry :(

 

 

I have several problems:

gml file looks like this (change extension, website did not allow me to upload gml):

1) FeatureCollection attributes.

snippet of xsd:

<xs:element name="FeatureCollection" type="eczr:FeatureCollectionType" substitutionGroup="gml:AbstractGML"/>
  <xs:complexType name="FeatureCollectionType">
    <xs:complexContent>
      <xs:extension base="gml:AbstractFeatureType">
        <xs:sequence minOccurs="0" maxOccurs="unbounded">
          <xs:element name="FeatureMember">
            <xs:complexType>
              <xs:complexContent>
                <xs:extension base="gml:AbstractFeatureMemberType">
                  <xs:sequence>
                    <xs:element ref="gml:AbstractFeature"/>
                  </xs:sequence>
                </xs:extension>
              </xs:complexContent>
            </xs:complexType>
          </xs:element>
        </xs:sequence>
        <xs:attribute name="constructionCode" type="xs:string" use="required"/>
        <xs:attribute name="schemaVersion" type="xs:string" use="required"/>
        <xs:attribute name="creatorName" type="xs:string" use="required"/>
        <xs:attribute name="creatorVersion" type="xs:string" use="required"/>
        <xs:attribute name="surveyedFrom" type="xs:date"/>
        <xs:attribute name="surveyedTo" type="xs:date"/>
      </xs:extension>
    </xs:complexContent>
  </xs:complexType>

Input gml file has this FeatureCollection element:

<FeatureCollection xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://example.xsd" schemaVersion="2018.0901.0" constructionCode="AAA" creatorName="SW" creatorVersion="1.6.2.101" surveyedFrom="2018-12-06" surveyedTo="2018-12-17" xmlns="http://www.customer.cz" gml:id="20">

in database, I have those attributes stored just fine, I read them just fine (constructionCode, schemaVersion, creatorName, creatorVersion, surveyedFrom, surveyedTo). In worbench I have featuretype "FeatureCollection", when I pass it to the writer with xsd, it manages to get gml:id attribute, however loses every other attribute. Is there a setting I am missing? Every other feature type is created with attributes just fine.

2) Empty attributes

snippet of xsd:

<xs:element name="stavzar" minOccurs="0" maxOccurs="1">
  <xs:simpleType>
    <xs:restriction base="xs:string">
      <xs:maxLength value="25"/>
      <xs:enumeration value="a"/>
      <xs:enumeration value="b­"/>
      <xs:enumeration value="c"/>
      <xs:enumeration value="d"/>
     </xs:restriction>
   </xs:simpleType>
</xs:element>

Means, that feature can have 0 occurences of the attribute, which happens. When I import a feature, that does not contain this attribute into the database, column "stavzar" is created with "null" value. However when I export this feature, element <stavzar/> is created (empty element) - however when I validate the file I get '...on line 3036, column 22: 'value '' not in enumeration' error, meaning the validator sees the empty value, checks enumeration where empty value is not defined and throws an error. Is there a way that gml writer does not create elements, when their value is "null"?

3) Final problem: namespace.

Is there a way in writer settings to replace / disable namespace from elements?

<eczr:FeatureMember>
  <eczr:Pomocne_popisy dmcode="POL.PomocnePopisy" gml:id="1">
    <eczr:OGC_GEOMETRY>
      <gml:Point gml:id="elid_17_4101" srsName="urn:ogc:def:crs:EPSG:5514">
        <gml:pos srsDimension="2">-624378.060000 -1177805.210000</gml:pos>
      </gml:Point>
    </eczr:OGC_GEOMETRY>
    <eczr:OGC_ANGLE uom="radian">1.229934</eczr:OGC_ANGLE>
    <eczr:OGC_TEXTJUST>LB</eczr:OGC_TEXTJUST>
    <eczr:OGC_TEXT>schody</eczr:OGC_TEXT>
    <eczr:emsid>2</eczr:emsid>
    <eczr:stav>stávající</eczr:stav>
  </eczr:Pomocne_popisy>
</eczr:FeatureMember>

I get that "eczr" is taken from xsd, however is there a way to generate that gml without "eczr" at the beggining of each element?

 

Thanks for any help in advance,

Radek

5 replies

deanatsafe
Rank
Badge +6

Hi Radek,

Thanks for your question. Given the in depth nature of your question, its probably best we follow up via our support system. I have created a new support case so you should receive an email shortly.

In the meantime, as you are fairly new to GML, I'll post a few links here for support resources in case you haven't seen them yet. While we don't have a generic GML tutorial yet, we do have a couple of related tutorials for AIXM and INSPIRE. As these readers/writers are based on the GML reader/writer, the basic principles are the same. So these examples should be helpful in terms of learning how to configure the GML writer. The AIXM article for example shows how to write to the dataset container - AIXMBasicMessage in this case and how to nest features using parent child gml ids.

https://knowledge.safe.com/articles/1312/aixm-xml-and-gml-formats-eu-airports-demo.html

https://knowledge.safe.com/articles/573/inspire-gml-reading-writing-and-validation-basics.html

https://docs.safe.com/fme/html/FME_Desktop_Documentation/FME_ReadersWriters/gml/GML_writer.htm

Dean

deanatsafe
Rank
Badge +6

Hi Radek,

I've managed to put together a simple example that illustrates an approach to each of the 3 challenges listed above - see: geonamesgmlwriter2018.zip . You are still welcome to follow up via the support system for questions specific to your dataset and schema.

Question 1: How to expose and preserve or modify the FeatureCollection attributes.

By default, FME's GML writer tends to hide the FeatureCollection container object. However, the following approach will let you expose and preserve or modify it as needed.

First, make sure you set the GML or in this case the INSPIRE reader parameter gml:FeatureCollection to gml.

Next, you will need to import FeatureCollection feature type on both the reader and writer. You may need to set import featureType from schema not dataset. Once you have the FeatureCollection feature type, then on the writer you will need to set these GML writer parameters:

FeatureCollection: FeatureCollection

FeatureCollection ID: AT.HR.IT.0

FeatureCollection NS URI: http://www.opengis.net/gml/3.2

In my example, I'm using the GeographicalNames.xsd application schema which doesn't have any additional FeatureCollection attributes, so I enabled Map Predefined Properties to Yes so I could add a value for the gml_description field to the FeatureCollection object. Notice how each of the NamedPlace features have their gml_parent_id set to AT.HR.IT.0

Question 2: How to suppress null elements in the output

At present, the GML writer does not have a setting for suppressing null outputs. FME does support nillable, so if an attribute is nillable then FME will set xsi:nil=true for null values.

To suppress null writing <gml:name/> in this case, there are 2 possible workarounds. You can use the GMLFeatureComposer to generate the GML, then pass it through the XMLFormatter and set it to 'remove empty elements to Yes. You would then need to use the TextFileWriter to write out your GML.

My preferred approach is to stick with the GML writer, and use a NullAttributeMapper to detect null attributes and map them to 'missing'. This effectively removes the field from the FME feature schema. In the attached example I use the NullAttributeMapper to scan all attributes and map any empty or null attributes to missing. This picks up the gml_name{2} list element and effectively removes it from the list.

Question 3: How to set the default namespace to remove namespace prefix from the element names

In your case, you have <eczr:FeatureMember> and you want this to show up as just <FeatureMember >. To do that you need to make eczr the default namespace.

In my case I have <gn:NamedPlace>. If I want to remove the gn namespace prefix, I need to make the namespace associated with gn the default namespace. Looking at the namespace header of my output I see that:

xmlns:gn="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"

So to remove this namespace we need to set:

Override XSD and Use Namespace Declarations to:

xmlns="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"

Thereafter, the output will have <NamedPlace> not <gn:NamedPlace>

Hopefully this gives you a few things to try. Because you can't share your application schema, I thought it might be useful to have a basic example of GML writing to play with. The above example is based on the INSPIRE GeoNames article. Feel free to edit this workspace for testing purposes and send back your modified version if you run into trouble, or manage to reproduce a problem with it.

If you have questions specific to your data feel free to follow up via the support system.

Dean

nielsgerrits
Rank
Userlevel 6
Badge +32

Hi Radek,

I've managed to put together a simple example that illustrates an approach to each of the 3 challenges listed above - see: geonamesgmlwriter2018.zip . You are still welcome to follow up via the support system for questions specific to your dataset and schema.

Question 1: How to expose and preserve or modify the FeatureCollection attributes.

By default, FME's GML writer tends to hide the FeatureCollection container object. However, the following approach will let you expose and preserve or modify it as needed.

First, make sure you set the GML or in this case the INSPIRE reader parameter gml:FeatureCollection to gml.

Next, you will need to import FeatureCollection feature type on both the reader and writer. You may need to set import featureType from schema not dataset. Once you have the FeatureCollection feature type, then on the writer you will need to set these GML writer parameters:

FeatureCollection: FeatureCollection

FeatureCollection ID: AT.HR.IT.0

FeatureCollection NS URI: http://www.opengis.net/gml/3.2

In my example, I'm using the GeographicalNames.xsd application schema which doesn't have any additional FeatureCollection attributes, so I enabled Map Predefined Properties to Yes so I could add a value for the gml_description field to the FeatureCollection object. Notice how each of the NamedPlace features have their gml_parent_id set to AT.HR.IT.0

Question 2: How to suppress null elements in the output

At present, the GML writer does not have a setting for suppressing null outputs. FME does support nillable, so if an attribute is nillable then FME will set xsi:nil=true for null values.

To suppress null writing <gml:name/> in this case, there are 2 possible workarounds. You can use the GMLFeatureComposer to generate the GML, then pass it through the XMLFormatter and set it to 'remove empty elements to Yes. You would then need to use the TextFileWriter to write out your GML.

My preferred approach is to stick with the GML writer, and use a NullAttributeMapper to detect null attributes and map them to 'missing'. This effectively removes the field from the FME feature schema. In the attached example I use the NullAttributeMapper to scan all attributes and map any empty or null attributes to missing. This picks up the gml_name{2} list element and effectively removes it from the list.

Question 3: How to set the default namespace to remove namespace prefix from the element names

In your case, you have <eczr:FeatureMember> and you want this to show up as just <FeatureMember >. To do that you need to make eczr the default namespace.

In my case I have <gn:NamedPlace>. If I want to remove the gn namespace prefix, I need to make the namespace associated with gn the default namespace. Looking at the namespace header of my output I see that:

xmlns:gn="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"

So to remove this namespace we need to set:

Override XSD and Use Namespace Declarations to:

xmlns="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"

Thereafter, the output will have <NamedPlace> not <gn:NamedPlace>

Hopefully this gives you a few things to try. Because you can't share your application schema, I thought it might be useful to have a basic example of GML writing to play with. The above example is based on the INSPIRE GeoNames article. Feel free to edit this workspace for testing purposes and send back your modified version if you run into trouble, or manage to reproduce a problem with it.

If you have questions specific to your data feel free to follow up via the support system.

Dean

@DeanAtSafe the attached zip got lost (403) could you please re upload the zip? Thanks.

drejkzet

Hi Radek,

I've managed to put together a simple example that illustrates an approach to each of the 3 challenges listed above - see: geonamesgmlwriter2018.zip . You are still welcome to follow up via the support system for questions specific to your dataset and schema.

Question 1: How to expose and preserve or modify the FeatureCollection attributes. 

By default, FME's GML writer tends to hide the FeatureCollection container object. However, the following approach will let you expose and preserve or modify it as needed.

First, make sure you set the GML or in this case the INSPIRE reader parameter gml:FeatureCollection to gml.

Next, you will need to import FeatureCollection feature type on both the reader and writer. You may need to set import featureType from schema not dataset. Once you have the FeatureCollection feature type, then on the writer you will need to set these GML writer parameters:

FeatureCollection: FeatureCollection

FeatureCollection ID: AT.HR.IT.0

FeatureCollection NS URI: http://www.opengis.net/gml/3.2

In my example, I'm using the GeographicalNames.xsd application schema which doesn't have any additional FeatureCollection attributes, so I enabled Map Predefined Properties to Yes so I could add a value for the gml_description field to the FeatureCollection object. Notice how each of the NamedPlace features have their gml_parent_id set to AT.HR.IT.0

Question 2: How to suppress null elements in the output

At present, the GML writer does not have a setting for suppressing null outputs. FME does support nillable, so if an attribute is nillable then FME will set xsi:nil=true for null values.

To suppress null writing <gml:name/> in this case, there are 2 possible workarounds. You can use the GMLFeatureComposer to generate the GML, then pass it through the XMLFormatter and set it to 'remove empty elements to Yes. You would then need to use the TextFileWriter to write out your GML.

My preferred approach is to stick with the GML writer, and use a NullAttributeMapper to detect null attributes and map them to 'missing'. This effectively removes the field from the FME feature schema. In the attached example I use the NullAttributeMapper to scan all attributes and map any empty or null attributes to missing. This picks up the gml_name{2} list element and effectively removes it from the list.

Question 3: How to set the default namespace to remove namespace prefix from the element names

In your case, you have <eczr:FeatureMember> and you want this to show up as just <FeatureMember >. To do that you need to make eczr the default namespace.

In my case I have <gn:NamedPlace>. If I want to remove the gn namespace prefix, I need to make the namespace associated with gn the default namespace. Looking at the namespace header of my output I see that:

xmlns:gn="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"

So to remove this namespace we need to set:

Override XSD and Use Namespace Declarations to: 

xmlns="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"

Thereafter, the output will have <NamedPlace> not <gn:NamedPlace>

Hopefully this gives you a few things to try. Because you can't share your application schema, I thought it might be useful to have a basic example of GML writing to play with. The above example is based on the INSPIRE GeoNames article. Feel free to edit this workspace for testing purposes and send back your modified version if you run into trouble, or manage to reproduce a problem with it.

If you have questions specific to your data feel free to follow up via the support system.

Dean

Hello @DeanAtSafe, thanks for this comprehensive answer!

Question 1: How to expose and preserve or modify the FeatureCollection attributes. 

This here is a bigger problem. I have tried your suggestion, it did not work. I have spent several hours trying to figure out what's wrong with my workspace. The problem that I had boils down to this issue: PostGIS dynamic writer forces lowercase attribute names

I have already reported this problem, it's known, and I can only hope that it gets fixed, because it seriously hinders my plans to do this dynamically, now with each xsd update I have to check, if there are any new attributes that are not lowercase

As you can see in the xsd snippet I have provided, all of the FeatureCollection attributes are snakeCase:

        <xs:attribute name="constructionCode" type="xs:string" use="required"/>
        <xs:attribute name="schemaVersion" type="xs:string" use="required"/>
        <xs:attribute name="creatorName" type="xs:string" use="required"/>
        <xs:attribute name="creatorVersion" type="xs:string" use="required"/>
        <xs:attribute name="surveyedFrom" type="xs:date"/>
        <xs:attribute name="surveyedTo" type="xs:date"/>

however when dynamically writing them do the database, they got converted to lowercase. Therefore when using xsd to map my data, those attributes didn't match, later failing validation.

Once I had renamed those columns in the database (which is still pretty bad solution), everything works.

Question 2: How to suppress null elements in the output 

Even though I have not exposed any attributes, NullAttributeMapper set to map all attributes NULL -> Missing did the trick.  

 

Question 3: How to set the default namespace to remove namespace prefix from the element names 

Overriding XSD and using namespace declarations also solved my problem, very nice!

 

Now to hope that the dynamic writer issue gets fixed. :(

deanatsafe
Rank
Badge +6

@DeanAtSafe the attached zip got lost (403) could you please re upload the zip? Thanks.

Thanks for noticing. Just re-loaded the attachment and tested it - looks fine now.

Reply


Community Stats

28,702
Posts
109,463
Replies
36,550
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.