Skip to main content

I have tested 7 different ways to download all WFS points within a geometry and all give an error using FME 2023.0

 

Tried the "Initiator OGC-Intersects Result", "Bounding Box OGC-Intersects",

Tried settings a Filter

<Filter><Within><PropertyName>Geometry</PropertyName> ... GML Polygon Data ... </Within><Filter>

 

Filter Overlaps etc.

Filter BBOX with and without namespace declarations

Filter Envelope

 

Nothing seems to be the right settings. I remember having the right parameters when using FME2019 but that doesn't seem to work anymore.

 

Maybe my service is not responding according to specs?

https://geodata.nationaalgeoregister.nl/inspire/ad/wfs?request=GetCapabilities&service=wfs

 

 

Maybe this has something to do with the fact that FME 2023.0 is not yet officially released, still in Beta?

What happens if you perform the same test in the latest official release, 2022.2?


Maybe this has something to do with the fact that FME 2023.0 is not yet officially released, still in Beta?

What happens if you perform the same test in the latest official release, 2022.2?

FME 2019 and FME2022.2.2 give the same errors.

 

But I made a little progress. This URL:

https://service.pdok.nl/lv/bag/wfs/v2_0?request=getCapabilities&service=WFS

gives back 'panden' and/or 'verblijfobjecten' when I request the data with the correct filter options.

 

So it seems the INSPIRE url is not working according to specs. QGis also has problems loading the data.

 


Aha, all right. So, it doesn't have anything to do with FME at all. The WFS service you tried to access in your original post has been discontinued altogether by the supplier as of January 19, 2023. The dataset you are after (Adressen (INSPIRE geharmoniseerd)) is no longer available as a WFS, but only as a WMS or as an ATOM Download. See these messages on the PDOK site for further information:

A good source to keep an eye on changes in the data offerings by PDOK might be this Geoforum (discussion mostly in Dutch)


Aha, all right. So, it doesn't have anything to do with FME at all. The WFS service you tried to access in your original post has been discontinued altogether by the supplier as of January 19, 2023. The dataset you are after (Adressen (INSPIRE geharmoniseerd)) is no longer available as a WFS, but only as a WMS or as an ATOM Download. See these messages on the PDOK site for further information:

A good source to keep an eye on changes in the data offerings by PDOK might be this Geoforum (discussion mostly in Dutch)

Thank you for the explanation. I use the QGis - PDOK Services Plugin a lot. So I looked up this URL there:

https://github.com/rduivenvoorde/pdokservicesplugin/blob/master/pdokservicesplugin/resources/layers-pdok.json

But that list is behind in maintenance.

 

Any change I can geofilter the Atom dataset? I will use the BAG WFS for now.

 

 


The whole idea behind this Atom service is that you have to download the dataset first to be able to process it locally. It is a zipped gml file, 800 MB in size.


Mmh, the WFS available under this url works (but not really fine):

https://geodata.nationaalgeoregister.nl/inspire/ad/wfs?

I can access the following feature types:

  1. Address (points)
  2. AdressAreaName
  3. PostalDescriptor
  4. ThoroughfareName

 

Imho, the service doesn’t return any useful attributes, expect of the postCode in PostalDescriptor.

 

But coming back to your question: most WFS are limited to a certain number of features. Most of them to 30.000 features. The reason is, that WFS are returning GML (XML) documents and these will get too large.

 

To bypass this limit, you’ll need a filter.

Either a spatial filter, or an attribute filter. You tried a spatial filter using a bounding box.

 

Sorry Safers, but the Spatial Filters in FeatureReader in combination with WFS don’t work as expected!

If you have a look into the logfile, you’ll see, that FME sends a BBOX requests in most cases.

And even the BBOX requests doesn’t work properly with this certain WFS.

 

But, this BBOX filter works as expected:

<fes:Filter xmlns:fes="http://www.opengis.net/fes/2.0" xmlns:gml="http://www.opengis.net/gml/3.2">
  <fes:BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </fes:BBOX>
</fes:Filter>

bboxOthers like „Within“ or „DWithin“ are returning errors - from the WFS. 

Maybe, it is not implemented properly?

Kind regards

Oliver


Here's a running DWithin Filter for another WFS.


Mmh, the WFS available under this url works (but not really fine):

https://geodata.nationaalgeoregister.nl/inspire/ad/wfs?

I can access the following feature types:

  1. Address (points)
  2. AdressAreaName
  3. PostalDescriptor
  4. ThoroughfareName

 

Imho, the service doesn’t return any useful attributes, expect of the postCode in PostalDescriptor.

 

But coming back to your question: most WFS are limited to a certain number of features. Most of them to 30.000 features. The reason is, that WFS are returning GML (XML) documents and these will get too large.

 

To bypass this limit, you’ll need a filter.

Either a spatial filter, or an attribute filter. You tried a spatial filter using a bounding box.

 

Sorry Safers, but the Spatial Filters in FeatureReader in combination with WFS don’t work as expected!

If you have a look into the logfile, you’ll see, that FME sends a BBOX requests in most cases.

And even the BBOX requests doesn’t work properly with this certain WFS.

 

But, this BBOX filter works as expected:

<fes:Filter xmlns:fes="http://www.opengis.net/fes/2.0" xmlns:gml="http://www.opengis.net/gml/3.2">
  <fes:BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </fes:BBOX>
</fes:Filter>

bboxOthers like „Within“ or „DWithin“ are returning errors - from the WFS. 

Maybe, it is not implemented properly?

Kind regards

Oliver

Thanks for finding the correct way to request the data. Seems the WFS is pretty strict in what it will answer to.

 

For additional information:

The FES namespace declaration may be omitted. But the GML namespace must be declared. Using the BoundsExtractor for the @Value(xmin) etc. in this example just like you did.

<Filter xmlns:gml="http://www.opengis.net/gml/3.2">
  <BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </BBOX>
</Filter>

It is important to use the WFS 2.0 specification and in this case setting the Start Index to 0 (zerro) and the Count to 500. The service responds with somewhere between 300 and 500 features. This settings will request multiple pages to receive all data.

 

As you pointed out the dataset does not respond with expected address information ether. Just a housenumber in this field locator{0}.AddressLocator.designator{0}.LocatorDesignator.designator

 

So the service has a pretty bad service.

 

But as you pointed out as well. It seems that the implementation of the Spatial Filter on the main page of the FeatureReader is not the best working implementation by FME Workbench for Spatial Filtering a WFS Service.

 

And I could really use some more working examples for the Filter function within the Parameters... section of the FeatureReader.


Thanks for finding the correct way to request the data. Seems the WFS is pretty strict in what it will answer to.

 

For additional information:

The FES namespace declaration may be omitted. But the GML namespace must be declared. Using the BoundsExtractor for the @Value(xmin) etc. in this example just like you did.

<Filter xmlns:gml="http://www.opengis.net/gml/3.2">
  <BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </BBOX>
</Filter>

It is important to use the WFS 2.0 specification and in this case setting the Start Index to 0 (zerro) and the Count to 500. The service responds with somewhere between 300 and 500 features. This settings will request multiple pages to receive all data.

 

As you pointed out the dataset does not respond with expected address information ether. Just a housenumber in this field locator{0}.AddressLocator.designator{0}.LocatorDesignator.designator

 

So the service has a pretty bad service.

 

But as you pointed out as well. It seems that the implementation of the Spatial Filter on the main page of the FeatureReader is not the best working implementation by FME Workbench for Spatial Filtering a WFS Service.

 

And I could really use some more working examples for the Filter function within the Parameters... section of the FeatureReader.

I think, I found the reason for the malfunction.

Just open the GetCapabilities document and search for <fes:Filter_Capabilities>

https://geodata.nationaalgeoregister.nl/inspire/ad/wfs?SERVICE=WFS&amp;REQUEST=GetCapabilities&amp;VERSION=2.0.0 

imageConcerning the FES namespace: yes you may omit the Filter Expression namespace, but as far as my experience goes, you‘ll better keep it. When you’re dealing with attribute filters, then you’ll need it.

 

And if you need more examples, just let me know the WFS url and the kind of filter you need. I’ll see, what I can do for you 😎 

 

Best regards

Oliver

 

 


I played with the BAG WFS, FeatureType "verblifjsobject" which seems to be implemented properly.

 

If you're interested in, just try these XML Filter Expressions:

# equal

<fes:Filter
xmlns:fes="http://www.opengis.net/fes/2.0"
xmlns:bag="http://bag.geonovum.nl">
<fes:PropertyIsEqualTo>
<fes:ValueReference>bag:woonplaats</fes:ValueReference>
<fes:Literal>Roermond</fes:Literal>
</fes:PropertyIsEqualTo>
</fes:Filter>

# like

<fes:Filter 
xmlns:fes="http://www.opengis.net/fes/2.0"
xmlns:bag="http://bag.geonovum.nl"><fes:PropertyIsLike wildCard='*' singleChar='.' escapeChar='!'><fes:PropertyName>bag:postcode</fes:PropertyName><fes:Literal>6041*</fes:Literal></fes:PropertyIsLike>
</fes:Filter>

 


Reply