Download Geometry Filtered WFS Data

Maybe this has something to do with the fact that FME 2023.0 is not yet officially released, still in Beta?

What happens if you perform the same test in the latest official release, 2022.2?

FME 2019 and FME2022.2.2 give the same errors.

But I made a little progress. This URL:

https://service.pdok.nl/lv/bag/wfs/v2_0?request=getCapabilities&service=WFS

gives back 'panden' and/or 'verblijfobjecten' when I request the data with the correct filter options.

So it seems the INSPIRE url is not working according to specs. QGis also has problems loading the data.

Thank you for the explanation. I use the QGis - PDOK Services Plugin a lot. So I looked up this URL there:

https://github.com/rduivenvoorde/pdokservicesplugin/blob/master/pdokservicesplugin/resources/layers-pdok.json

But that list is behind in maintenance.

Any change I can geofilter the Atom dataset? I will use the BAG WFS for now.

The whole idea behind this Atom service is that you have to download the dataset first to be able to process it locally. It is a zipped gml file, 800 MB in size.

Mmh, the WFS available under this url works (but not really fine):

https://geodata.nationaalgeoregister.nl/inspire/ad/wfs?

I can access the following feature types:

1. Address (points)
2. AdressAreaName
3. PostalDescriptor
4. ThoroughfareName

Imho, the service doesn’t return any useful attributes, expect of the postCode in PostalDescriptor.

But coming back to your question: most WFS are limited to a certain number of features. Most of them to 30.000 features. The reason is, that WFS are returning GML (XML) documents and these will get too large.

To bypass this limit, you’ll need a filter.

Either a spatial filter, or an attribute filter. You tried a spatial filter using a bounding box.

Sorry Safers, but the Spatial Filters in FeatureReader in combination with WFS don’t work as expected!

If you have a look into the logfile, you’ll see, that FME sends a BBOX requests in most cases.

And even the BBOX requests doesn’t work properly with this certain WFS.

But, this BBOX filter works as expected:

<fes:Filter xmlns:fes="http://www.opengis.net/fes/2.0" xmlns:gml="http://www.opengis.net/gml/3.2">
  <fes:BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </fes:BBOX>
</fes:Filter>

Others like „Within“ or „DWithin“ are returning errors - from the WFS. 

Maybe, it is not implemented properly?

Kind regards

Oliver

Here's a running DWithin Filter for another WFS.

<fes:Filter xmlns:fes="http://www.opengis.net/fes/2.0" xmlns:gml="http://www.opengis.net/gml/3.2">
  <fes:BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </fes:BBOX>
</fes:Filter>

Thanks for finding the correct way to request the data. Seems the WFS is pretty strict in what it will answer to.

For additional information:

The FES namespace declaration may be omitted. But the GML namespace must be declared. Using the BoundsExtractor for the @Value(xmin) etc. in this example just like you did.

<Filter xmlns:gml="http://www.opengis.net/gml/3.2">
  <BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </BBOX>
</Filter>

It is important to use the WFS 2.0 specification and in this case setting the Start Index to 0 (zerro) and the Count to 500. The service responds with somewhere between 300 and 500 features. This settings will request multiple pages to receive all data.

As you pointed out the dataset does not respond with expected address information ether. Just a housenumber in this field locator{0}.AddressLocator.designator{0}.LocatorDesignator.designator

So the service has a pretty bad service.

But as you pointed out as well. It seems that the implementation of the Spatial Filter on the main page of the FeatureReader is not the best working implementation by FME Workbench for Spatial Filtering a WFS Service.

And I could really use some more working examples for the Filter function within the Parameters... section of the FeatureReader.

<Filter xmlns:gml="http://www.opengis.net/gml/3.2">
  <BBOX>
        <gml:Envelope srsName="urn:ogc:def:crs:EPSG::28992">
          <gml:lowerCorner>@Value(xmin) @Value(ymin)</gml:lowerCorner>
          <gml:upperCorner>@Value(xmax) @Value(ymax)</gml:upperCorner>
       </gml:Envelope>
   </BBOX>
</Filter>

I think, I found the reason for the malfunction.

Just open the GetCapabilities document and search for <fes:Filter_Capabilities>

https://geodata.nationaalgeoregister.nl/inspire/ad/wfs?SERVICE=WFS&amp;REQUEST=GetCapabilities&amp;VERSION=2.0.0 

Concerning the FES namespace: yes you may omit the Filter Expression namespace, but as far as my experience goes, you‘ll better keep it. When you’re dealing with attribute filters, then you’ll need it.

And if you need more examples, just let me know the WFS url and the kind of filter you need. I’ll see, what I can do for you 😎 

Best regards

Oliver

I played with the BAG WFS, FeatureType "verblifjsobject" which seems to be implemented properly.

If you're interested in, just try these XML Filter Expressions:

# equal

<fes:Filter
	xmlns:fes="http://www.opengis.net/fes/2.0"
	xmlns:bag="http://bag.geonovum.nl">
	<fes:PropertyIsEqualTo>
		<fes:ValueReference>bag:woonplaats</fes:ValueReference>
		<fes:Literal>Roermond</fes:Literal>
	</fes:PropertyIsEqualTo>
</fes:Filter>

# like

<fes:Filter 
	xmlns:fes="http://www.opengis.net/fes/2.0"
	xmlns:bag="http://bag.geonovum.nl"><fes:PropertyIsLike wildCard='*' singleChar='.' escapeChar='!'><fes:PropertyName>bag:postcode</fes:PropertyName><fes:Literal>6041*</fes:Literal></fes:PropertyIsLike>
</fes:Filter>