+22
Hi list.
Does the FeatureReader reading WFS 2.0.0 data handle paging ?
If a service is known to be limited to return a maximum number of features per call, paging, i.e. fetching the complete dataset in chunks, is a way to handle this.
However, the only options I see are "Max Features" and "Count", but they don't incur paging, as far as I can tell.
So how does FME handle WFS 2.0.0 paging ?
Cheers
Lars I
Best answer by lifalin2016
View original
+19
From memory, you have the Start Index and the Count parameters. With those two you create the "pages".
If you want 5000 features from a WFS whose max features per page is 1000, you will need to perfomr 5 calls to the service:
Â
+22
From memory, you have the Start Index and the Count parameters. With those two you create the "pages".
If you want 5000 features from a WFS whose max features per page is 1000, you will need to perfomr 5 calls to the service:
Â
Can a WFS 2.0.0 service disable paging ?
I did find the start index and count (and max features) settings, and set them to appropriate values, but FME only fetches the first batch/page of features.
The help (under Count) does mention something along those lines:
"This is useful if the WFS server supports paging. The FME WFS client will keep fetching subsequent result documents as long as the current result document has a next url and the Max Features limit has not been reached."
Unfortunately the FME log doesn't include neither requests nor responses from the WFS service.
+19
Can a WFS 2.0.0 service disable paging ?
I did find the start index and count (and max features) settings, and set them to appropriate values, but FME only fetches the first batch/page of features.
The help (under Count) does mention something along those lines:
"This is useful if the WFS server supports paging. The FME WFS client will keep fetching subsequent result documents as long as the current result document has a next url and the Max Features limit has not been reached."
Unfortunately the FME log doesn't include neither requests nor responses from the WFS service.
That is a very good question. I hope someone else can give an answer to that, because I have no idea.
+25
Can a WFS 2.0.0 service disable paging ?
I did find the start index and count (and max features) settings, and set them to appropriate values, but FME only fetches the first batch/page of features.
The help (under Count) does mention something along those lines:
"This is useful if the WFS server supports paging. The FME WFS client will keep fetching subsequent result documents as long as the current result document has a next url and the Max Features limit has not been reached."
Unfortunately the FME log doesn't include neither requests nor responses from the WFS service.
I think WFS services can disable paging. I have run into one (not sure if it was 2.0.0 or older) which had that... ("oh no, you're actually trying to use our data, we can't have that" 😡 )
Can a WFS 2.0.0 service disable paging ?
I did find the start index and count (and max features) settings, and set them to appropriate values, but FME only fetches the first batch/page of features.
The help (under Count) does mention something along those lines:
"This is useful if the WFS server supports paging. The FME WFS client will keep fetching subsequent result documents as long as the current result document has a next url and the Max Features limit has not been reached."
Unfortunately the FME log doesn't include neither requests nor responses from the WFS service.
Looking at the OGC WFS 2.0.2 standard:
http://docs.opengeospatial.org/is/09-025r2/09-025r2.html#76
It seems paging is optional, although it might not be because of a configuration setting, but simply because paging hasn't been implemented by the developers of that particular WFS server software implementation.
However, since the standard stipulates that paging capabilities should be broadcast in the GetCapabilities response, that may be a way to tell if your particular WFS endpoint supports paging or not.
+22
I did some digging in the OGC document too. Paging is indeed optional, and it requires that the service returns a "next" parameter in its response, if enabled.
However, the (external) service I'm struggling with, an ArcGIS WFS service, does advertise that paging is indeed enabled, but doesn't add a "next" parameter in its response to a GetFeature request with a "count" setting.
So it seems that the service is the culprit, paging is simple not implemented correctly.
It being an external service, but an ArcGIS type of which I have next to no knowledge about, do any of y'all know whether this is a recognized bug, that's been fixed ?
Cheers
+22
Can a WFS 2.0.0 service disable paging ?
I did find the start index and count (and max features) settings, and set them to appropriate values, but FME only fetches the first batch/page of features.
The help (under Count) does mention something along those lines:
"This is useful if the WFS server supports paging. The FME WFS client will keep fetching subsequent result documents as long as the current result document has a next url and the Max Features limit has not been reached."
Unfortunately the FME log doesn't include neither requests nor responses from the WFS service.
The service fails with version 2.0.2, but works with 2.0.0. Do you think this has anything to do with paging ? Isn't paging part of 2.0.0 ?
Can a WFS 2.0.0 service disable paging ?
I did find the start index and count (and max features) settings, and set them to appropriate values, but FME only fetches the first batch/page of features.
The help (under Count) does mention something along those lines:
"This is useful if the WFS server supports paging. The FME WFS client will keep fetching subsequent result documents as long as the current result document has a next url and the Max Features limit has not been reached."
Unfortunately the FME log doesn't include neither requests nor responses from the WFS service.
Response paging seems to have been introduced in version 2.0.0 of the OGC Standard, but then again it's anyone's guess if, or how, it is actually implemented in either of the software implementations that purportedly support WFS 2. Unfortunately it is not been my experience that Esri has been the most ardent or strict implementer of OGC standards, to put it that way.
+22
I did some digging in the OGC document too. Paging is indeed optional, and it requires that the service returns a "next" parameter in its response, if enabled.
However, the (external) service I'm struggling with, an ArcGIS WFS service, does advertise that paging is indeed enabled, but doesn't add a "next" parameter in its response to a GetFeature request with a "count" setting.
So it seems that the service is the culprit, paging is simple not implemented correctly.
It being an external service, but an ArcGIS type of which I have next to no knowledge about, do any of y'all know whether this is a recognized bug, that's been fixed ?
Cheers
It would have been nice, if FME had dropped an error message about the missing "next" parameter, when tasked with doing a paged request cycle.
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
