Skip to main content
Solved

WFS returning 0 features

K
Forum|alt.badge.img

I'm having difficulty accessing data with a WFS reader.

I can read the feature types, and the translation is 'successful', but returns 0 features. I can load the WFS in QGIS too.

I've tried amending various parameters without success, including: connection timeout length, max features, ignore application schema (this worked with the previous version of the WFS), and using a search envelope.

The WFS is: https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs

Would really appreciate if anyone can nudge me in the right direction!

 

Best answer by david_r

Playing around a bit more I suspect that you will need to constrain the WFS query for it to work; the server times out if you query the layer without any criteria. I suspect the resulting dataset is too big.

You could try either setting "Max features" to some reasonable number, or defining a geographic search envelope. That seems to work fine for me.

View original
Did this help you find an answer to your question?

13 replies

david_r
Celebrity
  • david_rCelebrity
  • January 14, 2019

I tried the URL and got the same result. I then looked closely in the FME log and found the following line just under the line where it prints the full URL of the WFS request:

Received HTTP response header: 'HTTP/1.1 504 Gateway Time-out'

Unfortunately this is logged as INFO and not as WARN or ERROR as (I believe) it should, and therefore it's not obvious that there was an error.

Can you check your log and see if you find the same server error in there? 

david_r
Celebrity
  • david_rCelebrity
  • Best Answer
  • January 14, 2019

Playing around a bit more I suspect that you will need to constrain the WFS query for it to work; the server times out if you query the layer without any criteria. I suspect the resulting dataset is too big.

You could try either setting "Max features" to some reasonable number, or defining a geographic search envelope. That seems to work fine for me.

K
Forum|alt.badge.img
  • katiem
  • Author
  • January 15, 2019
david_r wrote:

Playing around a bit more I suspect that you will need to constrain the WFS query for it to work; the server times out if you query the layer without any criteria. I suspect the resulting dataset is too big.

You could try either setting "Max features" to some reasonable number, or defining a geographic search envelope. That seems to work fine for me.

Thanks! You're quite right that the dataset is large. This almost did the trick - I can get some features by constraining the max features to read.

I'd used a similar method to query the previous version of the WFS using search envelope coordinates, but that seems to not work this time. According to the capabilities document the data is in EPSG27700 but when I look at the wfs data with other data in that crs it's in completely the wrong place, hence why the bounding box method I'd already tried brought through 0 features. V odd!

K
Forum|alt.badge.img
  • katiem
  • Author
  • January 15, 2019
david_r wrote:

Playing around a bit more I suspect that you will need to constrain the WFS query for it to work; the server times out if you query the layer without any criteria. I suspect the resulting dataset is too big.

You could try either setting "Max features" to some reasonable number, or defining a geographic search envelope. That seems to work fine for me.

Update - apparently this is a known issue with this particular data, whereby the XY coordinates are inverted on the wfs. More info here if anyone else has come across this.

david_r
Celebrity
  • david_rCelebrity
  • January 15, 2019
katiem wrote:

Update - apparently this is a known issue with this particular data, whereby the XY coordinates are inverted on the wfs. More info here if anyone else has come across this.

It could perhaps be related to differences in how different fields interpret X as either northing or easting. More info here: https://gis.stackexchange.com/a/99862

K
Forum|alt.badge.img
  • katiem
  • Author
  • February 18, 2019

Update to this - still having issues unfortunately. Using the WFS reader with the following parameters:

This generates the following GetFeature URL, which seems to be invalid - I can get feature types, but 0 features are read: https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=ms%3AHistoric_Flood_Map&NAMESPACE=xmlns%28ms%3Dhttps%3A%2F%2Fenvironment.data.gov.uk%2Fspatialdata%2Fhistoric-flood-map%2Fwfs%29&BBOX=444018%2C321614%2C489124%2C401169%2Curn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700&MAXFEATURES=30000

I spoke to the data provider and the correct URL should be https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

This makes sense as it has the parameters for the bounding box CRS, and works as expected (when I use version 1.0.0 instead but that's a separate issue...)

I'm not sure where we're going wrong with the WFS reader. Whilst I can of course generate the URL myself it would be handy to be able to use the WFS reader parameters to do this, so would appreciate it if anyone can give any pointers.

Thanks

itay
Supporter
Forum|alt.badge.img+17
  • itaySupporter
  • Supporter
  • February 18, 2019

In the WFS reader under connection properties you have 2 settings (connection timeout length and transfer timeout length) that help with these kind of issues.

I usually set it up on higher values that the defaults.

Hope this helps.

Itay

K
Forum|alt.badge.img
  • katiem
  • Author
  • February 20, 2019
itay wrote:

In the WFS reader under connection properties you have 2 settings (connection timeout length and transfer timeout length) that help with these kind of issues.

I usually set it up on higher values that the defaults.

Hope this helps.

Itay

Thanks for the suggestion, we already tried that at the start. It was ultimately a combination of the WFS server timing out because our request was too large, and the WFS Reader generating an invalid GetFeature URL (see comment above) so the bounding box wasn't being recognised.

In the end I built the GetFeature URL from scratch rather than trying to do it with FME parameters. Also my area of interest was still too large so I chopped this up using a gridGenerator, got the coordinates into attributes using a BoundsExtractor, and fed this into a FeatureReader using the min/max XY attributes in the dataset URL.

chrisatsafe
Contributor
Forum|alt.badge.img+2
  • chrisatsafeContributor
  • Contributor
  • March 5, 2019

Hi @katiem,

Have you given this a try in our latest FME 2019.0 beta? There are a few fixes available for the WFS reader available in 2019 which may have resolved this issue. I was able to read 227 features using the parameters you supplied and the URL in FME 2019.0 build 19214:

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

0684Q00000ArKJGQA3.png

Hope this helps.

Download the latest FME 2019.0 beta here.

wfs.fmwt

chrisatsafe
Contributor
Forum|alt.badge.img+2
  • chrisatsafeContributor
  • Contributor
  • March 5, 2019
katiem wrote:

Update to this - still having issues unfortunately. Using the WFS reader with the following parameters: 

0684Q00000ArJEZQA3.png

This generates the following GetFeature URL, which seems to be invalid - I can get feature types, but 0 features are read:  https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=ms%3AHistoric_Flood_Map&NAMESPACE=xmlns%28ms%3Dhttps%3A%2F%2Fenvironment.data.gov.uk%2Fspatialdata%2Fhistoric-flood-map%2Fwfs%29&BBOX=444018%2C321614%2C489124%2C401169%2Curn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700&MAXFEATURES=30000

I spoke to the data provider and the correct URL should be https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700 

This makes sense as it has the parameters for the bounding box CRS, and works as expected (when I use version 1.0.0 instead but that's a separate issue...)

I'm not sure where we're going wrong with the WFS reader. Whilst I can of course generate the URL myself it would be handy to be able to use the WFS reader parameters to do this, so would appreciate it if anyone can give any pointers. 

Thanks

Hi @katiem,

Have you given this a try in our latest FME 2019.0 beta? There are a few fixes available for the WFS reader available in 2019 which may have resolved this issue. I was able to read 227 features using the parameters you supplied and the URL in FME 2019.0 build 19214:

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

0684Q00000ArMOdQAN.png

Hope this helps.

Download the latest FME 2019.0 beta here.

wfs.fmwt

danilo_fme
Evangelist
Forum|alt.badge.img+45
  • danilo_fmeEvangelist
  • Evangelist
  • March 5, 2019
chrisatsafe wrote:

Hi @katiem,

Have you given this a try in our latest FME 2019.0 beta? There are a few fixes available for the WFS reader available in 2019 which may have resolved this issue. I was able to read 227 features using the parameters you supplied and the URL in FME 2019.0 build 19214:

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

0684Q00000ArKJGQA3.png

Hope this helps.

Download the latest FME 2019.0 beta here.

wfs.fmwt

Hi @ChrisAtSafe

Thanks to inform us about this .

 

Danilo

U
  • union_jack
  • September 26, 2019

In FME2018.1 and 2019.1, I'm confused about reading WFS from environment.data.gov.uk.

 

FAQ's at environment.data.gov.uk give the following as an example GetFeature:-

 

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

I get/receive features with my WFS READER configured as follows, with a BBOX in LatLong coords :-

 

CRS=EPSG:27700/WFS=1.1.0/AxisOrder=2,1/max=5/bbox=51.4937388139627714/-0.3969194790841968/51.3866329950004612/-0.1233058624383356/BBOXCRS=bob

 

That's okay.

Note the order of the bbox coords, Latitute then Long, to match AXIS ORDER 2,1.

Sadly, FME2018.1 and 2019.1 fails to get any features if I use OSGB/Brit National Grid/EPSG27700 coords in my BBOX:- CRS=27700/WFS=1.1.0/AxisOrder=2,1/max=5/bbox=510500 165500 524500 179500/BBOXCRS=EPSG:27700

 

As a town council, we work in OSGB/Brit National Grid/EPSG27700 coords.

Also, I'm confused that FME seems a) to not include "&srsName=EPSG:27700" in the GetFeatures requests??

 

b) FME does suffix "EPSG:27700" to the BBOX cords, even though I set SEARCH ENVELOPE CRS to i) Mr Bean ii) LL-WGS84 or iii) any random string

In FME2019.1, I set "SEARCH ENVELOPE CRS" = mr bean

 

FME2019.1 log says:-

 

Sending HTTP 'GET' request to

 

https://environment.data.gov.uk:443/spatialdata/sites-of-special-scientific-interest-units-england/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=ms%3ASites_of_Special_Scientific_Interest_Units_England&NAMESPACE=xmlns%28ms%3Dhttps%3A%2F%2Fenvironment.data.gov.uk%2Fspatialdata%2Fsites-of-special-scientific-interest-units-england%2Fwfs%29&BBOX=-0.3969194790841968%2C51.38663299500046%2C-0.1233058624383356%2C51.49373881396277%2Curn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700&MAXFEATURES=5'

 

waiting for response ...

 

It gets 31 records/features. That's okay.

 

Part of the BBOX parameters includes:-%2Curn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700 ie urn ogc def crs EPSG 27700

 

It will get/receive features if, I use the bbox coords in Lat Long format. That's okay.

 

FME gets zero features if my bbox coords are in EPSG27700. Not okay.

I set "SEARCH ENVELOPE CRS" = LL-WGS84

 

FME2019.1 log says:-

 

Sending HTTP 'GET' request to

 

'https://environment.data.gov.uk:443/spatialdata/sites-of-special-scientific-interest-units-england/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=ms%3ASites_of_Special_Scientific_Interest_Units_England&NAMESPACE=xmlns%28ms%3Dhttps%3A%2F%2Fenvironment.data.gov.uk%2Fspatialdata%2Fsites-of-special-scientific-interest-units-england%2Fwfs%29&BBOX=-5599806.778056016%2C7442159.613065494%2C-5549611.459638113%2C7462169.2105717445%2Curn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700&MAXFEATURES=5', waiting for response ... Again, FME gets zero features.

@david_r @chrisatsafe @katiem

I can imagine lots of council staff in the UK spending loads of time trying and failing to connect their FME to environment.data.gov.uk

Attached are my workbenchs.

..Lat Long Coords.. works,

..27700 corods.. don't work.

 

 

 

 

U
  • union_jack
  • September 26, 2019

CLIP TO SEARCH ENVELOPE

As I said in my previous reply, below here..

For https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?

I get/receive features with my WFS READER configured as follows, with a BBOX in LatLong coords :-

 

CRS=EPSG:27700/WFS=1.1.0/AxisOrder=2,1/max=5/bbox=51.4937388139627714/-0.3969194790841968/51.3866329950004612/-0.1233058624383356/BBOXCRS=bob

 

That's okay.

Note the order of the bbox coords, Latitute then Long, to match AXIS ORDER 2,1.

But, if I simply tick/check CLIP TO SEARCH ENVELOPE with above settings, I get zero features. Not okay.

This is a pain because some of the floodzones are massive.

So, I need to clip them to my bounding box.

 

 

Reply


Helpful Members This Week

Community Stats

31,287
Posts
118,664
Replies
38,875
Members

Latest FME

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.