I tried the URL and got the same result. I then looked closely in the FME log and found the following line just under the line where it prints the full URL of the WFS request:

Received HTTP response header: 'HTTP/1.1 504 Gateway Time-out'

Unfortunately this is logged as INFO and not as WARN or ERROR as (I believe) it should, and therefore it's not obvious that there was an error.

Can you check your log and see if you find the same server error in there? 

Playing around a bit more I suspect that you will need to constrain the WFS query for it to work; the server times out if you query the layer without any criteria. I suspect the resulting dataset is too big.

You could try either setting "Max features" to some reasonable number, or defining a geographic search envelope. That seems to work fine for me.

Thanks! You're quite right that the dataset is large. This almost did the trick - I can get some features by constraining the max features to read.

I'd used a similar method to query the previous version of the WFS using search envelope coordinates, but that seems to not work this time. According to the capabilities document the data is in EPSG27700 but when I look at the wfs data with other data in that crs it's in completely the wrong place, hence why the bounding box method I'd already tried brought through 0 features. V odd!

Update - apparently this is a known issue with this particular data, whereby the XY coordinates are inverted on the wfs. More info here if anyone else has come across this.

It could perhaps be related to differences in how different fields interpret X as either northing or easting. More info here: https://gis.stackexchange.com/a/99862

Update to this - still having issues unfortunately. Using the WFS reader with the following parameters:

This generates the following GetFeature URL, which seems to be invalid - I can get feature types, but 0 features are read: https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=ms%3AHistoric_Flood_Map&NAMESPACE=xmlns%28ms%3Dhttps%3A%2F%2Fenvironment.data.gov.uk%2Fspatialdata%2Fhistoric-flood-map%2Fwfs%29&BBOX=444018%2C321614%2C489124%2C401169%2Curn%3Aogc%3Adef%3Acrs%3AEPSG%3A%3A27700&MAXFEATURES=30000

I spoke to the data provider and the correct URL should be https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

This makes sense as it has the parameters for the bounding box CRS, and works as expected (when I use version 1.0.0 instead but that's a separate issue...)

I'm not sure where we're going wrong with the WFS reader. Whilst I can of course generate the URL myself it would be handy to be able to use the WFS reader parameters to do this, so would appreciate it if anyone can give any pointers.

Thanks

In the WFS reader under connection properties you have 2 settings (connection timeout length and transfer timeout length) that help with these kind of issues.

I usually set it up on higher values that the defaults.

Hope this helps.

Itay

Thanks for the suggestion, we already tried that at the start. It was ultimately a combination of the WFS server timing out because our request was too large, and the WFS Reader generating an invalid GetFeature URL (see comment above) so the bounding box wasn't being recognised.

In the end I built the GetFeature URL from scratch rather than trying to do it with FME parameters. Also my area of interest was still too large so I chopped this up using a gridGenerator, got the coordinates into attributes using a BoundsExtractor, and fed this into a FeatureReader using the min/max XY attributes in the dataset URL.

Hi @katiem,

Have you given this a try in our latest FME 2019.0 beta? There are a few fixes available for the WFS reader available in 2019 which may have resolved this issue. I was able to read 227 features using the parameters you supplied and the URL in FME 2019.0 build 19214:

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

Hope this helps.

Download the latest FME 2019.0 beta here.

wfs.fmwt

Hi @katiem,

Have you given this a try in our latest FME 2019.0 beta? There are a few fixes available for the WFS reader available in 2019 which may have resolved this issue. I was able to read 227 features using the parameters you supplied and the URL in FME 2019.0 build 19214:

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

Hope this helps.

Download the latest FME 2019.0 beta here.

wfs.fmwt

https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&TYPENAME=Historic_Flood_Map&BBOX=444018,321614,489124,401169,EPSG:27700&srsName=EPSG:27700

Hi @ChrisAtSafe

Thanks to inform us about this .

Danilo

In FME2018.1 and 2019.1, I'm confused about reading WFS from environment.data.gov.uk.

I get/receive features with my WFS READER configured as follows, with a BBOX in LatLong coords :-

Note the order of the bbox coords, Latitute then Long, to match AXIS ORDER 2,1.

Sadly, FME2018.1 and 2019.1 fails to get any features if I use OSGB/Brit National Grid/EPSG27700 coords in my BBOX:- CRS=27700/WFS=1.1.0/AxisOrder=2,1/max=5/bbox=510500 165500 524500 179500/BBOXCRS=EPSG:27700

Also, I'm confused that FME seems a) to not include "&srsName=EPSG:27700" in the GetFeatures requests??

In FME2019.1, I set "SEARCH ENVELOPE CRS" = mr bean

I set "SEARCH ENVELOPE CRS" = LL-WGS84

@david_r @chrisatsafe @katiem

I can imagine lots of council staff in the UK spending loads of time trying and failing to connect their FME to environment.data.gov.uk

Attached are my workbenchs.

..Lat Long Coords.. works,

..27700 corods.. don't work.

CLIP TO SEARCH ENVELOPE

As I said in my previous reply, below here..

For https://environment.data.gov.uk/spatialdata/historic-flood-map/wfs?

I get/receive features with my WFS READER configured as follows, with a BBOX in LatLong coords :-

Note the order of the bbox coords, Latitute then Long, to match AXIS ORDER 2,1.

But, if I simply tick/check CLIP TO SEARCH ENVELOPE with above settings, I get zero features. Not okay.

This is a pain because some of the floodzones are massive.

So, I need to clip them to my bounding box.