Solved

I am using WFS reader. As data is fairly large and our firewall breaks the download process , I am trying to setup WFS Output Format as SHAPE-ZIP. But I can't work out how to get the zipped shapefile into the next step.


Badge

errorwfs setupwfs settingsI have tried to add feature reader after the WFS reader, it did not work. I then tried to use python caller to unzip the shapefile first. But it looks like the zipped shapefile gets deleted after wfs reader finishes.

Has anyone successfully used SHAPE-ZIP as WFS reader output format? how do you get that data into the next step?

 

 

 

icon

Best answer by kailinatsafe 4 March 2022, 20:12

View original

6 replies

Userlevel 2
Badge +13

Hello @lidiad​ , I think this is a bug that has been fixed in newer versions. You seem to be missing a parameter under the 'Constraints' window for Feature Types (this screenshot is from FME 2022.0):

 

WFS_MissingParameter 

I had a look at older versions, and was also able to see the missing parameter in FME 2021.2 b21766, see image:

 

image 

This should be an easy fix, just need to upgrade. You can find new versions on our Downloads Page (should be okay with FME 2021.2 b21812) .

 

As for the 'WFS Output Format', the deletion of the output is expected I think. If we take a look at the documentation, it tells us: 'If a non-GML fomat is selected, an additional Non_GML_GetFeature will be returned by the WFS reader' and carry additional attributes through the translation. One of these attributes is 'wfs_temp_datafile' , and the doc tells us this file will be deleted at the end of the translation. So this is likely why you're seeing it deleted!

 

I wonder if using a FeatureWriter will be more useful here? Best, Kailin.

Badge

Hello @lidiad​ , I think this is a bug that has been fixed in newer versions. You seem to be missing a parameter under the 'Constraints' window for Feature Types (this screenshot is from FME 2022.0):

 

WFS_MissingParameter 

I had a look at older versions, and was also able to see the missing parameter in FME 2021.2 b21766, see image:

 

image 

This should be an easy fix, just need to upgrade. You can find new versions on our Downloads Page (should be okay with FME 2021.2 b21812) .

 

As for the 'WFS Output Format', the deletion of the output is expected I think. If we take a look at the documentation, it tells us: 'If a non-GML fomat is selected, an additional Non_GML_GetFeature will be returned by the WFS reader' and carry additional attributes through the translation. One of these attributes is 'wfs_temp_datafile' , and the doc tells us this file will be deleted at the end of the translation. So this is likely why you're seeing it deleted!

 

I wonder if using a FeatureWriter will be more useful here? Best, Kailin.

Thanks for your reply! I do have that part of the menu, it is all menu options folded somehow and hid it in my previous screenshot.

 

wfs settings2

Badge

Hello @lidiad​ , I think this is a bug that has been fixed in newer versions. You seem to be missing a parameter under the 'Constraints' window for Feature Types (this screenshot is from FME 2022.0):

 

WFS_MissingParameter 

I had a look at older versions, and was also able to see the missing parameter in FME 2021.2 b21766, see image:

 

image 

This should be an easy fix, just need to upgrade. You can find new versions on our Downloads Page (should be okay with FME 2021.2 b21812) .

 

As for the 'WFS Output Format', the deletion of the output is expected I think. If we take a look at the documentation, it tells us: 'If a non-GML fomat is selected, an additional Non_GML_GetFeature will be returned by the WFS reader' and carry additional attributes through the translation. One of these attributes is 'wfs_temp_datafile' , and the doc tells us this file will be deleted at the end of the translation. So this is likely why you're seeing it deleted!

 

I wonder if using a FeatureWriter will be more useful here? Best, Kailin.

Is there a way to read SHAPE-ZIP in the same workbench? In one of screenshots that I posted above, reader for non gml feature (which is the shape-zip) is followed by a feature reader, but I still can't get the shape-zip read.

Userlevel 2
Badge +13

Is there a way to read SHAPE-ZIP in the same workbench? In one of screenshots that I posted above, reader for non gml feature (which is the shape-zip) is followed by a feature reader, but I still can't get the shape-zip read.

Hey @lidiad​ , I am honestly not too sure why its not making it to the FeatureReader...

Is SHAPE-ZIP supported by your WFS server?

 

Might be worth double checking the path to your downloaded SHAPE-ZIP file too. I think you can find the path as an attribute (wfs_temp_datafile) on the Non_GML_GetFeature (try looking in the FeatureInformation window or logfile). Ideally this path should be present in your FeatureReader for the source dataset! If this doesn't work still, I may suggest we create a case for you so we can look at your specific WFS.

 

Another thought, you could use a SystemCaller/FileCopy Writer/FeatureWriter to copy the file out of the temp location, so it can be used downstream? (you'd likely still need the path from wfs_temp_datafile).

 

Let me know if you can't find it / have questions. Best, Kailin.

Badge

Is there a way to read SHAPE-ZIP in the same workbench? In one of screenshots that I posted above, reader for non gml feature (which is the shape-zip) is followed by a feature reader, but I still can't get the shape-zip read.

I checked the path and file is never there. Maybe WFS does not have it. I actually just added a FeatureHolder to WFS reader and this works well to download all WFS feature without fail. For some reason the processing fails if it starts to process while still downloading.

thank you for your help!

Userlevel 2
Badge +13

Is there a way to read SHAPE-ZIP in the same workbench? In one of screenshots that I posted above, reader for non gml feature (which is the shape-zip) is followed by a feature reader, but I still can't get the shape-zip read.

@lidiad​  That's great to hear. So the translation was moving faster than the data, good catch! Best, Kailin

Reply