Skip to main content
Solved

How to check if all parts of a shape file were uploaded to FME server?

  • March 28, 2018
  • 9 replies
  • 61 views

Forum|alt.badge.img

If we use a shape file reader in FME desktop we only need to pick the shp-file but not the other parts like dbf, shx etc. FME desktop recognizes all parts of the shape file and we can start our workflow.

If we publish the same workspace to the server and use a published parameter for the shape file then we need to upload all parts of the shape file as a temporary data upload to FME server. If one of the parts is missing then our workflow will not run. How can we check (in advance?) if all necessary parts of a shape file were temporary uploaded to FME server?

Best regards,

Frank

Best answer by lars_de_vries

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

9 replies

jneujens
Forum|alt.badge.img
  • March 28, 2018

You could use the FME Server REST API to check what files are in a certain location.

The structure could be:

https://<your server>/fmerest/v3/resources/connections/FME_SHAREDRESOURCE_DATA/<files or folders>

 

 

With that response, you can check that all files belonging to the shapefile are present by using an Aggregator and a TestFilter.

For example: 'test.shp', 'test.dbf' and 'test.shx'. Extract the filename (test) and the extension (.shp) with a stringsearcher and regular expressions. Then use an Aggregator based on the filename and concatenate all the extensions in one attribute. Then use a TestFilter to check if the extensions (.shp, .dbf, ...) are in this concatenated string.

 

 

If all this is succesfull, you can be sure that all files are present.

lars_de_vries
Forum|alt.badge.img+10

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.


Forum|alt.badge.img
  • Author
  • March 28, 2018

You could use the FME Server REST API to check what files are in a certain location.

The structure could be:

https://<your server>/fmerest/v3/resources/connections/FME_SHAREDRESOURCE_DATA/<files or folders>

 

 

With that response, you can check that all files belonging to the shapefile are present by using an Aggregator and a TestFilter.

For example: 'test.shp', 'test.dbf' and 'test.shx'. Extract the filename (test) and the extension (.shp) with a stringsearcher and regular expressions. Then use an Aggregator based on the filename and concatenate all the extensions in one attribute. Then use a TestFilter to check if the extensions (.shp, .dbf, ...) are in this concatenated string.

 

 

If all this is succesfull, you can be sure that all files are present.
This sounds a little bit trick but I will try it...

 

Frank

 

 


Forum|alt.badge.img
  • Author
  • March 28, 2018

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.

This sounds very easy - I will try it tomorrow...

 

Frank

 

 


redgeographics
Celebrity
Forum|alt.badge.img+62

Shapefile is a nasty format in that respect, you need the .shp, .shx and .dbf and the rest is optional. I agree with @lars_de_vries that a .shz (or a .zip) is the easiest way to solve this. At least it'll solve the "is it uploaded or not" problem. You don't need to change anything to your workspace even, just make sure you let your users know they should upload a zipfile. However, it's probably wise to add a message when the workspace fails to explain what the workspace expects in terms of input.


lars_de_vries
Forum|alt.badge.img+10

Shapefile is a nasty format in that respect, you need the .shp, .shx and .dbf and the rest is optional. I agree with @lars_de_vries that a .shz (or a .zip) is the easiest way to solve this. At least it'll solve the "is it uploaded or not" problem. You don't need to change anything to your workspace even, just make sure you let your users know they should upload a zipfile. However, it's probably wise to add a message when the workspace fails to explain what the workspace expects in terms of input.

You could change the filter of the parameter, so that only .shz-files are accepted! That'll solve your wrong input selection issue.

 

You could set 'Configuration' in the Parameter definition to: 

 

Shape_zips(*.shz)

jneujens
Forum|alt.badge.img
  • March 30, 2018
This sounds a little bit trick but I will try it...

 

Frank

 

 

Let me know if you need more help with implementing that!

 


Forum|alt.badge.img
  • Author
  • April 5, 2018

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.

We tested shz and zip and both works like a charm :-) Thank you for your suggestion!

 

 


Forum|alt.badge.img
  • Author
  • April 5, 2018

We have not tested this so far because uploading a shz or zip works very well. But this is on my todo list... At the moment I have no experiences with FME Server REST API but this should change in the future...