Solved

How to check if all parts of a shape file were uploaded to FME server?

  • 28 March 2018
  • 9 replies
  • 8 views

Badge

If we use a shape file reader in FME desktop we only need to pick the shp-file but not the other parts like dbf, shx etc. FME desktop recognizes all parts of the shape file and we can start our workflow.

If we publish the same workspace to the server and use a published parameter for the shape file then we need to upload all parts of the shape file as a temporary data upload to FME server. If one of the parts is missing then our workflow will not run. How can we check (in advance?) if all necessary parts of a shape file were temporary uploaded to FME server?

Best regards,

Frank

icon

Best answer by lars_de_vries 28 March 2018, 16:03

View original

9 replies

Badge

You could use the FME Server REST API to check what files are in a certain location.

The structure could be:

https://<your server>/fmerest/v3/resources/connections/FME_SHAREDRESOURCE_DATA/<files or folders>

 

 

With that response, you can check that all files belonging to the shapefile are present by using an Aggregator and a TestFilter.

For example: 'test.shp', 'test.dbf' and 'test.shx'. Extract the filename (test) and the extension (.shp) with a stringsearcher and regular expressions. Then use an Aggregator based on the filename and concatenate all the extensions in one attribute. Then use a TestFilter to check if the extensions (.shp, .dbf, ...) are in this concatenated string.

 

 

If all this is succesfull, you can be sure that all files are present.
Badge +3

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.

Badge

You could use the FME Server REST API to check what files are in a certain location.

The structure could be:

https://<your server>/fmerest/v3/resources/connections/FME_SHAREDRESOURCE_DATA/<files or folders>

 

 

With that response, you can check that all files belonging to the shapefile are present by using an Aggregator and a TestFilter.

For example: 'test.shp', 'test.dbf' and 'test.shx'. Extract the filename (test) and the extension (.shp) with a stringsearcher and regular expressions. Then use an Aggregator based on the filename and concatenate all the extensions in one attribute. Then use a TestFilter to check if the extensions (.shp, .dbf, ...) are in this concatenated string.

 

 

If all this is succesfull, you can be sure that all files are present.
This sounds a little bit trick but I will try it...

 

Frank

 

 

Badge

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.

This sounds very easy - I will try it tomorrow...

 

Frank

 

 

Userlevel 4
Badge +25

Shapefile is a nasty format in that respect, you need the .shp, .shx and .dbf and the rest is optional. I agree with @lars_de_vries that a .shz (or a .zip) is the easiest way to solve this. At least it'll solve the "is it uploaded or not" problem. You don't need to change anything to your workspace even, just make sure you let your users know they should upload a zipfile. However, it's probably wise to add a message when the workspace fails to explain what the workspace expects in terms of input.

Badge +3

Shapefile is a nasty format in that respect, you need the .shp, .shx and .dbf and the rest is optional. I agree with @lars_de_vries that a .shz (or a .zip) is the easiest way to solve this. At least it'll solve the "is it uploaded or not" problem. You don't need to change anything to your workspace even, just make sure you let your users know they should upload a zipfile. However, it's probably wise to add a message when the workspace fails to explain what the workspace expects in terms of input.

You could change the filter of the parameter, so that only .shz-files are accepted! That'll solve your wrong input selection issue.

 

You could set 'Configuration' in the Parameter definition to: 

 

Shape_zips(*.shz)
Badge
This sounds a little bit trick but I will try it...

 

Frank

 

 

Let me know if you need more help with implementing that!

 

Badge

Another solution would be to start working with SHZ-files, i.e. a ZIP-file of the necessary files of a shapefile, with the extension changed to SHZ in stead of ZIP. This way only a single file needs to be uploaded.

We tested shz and zip and both works like a charm :-) Thank you for your suggestion!

 

 

Badge

We have not tested this so far because uploading a shz or zip works very well. But this is on my todo list... At the moment I have no experiences with FME Server REST API but this should change in the future...

Reply