Question

Scanning an uploaded zip and subfolders for geodatabases on FME Server

  • 24 July 2017
  • 7 replies
  • 20 views
chriswilson
Rank
Badge +11

I have been using FME server to run a workspace that reads an uploaded zip containing - a couple of directories down - a single geodatabase. The zip is uploaded via a published parameter. I now need the uploaded zip to be scanned for multiple geodatabases and for all of those to be read. Additionally the geodatabases are now a couple of extra subdirectories down and the zip is now large (130mb).

The desktop solution would be something like this:

C:\\Folder\\Subfolder1\\Subfolder2\\Subfolder3\\temp.zip\\**\\*.gdb

As the workspace on FME Server will require an uploaded zip it will not be reading from a network drive so the published parameter by default is blank. How do I ensure that the zip will be scanned on the server for folders and subfolders, and any geodatabase within that read?

I did try uploading the new zip with multiple geodatabases via the original published parameter, with the error message causing a subsequent process failure being:

Unable to connect to the File Geodatabase at 'D:\\FMETemp\\fmeengines\\AUSYD0-APP022_Engine1\\temp.zip_1500851858307_1164'. Make sure the correct filename was specified, and that the geodatabase wasn't saved with a newer version of ArcGIS than the one installed locally. The error number from ArcObjects is: '-2147024894'. The error message from ArcObjects is: {}

7 replies

todd_davis
Rank
Userlevel 1
Badge +12

Hi Chris,

You should be able to just use the normal gdb reader and not do any advanced browsing. Pass it the zip and it will go through the file itself and pull all the featureclasses.

stalknecht
Rank
Userlevel 1
Badge +18
Can you share your workspace?

 

 

david_r
Rank
Userlevel 4

If your zip file may contain multiple file geodatabases in an unknown multi-level structure you may have to manually unzip it to a temporary location before reading the contents.

A possible pattern could be using a

Creator -> ParameterFetcher -> TempPathnameCreator -> Unzipper (FME Hub) -> FeatureReader

The FeatureReader can use the "swizzler" wildcard, e.g.

@Value(_pathname)\\**\\*.gdb

Where _pathname is the temp path created by the TempPathnameCreator. The beauty of using the TempPathnameCreator is that FME will automatically delete everything inside when your workspace terminates, you don't have to worry about that yourself.

For the upload service to work you can replace your File Geodatabase reader with a Null reader.

chriswilson
Rank
Badge +11

Hi Chris,

You should be able to just use the normal gdb reader and not do any advanced browsing. Pass it the zip and it will go through the file itself and pull all the featureclasses.

Hey Todd. Doing this resulted in the error I posted above, which occurred twice before failure.

 

 

chriswilson
Rank
Badge +11
Hey Todd. Doing this resulted in the error I posted above, which occurred twice before failure.

 

 

However, after all this the zip-reading doesn't seem to be the issue - my mistake!
chriswilson
Rank
Badge +11

As just mentioned to @todd_davis it turns out my issue is not with the zip file reading after all, something else. However, the above info about the geodatabase reader and unzipper from Todd and @david_r is nonetheless useful so thanks for that.

I did end up adjusting my workspace anyway to use the File and Pathnames reader, linked to the uploaded zip file parameter, to unzip and then scan using the **/*.gdb filter for geodatabases.

chriswilson
Rank
Badge +11

If your zip file may contain multiple file geodatabases in an unknown multi-level structure you may have to manually unzip it to a temporary location before reading the contents.

A possible pattern could be using a

Creator -> ParameterFetcher -> TempPathnameCreator -> Unzipper (FME Hub) -> FeatureReader

The FeatureReader can use the "swizzler" wildcard, e.g.

@Value(_pathname)\\**\\*.gdb

Where _pathname is the temp path created by the TempPathnameCreator. The beauty of using the TempPathnameCreator is that FME will automatically delete everything inside when your workspace terminates, you don't have to worry about that yourself.

For the upload service to work you can replace your File Geodatabase reader with a Null reader.

Hi @david_r, as I've just commented I'm not sure the zip scanning was the issue for my process in the end. However, your information is helpful and allowed me to unzip, search and find geodatabases throughout the zip file using only the File and Pathnames reader and the **/*.gdb path filter which is very useful to know. So thanks!

 

Reply


Community Stats

28,688
Posts
109,430
Replies
36,542
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.