Solved

ArcGISOnlineConnector - Export and Download?

  • 6 December 2021
  • 9 replies
  • 4 views
R

On the item page of a hosted service you can export to fgdb and other formats.

The benefit of this over an fme reader and writer workflow is the easy preservation of unique ids, relationships and attachments. The also take no local compute power/time.

 

Is there a way perform the "Export To FGDB" or other formats within FME?

I see that there is a method to download items from arcgis online with the use of the ArcGISOnlineConnector (https://docs.safe.com/fme/2020.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/arcgisonlineconnector.htm)

If I could create the fgdb through fme, I'm pretty sure I could also download and manipulate it within a workspace.

 

I'm doing some lengthy transformations on a million+ records. Performing these actions through the Online Readers/Writers is taking hours (90% is reading from the REST API) and I need to reduce that time.

 

Thanks!

icon

Best answer by todd_davis 10 December 2021, 02:02

View original

9 replies

nielsgerrits
Rank
Userlevel 6
Badge +32

Good suggestion. Please create an Idea for this and I will vote for it.

todd_davis
Rank
Userlevel 1
Badge +12

@hkingsbury​  forwarded me this question and now he has just made some more work for himself, as he now has to QA the transformer I have put together. It was a good question and not something we had previously created, but once checked we can release it to the Hub and we will provide an update here. Saying that, it may take @hkingsbury​  a bit of time to get to it, as he is busy at the moment...but it is coming.

 

Here is an early look....so putting the relevant item id, the folder that you want it downloaded to and the webconnection. It will still hold the GDB as a new item under the user, as I have not built it delete it after it is downloaded (because we already have that as a separate transformer).

 

Interface

R

This looks very promising! Would this work with the other "Export to" formats (Shapefile, CSV, KML, etc.) ? This isn't something needed in my workflow, but I could see it being useful for other users.

Also, will the resulting item id attributes be definable (Title, Tags, Summary,User Content Folder)?

 

Thanks so much!ExportOptionsExportToFGDB

todd_davis
Rank
Userlevel 1
Badge +12

This looks very promising! Would this work with the other "Export to" formats (Shapefile, CSV, KML, etc.) ? This isn't something needed in my workflow, but I could see it being useful for other users.

Also, will the resulting item id attributes be definable (Title, Tags, Summary,User Content Folder)?

 

Thanks so much!ExportOptionsExportToFGDB

Well it could do all those things but I won't aim for that on the first release, as we would need to study all the calls for the other formats and we would need to build some extra logic around inputs e.g. make sure that titles are unique so that we don't overwrite the output file if passing in 50 features, etc. So currently the new item containing the File Geodatabase just gets created as FGDB_<original item no>, but I can't control the name of the file geodatabase as this defaults to a "JobId"

todd_davis
Rank
Userlevel 1
Badge +12

Hi, the custom transformer is up at and can be found here. If there are any issues feel free to comment away, but @hkingsbury​ has put it through quite a few tests.

 

Will look at other formats and other options in future

 

R

Hi, the custom transformer is up at and can be found here. If there are any issues feel free to comment away, but @hkingsbury​ has put it through quite a few tests.

 

Will look at other formats and other options in future

 

I attempted to run the transformer and got the following error message:

{"code":403,"messageCode":"DS_0003","message":"User not allowed for this account","details":[]}

 

I think the organization url is hardcoded in due to the following log line:

Host: interpret.maps.arcgis.com

FleGDBExport_TransformerThanks. I'm a little too new with this to change the .fmx directly or know if that is even possible.

todd_davis
Rank
Userlevel 1
Badge +12

I attempted to run the transformer and got the following error message:

{"code":403,"messageCode":"DS_0003","message":"User not allowed for this account","details":[]}

 

I think the organization url is hardcoded in due to the following log line:

Host: interpret.maps.arcgis.com

FleGDBExport_TransformerThanks. I'm a little too new with this to change the .fmx directly or know if that is even possible.

Sorry, that slipped through in one caller. Updated now. So download the lastest version and it should, fingers-crossed, work.

 

It was related to when we get the user that is authenticated, and we didn't have that process defined previously either and I just coped in what it returned for me, rather than converting it. So it was just a case of changing that url.

R

I attempted to run the transformer and got the following error message:

{"code":403,"messageCode":"DS_0003","message":"User not allowed for this account","details":[]}

 

I think the organization url is hardcoded in due to the following log line:

Host: interpret.maps.arcgis.com

FleGDBExport_TransformerThanks. I'm a little too new with this to change the .fmx directly or know if that is even possible.

This is working great now! Thanks.

todd_davis
Rank
Userlevel 1
Badge +12

I attempted to run the transformer and got the following error message:

{"code":403,"messageCode":"DS_0003","message":"User not allowed for this account","details":[]}

 

I think the organization url is hardcoded in due to the following log line:

Host: interpret.maps.arcgis.com

FleGDBExport_TransformerThanks. I'm a little too new with this to change the .fmx directly or know if that is even possible.

glad it is working for you

Reply


Community Stats

28,715
Posts
109,490
Replies
36,561
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.