Question

Best practice for streamlining data flow to/from AGOL

  • 15 February 2023
  • 1 reply
  • 2 views
M
Badge

I am currently working with about 60 towns (folders) on AGOL which all contain 10+ feature layers. The data is read from all these layer (reader per layer), compared with data in my db, then the data is updated/inserted/deleted in our db.

 

After this, let's say some work is done on the db data . I then have another workspace which is writing the updates back up to AGOL for the same town.

 

I currently have 2 workspaces per town which take care of all the reading/writing/processing for all the layers. This has resulted in 120 workbenches which, when an update has to be made to the way data is processed, I have to go back and update all of those workspaces. Also more towns are being added every week so the number of workbenches is increasing.

 

The screenshot below is taken from one layer where data is read from both AGOL and the db, compared and then written out. I have a section like this for every layer where the processing is largely the same, with small differences depending on if it's a point, line or polygon.

 

Would anybody have any suggestions on streamlining this process so that I don't have to go back and change hundreds of workspaces when an update goes out? Any help would be greatly appreciated.

 

 

image

1 reply

M
Badge +2

@manca_ent​ You might want to talk to your local FME reseller for a really deep dive into this workflow. But here are some possible ideas:

  • have you tried ChangeDetector instead of FeatureMerger
  • Try using fme_db_operation on the writer feature types : Updating Databases: Using Change Detection to Set Operation Type. This should allow you to have only one writer / feature type for each feature class which will simplify the workspace a little.
  • Have a look at the dynamic workflows. This may allow you to have the same workspace work with several different feature classes: Tutorial: Dynamic Workflows. A dynamic workflow might whittle your 120 workspaces to 2 if it can be made to work with this data.

If you can attach a small sample of the data and your workspace someone might take a look.

 

 

Reply


Community Stats

28,651
Posts
109,272
Replies
36,525
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.