Skip to main content
  • EN

Hello,

I'm working on a workspace that will update a AGOL Feature Service. I need to pull from two different data sources, normalize the attributes and do some additional processing in the middle then write updates to the master. Both source data sets are fairly large so I am trying to not have to read each of them in their entirety every time the workspace runs. Rather I would like to leverage a "Modified On" field in the source data to only read in features that have been modified since the last time my workspace ran.

I plan to push the workspace I'm building to FME Server and then sschedule it to run nightly. Currently I use the "time stamper" transformer to park the current data/time when the workspace runs in a .csv file. My thought is that I could then read in that timestamp and use it in my WHERE Clause. That's the thought however, I cannot seem to find a way to leverage that timestamp in my workspace parameters.

 

Does anyone have some experience trying to accomplish what I've outlined above? If so how are you tackling this problem?

Hi @bbbutler

Many different approaches you could use. Here's one suggestion:

Read value out of database (or CSV if you want) and use that value in a FeatureReader rather than using a standard Reader with parameter.

I just like databases but here is one option. updatesincelastrun.fmw

You could create a table layer in your feature service with one row and attributes for created and modified and read then update the modified value to the UTC timestamp of the last update. I have done this before.

Hi @bbbutler

Many different approaches you could use. Here's one suggestion:

Read value out of database (or CSV if you want) and use that value in a FeatureReader rather than using a standard Reader with parameter.

I just like databases but here is one option. updatesincelastrun.fmw

Thanks Mark,

 

I hadn't thought to use a feature writer. I'm working with the option now and think it'll work. Thanks for the quick response!

 

 

You could create a table layer in your feature service with one row and attributes for created and modified and read then update the modified value to the UTC timestamp of the last update. I have done this before.

This is slick! I like how it keep everything all in one location

 

 

Reply


Community Stats

30,092
Posts
114,247
Replies
37,720
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.