Skip to main content
Question

How Do I Set the Order of Operations?

  • October 13, 2021
  • 3 replies
  • 312 views

Forum|alt.badge.img

I need the operations to work in the following order:

  • Terminator
  • DatabaseDeleter
  • Data Load (rest of workspace)

 

I tried using a WorkspaceRunner transformer, having split out the three functions. It works fine if I'm in FME, but if I try to run it using a batch file, it errors out.

 

Ultimately, I want to be able to schedule this through Windows Task Scheduler to run nightly. It will compare the last updated date between two datasets, and terminate if the destination dataset is already up-to-date. Otherwise, it will proceed to delete a subset of the destination dataset (a "partial truncate" for lack of a better term), and then re-load the new data.Workbench

3 replies

hkingsbury
Celebrity
Forum|alt.badge.img+53
  • Celebrity
  • October 13, 2021

A better approach would be to make use of featurereaders and featurewriters. So the workflow (in one 'branch') would be something like:

  1. Checkdates
    1. If the match no update (i wouldnt then pass it to a terminator as thats going to 'fail' your workbench, leave it empty and have the wb finish successfuly)
    2. if the dates dont match, proceed
  2. Using a featurewriter delete the required rows
  3. once the deletes are complete, use a featurereader to read the data and pass through the rest of the logic

The advantage of using feautrereaders and writers is that you can conditionally control exaclty when data is read (and written/deleted)

In your current workflow, all the readers essentially run reading in all you data irrespective of if you need it or not


Forum|alt.badge.img
  • Author
  • October 14, 2021
hkingsbury wrote:

A better approach would be to make use of featurereaders and featurewriters. So the workflow (in one 'branch') would be something like:

  1. Checkdates
    1. If the match no update (i wouldnt then pass it to a terminator as thats going to 'fail' your workbench, leave it empty and have the wb finish successfuly)
    2. if the dates dont match, proceed
  2. Using a featurewriter delete the required rows
  3. once the deletes are complete, use a featurereader to read the data and pass through the rest of the logic

The advantage of using feautrereaders and writers is that you can conditionally control exaclty when data is read (and written/deleted)

In your current workflow, all the readers essentially run reading in all you data irrespective of if you need it or not

Is it going to be something like this? Or is it going to start deleting records from EASTERN_STATES regardless of if the Tester passes or fails? I need to delete only if the Tester passes.

 

Screen Shot 2021-10-14 at 11.02.53 AM


hkingsbury
Celebrity
Forum|alt.badge.img+53
  • Celebrity
  • October 14, 2021
zzordo wrote:

Is it going to be something like this? Or is it going to start deleting records from EASTERN_STATES regardless of if the Tester passes or fails? I need to delete only if the Tester passes.

 

Screen Shot 2021-10-14 at 11.02.53 AM

Your tester output should be either:

  1. going into a featurereader to read in the EASTERN_STATES data. Then the feature that are read go into a featurewriter to do the delete.
  2. tester output goes straight into the featurewriter and triggers a delete using WHERE 1=1 on the EASTERN_STATES feature class

 

#1 would be more appropriate if you needed to select exactly which features to delete, but will be slower as you need to read all features then delete each one one-by-one

#2 would be faster as you don't need to read in each feature you want to delete and you're pass the load off to the database. However it doesnt make it more difficult to specify which features need deleting


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings