Idea: cancelling a job will rollback the whole transaction of all features for Snowflake Writer

​@lauryn00789 Hello and happy new year!

After a quick review with the developers the discrepancy between the number of features written in the job log and the actual features written by Snowflake sounds like a defect. We would be interested in a workspace to reproduce this issue if you can share something with us? Feel free to dm me.

The translation rollback would be a bit trickier and might be achieved using the Advanced → Features per Transaction parameter in the writer, but that depends on the performances of the machine running the translation and how many features you want to write.

Not sure how feasible it would be to rollback transactions that have already been committed to Snowflake. The Job cancellation is more of a “Job stop” in that regard, not a “Job stop & rollback”.

I’m also curious to know what is the reason leading you to cancel a translation on Flow in a normal application? Is it for workspace development/testing purposes?

Cheers

Hi ​@PierreAtSafe, thanks for your reply.

It sounds like this is the expected behavior by design when Features per Transaction is set lower than the overall number of records to be written to Snowflake. It would make sense from a database perspective if cancelled jobs were rolled back completely rather than stopped (for atomicity). If the job is cancelled and data is not rolled back, then the target tables are not usable. It sounds like we may be able to get around this by setting Features per Transaction to a higher value than the expected number of records.

The desire to cancel a job can be for a variety of reasons. For example, we find that sometimes jobs get ‘stuck’ in the running state. Other times, the job may have been accidentally triggered (if a user accidentally re-submits a failed job that’s part of an automation chain, then FME Flow submits all subsequent jobs in the automation chain).