Question

Lost records using Workspace runner

  • 19 November 2022
  • 1 reply
  • 4 views

Badge +1

I have been tweaking workspace briefly described in this question for better performance and encountered weird behaviour on setting up worskpace runner parameters. The workspace is loading XML data, parsing and fetching results to PostGIS database. When I setup the workspace runner to 4 concurrent processes and 4 workspace runs per process I end up with some features not being written. On contrary running directly the child workspace or setting up the workspace runner to just 1 concurrent proccess, all features are being written.

On the comparison you can see limited features written (left, import_RUIAN_c4_w4.log) and all features (right, import_runner_c1_w1.log) from the same XML:

logs.pngLooks like some translations fails in concurrent processes trying to delete some file:

ERROR |AttributeKeeper_6 (AttributeKeeperFactory): Failed to delete the following files: 'D:\Zakazky\APHA\FME\per_partes\import_RUIAN_1b_zapis_log.ffs'.  Please ensure that the files are not in use and that you have sufficient privileges to delete them.

How to prevent failing some of the workspace with more concurrent processes? It would greatly improve the performance. Mentioned logs attached.

EDIT:

I would need the workspace runner to stop writing FFS files. Feature caching is off. How to prevent this?


1 reply

Badge +1

I've probably found out a solution in better definition of Attribute Creator - adding up a condition to write null in case attibute is missing, so FME won't raise a warning in that case and stops to write FFS file.

Reply