Question

Slow reader creation?

  • 26 April 2017
  • 5 replies
  • 8 views
rclarkenz
Rank
Badge +1

I'm new to FME so would like some help understanding what is happening in the backend. I am running FME 2017 Desktop.

I have the below workspace which is extracting the min and max band figures of an ECW and logging whether all of the bands have a min of 0. At present I am testing with only 3 ECW files.

0684Q00000ArExdQAF.png

I have been watching the process run, and I saw the the RasterBandMinMaxExtractor took around half an hour which I expected. However, once the first feature reached the appropriate logger, there was then nearly a 45 minutes wait before the next feature was run through the process.

This is what appears in the Translation Log following the final Logger line.

2017-04-26 14:23:04|2301.7|  0.0|INFORM|================================================
2017-04-26 15:07:26|3096.3|794.6|INFORM|Creating reader for format: ER Mapper ECW

I checked the temp directory and there were no temp files being created during the time. So I'm a bit stumped on what's going on.

TIA

5 replies

itay
Rank
Badge +16

by temp location do you mean the one FME uses or you own temp location?

mmijn-documentenmijn-afbeeldingentemplocation.png

rclarkenz
Rank
Badge +1

The one FME uses. 

2017-04-26 13:12:56|   0.2|  0.0|INFORM|FME Configuration: Temporary folder is `H:\FME\Temp', set from environment variable `FME_TEMP'
itay
Rank
Badge +16

Ok, if you don't need the geometry after extracting the min and max, does dropping it speed up the process?

L
Badge

When you use the logger, feature information goes in the log file (MyWorkspace.log) but there is also a FFS (FME Feature Store) file created to write the feature (MyWorkspace_log.ffs). I assume that this is the case and that your raster data is written to this FFS file. Check the folder where your log file is written and see if you have a large FFS file sitting there. To work around that, drop the geometry as @itay suggested or use something else than a Logger to store your information in the log file as described here: Logging with Python scripts

rclarkenz
Rank
Badge +1

I tried removing the geometry and that made no difference. I checked the Temp directory and the log file location and there were no FFS files being created. I found by removing RasterBandMinMaxExtractor that the speed was fine. I had originally been using this transformer to see if there was any black in the raster, which would be a good indicator that there is NODATA. As this was taking too long I decided to take another approach.

Reply


Community Stats

28,661
Posts
109,324
Replies
36,528
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.