Skip to main content
Solved

FeatureReader Cache

  • April 10, 2020
  • 2 replies
  • 228 views
arnold_bijlsma
Enthusiast
Forum|alt.badge.img+15

I just found the "Enable Cache" option on the FeatureReader, but it's not entirely clear from the documentation when this cache is renewed.

Assume I have set the "Cache Timeout" parameter to a very long time, say a week or longer.

  1. Will the cache only remain during a session, or will it still exist when I close the workbench, and reopen it later that day?
  2. Will the cache still exist when I close the workbench, close down the machine, and reopen the workbench the next day?

The reason for asking is that some remote datasets are pretty heavy and only change occasionally, so I wish I could minimise the number of times reading it.

Note: I am not talking about workbench feature caching here!

Best answer by bwn

I agree the documentation is not clear, but it is fairly easy to test. Surprisingly, it appears the cache IS permanent between FME sessions, and so this option is smarter than I thought.

In my testing, the first run of the FeatureReader will build the cache in the FME Temp folder.

Every subsequent run WorkBench reads from the cache, even if:

  • You run the Workflow again from scratch a second time
  • You close the project and re-open it and run from scratch
  • You close FME Workbench, re-open the application, re-open the Project, and run form scratch

The documentation hints at this by suggesting that the expiry time on the cache is actually stored as a physical data attribute within the cache files themselves rather than being only stored during run-time.

That's actually quite handy from a development perspective for design/testing purposes, as that means your test Workspace working with a static offline data extract doesn't need to re-read the data on every session, saving test time. I almost wish the plain Readers would have something like this, because there are things the Readers can do (like limiting which attributes to extract from the Source with SQL), that FeatureReader can't.

This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

2 replies

bwn
Evangelist
Forum|alt.badge.img+26
  • bwnEvangelist
  • Evangelist
  • Best Answer
  • April 11, 2020

I agree the documentation is not clear, but it is fairly easy to test. Surprisingly, it appears the cache IS permanent between FME sessions, and so this option is smarter than I thought.

In my testing, the first run of the FeatureReader will build the cache in the FME Temp folder.

Every subsequent run WorkBench reads from the cache, even if:

  • You run the Workflow again from scratch a second time
  • You close the project and re-open it and run from scratch
  • You close FME Workbench, re-open the application, re-open the Project, and run form scratch

The documentation hints at this by suggesting that the expiry time on the cache is actually stored as a physical data attribute within the cache files themselves rather than being only stored during run-time.

That's actually quite handy from a development perspective for design/testing purposes, as that means your test Workspace working with a static offline data extract doesn't need to re-read the data on every session, saving test time. I almost wish the plain Readers would have something like this, because there are things the Readers can do (like limiting which attributes to extract from the Source with SQL), that FeatureReader can't.

arnold_bijlsma
Enthusiast
Forum|alt.badge.img+15

I agree the documentation is not clear, but it is fairly easy to test. Surprisingly, it appears the cache IS permanent between FME sessions, and so this option is smarter than I thought.

In my testing, the first run of the FeatureReader will build the cache in the FME Temp folder.

Every subsequent run WorkBench reads from the cache, even if:

  • You run the Workflow again from scratch a second time
  • You close the project and re-open it and run from scratch
  • You close FME Workbench, re-open the application, re-open the Project, and run form scratch

The documentation hints at this by suggesting that the expiry time on the cache is actually stored as a physical data attribute within the cache files themselves rather than being only stored during run-time.

That's actually quite handy from a development perspective for design/testing purposes, as that means your test Workspace working with a static offline data extract doesn't need to re-read the data on every session, saving test time. I almost wish the plain Readers would have something like this, because there are things the Readers can do (like limiting which attributes to extract from the Source with SQL), that FeatureReader can't.

Thanks for checking this out. I probably could (should?) have tested it myself as well. At the same time, it's probably worth putting it out here in the open, as I think it's indeed a very underrated feature, which more people should know about!

Community Stats

32,219
Posts
122,352
Replies
39,914
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.