Skip to main content
Archived

InlineQuerier: Expose Virtual RAM Option

Related products:FME Form

bwn
Evangelist
  • Evangelist

This should be a fairly simple addition, but it is almost certain that the underlying code in InlineQuerier has an inbuilt option for creating and executing the DB in RAM. All it needs is to be exposed to the user in the Transformer Parameters.

Currently InlineQuerier creates a temporary SQLite database file on disk, and executes SQL queries against this. This makes the Transformer dependent on both the available disk space and the speed of writing/reading from that disk.

However the underlying SQLite engine used by the Transformer has In-memory database creation options as a standard part SQLite. There are three ways of creating different databases, but all use the same base functions and commands.

I've tested all 3 methods with the similar SQLite FeatureWriter, and all 3 seem to work which seems to indicate that FME can already do these, it is just lacking a Parameter in InlineQuerier to expose the different performance options

If you call

"sqlite.exe filepathname" or "rc = sqlite3_open(filepathname, &db)"  depending on accessor being used

then this creates the database on disk. This is what InlineQuerier does now.

However if this is altered to:

"sqlite.exe :memory:" or "rc = sqlite3_open(":memory:", &db)"  depending on accessor being used

This creates the database in RAM, if available.

A third hybrid option is available that again uses the same command, and creates a temporary "pagefile" like file that if allocated RAM is consumed then it will mix data between RAM and on-disk:

"sqlite.exe "" " or "rc = sqlite3_open("", &db)"  depending on accessor being used

SQLite technical documentation here:

https://www.sqlite.org/inmemorydb.html

This post is closed to further activity.
It may be a question with a best answer, an implemented idea, or just a post needing no comment.
If you have a follow-up or related question, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings