Question

InLineQuerier Data Types, Numeric, Real or Float?


Badge +2

Per SQLite documentation, Real and Numeric are supported and Real is of a Float type.

However in InLineQuerier Numeric is not an option, only Real and Float to pick.

Due to precision issues of floating point, we would prefer to use Numeric in several cases.

Please advise.


4 replies

Badge +5

I think you should file a case with the support team to ask about this (safe.com/support) - nothing I've searched for has led me to an answer.

Badge +2

Thanks, I filed the support ticket.

Userlevel 4

While waiting for a reply / fix from Safe, you can use the SQLite built-in round() function with a specified precision.

Example where you want to join two tables based on floats with two decimals of precision:

select *
from table1 join table2 on 
  round(table1.value1, 2) = round(table2.value2, 2)

David

Badge +3

Old thread, but noting that SQLite is weakly typed and so it may not matter what you declare in InlineQuerier (and I kind of wonder what having the Field Data type options in the Transformer do as a result). Seeing as most downstream transformers in FME seem to work without needing a particular data type as well you could just leave everything as VarChar and it shouldn't matter? The data is stored exactly the same way regardless of the field data type declaration. You can declare a field as INTEGER in SQLite and yet still store "abc" or 1.245 without any issues (tested in 2018 InlineQuerier), the Type declarations are more to provide information to any external interfaces or functions as to what type of data it should expect to retrieve through the connection to the DB. You can create and store data in SQLite tables without any field Type declaration at all. As David_r suggests, if you want numerical data to a particular precision, this needs to be done before pushing it into InlineQuerier/SQLite as SQLite just takes whatever raw data it gets and stores that value in whatever it thinks is the most efficient, regardless of field type declaration. As a by-product, this is also why SQlite DBs are extremely space efficient. The DBs hold each individual data value in its most efficient binary format, decided value-by-value when written to the DB, regardless of field data type declaration, although this does create some traps in some SQL statements not behaving quite as you would expect for those used to working with strongly typed DBs that "force" each data value to be stored in a particular way.

Reply