Skip to main content
Question

Mapping data types stored as attributes


hkingsbury
Celebrity
Forum|alt.badge.img+54

I feel like I’ve seen this done before, but can’t put my finger on how to do it…..

A warning, I’m working with 2020.2 so cast your mind back a few years…..

 

Essentially I have a workspace that is taking data from MSSQL and writing it out to SQLite. In theory this is just a straight Reader to Writer.

However, the SQLite file is being created fresh each time the process runs and it’s creating any number of tables based on relationships within the MSSQL database - these relationships need to be maintained. Therefore, the schema in the SQLite table is being created with an SQLExecutor and CREATE TABLE statements.

 

Currently, I’m reading the schema from MSSQL and manually mapping each attributes data type from the MSSQL definition to SQLite using an AttributeValueMapper.

 

Now it wouldn’t take too much effort to make this list exhaustive, but there is surely a better way. 

I initially thought of using a schema attribute and let FME do the MSSQL→ FME→ SQLite mapping with a dynamic writer, and that worked. But in SQLite you can only add relationships when you’re creating the table, you can’t add a relationship retrospectively. So that option doesn’t actually get me where I want to be.

FME knows what MSSQL types map to what SQLite types when using readers and writers, so this information must be somewhere. I had a bit of a look through the function and factories and python API to see if anything jumped out at me that would facilitate this - I couldn't see anything immediately.

 

Thoughts?

2 replies

takashi
Influencer
  • May 20, 2025

Hi ​@hkingsbury ,

Data type mapping between any data format and FME is generally defined in a format specific resource file called "metafile" which have been installed into {FME_HOME}/metafile folder. In my quick check, it seems that the mapping definitions for MSSQL and SQLite are defined as ATTR_TYPE_MAP macro in these files, FME 2020.2

  • {FME_HOME}/metafile/mssql_azure.fmf
  • {FME_HOME}/metafile/sqlite3.fmf

I think you can use the mapping definitions to define comprehensive data type mapping between MSSQL and SQLite.

Just be aware that the mapping definition and/or the metafile name could be changed when the implementation of the format reader/writer was updated. 


hkingsbury
Celebrity
Forum|alt.badge.img+54
  • Author
  • Celebrity
  • May 20, 2025
takashi wrote:

Hi ​@hkingsbury ,

Data type mapping between any data format and FME is generally defined in a format specific resource file called "metafile" which have been installed into {FME_HOME}/metafile folder. In my quick check, it seems that the mapping definitions for MSSQL and SQLite are defined as ATTR_TYPE_MAP macro in these files, FME 2020.2

  • {FME_HOME}/metafile/mssql_azure.fmf
  • {FME_HOME}/metafile/sqlite3.fmf

I think you can use the mapping definitions to define comprehensive data type mapping between MSSQL and SQLite.

Just be aware that the mapping definition and/or the metafile name could be changed when the implementation of the format reader/writer was updated. 

Thanks ​@takashi, that was a direction I was exploring after posting the above.

 

I’m now wondering if there is a cleaner way of mapping this that opening up the individual mapping files and ‘manually’ matching them.

I’m going to take a deeper look into some of the fme objects and python api functions


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings