Skip to main content
Question

FME "BatchMode" Missing vs Null - Performance


virtualcitymatt
Celebrity
Forum|alt.badge.img+35

Is there a difference in performance when it comes to working with Missing vs Null values in FME?

 

I'm working with some database tables which have a lot of null values. The output is something like JSON so In the end I want the null values to be missing in the output rather than a null or an empty string.

 

but when it comes for manipulating the data is there a difference between missing and null values?

4 replies

hkingsbury
Celebrity
Forum|alt.badge.img+54
  • Celebrity
  • June 15, 2023

Not a definative answer, but my thought process would be that missing is faster as at the feature schema level, it doesn't exist, fewer fields, less data, faster processing....

 

But I might be wrong!


virtualcitymatt
Celebrity
Forum|alt.badge.img+35
hkingsbury wrote:

Not a definative answer, but my thought process would be that missing is faster as at the feature schema level, it doesn't exist, fewer fields, less data, faster processing....

 

But I might be wrong!

Yeah thanks Hamish - My thinking was that with Bulk Mode it's better to have all the features share the same table structure. I'm not sure if having a lot of missing attributes messes that up. Any ideas @mark2atsafe​ ?

I did some minor testing but results were inconclusive


mark2atsafe
Safer
Forum|alt.badge.img+44
virtualcitymatt wrote:

Yeah thanks Hamish - My thinking was that with Bulk Mode it's better to have all the features share the same table structure. I'm not sure if having a lot of missing attributes messes that up. Any ideas @mark2atsafe​ ?

I did some minor testing but results were inconclusive

I asked around, and we don't really think there would be a huge difference. You'd need to have hundreds of null/empty attributes per feature to even make it worth testing. But if I hear different from another source, I'll let you know.


virtualcitymatt
Celebrity
Forum|alt.badge.img+35
virtualcitymatt wrote:

Yeah thanks Hamish - My thinking was that with Bulk Mode it's better to have all the features share the same table structure. I'm not sure if having a lot of missing attributes messes that up. Any ideas @mark2atsafe​ ?

I did some minor testing but results were inconclusive

Awesome thanks mark. definitely not hundreds - more in the dozens. Seems to be working well either way so I'm happy.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings