How to keep full number accuracy in JSON contents, when using JSONiq to query a JSON Array?

I don't know how it would compare performance wise, but if you just need the BAG_ID couldn't you just do this in a JSON Fragmenter

json[*]["attributes"]["BAG_ID"]

json[*]["attributes"]["BAG_ID"]

Hi @ebygomm,

Thanks for your reply. If I would only need the BAG_ID that would indeed be a good option. 

In my case however I need to extract all (whole) JSON Objects in the array, which contain a BAG_ID attribute (and then I also need to retain the full accuraccy of the BAG_ID).

So basically I need to thin out the array for only relevant JSON Objects, since in the true datasets these contain several attributes/objects that need to be processed (like building height, related adresses, building function, geometry, etc).

Probably one for safe I think, it doesn't seem to matter how you process the data within a XMLXQueryExtractor, you always end up with the truncated value. I'd be interested to know the outcome.

If performance was essential I'd probably end up resorting to python

import fme
import fmeobjects
import json
 
class FeatureProcessor(object):
    def __init__(self):
        pass
    def input(self,feature):
        self.js = json.loads(feature.getAttribute('sample_json'))
        
    def close(self):
        for item in self.js:
            attrs = item["attributes"]
            if "BAG_ID" in attrs:
                feature = fmeobjects.FMEFeature()
                feature.setAttribute("BAG_ID",attrs["BAG_ID"])
                feature.setAttribute("json_fragment",json.dumps(attrs))
                self.pyoutput(feature)
            else:
                pass

Hi @ebygomm​ ,

I'm a bit of a Python noob, but I am interested in learning and doing more with it, specifically in cases like these where you just want to do a bit more with your data. That code snippet is really insightfull, and indeed it works!

One minor thing, in my case I wanted the json_fragment to be the 'item' and not the 'attrs' object, but thats an easy fix.

Although I somewhat still prefer the JSONiq code (a. I'm just more familiar with it, and b. I think the code is a bit shorter/cleaner), its really nice to see and learn you have options. 🙂

Although you provided a very good solution just now, I hope Safe (or a fellow FME'er) can have a look at the main question 'How to keep full number accuracy in JSON contents, when using JSONiq to query a JSON Array?', as it's more on JSONiq and its possibilites in FME. So I'm not selecting it as best answer yet if you don't mind.

Even if you just put the following in a JSONTemplater, that value gets truncated in the output

[
	{
		"id": 1,
		"name": null,
		"attributes": {
			"type": "Bridge"
		}
	},
	{
		"id": 2,
		"name": "just a shed",
		"attributes": {
			"type": "Shed"
		}
	},
	{
		"id": 3,
		"name": "just a house",
		"attributes": {
			"type": "Building",
			"BAG_ID": 1.5010000000035E14
		}
	}
]

Yes, I noticed, see also my earlier screenshot with the disabled JSONTemplator (and the annotation at it). 

However, that problem can quite easily be circumvented by adding a 'type declaration' in the JSONTemplator, i.e. replace the above line 21 with:

"BAG_ID": xs:decimal(1.5010000000035E14)