Skip to main content
Open

JSON processing speed

Related products:Transformers
  • June 25, 2019
  • 1 reply
  • 14 views
S
Forum|alt.badge.img

Please please please :-) do some work to improve processing speed in FME of JSON.


Using the JSON fragmentor I'm processing a JSON document with 20k simple records. FME processes each record in ~2-3 seconds. Unusably slow.

Using Python I can process to document is ~5 seconds.

A sample document is available here and I'm interested in the attributes of the feature array.


1 reply

revesz
Supporter
Forum|alt.badge.img+21
  • reveszSupporter
  • Supporter
  • July 1, 2019

I would add XMLXQueryExploder to the improve list. I have a feeling that the whole XML is read and processed at every feature extracted.

I guess the the problem is the same with XMLFragmenter.

 

Community Stats

32,707
Posts
124,102
Replies
41,180
Members
Terms & ConditionsAccessibility statement
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.