Question

Slow XMLUpdater - do others experience this?


Badge +2

Hi all,

 

I have an XMLUpdater setup in my workspace, but it's painfully slow.

 

I'm trying to update XML documents with 10k+ updates in some cases, but it will hang and take forever to complete. On average it's applying approximately 1 "Update" feature per second, meaning 10k "Update" features is taking close to 4 hours (with caching on but it's also slow without cache). The XML has around 200k lines (10 MB file) so that's not helping.

 

I have tried obvious steps such as removing unnecessary attributes and the geometry from the update feature, sorting the order first, and limiting the numbers of calculations/swaps that the XMLUpdater performs, but it still seems slower than it should be...

 

I'm applying a "Replace contents" update type, with XML path of the format "//Element1[@name=fme:get-attribute("_att1")]/Element2/@_att2".

 

The only way I can get a performance improvement is to fragment the XML, extract the attribute I'm matching against, and send the fragments individually to the XMLUpdater and enable group processing. BUT that means it's going to be a headache to stitch back together again which I would prefer not to do (as the original XML structure can change). The XMLUpdater workflow would be perfect if it weren't so slow.

 

Any tips, discussion, or other possible workarounds would be welcome!

 

FME(R) 2021.0.0.0 (20210305 - Build 21302 - WIN64)


0 replies

Be the first to reply!

Reply