Anyone notice that spatial index is being built on a non-geometric feature class using 2019.1?

Just to confirm, you are using the Postgres writer, not the PostGIS one, right? Is it also creating a geometry column in the output?

Yes, Postgres (no geom column), not PostGIS.

Could you post the log messages -- I'm wondering if this is coming from our internal feature caching storage and is unrelated to your source. Having the full log or at least the section in question could help. I agree there shouldn't be a spatial index being created, but seeing what is going on will help alot to figure out what is actually happening.

Test_noFeatureCache.log.txtTest-withFeatureCache.log.txtHere are the log files. It looks like it only happens when enabling Feature Caching. Postgres table attributes are being joined with Shapefile but Postgres rows are also being spatially indexed (how?). When a lot (>13M) of rows are being read from the Postgres table, then this takes quite some time...

Here's a full run including 15M+ postgres rows.

Excerpt from the log:

Attached is the "Mega" log.Test-fullRunWithFeatureCache.log.txt

Please let me know if I'm not grasping something here. A.) there is a substantial lag at the end of adding features to a spatial index. And of course, the spatial index is invalid for a non geometric table.

Thanks for the logs. That clears it up a lot. I can recreate the issue now. I'll get in touch with the developers and let you know what they say,

Regards,

Mark

For your reference this is filed with the developers under the ID FMEENGINE-60953.

Is there any way you can share this workspace with us at Safe? I can replicate the messages easily enough, but I can't create a workspace where the time taken under those messages is as long as you experience. I'm using the same number of features, but I just can't get the same results. I think the best way is for us to try and duplicate your exact scenario, here at Safe. Safe.com/support would be the best place to create a case, and submit your workspace. I'm really hoping that we don't need the data as well. I think if we can inspect the workspace then that will be enough to put us on the right track. Thanks for your patience.

Done. The case number is C148442 . The workspace is not complex. I attached it to the case. We deal with a huge amount of records on almost a daily basis, there is always a need for shorten processing time (like I said, there were >13M rows read and then a significant number of time was spent creating a Spatial Index where none was necessary). Let me know if I can help some more...

@mark2atsafe​ any updates on this? I have a feature joiner taking in tabular data that is resulting in a message 'Spatial Index: Adding 19530 features'. Is this to be expected. Using FME 2023

Yes, that's absolutely fine. I think the message is badly worded because it includes tabular as well as spatial data in the index, but it's definitely OK behaviour. You should only need to worry if that message was seen to be taking a substantial amount of time to process. If the log shows it was just a fraction of a second of processing time, then all should be fine.

I see. My workspace was failing after the message I included, but I've been having some processing issues that I think are mostly tied to my PC's memory capacity. Closing programs and disabling unnecessary sections of the workspace tends to help when I need to run it with feature caching enabled.

I installed 2023.1.1 yesterday and the workspace is running great, so in the end, all is good for now. Thanks for your response.