Hi FME world,
I have two tables in a database (ArcGIS SDE). They are effectively a 1:1 match, linked via a common ID. Table 1 features are always present in the flow (filtered prior) and I want to read in ONLY the features from table 2, where match with the Table 1 filtered features (because there are 1m+).
I’ve tried options like aggregating the Table 1 features and concatenating the ID, passing into Table 2 feature reader with a where clause. However, there is a maximum limit of queries the database can do (somewhere between 10,000-15,000 - and often I will need to read in more than this.
My next thought is to create a looping transformer where I sample the T1 features into groups of 10k, aggregate, pass to FeatureReader, reads in associated 10k T2 features, then loop back to read in the next 10k, until there are no more features to be read (using a NoFeaturesTester to exit the loop.
I’m a bit stuck on how to actually implement this though. Any ideas/thoughts much appreciated! Or if i’m missing something really obvious, please tell!
Thanks!
