Okay newbie here (sorry in advance)--so I plopped an MSSQL reader into my workspace and set the max count for each feature paramater to read to 10. And indeed, the cache count stops at 10. However, in the log I can see that it is still going to town on 4 million spatial records in the SQL DB and the bucket keeps filling up even after it reaches the max 10 count. In other words, setting the max features to 10, 100, or 1,000,000 takes exactly the same amount of time (and resources) for the workspace to complete its processing. What am I doing wrong here? THANKS!
Question
Max Features to Read in name only?
This post is closed to further activity.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
It may be an old question, an answered question, an implemented idea, or a notification-only post.
Please check post dates before relying on any information in a question or answer.
For follow-up or related questions, please post a new question or idea.
If there is a genuine update to be made, please contact us and request that the post is reopened.
+26- Contributor
Great question!
What FME version are you on?
If this is the case, it seems like a bug to me and I recommend submitting it to https://www.safe.com/support/report-a-problem/
A work around would be adding a WHERE clause to the table to limit the number of records to 10, which is what the mentioned setting should do.
Or using the SQLCreator transformer to post a specific SQL statement, but only as work around.
Thanks for posting the question.
+5- Contributor
Great question!
What FME version are you on?
If this is the case, it seems like a bug to me and I recommend submitting it to https://www.safe.com/support/report-a-problem/
A work around would be adding a WHERE clause to the table to limit the number of records to 10, which is what the mentioned setting should do.
Or using the SQLCreator transformer to post a specific SQL statement, but only as work around.
Thanks for posting the question.
That does work thank you for that suggestion (a little extra step but thats ok). I am using 2019.1 --maybe an upgrade will help fix this. Thank you again! Looking forward to using FME more!
+11- Safer
Hi @louielou1234,
If I remember correctly, and for databases especially, it's not a true Max Features to Read, but as you've found reads the whole table and then only offers up the requested feature counts, so this is expected behavior.
@erik_jan's workaround would be recommended in this case to let the database do the work for better performance, with a WHERE clause in the SQLExecutor/Creator, or perhaps even the FeatureReader. There's a mention here in the doc on Limiting the Number of Features to Read for a bit more information.