Hi, wondering if anyone know how to limit the featurereader, I am trying to read only the first 10 rows from each table in postgres and it is still pulling back every row before passing on only 10. What am I doing wrong?
Thank you in advance
Hi, wondering if anyone know how to limit the featurereader, I am trying to read only the first 10 rows from each table in postgres and it is still pulling back every row before passing on only 10. What am I doing wrong?
Thank you in advance
Hi @oliver.morris, it's a bit odd that FME is not respecting these settings. I just tested this myself and these settings limit the reader to only reading in 10 features overall, not 10 features from each table. In this situation I would recommend using the SQLCreator transformer to run a SQL query on the data, bringing in only the first 10 rows. This thread on stackoverflow has some great ideas on how to go about this. This page on W3S provides more information on the syntax.
thanks @danminneyatsaf
Hi
What version of FME are you using? In 2024.1 I see no evidence that all features in a feature service are being read before returning a subset in the featurereader. Only this sort of line “Getting feature results 0 up to 1000” where 1000 is the number of features per request in the writer setup
However, in the featurereader there is no max features to read per feature type as there is in the regular reader so you can’t return 10 features per feature type
Replying with Safe support’s response. TL;DR, use Where filter on ObjectID
I understand that you've noticed some interesting behavior with the FeatureReader, and the way it works with the Max Features To Read parameter. The behavior that you're describing is expected behavior in FME, as FME will first pull all features of a source dataset into memory, and then will chop the features in memory down to meet the Max Features to Read parameter's value.
If you would like FME to avoid initially pulling all source features into memory, and just pull the first 100 into memory, you could try taking advantage of the FeatureReader's WHERE clause. This method is a bit tricky with formats that don't natively support typical SQL key words like LIMIT or TOP in the WHERE clause, and the ArcGIS Online Feature Service format is one of these formats.
Luckily most (if not all!) ArcGIS Feature Services do have an OBJECTID field that uniquely identifies each feature, typically in the order that the features were created. Something you could try would be to set the FeatureReader's WHERE clause to OBJECTID <= 100. This WHERE clause should result in FME only committing the first 100 features of the feature service layer to memory.
Give the WHERE clause a try on the FeatureReader, and see if that results in better behavior for your workflow while giving you the results you expect to see!