Skip to main content
Solved

Feature Reader not respecting max features to read

  • October 27, 2021
  • 7 replies
  • 244 views

oliver.morris
Contributor
Forum|alt.badge.img+12

Hi, wondering if anyone know how to limit the featurereader, I am trying to read only the first 10 rows from each table in postgres and it is still pulling back every row before passing on only 10. What am I doing wrong?

 

Thank you in advancerecord limit

Best answer by danminneyatsaf

Hi @oliver.morris​, it's a bit odd that FME is not respecting these settings. I just tested this myself and these settings limit the reader to only reading in 10 features overall, not 10 features from each table. In this situation I would recommend using the SQLCreator transformer to run a SQL query on the data, bringing in only the first 10 rows. This thread on stackoverflow has some great ideas on how to go about this. This page on W3S provides more information on the syntax.

View original
Did this help you find an answer to your question?

danminneyatsaf
Safer
Forum|alt.badge.img+12

Hi @oliver.morris​, it's a bit odd that FME is not respecting these settings. I just tested this myself and these settings limit the reader to only reading in 10 features overall, not 10 features from each table. In this situation I would recommend using the SQLCreator transformer to run a SQL query on the data, bringing in only the first 10 rows. This thread on stackoverflow has some great ideas on how to go about this. This page on W3S provides more information on the syntax.


oliver.morris
Contributor
Forum|alt.badge.img+12

thanks @danminneyatsaf​ 


gracecai
Contributor
Forum|alt.badge.img+3
  • Contributor
  • February 22, 2025

@oliver.morris did you ever figure out the root cause? I'm seeing the same behaviour with feature services! 


oliver.morris
Contributor
Forum|alt.badge.img+12

Hi ​@gracecai it was a few years back but from what I remember I ended up using the sqlexecutor in the end to get the given number of records.


ebygomm
Influencer
Forum|alt.badge.img+32
  • Influencer
  • February 24, 2025
gracecai wrote:

@oliver.morris did you ever figure out the root cause? I'm seeing the same behaviour with feature services! 

What version of FME are you using? In 2024.1 I see no evidence that all features in a feature service are being read before returning a subset in the featurereader. Only this sort of line “Getting feature results 0 up to 1000” where 1000 is the number of features per request in the writer setup

 

However, in the featurereader there is no max features to read per feature type as there is in the regular reader so you can’t return 10 features per feature type

 

 

 


gracecai
Contributor
Forum|alt.badge.img+3
  • Contributor
  • February 28, 2025

@oliver.morris okay noted, thanks! ​@ebygomm I’m on 2023.2.2


gracecai
Contributor
Forum|alt.badge.img+3
  • Contributor
  • April 7, 2025

Replying with Safe support’s response. TL;DR, use Where filter on ObjectID

I understand that you've noticed some interesting behavior with the FeatureReader, and the way it works with the Max Features To Read parameter. The behavior that you're describing is expected behavior in FME, as FME will first pull all features of a source dataset into memory, and then will chop the features in memory down to meet the Max Features to Read parameter's value. 
 
If you would like FME to avoid initially pulling all source features into memory, and just pull the first 100 into memory, you could try taking advantage of the FeatureReader's WHERE clause. This method is a bit tricky with formats that don't natively support typical SQL key words like LIMIT or TOP in the WHERE clause, and the ArcGIS Online Feature Service format is one of these formats.
 
Luckily most (if not all!) ArcGIS Feature Services do have an OBJECTID field that uniquely identifies each feature, typically in the order that the features were created. Something you could try would be to set the FeatureReader's WHERE clause to  OBJECTID <= 100. This WHERE clause should result in FME only committing the first 100 features of the feature service layer to memory.
 
Give the WHERE clause a try on the FeatureReader, and see if that results in better behavior for your workflow while giving you the results you expect to see!

 


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings