Question

How to enable "Max Features to read per Feature Type" with database tables

  • 7 November 2022
  • 4 replies
  • 11 views
whkarto
Rank
Badge +11

Hi FME-Experts,

I'm trying to read features from an Oracle Spatial Table by the corresponding reader. Although reader option "Max Features to Read per Feature Type" has been set to 20, FME didn't stop reading. (See Screenshot) Does anybody have an explanation why this behaviour occurs?

Actually I'm not reading only one table but three, so switching over to "Max Features to Read" is not really an option. With this option, reading would really stop after the given number of features has been reached. (but only for one Feature Type)

 

Looking forward to your answers!

(FME Desktop 2022.1.1 Build 20220728, identical behaviour in FME 2021.2)

1 Attachment

4 replies

f.kemminje
Badge +3

Read oracle spatial table from SQL executor transformer.

use rownum in the statement

 

select * from tablename where rownum < 1000. something like this

whkarto
Rank
Badge +11

Read oracle spatial table from SQL executor transformer.

use rownum in the statement

 

select * from tablename where rownum < 1000. something like this

Sure I could do this (and have actually done), but let's imagine we don't have three but thirty, or even more Feature Types to handle. Equipping everyone with a Where clause instead of setting one single parameter on the Reader is truly not an option, especially if we're just up for a sneak preview of the data.

virtualcitymatt
Rank
Userlevel 4
Badge +26

Haha, oh dear - this seems a but silly. Looks like a bit of a bug to me. Have you tried it with any other formats at all?

In your screen shot there seems to be an option to specify the feature types​ to read. Does the behavior change if you set that?

Either way this isn't really working as I would expect to.

@steveatsafe​ ? Any ideas? ​

steveatsafe
Rank
Badge +11

Hi Guys,

Oh Dear, and now I must come clean.

Thanks for sharing and I understand your frustration.

Unfortunately, this feature/parameter is a little misleading. FME has actually always read all the data from the tables but then only spit out the "Max Features to Read per Feature Type"  or "Max Features to Read"  (whichever is in use) the number of records to the rest of the workspace.

 

That would still be helpful and it appears that might be still happening (at least in the canvas - just not in the logging). So you can safely ignore the logging... (yes that might be new). My repro shows the count is not reliable... for I had 870,000 records but only saw counters for max 100,000... I'm filing an issue to get this sorted out.

Question: Do you find that more than 20 features actually get passed to the next transformer (Counter?)? I could not reproduce this.

 

I could reproduce the behaviour that FME continues to read the features of large tables.

 

For your workaround I would make use of the "WHERE Clause" on the Reader (not each Feature Type) and set this to ROWNUM < # as seen in the attached screen grab.

 

Where ClauseWhat looks like a change of behaviour is that we are now logging the features being read out to the log, exposing our secret "full" read.

 

Thanks for sharing.

SteveatSafe

Reply


Community Stats

28,723
Posts
109,511
Replies
36,560
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.