Skip to main content
Question

How to store the result of a where close to use in the where clause of a reader ?


tsirkuse

I use a SQL creator on a Oracle Spatial database extracting around 40 columns.

I would want to score the fid of this extraction in order to use it in the where clause of a post-gis reader. This to avoid the post-gis reader to read all 1 000 000 records when I only need 300 of those.

How can I do this ? I did some research but seems I am using the wrong key words as I can't find it.

Thank you

 

 

9 replies

david_r
Evangelist
  • March 7, 2019

Rather than using the regular reader, you could simply connect your SQLCreator to a FeatureReader, with the FID in the where-clause.


arnovananrooij
Contributor
Forum|alt.badge.img+5

Hi @tsirkuse, I would use the DatabaseJoiner after the SQLCreator to read the PostGIS database.


itay
Supporter
Forum|alt.badge.img+16
  • Supporter
  • March 7, 2019

A nice trick on how to create the where IN clause (something I learnt a long time ago from @erik_jan and I am still using) is:

If you have >1000 features (ID's) you will need to do the following:

  • Add a Counter starting with 0
  • Add an ExpressionEvaluator with floor(@Value(_count)/1000,0).
  • Add an Aggregator with group by on _result and concatenate the attributes you want in the where clause.

Hope this helps.

Itay


tsirkuse
  • Author
  • March 8, 2019
arnovananrooij wrote:

Hi @tsirkuse, I would use the DatabaseJoiner after the SQLCreator to read the PostGIS database.

Ah, tried that first, not possible cause what I want to join is the geometry. Except if there is a work around for that problem?


tsirkuse
  • Author
  • March 8, 2019
david_r wrote:

Rather than using the regular reader, you could simply connect your SQLCreator to a FeatureReader, with the FID in the where-clause.

I don't see how that helps me not reading half a milion post gis records I don't need.... Maybe I am missing something, but I really don't see it. I want to use the id's selected by my sqlcreator (oracle) in my post-gis reader. So I need to somehow store the values in a list that I can use in the where clause...


ebygomm
Influencer
Forum|alt.badge.img+31
  • Influencer
  • March 8, 2019

I've always found the most efficient way to do this is to write a temporary table in the POSTGIS database then use an sqlexecutor to return the matched records.

 

 


arnovananrooij
Contributor
Forum|alt.badge.img+5
tsirkuse wrote:

Ah, tried that first, not possible cause what I want to join is the geometry. Except if there is a work around for that problem?

Aha @tsirkuse, then you should use the FeatureReader. This transformer is the same as the DatabaseJoiner except that you can also do a spatial join..


david_r
Evangelist
  • March 12, 2019
ebygomm wrote:

I've always found the most efficient way to do this is to write a temporary table in the POSTGIS database then use an sqlexecutor to return the matched records.

 

 

I agree, in particular if there's a non-trivial number of records to join. Although it does depend on having sufficient rights.


david_r
Evangelist
  • March 12, 2019
tsirkuse wrote:

I don't see how that helps me not reading half a milion post gis records I don't need.... Maybe I am missing something, but I really don't see it. I want to use the id's selected by my sqlcreator (oracle) in my post-gis reader. So I need to somehow store the values in a list that I can use in the where clause...

This is what I had i mind, it will prevent you from reading the entire postgis table and as long as the FID column is indexed it should be fairly quick:

Although if you have a "large" number of FIDs, then you should look into the suggestion from @egomm, it will scale much better (but requires that you have sufficient rights).


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings