Question

FeatureReader & clip search envelope on raster dataset

  • 29 June 2020
  • 9 replies
  • 38 views
pdorio
Rank
Userlevel 2
Badge +4

The feature reader does not allow you to make a clip on the initiator geometry.

 

I need to read a fairly large raster (13GB). Input I get some small zone geometries to analyze. The idea was to create a BBOX on the geometry and use the feature "Spatial filter: intersects or bbox intersects" to load only the part of the raster needed for the analysis. Even configuring intersects or bbox on the feature reader, in output there is always and anyway all the raster. This problem is great because it slows me down considerably the server side script.

 

If you add a raster reader instead you can check the "Clip on search envelope" flag and it works. But without FeatureReader i can't give my search envelope by the initiator.

 

Could this be a missing-feature?

Does anyone have a workaround?

 

I use the FME Desktop version 2019.1

9 replies

ebygomm
Rank
Userlevel 1
Badge +10

In the past I've ended up using a workspace runner to get round this issue, so a second workspace with a normal reader and clip on search envelope. Set the BBOX extents as user parameters

Idea here

https://knowledge.safe.com/content/idea/43982/featurereader-option-to-clip-to-spatial-extent-whe.html

nielsgerrits
Rank
Userlevel 6
Badge +32

In the past I've ended up using a workspace runner to get round this issue, so a second workspace with a normal reader and clip on search envelope. Set the BBOX extents as user parameters

Idea here

https://knowledge.safe.com/content/idea/43982/featurereader-option-to-clip-to-spatial-extent-whe.html

Great alternative, but for desktop only. When on server, you need at least 2 engines (and a FMEServerJobSubmitter) to do this.

ebygomm
Rank
Userlevel 1
Badge +10

Great alternative, but for desktop only. When on server, you need at least 2 engines (and a FMEServerJobSubmitter) to do this.

I don't believe you do need 2 engines to use a job submitter

When Wait for Server Job to Complete is set to Yes, and multiple FMEServerJobSubmitter transformers are linked to manage the workflow of multiple jobs on the same FME Server, all jobs run as child processes of the same FME engine that is engaged by FME Server to run the first job. No other FME engine is engaged. As a result, child jobs do not appear in the FME Server job queue or job history, and do not have their own job_id's. Additionally, all job logs are included in the job log of the parent (controller) job. For more information, see "FME Server Workflow Management,"

http://docs.safe.com/fme/2016.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/fmeserverjobsubmitter.htm

nielsgerrits
Rank
Userlevel 6
Badge +32

I don't believe you do need 2 engines to use a job submitter

When Wait for Server Job to Complete is set to Yes, and multiple FMEServerJobSubmitter transformers are linked to manage the workflow of multiple jobs on the same FME Server, all jobs run as child processes of the same FME engine that is engaged by FME Server to run the first job. No other FME engine is engaged. As a result, child jobs do not appear in the FME Server job queue or job history, and do not have their own job_id's. Additionally, all job logs are included in the job log of the parent (controller) job. For more information, see "FME Server Workflow Management,"

http://docs.safe.com/fme/2016.1/html/FME_Desktop_Documentation/FME_Transformers/Transformers/fmeserverjobsubmitter.htm

I stand corrected, thanks! I do read quite a bit of documentation but still not enough apparently...
pdorio
Rank
Userlevel 2
Badge +4

If, as I understand it, this is a missing-feature, I hope it can be taken into consideration for future developments

pdorio
Rank
Userlevel 2
Badge +4

Does not work also with 2020.0.3

ebygomm
Rank
Userlevel 1
Badge +10

If, as I understand it, this is a missing-feature, I hope it can be taken into consideration for future developments

The more people vote for the idea, the more likely it will be given consideration

F

I voted for the idea (which is more general than the current topic as it suggests adding the feature for any type -raster and vector- of input data) and posted a comment, as my real requirement would be to perform a RasterSubsetter on an approximation (bounding box for example) of the shape of the Initiator feature, rather than an acurate Clipper. This would allow for avoiding any data transformation apart from selecting a subset of the source image.

amdubois

I'm using FME 2020.2 and it's still not an option. I can't believe there is no going around this? My raster is almost 30 Go and I must read it (a process taking around 3-4 mins) everytime it loops in the workspace runner. Which is looping more than 2000. This really is not optimal at all ...

Reply


Community Stats

28,669
Posts
109,361
Replies
36,531
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.