Skip to main content
Question

Raster mosaicking performance tips on FME Server

nic_ran
Contributor
Forum|alt.badge.img+16

I'm looking for any performance tips for mosaicking a large number of rasters.

I already know about parallel processing and don't need any further advice on that aspect (thanks!).

I'm wondering if anything can be gained from using a large number of FME Server Engines or FME Cloud instances and if there's a simple workflow for achieving this. My initial thought is to build my own version of parallel processing and split the mosaicking into several workspaces for submission by the FMEServerJobSubmitter. Anyone got any other ideas?

3 replies

nic_ran
Contributor
Forum|alt.badge.img+16
  • nic_ranContributor
  • Author
  • Contributor
  • April 5, 2018
PS. I'd be particularly interested in hearing from @donatsafe if there's some way of using Docker Swarms or Kubernetes to help with this. :)

 

jdh
Contributor
Forum|alt.badge.img+28
  • jdhContributor
  • Contributor
  • April 5, 2018

I have in the past when dealing with mosaicking very large amounts of files, created a child workspace the would mosaic a subset of the data (say 512 files) in either a strip or tile (depending on how the input data was structured), and a parent workspace that would call the child workspace for each subset, and then read in the resultant files and mosaic them.

nic_ran
Contributor
Forum|alt.badge.img+16
  • nic_ranContributor
  • Author
  • Contributor
  • April 6, 2018
jdh wrote:

I have in the past when dealing with mosaicking very large amounts of files, created a child workspace the would mosaic a subset of the data (say 512 files) in either a strip or tile (depending on how the input data was structured), and a parent workspace that would call the child workspace for each subset, and then read in the resultant files and mosaic them.

Thanks @jdh. Given the 512 open file limit, this might be the way to go. The child workspaces would each be sent to different Engines, which would help with performance.

 

Reply


Helpful Members This Week

Community Stats

31,478
Posts
119,334
Replies
39,092
Members

Latest FME

Terms & ConditionsCookie settingsAccessibility statement

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.