Skip to main content
  • EN

Hi, I'm running a modified version of Dmitri Bagh's LAS to DEM template (https://hub.safe.com/templates/las-to-dem), as I am generating hillsides (and other LAS based rasters) from approximately 2700 LAS files. While this works great when feeding one LAS file at a time in, when batching the process using the Batch Deploy wizard, it becomes very inefficient.

The workspace by default will load all 2700 potential surrounding LAS files to the tile I'm working on, buffer, select only the data it needs, and then run. When batching it, it will load the 2700 LAS files to memory each iteration. This iterates 2700 times. Not the best method to do this.

Is there any way to run this in such a fashion that it only loads the LAS dataset once, then loops through the dataset one LAS tile at a time, and processes that? I've tried modifying this to use the WorkspaceRunner (and take advantage of parallel processing), but everything I've tried runs into it reloading the whole dataset each iteration.

Thanks!

My first tip would be to skip the batch deployment wizard and go straight for the WorkspaceRunner.

Can you be a bit more specific about what exactly you tried with the WorkspaceRunner?

My first tip would be to skip the batch deployment wizard and go straight for the WorkspaceRunner.

Can you be a bit more specific about what exactly you tried with the WorkspaceRunner?

I had tried using the workspace runner to process this workbench, and used a Directory and File Pathnames reader as the source. I've attached screenshots below.

Here is my main workbench:

And here is the workspace runner workbench, with parameters shown:

 

Thanks for your assistance!

 

- James

Thanks for posting the screenshots, that's very helpful.

In the WorkspaceRunner transformer you're transmitting the value of path_directory_windows which will contain the directory name, not the LAS file name. That would explain why the LAS reader in the child workspace reads the entire directory each time it's invoked.

Rather than using path_directory_windows in the WorkspaceRunner, try using path_windows, which contains the LAS filename.

Thanks for posting the screenshots, that's very helpful.

In the WorkspaceRunner transformer you're transmitting the value of path_directory_windows which will contain the directory name, not the LAS file name. That would explain why the LAS reader in the child workspace reads the entire directory each time it's invoked.

Rather than using path_directory_windows in the WorkspaceRunner, try using path_windows, which contains the LAS filename.

My apologies for this, but in my haste to respond I had built the example incorrectly. I had used path_windows in my tests. I rebuilt this example in my response to you, and picked the wrong variable.

 

 

While it only passes one file at a time into the WorkspaceRunner, the workspace inside the WorkspaceRunner will load the entire 2700 LAS files each time. The problem isn't with the "top" reader (the reader which receives the filename specified in path_windows) in my main workspace, it's the "bottom" one.

 

 

From an optimization standpoint, re-reading 2700 files for each iteration of the WorkspaceRunner is very inefficient, and am hoping I can rejig this process to be better.

 

 

My apologies for this, but in my haste to respond I had built the example incorrectly. I had used path_windows in my tests. I rebuilt this example in my response to you, and picked the wrong variable.

 

 

While it only passes one file at a time into the WorkspaceRunner, the workspace inside the WorkspaceRunner will load the entire 2700 LAS files each time. The problem isn't with the "top" reader (the reader which receives the filename specified in path_windows) in my main workspace, it's the "bottom" one.

 

 

From an optimization standpoint, re-reading 2700 files for each iteration of the WorkspaceRunner is very inefficient, and am hoping I can rejig this process to be better.

 

 

I can't see which parameter is linked to what reader or setting, so it's difficult to say, but it's clearly a configuration issue. In the WorkspaceRunner, try setting the second parameter to path_directory_windows and the third parameter to path_windows or path_filename.

 

 

Reply


Community Stats

30,162
Posts
114,480
Replies
37,807
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.