Skip to main content
Question

Workspacerunner not working for different clippers

  • September 16, 2019
  • 5 replies
  • 16 views

margaret_wei
Forum|alt.badge.img

My workflow is simple. I use shp files generated from Land Cover metadata to clip the ERDAS Imagine file and convert it to polygons. Each shapefile contains a polygon as a clipper.

As instructed by the forum discussion,

1) In Workspace1,

  • I set the workflow options as "Single Merged Feature Type" and I chose one of the shape files in my sample data folder (which contains three sample shp files).
  • The output file is named based on the input shapefile name.

2) In Workspace2 - Path Reader setting, I set the dataset folder to sample folder and only read in shp files.

3) In Workspace2 WorkspaceRunner setting,

  • pointed the FME Workspace to Workspace1 - tile2polygon.fmw
  • set the shapefile to path_windows
  • set the erdas img file to the corresponding img file
  • set the destination folder to an empty folder

 

I run the workspace runner without any issue reported and as I checked the destination folder, there are three files with name corresponding to the input shapefile.

However, all three shape files have the same size and records, i.e, same result. And the result is based on what I chose as a clipper in Workspace1. I doubled checked the three input shapefiles to make sure they have different geometries.

I am wondering if anyone had a similar issue or can help me figure out what is wrong with my configuration.

Thank you!

5 replies

gio
Contributor
Forum|alt.badge.img+15
  • Contributor
  • September 17, 2019

@margaret_wei

 

Afais, you are having workspace1 executed numberofshapefilesinfolder-times, and everytime it reads all shapefiles in the folder of which you choose one, as you say..

AS you say you choose one you will be running the same clipping times numberofshapefilesinfolder"".

So you clip 3x the same extent.

 

You might want to use a FeatureReader in stead of the dynamic reader in Workspace1 for the clipper.

The path_windows attribute would be used by this featureReader to actually clip the img shapfile by shapefile.


margaret_wei
Forum|alt.badge.img
gio wrote:

@margaret_wei

 

Afais, you are having workspace1 executed numberofshapefilesinfolder-times, and everytime it reads all shapefiles in the folder of which you choose one, as you say..

AS you say you choose one you will be running the same clipping times numberofshapefilesinfolder"".

So you clip 3x the same extent.

 

You might want to use a FeatureReader in stead of the dynamic reader in Workspace1 for the clipper.

The path_windows attribute would be used by this featureReader to actually clip the img shapfile by shapefile.

Thank you for your suggestion. I replaced the reader with feature reader.As for Workspace2, I believe I should not change any parameters as I removed the initial reader so the source ESRI shapefiles should still be the same "path_windows".

I am still getting the same issue. All of them getting the result from the original 1028.shp extent.

Is there anything I did incorrectly?

Thank you!


mark2atsafe
Safer
Forum|alt.badge.img+43
  • Safer
  • September 17, 2019

It took me a while - at first I too was a little confused - but the reason the same file size appears is simple. The Clipper transformer has a parameter called Preserve Clippee Extents for raster data. If you have this set to Yes, then the output will always be the same size as the input, so each run gets the same output size.

In other words, the size of your output is always the same because the geographic extent of your data is always the same, which is because it's extents are based on the same raster input unclipped.

So the fact that the files are all the same size isn't a bug to me; the results (cell label/value) will be different for each RasterToPolygonCoercer output, they will just be the same size.

How do we fix this? Well I know I said it's because of the Preserve Clippee Extents setting, but because you eventually convert to vector, it's not the full answer. I'm guessing you want to limit the vector output to the clipped raster data and even using this parameter you would still end up with a rectangle (the bounds of the clipped raster) because a raster dataset is rectangular by necessity, clipped or not.

So... simply add a Tester transformer after the RasterToPolygonCoercer and test for _label = 0,0,0 (or whatever your Nodata value is). Discard that feature and keep the rest. That should give you the correct result.

By the way, I'm not sure if you want all of the cells as polygons in the output, or just one single polygon covering the entire raster. If you do only want a single polygon, then use a Clipper > RasterBandNodataSetter > RasterExtentsCoercer combination in your workspace instead of what you have. Set Nodata to 0 and the extents to the Data Extents.

I hope this helps. In short, your original design is working fine, it's just not doing what you expected because it outputs the entire raster extents. Drop the unwanted part and the results will be fine.


mark2atsafe
Safer
Forum|alt.badge.img+43
  • Safer
  • September 17, 2019
gio wrote:

@margaret_wei

 

Afais, you are having workspace1 executed numberofshapefilesinfolder-times, and everytime it reads all shapefiles in the folder of which you choose one, as you say..

AS you say you choose one you will be running the same clipping times numberofshapefilesinfolder"".

So you clip 3x the same extent.

 

You might want to use a FeatureReader in stead of the dynamic reader in Workspace1 for the clipper.

The path_windows attribute would be used by this featureReader to actually clip the img shapfile by shapefile.

I too was thinking the same thing. But here Workspace2 is the Control (Parent) workspace and Workspace1 is the Helper (Child). So it's acceptable to read a folder because that's the File/Directory reader, not the Shapefile reader! So the original design is sound, it just doesn't do what the user expected.


margaret_wei
Forum|alt.badge.img
mark2atsafe wrote:

It took me a while - at first I too was a little confused - but the reason the same file size appears is simple. The Clipper transformer has a parameter called Preserve Clippee Extents for raster data. If you have this set to Yes, then the output will always be the same size as the input, so each run gets the same output size.

In other words, the size of your output is always the same because the geographic extent of your data is always the same, which is because it's extents are based on the same raster input unclipped.

So the fact that the files are all the same size isn't a bug to me; the results (cell label/value) will be different for each RasterToPolygonCoercer output, they will just be the same size.

How do we fix this? Well I know I said it's because of the Preserve Clippee Extents setting, but because you eventually convert to vector, it's not the full answer. I'm guessing you want to limit the vector output to the clipped raster data and even using this parameter you would still end up with a rectangle (the bounds of the clipped raster) because a raster dataset is rectangular by necessity, clipped or not.

So... simply add a Tester transformer after the RasterToPolygonCoercer and test for _label = 0,0,0 (or whatever your Nodata value is). Discard that feature and keep the rest. That should give you the correct result.

By the way, I'm not sure if you want all of the cells as polygons in the output, or just one single polygon covering the entire raster. If you do only want a single polygon, then use a Clipper > RasterBandNodataSetter > RasterExtentsCoercer combination in your workspace instead of what you have. Set Nodata to 0 and the extents to the Data Extents.

I hope this helps. In short, your original design is working fine, it's just not doing what you expected because it outputs the entire raster extents. Drop the unwanted part and the results will be fine.

Thank you for the tips about setting the NoData value.

I checked my setting, the Preserve Clippee Extents for raster data is set to no.

 

I also did an experiment. In my Workspace1, I set the input shapefile in some other folder and keep the rest setting unchanged. It turns out I got three same files with the input shape file extent but names from the original source file.

I am using Mac, so I also tried using path_unix instead of path_windows. But I still get three same files with different names but the extent from the shape file in Workspace 1 .

Then for Workspace 1,

1) I change the ESRI shp file source to a published parameter with no default value,

2) I change the destination file name referring to $(esri_source);

for Workspace 2, I set the two source to path_unix.

 

Right now, it is working. Although to be frank, I don't know which setting makes it work.

You are right about the nodata, I only need the polygons with non-zero code inside this rectangle.


Reply


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings