Solved

Iterate features in FME ?

  • 29 October 2018
  • 6 replies
  • 40 views

Badge +1

Hi, I have a model in ARCGIS model builder that perfectly iterates through features. What i do: I grab a polygone from a dataset make buffer around and than clip over it 3 other shapes - I would like to do this for 30 polygones (each with uniq ID).

In FME workspace i built a complex model with a tester transformer, but it it is too complex, because one of the shapes that overlays and have to be cliped has over 7.5 million of rows.

Any solution? Thanks

icon

Best answer by redgeographics 29 October 2018, 10:02

View original

6 replies

Userlevel 5
Badge +25

The 7.5 million features might pose a problem, but in general terms you could set up a workspace that takes your clipper features as well as the 3 other inputs. Run them through a Clipper which you've set to Clippers First (and of course make sure the clipper features are being read first by dragging that reader to the top of the Navigator panel)

Then set a fanout on your output feature type name and it will generate separate files with those unique id's.

Key here is to make sure to put the clipper features in first to cut down on memory usage.

Badge +1

The 7.5 million features might pose a problem, but in general terms you could set up a workspace that takes your clipper features as well as the 3 other inputs. Run them through a Clipper which you've set to Clippers First (and of course make sure the clipper features are being read first by dragging that reader to the top of the Navigator panel)

Then set a fanout on your output feature type name and it will generate separate files with those unique id's.

Key here is to make sure to put the clipper features in first to cut down on memory usage.

 

Hi, Can you help me a little bit more with writter settings. I am confused, how to set a writer to make several shapefiles from one database ? What shapefile definition do i use ?
Badge +1

The 7.5 million features might pose a problem, but in general terms you could set up a workspace that takes your clipper features as well as the 3 other inputs. Run them through a Clipper which you've set to Clippers First (and of course make sure the clipper features are being read first by dragging that reader to the top of the Navigator panel)

Then set a fanout on your output feature type name and it will generate separate files with those unique id's.

Key here is to make sure to put the clipper features in first to cut down on memory usage.

model-ehis.png

 

 

I have set the model in FME like this, using tester transformer, to grab each shape and make intersections and write down dwg files, but it is not working due to memory limitations - i use 32 bit FME.

 

 

Userlevel 5
Badge +25
model-ehis.png

 

 

I have set the model in FME like this, using tester transformer, to grab each shape and make intersections and write down dwg files, but it is not working due to memory limitations - i use 32 bit FME.

 

 

If you have the option to go to 64 bit FME I would highly recommend that. In the meantime, I've put together a small sample for you to look at.

 

clipping.fmwt

 

 

Note that I'm reading in 4 shapefiles, one of which, the ForwardSortationAreas has a different schema than the other 3. I clip all of that with the neighborhood boundaries and use a Shapefile writer with Dynamic Schema Definition. It looks at the original shapefiles for their schema's and applies those, then sets the output filename to be the original feature type plus the neighborhood name.

 

 

Hope this helps.

 

Badge +1
If you have the option to go to 64 bit FME I would highly recommend that. In the meantime, I've put together a small sample for you to look at.

 

clipping.fmwt

 

 

Note that I'm reading in 4 shapefiles, one of which, the ForwardSortationAreas has a different schema than the other 3. I clip all of that with the neighborhood boundaries and use a Shapefile writer with Dynamic Schema Definition. It looks at the original shapefiles for their schema's and applies those, then sets the output filename to be the original feature type plus the neighborhood name.

 

 

Hope this helps.

 

 

Hi,

 

I have managed to set the writter to export several shapefiles in a file. The problem in reading 7.500.000 features was, that i didnt have spatial indexes of the imported shapefile. As soon as we calculate the spatial indexes, process runed smootly threw the model and it was much faster.

 

 

Thanks a lot in your help!
Userlevel 5
Badge +25

 

Hi,

 

I have managed to set the writter to export several shapefiles in a file. The problem in reading 7.500.000 features was, that i didnt have spatial indexes of the imported shapefile. As soon as we calculate the spatial indexes, process runed smootly threw the model and it was much faster.

 

 

Thanks a lot in your help!
Glad to hear that!

 

 

Reply