Skip to main content

Hi, does anyone know of a nice trick to delete features (rows/records) from a shapefile. We currently do this using the delete property in feature writer but this isn't available with shapefile.

We basically want to be able to delete where a property = true, the only other thing to mention is this might leave the shapefile empty, so we cant simply use a filter and a writer because FME wont write out an empty shapefile.

Many Thanks,

Oliver

I don't think there is any way to either delete records from an existing Shapefile dataset or create an empty Shapefile dataset, with FME regular functionalities unfortunately.


As @takashi has mentioned, there is not a "out-of-box" FME way to do this. If you do have Esri installed on the same machine you can access the arcpy - data access cursor and call the delete from a Python caller.

The other option would be to create an "intermediate" format that does allow for deletion - empty files (I would look at something like Spatial Light or MySQL Spatial) then once completed add a second step to export to the shape file. If the result it empty you can have a "template" empty shape file that can be copied into place.


What about having a template empty shapefile (created in ArcGIS), in the workspace use a tester and a NoFeaturesTester. If there is input, write the shapefile as normal, if there is no input, send the trigger feature to a filecopy writer to copy (and rename as necessary) the template file to the destination directory.


I don't think there is any way to either delete records from an existing Shapefile dataset or create an empty Shapefile dataset, with FME regular functionalities unfortunately.

Thanks Takashi


As @takashi has mentioned, there is not a "out-of-box" FME way to do this. If you do have Esri installed on the same machine you can access the arcpy - data access cursor and call the delete from a Python caller.

The other option would be to create an "intermediate" format that does allow for deletion - empty files (I would look at something like Spatial Light or MySQL Spatial) then once completed add a second step to export to the shape file. If the result it empty you can have a "template" empty shape file that can be copied into place.

Thanks @dellerbeck - we are using FME cloud so unfortunately this arcpy option isnt available to us.

The template shapefile is a good idea, unfortunately most of our content is so dynamic that managing the templates would be an ongoing pain.


What about having a template empty shapefile (created in ArcGIS), in the workspace use a tester and a NoFeaturesTester. If there is input, write the shapefile as normal, if there is no input, send the trigger feature to a filecopy writer to copy (and rename as necessary) the template file to the destination directory.

Thanks @jdh as most of the content we are playing with is dynamic having templates for each of our feature classes would be a pain but it is something we may have to investigate doing.


If your data is dynamic you could use the Feature Reader and set the response to Schema and Data. This will return a schema "object" that will create the output to be identical to the input. It's quite powerful and with some tweaks you can do quite a lot with the schema and make it dynamic. I have used this for a lot of applications where the end users can add \\ remove schema items.

One of the approaches that I have taken before, use 2 feature readers and return the schema object from each. You can use the change detector to compare the schemas at this point and then decide what to do with the output. If the change detector seems to not be a good fit, check the Safe forums - There are hundreds of ideas to compare objects that might also be a better fit.

That "should" allow you to add \\ delete rows and update schema if needed without the Esri licensing.


If your data is dynamic you could use the Feature Reader and set the response to Schema and Data. This will return a schema "object" that will create the output to be identical to the input. It's quite powerful and with some tweaks you can do quite a lot with the schema and make it dynamic. I have used this for a lot of applications where the end users can add \\ remove schema items.

One of the approaches that I have taken before, use 2 feature readers and return the schema object from each. You can use the change detector to compare the schemas at this point and then decide what to do with the output. If the change detector seems to not be a good fit, check the Safe forums - There are hundreds of ideas to compare objects that might also be a better fit.

That "should" allow you to add \\ delete rows and update schema if needed without the Esri licensing.

Hi @dellerbeck I appreciate the help on this. It took me a while to understand what you were proposing but I now think that the end result would still need a template empty shapefile. Even with using the change detector the output you could use would be the insert/new/changed or delete. In the case where an empty shapefile would be the output nothing would come through the insert/new/changed and so could not be used and even though the row/s would be identified for deletion you would then need a means to delete which isnt available in the feature writer for shapefile.

Sorry for the ramble, hope it makes sense. Thanks


Back in the days there was a similar question regarding the creation of empty shapefiles. This was my take then: if your shapefile has point or polygon geometry type, then build a non-closed line with a Creator. If it has a polyline geometry type, then build a point with a Creator and pass it directly into the shapefile.

FME sees you want to add a feature, so it's creating the shapefile. But the incompatible geometry type will reject that incoming feature, which you see as warning in your log file.


Hi @dellerbeck I appreciate the help on this. It took me a while to understand what you were proposing but I now think that the end result would still need a template empty shapefile. Even with using the change detector the output you could use would be the insert/new/changed or delete. In the case where an empty shapefile would be the output nothing would come through the insert/new/changed and so could not be used and even though the row/s would be identified for deletion you would then need a means to delete which isnt available in the feature writer for shapefile.

Sorry for the ramble, hope it makes sense. Thanks

Back in the days there was a similar question regarding the creation of empty shapefiles. This was my take then: if your shapefile has point or polygon geometry type, then build a non-closed line with a Creator. If it has a polyline geometry type, then build a point with a Creator and pass it directly into the shapefile.

FME sees you want to add a feature, so it's creating the shapefile. But the incompatible geometry type will reject that incoming feature, which you see as warning in your log file.


Back in the days there was a similar question regarding the creation of empty shapefiles. This was my take then: if your shapefile has point or polygon geometry type, then build a non-closed line with a Creator. If it has a polyline geometry type, then build a point with a Creator and pass it directly into the shapefile.

FME sees you want to add a feature, so it's creating the shapefile. But the incompatible geometry type will reject that incoming feature, which you see as warning in your log file.

thanks @klaas, we will take a look at this, seems a nice solution!


Reply