Question

How do I sort out from two polygon shape files(let's call them A and B) Where A exist but not B and Where B exist but not A?


Badge +1

Hello!

I have two different layers with polygon shapefiles. My goal is to create a workbench where Fastkart.by(A) and Map_polygon(B) where one exist but not the other. I will need to check both layers for this. So when I test A I need to check where buildings exist that are not in B.

A is esri shapefile. B is originally from CAD and has been saved as shapefile(Oracle spatial)

Also from B I need to select distinct values from attribute where I can filter out some kinds of houses from "gammelkod" and from "metod"(see picture 1). I need to do this to sort out information I don't need. Both these are varchar2.

When I run my workbench (se picture 2) first I need to sort out information to only have date in investigation areas. I run both these trough spatialfilter and have intersects with filter as filter.

Then I run both these in two different spatialfilter where one is filter and one is candidate and then the reverse order. This I think solves the rule "where A exist and not B" and "where B exist and not A". But still I have problem that some buildings intersects with layer but are not removed for some reason.

Also I have not figured out how to filter buildings I don't want from B.

Can anyone help me with this? If you don’t understand please write question and I will answer.

Best regards

Jesper


25 replies

Userlevel 2
Badge +17
I have two questions.

 

  1. What is the third source feature type (layer) within the bookmark "Ansvarsomrade" in your screenshot?
  2. What is the criteria that you determine if "an A polygon exists in B layer"? e.g. exactly equal polygon exists in B layer; the A polygon overlaps a B polygon; the A polygon contains a B polygon; etc. Could you please clarify the criteria using predicates on Spatial Relations? cf. Spatial Relations Defined

 

Badge +1
I have two questions.

 

  1. What is the third source feature type (layer) within the bookmark "Ansvarsomrade" in your screenshot?
  2. What is the criteria that you determine if "an A polygon exists in B layer"? e.g. exactly equal polygon exists in B layer; the A polygon overlaps a B polygon; the A polygon contains a B polygon; etc. Could you please clarify the criteria using predicates on Spatial Relations? cf. Spatial Relations Defined

 

  1. What is the third source feature type (layer) within the bookmark "Ansvarsomrade" in your screenshot?
The third source is the areas I will investigate. It’s a esri shapefile with polygons.

 

 

  1. What is the criteria that you determine if "an A polygon exists in B layer"? e.g. exactly equal polygon exists in B layer; the A polygon overlaps a B polygon; the A polygon contains a B polygon; etc. Could you please clarify the criteria using predicates on Spatial Relations? cf. Spatial Relations Defined
The criteria I have used in spatialrelations is intersects, within, touches and contains. I have selected pass against one filter but maybe this is the problem? I want the end result to be only buildings that do not touch, intersect, within or contain. It should be completely separated from each other. Layer A have buildings that are missing in B and layer B have buildings that are missing in A. I need to find all these buildings.

 

From B I need to filter so ONLY buildings that are of interest are the ones that are shown in end result.

 

 

 

Userlevel 2
Badge +17
OK. I think your workspace works as your expected, i.e. the polygons in the two destination layers ('fastkart.by' and 'cad_fail') would be completely separated. Did you check the result with FME Data Inspector?

 

 

Hi did you try the disjoint relation? If I understood correctly this is the one you need.

I also had similar issues in the past, when testing for multiple relations, to solve that I was using other transfromers. In your case I would suggest the clipper, the "outside" port would give the polygons you are looking for.

Badge +1

Hi did you try the disjoint relation? If I understood correctly this is the one you need.

I also had similar issues in the past, when testing for multiple relations, to solve that I was using other transfromers. In your case I would suggest the clipper, the "outside" port would give the polygons you are looking for.

  1. I tried the disjoint but for some reason I don't understand I don't get any separated buildings. When I have double spatialfilter I get the right result almost. For some reason some buildings that clearly intersects, touches etc are not treated as such. The spatialfilter don’t catch all the buildings for some reason. Any ideas why?
Tried the clipper but it does not work because buildings are not at exactly the same place. One layer is made from photos and the other one is measured with gps.

 

 

Any ideas why Spatialfilter don't remove all?

 

 

 

Badge +1
OK. I think your workspace works as your expected, i.e. the polygons in the two destination layers ('fastkart.by' and 'cad_fail') would be completely separated. Did you check the result with FME Data Inspector?

 

 

All except 10 attributes are completely separated. For some reason Spatialfilter don’t remove all even though they should have been removed. Also, do you know how I can filter distinct values from “metod” and from “Gammalkod”? I need to remove those who should not be in investigation.

 

 

 

  1. I tried the disjoint but for some reason I don't understand I don't get any separated buildings. When I have double spatialfilter I get the right result almost. For some reason some buildings that clearly intersects, touches etc are not treated as such. The spatialfilter don’t catch all the buildings for some reason. Any ideas why?
Tried the clipper but it does not work because buildings are not at exactly the same place. One layer is made from photos and the other one is measured with gps.

 

 

Any ideas why Spatialfilter don't remove all?

 

 

 

Is it possible to send few polygons so I can test, because I do not see why it doesnt work.

 

 

SpatialFilter does not work in most cases because of the number of records in "filter" and "candidate".
Is it possible to send few polygons so I can test, because I do not see why it doesnt work.

 

 

SpatialFilter does not work in most cases because of the number of records in "filter" and "candidate".
Also to filter distinct values in attributes you can use the tester or testfilter.

 

Do you already know the values you want to keep?

 

Userlevel 2
Badge +17
OK. I think your workspace works as your expected, i.e. the polygons in the two destination layers ('fastkart.by' and 'cad_fail') would be completely separated. Did you check the result with FME Data Inspector?

 

 

If you need to select just a single feature from those which have the same value in a specific attribute, the DuplicateFilter might help you.

 

 

Badge +1
Also to filter distinct values in attributes you can use the tester or testfilter.

 

Do you already know the values you want to keep?

 

Yes, I can send you a sample of the data and also my workbench if you want. How do I send it?

 

 

Yes, I already know what attributes I want to use but don't know how to add from an excel file or even manually.

 

 

Also to filter distinct values in attributes you can use the tester or testfilter.

 

Do you already know the values you want to keep?

 

You can zip them and upload them here using Insert File in the comment section.

 

 

Regarding how to filter using the excel file, you could read the excel and then use a featuremerger to merge based on the attributes in question. The merged port will give the records you want. That's the best way, so you do not have to write manually all the values you want to keep.

 

To do that manually you could use a tester and then add for left value the attribute name, oporator "=" and left value the unique values, pass critiria should be OR.

 

 

Userlevel 2
Badge +12

Have you tried the ChangeDetector transformer.

If I read the question right, this transformer will do exactly what you need.

Comparing on geometry and a selection of attributes.

A as Original and B as Revised will result in Added (in B not in A) and Deleted (in A not in B).

Badge +1

Hi did you try the disjoint relation? If I understood correctly this is the one you need.

I also had similar issues in the past, when testing for multiple relations, to solve that I was using other transfromers. In your case I would suggest the clipper, the "outside" port would give the polygons you are looking for.

Ok, I have zipped on of the areas of intrest and added here. Also added my most recent workbench.shape.zip.

 

 

I used attributemapper to define one of the attributes. The other one is "gammelkod" where I want to add one new values to distinct values. All that begin with bd and BM should get number 1 and all the rest 0.

 

After spatialfilter 2 i only want informtion that have vaules number 1 from the values and filter away the rest.

 

 

The buildings in map_abt are sepeerated now but need somehow to create polygon from map_abt_by when it's small gaps in between the lines.

 

Also i need to do the spatialfilter before I filter away data in gammelkod.

 

 

Badge +1

Have you tried the ChangeDetector transformer.

If I read the question right, this transformer will do exactly what you need.

Comparing on geometry and a selection of attributes.

A as Original and B as Revised will result in Added (in B not in A) and Deleted (in A not in B).

Yes, I have tried this but it does not work because both layers contain new buildings and a lot of buildings are overlapping some. This makes the result difficult to investigate.

 

Also I need to sort out the attribute from distinct values in 2 diffrent attributes.

 

 

Userlevel 2
Badge +17
Ok, I have zipped on of the areas of intrest and added here. Also added my most recent workbench.shape.zip.

 

 

I used attributemapper to define one of the attributes. The other one is "gammelkod" where I want to add one new values to distinct values. All that begin with bd and BM should get number 1 and all the rest 0.

 

After spatialfilter 2 i only want informtion that have vaules number 1 from the values and filter away the rest.

 

 

The buildings in map_abt are sepeerated now but need somehow to create polygon from map_abt_by when it's small gaps in between the lines.

 

Also i need to do the spatialfilter before I filter away data in gammelkod.

 

 

Hi @flugan87, a Shapefile dataset should consist of at least three files - *.shp, *.shx, and *.dbf. Since *.shx files are missing in your attachment, we cannot read data contents unfortunately. Can you please re-post complete Shapefile datasets?
Badge +1

Hello! Ok, om sorry :(. I try too add new document now. zipp.zip

Userlevel 2
Badge +17

I understood that you intend to find buildings which aren't duplicated between A and B. i.e. "A buildings that are not equal to any B building" and "B buildings that are not equal to any A building".

Looks like many buildings are duplicated between A and B, but there are slight difference in their coordinates in fact. Because of that, adjacent A building and B building could overlap. I think it's the reason why the spatial filtering operations in your workspace did not work as expected.

A workaround I can think of is: slightly shrink the geometries before spatial filtering in order to resolve the overlaps between adjacent A building and B building. See the attached workspace example.

esrishape-esrishape-esrishape2none.fmwt (FME 2017.1.2)

Badge +1

I understood that you intend to find buildings which aren't duplicated between A and B. i.e. "A buildings that are not equal to any B building" and "B buildings that are not equal to any A building".

Looks like many buildings are duplicated between A and B, but there are slight difference in their coordinates in fact. Because of that, adjacent A building and B building could overlap. I think it's the reason why the spatial filtering operations in your workspace did not work as expected.

A workaround I can think of is: slightly shrink the geometries before spatial filtering in order to resolve the overlaps between adjacent A building and B building. See the attached workspace example.

esrishape-esrishape-esrishape2none.fmwt (FME 2017.1.2)

Thank you Takashi! The result got alot better and made me realise the main problem with my data. Both A and B layer sometimes are two polygons when they should be one. Small gaps between them creates two polygons.

 

Mainly the problem is in B and I somehow need to adress this.

 

 

Badge +3

byggnader-test-gio.fmw

takashi

you can read the data. (at least I can)

Just remove readers after download and add the readers from the download location.

He has references to oracle and postgis in the fmw. Need to remove those.

flugan87

Use spatial filter with calculate cardinality. Listconcatenate relationshisp{}.card_area

Test if value 1 is in the list.

After relating all of them all the GAMMALKOD start with BD or BM... ( of course I have no idea what a GAMMLKOD is.. sound like fish ;))

I uploaded a fmw.

Also there are pretty severe differences in overlapping objects. Negative buffering will not take care of that.

You couldn use a AreaOnArea overlayer and calculate the percentage of overlap. (both ways) and check which have a significant overlap.

You will end up with some manual/eyeball work.

Badge +3

byggnader-test-gio.fmw

takashi

you can read the data. (at least I can)

Just remove readers after download and add the readers from the download location.

He has references to oracle and postgis in the fmw. Need to remove those.

flugan87

Use spatial filter with calculate cardinality. Listconcatenate relationshisp{}.card_area

Test if value 1 is in the list.

After relating all of them all the GAMMALKOD start with BD or BM... ( of course I have no idea what a GAMMLKOD is.. sound like fish ;))

I uploaded a fmw.

Also there are pretty severe differences in overlapping objects. Negative buffering will not take care of that.

You couldn use a AreaOnArea overlayer and calculate the percentage of overlap. (both ways) and check which have a significant overlap.

You will end up with some manual/eyeball work.

ok...google says it's gammal kod = old code

 

 

All hail google!

 

 

Badge +3

I understood that you intend to find buildings which aren't duplicated between A and B. i.e. "A buildings that are not equal to any B building" and "B buildings that are not equal to any A building".

Looks like many buildings are duplicated between A and B, but there are slight difference in their coordinates in fact. Because of that, adjacent A building and B building could overlap. I think it's the reason why the spatial filtering operations in your workspace did not work as expected.

A workaround I can think of is: slightly shrink the geometries before spatial filtering in order to resolve the overlaps between adjacent A building and B building. See the attached workspace example.

esrishape-esrishape-esrishape2none.fmwt (FME 2017.1.2)

@takashi

 

 

 

I think safe should not have removed the predicates overlaps and crosses.

 

 

Using the DE9IM is like trying to perform magic...lol.

 

 

Luckily cardinality is preserved.
Userlevel 2
Badge +17
@takashi

 

 

 

I think safe should not have removed the predicates overlaps and crosses.

 

 

Using the DE9IM is like trying to perform magic...lol.

 

 

Luckily cardinality is preserved.
@gio, you can still select Overlaps and Crosses if you change the Support Mode parameter to "Support All Predicates".

 

Badge +1

byggnader-test-gio.fmw

takashi

you can read the data. (at least I can)

Just remove readers after download and add the readers from the download location.

He has references to oracle and postgis in the fmw. Need to remove those.

flugan87

Use spatial filter with calculate cardinality. Listconcatenate relationshisp{}.card_area

Test if value 1 is in the list.

After relating all of them all the GAMMALKOD start with BD or BM... ( of course I have no idea what a GAMMLKOD is.. sound like fish ;))

I uploaded a fmw.

Also there are pretty severe differences in overlapping objects. Negative buffering will not take care of that.

You couldn use a AreaOnArea overlayer and calculate the percentage of overlap. (both ways) and check which have a significant overlap.

You will end up with some manual/eyeball work.

Haha I would not eat a fish called "GAMMLKOD" I can't open the fnw you sent for some reason. Can you send it again?

 

 

Badge +3

@flugan87 see it (firefox) for some reason changes extension to xml.

Just right-click download on link then change it to fmw

Badge +1

This is my latest workbench that does not work. I need too create polygon from CAD_lines. The ones that are connected to CAD_buildings are supposed to be one polygon.

 

Then I need to run the filters where A is but not B.

Any ideas why it does not work? zipp.zip

Reply