Skip to main content
  • EN

I have two sets of address data that I'm trying to match up using the CRCDetector as described here https://community.safe.com/s/article/creating-a-unique-identifier-crccalculator.

 

Geometry isn't a concern so there are only the 3 fields to match up but I'm getting different CRC values for things that should be the same.

imageSome of the algorithms will give a handful of matches but no more that a couple of dozen (out of about 12,000).

 

I've checked for trailing white spaces. What else is making it think these things are different?

 

Thanks

What is the configuration of the CRCCalculator?

A couple of things

  • It seems you're using a 16-bit CRC. If you have more than a handful of features you should consider using a different algorithm to avoid collisions (false positives), e.g. MD5
  • In the configuration, make sure to set "Calculate CRC on" = "Selected attributes only" and then manually select the attributes you want included. Otherwise there's a small chance that unexposed attributes (e.g. format attributes) are included, throwing off the results.

What is the configuration of the CRCCalculator?

image

A couple of things

  • It seems you're using a 16-bit CRC. If you have more than a handful of features you should consider using a different algorithm to avoid collisions (false positives), e.g. MD5
  • In the configuration, make sure to set "Calculate CRC on" = "Selected attributes only" and then manually select the attributes you want included. Otherwise there's a small chance that unexposed attributes (e.g. format attributes) are included, throwing off the results.

Hi @david_r​ 

 

I have tried all the algorithms. The most matches any of them managed was 32 out of twelve thousand records. I've also tried the ChangeDetector and I'm getting around eleven thousand matches (which is about what I would've expected).

 

It is set to calculate on the 3 address fields and nothing else.

Hi @david_r​ 

 

I have tried all the algorithms. The most matches any of them managed was 32 out of twelve thousand records. I've also tried the ChangeDetector and I'm getting around eleven thousand matches (which is about what I would've expected).

 

It is set to calculate on the 3 address fields and nothing else.

I wonder if a) the data types of the fields are different between the two data sets and b) if this would even effect anything.

 

Can you share a subset of your data?

Hi @david_r​ 

 

I have tried all the algorithms. The most matches any of them managed was 32 out of twelve thousand records. I've also tried the ChangeDetector and I'm getting around eleven thousand matches (which is about what I would've expected).

 

It is set to calculate on the 3 address fields and nothing else.

Thanks @hkingsbury​ 

 

So it wasn't that the data was different types, it was that the source fields were different lengths (even though the extra space was unused). I concatenated the six original fields into two new ones and then everything joined up.

Reply


Community Stats

30,108
Posts
114,295
Replies
37,744
Members
Terms & ConditionsCookie settings
A product of
Safe Software respectfully acknowledges that we live, learn and work on the traditional and unceded territories of the Kwantlen, Katzie, and Semiahmoo First Nations.